Portal | Manuals | References | Downloads | Info | Programs | JCLs | Mainframe wiki | Quick Ref
IBM Mainframe Forum Index
 
Register
 
IBM Mainframe Forum Index Mainframe: Search IBM Mainframe Forum: FAQ Memberlist Profile Log in to check your private messages Log in
 
Pipe Delimited File to xpand

 
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> COBOL Programming
View previous topic :: :: View next topic  
Author Message
thesumitk

Active User


Joined: 24 May 2013
Posts: 147
Location: INDIA

PostPosted: Tue Apr 02, 2019 1:02 pm    Post subject: Pipe Delimited File to xpand
Reply with quote

Hi All,

Recently I joined a new team where we are using the Pipe Delimited files and we all struggle to verify the data if require. For example if we have to check a certain field then we use to download the file and map it to excel because the there are so many fields which are not populated so we have to map it to PIPE delimited in Excel to get that particular field number.

For Example:

Code:
AAAA|BBB||||||CCCC||||||DDDD

AAAA- 1st Field
BBB - 2nd field
CCCC- 7th Field
DDDD-13th field


Code:
3rd,4th,5th,6th,8th,9th,10,th,11th,12th field were  not populated so there are only pipes  no default values



I am thinking to automate this process so if we have to map any file we can do in mainframe itself. Thinking to write a Cobol program which will unstring the file in to the copybook provided only challenge is that how I will take the copybook for file in run time, Files we can pass to the jcl thru a rexx exec or so but how we will use the copybook against the file passed for expanding.

Does any one have any idea on that , Please let me know .

Please let me know if I am not clear in my requirement or need to give more examples.

Thanks Much in advance.

Sumit
Back to top
View user's profile Send private message

Nic Clouston

Global Moderator


Joined: 10 May 2007
Posts: 2225
Location: Hampshire, UK

PostPosted: Tue Apr 02, 2019 5:29 pm    Post subject: Reply to: Pipe Delimited File to xpand
Reply with quote

Quote:
I am thinking to automate this process so if we have to map any DATA SET we can do ON the mainframe itself.


Quote:
Thinking to write a Cobol program which will unstring the DATA SET in to the copybook provided.

This translates to..
Quote:
unstring the DATA SET in to the code provided

as a copybook contains code - maybe declarations, maybe executable statements

Quote:
only challenge is that how I will take the copybook for file in run time, DATA SETS we can pass to the JCL thru a rexx exec or so but how we will use the copybook against the DATA SETpassed for expanding.

Clear as mud - mud may be clearer!
Back to top
View user's profile Send private message
thesumitk

Active User


Joined: 24 May 2013
Posts: 147
Location: INDIA

PostPosted: Tue Apr 02, 2019 5:55 pm    Post subject:
Reply with quote

HI Nick,

Since the copybook fields declaration we can provide in the Program only in working storage for any particular file , so I am looking the way that we can pass this or may be change it in run time to make it generic.

If you are clear or know the way , please help.
Back to top
View user's profile Send private message
thesumitk

Active User


Joined: 24 May 2013
Posts: 147
Location: INDIA

PostPosted: Tue Apr 02, 2019 6:01 pm    Post subject:
Reply with quote

My Approach which I am thinking is below

1- get the file name and copybook(field declaration of file ) name from user
2- Get the file name overridden in the Skelton JCL
3- Override(Some how) the copybook name of working storage in Cobol program and unstring the record for all the fields mention in copybook and write it to output (this file can be map in file manager) - This is the challenge for me as of now
4-create a temp copybook(without the pipes) in user's RACFID.TEMP.DSN to map the file

Please let me know your thoughts on this . Many Thanks
Back to top
View user's profile Send private message
thesumitk

Active User


Joined: 24 May 2013
Posts: 147
Location: INDIA

PostPosted: Tue Apr 02, 2019 6:39 pm    Post subject:
Reply with quote

I have got below step to change it in run time but bad luck that it is not working

Code:
//A1     EXEC PGM=IGYCRCTL,                                     
// PARM=('APOST,RENT,NOCOMPILE(S),NOSEQ,MAP',                     
// 'BUF(20K),OPT(STD),TERM,LIB'),COND=(0,NE)                     
//STEPLIB  DD  DISP=SHR,                                         
//             DSN=XXXX.XXXXX                                 
//SYSPRINT DD DSN=XXX.XXXX,DISP=(,CATLG,DELETE),   
//            DCB=(LRECL=133,RECFM=FBA),                         
//            UNIT=SYSDA,SPACE=(CYL,(1,1),RLSE)                   
//SYSTERM DD SYSOUT=*                                             
//SYSIN DD *                                                     
        PROCESS ARITH(EXTEND)                                     
       IDENTIFICATION DIVISION.                                   
       PROGRAM-ID. XXXXXX.                                     
       ENVIRONMENT DIVISION.                                     
       DATA DIVISION.                                             
       WORKING-STORAGE SECTION.                                   
        01 XXXXXXXXX  PIC X(1).                               
        COPY &COPYB..     - This is overridden by rexx                                           
        01 XXXXXX PIC X(1).                             
       PROCEDURE DIVISION.                                       
           STOP RUN.   
//SYSLIB   DD DSN=XXXXXXXX,DISP=SHR     
//SYSLIN   DD DUMMY                                     
//SYSUT1   DD UNIT=DISK,SPACE=(CYL,(5,1))               
//SYSUT2   DD  UNIT=DISK,SPACE=(CYL,(1,1))             
//SYSUT3   DD  UNIT=DISK,SPACE=(CYL,(1,1))             
//SYSUT4   DD  UNIT=DISK,SPACE=(CYL,(1,1))             
//SYSUT5   DD  UNIT=DISK,SPACE=(CYL,(2,1))             
//SYSUT6   DD  UNIT=DISK,SPACE=(CYL,(1,1))             
//SYSUT7   DD  UNIT=DISK,SPACE=(CYL,(1,1))             
//SYSUDUMP DD  SYSOUT=*


S306-X'34' abend which means below


Code:
<H3> Abend Code S306-X'34' Explanation                                       
                                                                             
  306                                                                         
                                                                             
  Explanation: The error occurred during processing of a LINK(X), XCTL(X),   
  ATTACH(X), or LOAD macro.                                                   
                                                                             
  An incorrect load to global request was attempted, or the authorized routine
  requested a module that could not be found in an authorized library. The   
  module was found either in an unauthorized library or already loaded in     
  storage but marked as coming from an unauthorized library.                 
                                                                             
Code Explanation
 34 The user attempted to use a program while a program-accessed data set was 
       open.           


Anyone aware of doing this please?
Back to top
View user's profile Send private message
Garry Carroll

Senior Member


Joined: 08 May 2006
Posts: 1053
Location: Dublin, Ireland / Edinburgh, Scotland

PostPosted: Tue Apr 02, 2019 6:52 pm    Post subject:
Reply with quote

I don't believe your approach is feasible. You can't dynamically change the copybook in a pre-compiled program. If you re-compile for each time, your field names change, so you have major code changes.

What might work - but would involved a lot of work - is to have your Cobol program read in your pipelined dataset and also read in the copybook as a dataset. You then have to match the pipelined field in the first dataset with the appropriate copybook field in the second dataset. The field name from the copybok can then be output with the related value for the pipelined dataset

Things get complicated when you have nested fields and in bypassing lines in
the copybook such as comments, values etc.

Garry.
Back to top
View user's profile Send private message
prino

Senior Member


Joined: 07 Feb 2009
Posts: 1161
Location: Vilnius, Lithuania

PostPosted: Tue Apr 02, 2019 7:29 pm    Post subject:
Reply with quote

thesumitk wrote:
I have got below step to change it in run time but bad luck that it is not working

Don't spout bovine excrement, you have written some code that does nothing like changing a copybook at runtime, because that is not possible without recompiling the entire program.

In PL/I the problem would be (relatively) easy to solve by just passing your delimited file and a file with offsets of all fields to the program, which would allow you to build the "copybook" on-the-fly in an internal procedure, been there done that.
Back to top
View user's profile Send private message
thesumitk

Active User


Joined: 24 May 2013
Posts: 147
Location: INDIA

PostPosted: Tue Apr 02, 2019 7:36 pm    Post subject:
Reply with quote

Thanks Garry for your suggestion .

The comments and complicated tasks I will handle in rexx exec which will run before this program step , Is this a way to compile my program in JCL itself instead of ENDEVOR , I am thinking to read my program in REXX step each time and change the copybook name and then compile in next step and run in next step. will that be fine?


Please let me know if I am not clear. Thanks

Sumit
Back to top
View user's profile Send private message
thesumitk

Active User


Joined: 24 May 2013
Posts: 147
Location: INDIA

PostPosted: Tue Apr 02, 2019 7:37 pm    Post subject:
Reply with quote

prino wrote:


In PL/I the problem would be (relatively) easy to solve by just passing your delimited file and a file with offsets of all fields to the program, which would allow you to build the "copybook" on-the-fly in an internal procedure, been there done that.


Thanks Robert for your suggestion , I don't know Assembler but if you show me a way I can try that for sure , Please let me know if you have anything handy , Thanks
Back to top
View user's profile Send private message
Garry Carroll

Senior Member


Joined: 08 May 2006
Posts: 1053
Location: Dublin, Ireland / Edinburgh, Scotland

PostPosted: Tue Apr 02, 2019 7:54 pm    Post subject:
Reply with quote

Quote:
I am thinking to read my program in REXX step each time and change the copybook name and then compile in next step and run in next step. will that be fine?


No, that's an extremely complex and costly approach. How do you cater for copybooks where there are different numbers of variables, for example? The approach I suggested does not require recompile for each copybook. The one version of the program can cater for all copybooks, if coded correctly.

Garry.
Back to top
View user's profile Send private message
prino

Senior Member


Joined: 07 Feb 2009
Posts: 1161
Location: Vilnius, Lithuania

PostPosted: Tue Apr 02, 2019 8:18 pm    Post subject:
Reply with quote

Written on-the-fly, totally untested, other than that I know I did a similar thing in the past, and the second file contains length of fields, not offsets.

Code:
 /* Roberts Q&D PL/I test program                              */
 rahp: proc(parm) options(main) reorder;
 dcl parm char     (100) var;
 dcl sysprint file;
 dcl pipein   file record input  env(fb recsize (80));
 dcl flenin   file record input;

 dcl eof bit (1) aligned init ('0'b);
 dcl 1 flens,
       2 flen   pic    'zzz9',
       2 filler char     (76);

 dcl a_flen(10) fixed bin (31) init ((10)0);

 on endfile(flenin) eof = '1'b;
 open file(flenin);

 read file(flenin) into (flens);
 do while (^eof);
   i         = i + 1;
   a_flen(i) = flen;
   read file(flenin) into (flens);
 end;

 call flenner();

 flenner: proc;
 on endfile(pipein) eof = '1'b;
 dcl rec_80 char(80);
 dcl 1 inrec,
       2 fld_01 char(a_flen(01)),
       2 fld_02 char(a_flen(02)),
       2 fld_03 char(a_flen(03)),
       2 fld_04 char(a_flen(04)),
       2 fld_05 char(a_flen(05)),
       2 fld_06 char(a_flen(06)),
       2 fld_07 char(a_flen(07)),
       2 fld_08 char(a_flen(08)),
       2 fld_09 char(a_flen(09)),
       2 fld_10 char(a_flen(10)),
       2 filler char(80 - sum(a_flen));
 eof = '0'b;
 open file(pipein);
 read file(pipein) into(rec_80);
 do while(^eof);
   /*
    parse your inrec, splitting it on whatever character
    and pit the pieces into fld_01 to fld_nn

    verify the fld_nn fields

    the first record of flenin can be set up to tell you the type
    of file you're receiving, and even the LRECL and RECFM...
    or use the parm passed to the main procedure
   */

   read file(pipein) into(rec_80);
 end;

 end rahp;
Back to top
View user's profile Send private message
Rohit Umarjikar

Senior Member


Joined: 21 Sep 2010
Posts: 2247
Location: NY,USA

PostPosted: Wed Apr 03, 2019 6:17 am    Post subject:
Reply with quote

Such tasks are difficult to automate and why automate? How much time does it take to get this PIPE delimited values to a excel cells ? 5 minutes or less. Once you make a shell of each data set ( you haven’t specified how many such you have) but you can do it once and reuse it.

Please explain in more detail why you need to unload it to a copybook and achieve what ?
Back to top
View user's profile Send private message
thesumitk

Active User


Joined: 24 May 2013
Posts: 147
Location: INDIA

PostPosted: Wed Apr 03, 2019 3:16 pm    Post subject:
Reply with quote

Rohit Umarjikar wrote:
Such tasks are difficult to automate and why automate? How much time does it take to get this PIPE delimited values to a excel cells ? 5 minutes or less. Once you make a shell of each data set ( you haven’t specified how many such you have) but you can do it once and reuse it.

Please explain in more detail why you need to unload it to a copybook and achieve what ?


This is to make our daily life easy and save manual hours each day our entire team is spending such scenarios when they/we have to research an issue/item and goin to those big/large files and to see the values of certain fields ,which require to download them andmap them in excel.

5 minutes or more , that depends on the size of the file .
Back to top
View user's profile Send private message
Marso

REXX Moderator


Joined: 13 Mar 2006
Posts: 1327
Location: Israel

PostPosted: Wed Apr 03, 2019 6:52 pm    Post subject: Re: Pipe Delimited File to xpand
Reply with quote

thesumitk wrote:
For Example:

Code:
AAAA|BBB||||||CCCC||||||DDDD

AAAA- 1st Field
BBB - 2nd field
CCCC- 7th Field
DDDD-13th field


Code:
3rd,4th,5th,6th,8th,9th,10,th,11th,12th field were  not populated so there are only pipes  no default values



The easiest way, by far, is to use your SORT product.
First of all, get the Smart DFSORT Tricks pdf book (even if you have the other SORT product)
Next, read paragraph "Deconstruct and reconstruct CSV records"
Last, make your own (hint: drop the INREC and use PARSE in the OUTREC statement).
Back to top
View user's profile Send private message
Marso

REXX Moderator


Joined: 13 Mar 2006
Posts: 1327
Location: Israel

PostPosted: Wed Apr 03, 2019 8:59 pm    Post subject:
Reply with quote

I've read all the posts again, from top to bottom and back, and I think I understand what you want to achieve.

1. You have many different CSV files (the fact that the fields are separated by a vertical bar instead of a comma is minor).
2. For each of these files you have a corresponding copybook.
3. You want to reformat these CSV files so each field appear in fixed columns (just like Excel does).
4. You want this process to be as automatic as possible.

Here is what I would do:

1. Take one file and its corresponding copybook.
2. Manually write the sort step as explained in paragraph "Deconstruct and reconstruct CSV records"

After this stage, I have a step looking like this:
Code:
//*==============================================
//*   REFORMAT CSV TO FIXED COLUMNS             
//*==============================================
//STEP010  EXEC PGM=SORT                         
//SYSOUT   DD   SYSOUT=*                         
//SORTIN   DD   *                               
AAAA|BBB||||||CCCC||||||DDDD                     
//SORTOUT  DD   SYSOUT=*                         
//SYSIN    DD   *                               
  OPTION COPY                                 
  INREC  PARSE=(%01=(ENDBEFR=C'|',FIXLEN=8), 
                %02=(ENDBEFR=C'|',FIXLEN=12),
                %03=(ENDBEFR=C'|',FIXLEN=2), 
                %04=(ENDBEFR=C'|',FIXLEN=4), 
                %05=(ENDBEFR=C'|',FIXLEN=8), 
                %06=(ENDBEFR=C'|',FIXLEN=10),
                %07=(ENDBEFR=C'|',FIXLEN=6), 
                %08=(ENDBEFR=C'|',FIXLEN=8), 
                %09=(ENDBEFR=C'|',FIXLEN=3), 
                %10=(ENDBEFR=C'|',FIXLEN=12),
                %11=(ENDBEFR=C'|',FIXLEN=6), 
                %12=(ENDBEFR=C'|',FIXLEN=4), 
                %13=(ADDPOS=1,FIXLEN=4)),
         BUILD=(%01,C'|', 
                %02,C'|', 
                %03,C'|', 
                %04,C'|', 
                %05,C'|', 
                %06,C'|', 
                %07,C'|', 
                %08,C'|', 
                %09,C'|', 
                %10,C'|', 
                %11,C'|', 
                %12,C'|', 
                %13,C'|') 
and my output looks like this:
Code:
AAAA    |BBB         |  |    |        |          |      |CCCC    |   |            |      |    |DDDD|
I used random values for the FIXLEN= parameter. You should manually enter the values found in the copybook.

I think you have already met the paragraph "Create DFSORT Symbols from COBOL Copybook" from the same book, so:

3. Copy the JCL and the REXX program.
4. Make necessary modifications so it runs on your computer.
5. Run it with the copybook you have choosen.

The output will look like this:
Code:
CUSTOMER-GROUP,1,8,CH
CG-NAME,9,12,CH
CG-COUNT,21,2,CH
CG-DATE,23,4,CH
CG-TIME,27,8,CH
and so on...
If everything is OK, the length (2nd number on each line) should be the same as the one found in the FIXLEN= parameter.

6. Modify the REXX program so that the output is
Code:
   %01=(ENDBEFR=C'|',FIXLEN=8),
   %02=(ENDBEFR=C'|',FIXLEN=12),
instead of
Code:
CUSTOMER-GROUP,1,8,CH
CG-NAME,9,12,CH
That should not be too hard...

7. Rerun your job again. Output should be now:
Code:
   %01=(ENDBEFR=C'|',FIXLEN=8), 
   %02=(ENDBEFR=C'|',FIXLEN=12),
   %03=(ENDBEFR=C'|',FIXLEN=2), 
   %04=(ENDBEFR=C'|',FIXLEN=4), 
   %05=(ENDBEFR=C'|',FIXLEN=8), 
   %06=(ENDBEFR=C'|',FIXLEN=10),
   %07=(ENDBEFR=C'|',FIXLEN=6), 
   %08=(ENDBEFR=C'|',FIXLEN=8), 
   %09=(ENDBEFR=C'|',FIXLEN=3), 
   %10=(ENDBEFR=C'|',FIXLEN=12),
   %11=(ENDBEFR=C'|',FIXLEN=6), 
   %12=(ENDBEFR=C'|',FIXLEN=4), 
   %13=(ENDBEFR=C'|',FIXLEN=4),

8. Add some more code in the REXX program in order to deal with the 1st line (starting with INREC PARSE)
and the last one (ADDPOS instead of ENDBEFR, extra parenthesis).
9. Test again and again.
10. Repeat the write loop to generate the second part of your output. Generate now:
Code:
  %01,C'|',
  %02,C'|',
  %03,C'|',

11. Modify the REXX program again to include the BUILD=( and the last parenthesis.

If you survive this, you will have the INREC PARSE and BUILD that will be tailored according to the copybook.

The whole job runs in 3 steps:
1. the cobol compile of the copybook,
2. the rexx program used to generate the INREC statement,
3. the sort processing the CSV file
The final output will be a nicely formatted list.

Before submitting the job, you will have to modify the COPY statement in the dummy cobol program and the file name in SORTIN of step 3.

I usually get paid for this kind of job. I will contact you before my next travel to your beautiful country icon_biggrin.gif
Back to top
View user's profile Send private message
Rohit Umarjikar

Senior Member


Joined: 21 Sep 2010
Posts: 2247
Location: NY,USA

PostPosted: Fri Apr 05, 2019 1:23 am    Post subject:
Reply with quote

Quote:
5 minutes or more , that depends on the size of the file

I thought one would only pick up bad ones only to troubleshoot and download them instead of the whole massive data set.

Second , when you transfer this pipe delimited data set to (from notepad) to CSV/Excel, don't that go to each cell when you say so ? I am sure it goes, and then take your copybook and transpose is horizontally as first line (header) to the same CSV/Excel, then you will have the mapping that you want with copybook name and data. I assume its readable data set and not a encrypted pipe delimited. To automate further , You can set up a FTP job to get into notepad and then a macro that pick up this notepad and put it into Excel/CSV and same goes for header.

Repeat the above process just once for other files and then you are done. We used to do this and this don't really consume time at all at least in our case.
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> COBOL Programming All times are GMT + 6 Hours
Page 1 of 1

 

Search our Forum:

Similar Topics
Topic Author Forum Replies Posted
No new posts What 'EF' in the CAFC command REGION,... vrkratheesh CICS 2 Wed Sep 18, 2019 4:52 pm
No new posts Can we pass the value of Symbolic par... Poha Eater JCL & VSAM 5 Thu Sep 05, 2019 2:21 pm
No new posts Search the strings present in ps file... Bhuvana Narasimhan All Other Mainframe Topics 8 Thu Aug 22, 2019 8:20 pm
No new posts Concatenate two files of variable rec... Thiru S DFSORT/ICETOOL 3 Wed Aug 21, 2019 11:53 pm
No new posts CICS VSAM file max volumes socker_dad JCL & VSAM 4 Tue Aug 20, 2019 1:28 am

Facebook
Back to Top
 
Job Vacancies | Forum Rules | Bookmarks | Subscriptions | FAQ | Polls | Contact Us