|
View previous topic :: View next topic
|
| Author |
Message |
thesumitk
Active User
Joined: 24 May 2013
Posts: 156
Location: INDIA
|
|
|
|
Hi All,
Recently I joined a new team where we are using the Pipe Delimited files and we all struggle to verify the data if require. For example if we have to check a certain field then we use to download the file and map it to excel because the there are so many fields which are not populated so we have to map it to PIPE delimited in Excel to get that particular field number.
For Example:
| Code: |
AAAA|BBB||||||CCCC||||||DDDD
AAAA- 1st Field
BBB - 2nd field
CCCC- 7th Field
DDDD-13th field |
| Code: |
| 3rd,4th,5th,6th,8th,9th,10,th,11th,12th field were not populated so there are only pipes no default values |
I am thinking to automate this process so if we have to map any file we can do in mainframe itself. Thinking to write a Cobol program which will unstring the file in to the copybook provided only challenge is that how I will take the copybook for file in run time, Files we can pass to the jcl thru a rexx exec or so but how we will use the copybook against the file passed for expanding.
Does any one have any idea on that , Please let me know .
Please let me know if I am not clear in my requirement or need to give more examples.
Thanks Much in advance.
Sumit |
|
| Back to top |
|
 |
Nic Clouston
Global Moderator
Joined: 10 May 2007
Posts: 2454
Location: Hampshire, UK
|
|
|
|
| Quote: |
| I am thinking to automate this process so if we have to map any DATA SET we can do ON the mainframe itself. |
| Quote: |
| Thinking to write a Cobol program which will unstring the DATA SET in to the copybook provided. |
This translates to..
| Quote: |
| unstring the DATA SET in to the code provided |
as a copybook contains code - maybe declarations, maybe executable statements
| Quote: |
| only challenge is that how I will take the copybook for file in run time, DATA SETS we can pass to the JCL thru a rexx exec or so but how we will use the copybook against the DATA SETpassed for expanding. |
Clear as mud - mud may be clearer! |
|
| Back to top |
|
|
thesumitk
Active User
Joined: 24 May 2013
Posts: 156
Location: INDIA
|
|
|
|
HI Nick,
Since the copybook fields declaration we can provide in the Program only in working storage for any particular file , so I am looking the way that we can pass this or may be change it in run time to make it generic.
If you are clear or know the way , please help. |
|
| Back to top |
|
|
thesumitk
Active User
Joined: 24 May 2013
Posts: 156
Location: INDIA
|
|
|
|
My Approach which I am thinking is below
1- get the file name and copybook(field declaration of file ) name from user
2- Get the file name overridden in the Skelton JCL
3- Override(Some how) the copybook name of working storage in Cobol program and unstring the record for all the fields mention in copybook and write it to output (this file can be map in file manager) - This is the challenge for me as of now
4-create a temp copybook(without the pipes) in user's RACFID.TEMP.DSN to map the file
Please let me know your thoughts on this . Many Thanks |
|
| Back to top |
|
|
thesumitk
Active User
Joined: 24 May 2013
Posts: 156
Location: INDIA
|
|
|
|
I have got below step to change it in run time but bad luck that it is not working
| Code: |
//A1 EXEC PGM=IGYCRCTL,
// PARM=('APOST,RENT,NOCOMPILE(S),NOSEQ,MAP',
// 'BUF(20K),OPT(STD),TERM,LIB'),COND=(0,NE)
//STEPLIB DD DISP=SHR,
// DSN=XXXX.XXXXX
//SYSPRINT DD DSN=XXX.XXXX,DISP=(,CATLG,DELETE),
// DCB=(LRECL=133,RECFM=FBA),
// UNIT=SYSDA,SPACE=(CYL,(1,1),RLSE)
//SYSTERM DD SYSOUT=*
//SYSIN DD *
PROCESS ARITH(EXTEND)
IDENTIFICATION DIVISION.
PROGRAM-ID. XXXXXX.
ENVIRONMENT DIVISION.
DATA DIVISION.
WORKING-STORAGE SECTION.
01 XXXXXXXXX PIC X(1).
COPY ©B.. - This is overridden by rexx
01 XXXXXX PIC X(1).
PROCEDURE DIVISION.
STOP RUN.
//SYSLIB DD DSN=XXXXXXXX,DISP=SHR
//SYSLIN DD DUMMY
//SYSUT1 DD UNIT=DISK,SPACE=(CYL,(5,1))
//SYSUT2 DD UNIT=DISK,SPACE=(CYL,(1,1))
//SYSUT3 DD UNIT=DISK,SPACE=(CYL,(1,1))
//SYSUT4 DD UNIT=DISK,SPACE=(CYL,(1,1))
//SYSUT5 DD UNIT=DISK,SPACE=(CYL,(2,1))
//SYSUT6 DD UNIT=DISK,SPACE=(CYL,(1,1))
//SYSUT7 DD UNIT=DISK,SPACE=(CYL,(1,1))
//SYSUDUMP DD SYSOUT=* |
S306-X'34' abend which means below
| Code: |
<H3> Abend Code S306-X'34' Explanation
306
Explanation: The error occurred during processing of a LINK(X), XCTL(X),
ATTACH(X), or LOAD macro.
An incorrect load to global request was attempted, or the authorized routine
requested a module that could not be found in an authorized library. The
module was found either in an unauthorized library or already loaded in
storage but marked as coming from an unauthorized library.
Code Explanation
34 The user attempted to use a program while a program-accessed data set was
open. |
Anyone aware of doing this please? |
|
| Back to top |
|
|
Garry Carroll
Senior Member
Joined: 08 May 2006
Posts: 1216
Location: Dublin, Ireland
|
|
|
|
I don't believe your approach is feasible. You can't dynamically change the copybook in a pre-compiled program. If you re-compile for each time, your field names change, so you have major code changes.
What might work - but would involved a lot of work - is to have your Cobol program read in your pipelined dataset and also read in the copybook as a dataset. You then have to match the pipelined field in the first dataset with the appropriate copybook field in the second dataset. The field name from the copybok can then be output with the related value for the pipelined dataset
Things get complicated when you have nested fields and in bypassing lines in
the copybook such as comments, values etc.
Garry. |
|
| Back to top |
|
|
prino
Senior Member
Joined: 07 Feb 2009
Posts: 1323
Location: Vilnius, Lithuania
|
|
|
|
| thesumitk wrote: |
| I have got below step to change it in run time but bad luck that it is not working |
Don't spout bovine excrement, you have written some code that does nothing like changing a copybook at runtime, because that is not possible without recompiling the entire program.
In PL/I the problem would be (relatively) easy to solve by just passing your delimited file and a file with offsets of all fields to the program, which would allow you to build the "copybook" on-the-fly in an internal procedure, been there done that. |
|
| Back to top |
|
|
thesumitk
Active User
Joined: 24 May 2013
Posts: 156
Location: INDIA
|
|
|
|
Thanks Garry for your suggestion .
The comments and complicated tasks I will handle in rexx exec which will run before this program step , Is this a way to compile my program in JCL itself instead of ENDEVOR , I am thinking to read my program in REXX step each time and change the copybook name and then compile in next step and run in next step. will that be fine?
Please let me know if I am not clear. Thanks
Sumit |
|
| Back to top |
|
|
thesumitk
Active User
Joined: 24 May 2013
Posts: 156
Location: INDIA
|
|
|
|
| prino wrote: |
In PL/I the problem would be (relatively) easy to solve by just passing your delimited file and a file with offsets of all fields to the program, which would allow you to build the "copybook" on-the-fly in an internal procedure, been there done that. |
Thanks Robert for your suggestion , I don't know Assembler but if you show me a way I can try that for sure , Please let me know if you have anything handy , Thanks |
|
| Back to top |
|
|
Garry Carroll
Senior Member
Joined: 08 May 2006
Posts: 1216
Location: Dublin, Ireland
|
|
|
|
| Quote: |
| I am thinking to read my program in REXX step each time and change the copybook name and then compile in next step and run in next step. will that be fine? |
No, that's an extremely complex and costly approach. How do you cater for copybooks where there are different numbers of variables, for example? The approach I suggested does not require recompile for each copybook. The one version of the program can cater for all copybooks, if coded correctly.
Garry. |
|
| Back to top |
|
|
prino
Senior Member
Joined: 07 Feb 2009
Posts: 1323
Location: Vilnius, Lithuania
|
|
|
|
Written on-the-fly, totally untested, other than that I know I did a similar thing in the past, and the second file contains length of fields, not offsets.
| Code: |
/* Roberts Q&D PL/I test program */
rahp: proc(parm) options(main) reorder;
dcl parm char (100) var;
dcl sysprint file;
dcl pipein file record input env(fb recsize (80));
dcl flenin file record input;
dcl eof bit (1) aligned init ('0'b);
dcl 1 flens,
2 flen pic 'zzz9',
2 filler char (76);
dcl a_flen(10) fixed bin (31) init ((10)0);
on endfile(flenin) eof = '1'b;
open file(flenin);
read file(flenin) into (flens);
do while (^eof);
i = i + 1;
a_flen(i) = flen;
read file(flenin) into (flens);
end;
call flenner();
flenner: proc;
on endfile(pipein) eof = '1'b;
dcl rec_80 char(80);
dcl 1 inrec,
2 fld_01 char(a_flen(01)),
2 fld_02 char(a_flen(02)),
2 fld_03 char(a_flen(03)),
2 fld_04 char(a_flen(04)),
2 fld_05 char(a_flen(05)),
2 fld_06 char(a_flen(06)),
2 fld_07 char(a_flen(07)),
2 fld_08 char(a_flen(08)),
2 fld_09 char(a_flen(09)),
2 fld_10 char(a_flen(10)),
2 filler char(80 - sum(a_flen));
eof = '0'b;
open file(pipein);
read file(pipein) into(rec_80);
do while(^eof);
/*
parse your inrec, splitting it on whatever character
and pit the pieces into fld_01 to fld_nn
verify the fld_nn fields
the first record of flenin can be set up to tell you the type
of file you're receiving, and even the LRECL and RECFM...
or use the parm passed to the main procedure
*/
read file(pipein) into(rec_80);
end;
end rahp; |
|
|
| Back to top |
|
|
Rohit Umarjikar
Global Moderator
Joined: 21 Sep 2010
Posts: 3109
Location: NYC,USA
|
|
|
|
Such tasks are difficult to automate and why automate? How much time does it take to get this PIPE delimited values to a excel cells ? 5 minutes or less. Once you make a shell of each data set ( you haven’t specified how many such you have) but you can do it once and reuse it.
Please explain in more detail why you need to unload it to a copybook and achieve what ? |
|
| Back to top |
|
|
thesumitk
Active User
Joined: 24 May 2013
Posts: 156
Location: INDIA
|
|
|
|
| Rohit Umarjikar wrote: |
Such tasks are difficult to automate and why automate? How much time does it take to get this PIPE delimited values to a excel cells ? 5 minutes or less. Once you make a shell of each data set ( you haven’t specified how many such you have) but you can do it once and reuse it.
Please explain in more detail why you need to unload it to a copybook and achieve what ? |
This is to make our daily life easy and save manual hours each day our entire team is spending such scenarios when they/we have to research an issue/item and goin to those big/large files and to see the values of certain fields ,which require to download them andmap them in excel.
5 minutes or more , that depends on the size of the file . |
|
| Back to top |
|
|
Marso
REXX Moderator
Joined: 13 Mar 2006
Posts: 1356
Location: Israel
|
|
|
|
| thesumitk wrote: |
For Example:
| Code: |
AAAA|BBB||||||CCCC||||||DDDD
AAAA- 1st Field
BBB - 2nd field
CCCC- 7th Field
DDDD-13th field |
| Code: |
| 3rd,4th,5th,6th,8th,9th,10,th,11th,12th field were not populated so there are only pipes no default values |
|
The easiest way, by far, is to use your SORT product.
First of all, get the Smart DFSORT Tricks pdf book (even if you have the other SORT product)
Next, read paragraph "Deconstruct and reconstruct CSV records"
Last, make your own (hint: drop the INREC and use PARSE in the OUTREC statement). |
|
| Back to top |
|
|
Marso
REXX Moderator
Joined: 13 Mar 2006
Posts: 1356
Location: Israel
|
|
|
|
I've read all the posts again, from top to bottom and back, and I think I understand what you want to achieve.
1. You have many different CSV files (the fact that the fields are separated by a vertical bar instead of a comma is minor).
2. For each of these files you have a corresponding copybook.
3. You want to reformat these CSV files so each field appear in fixed columns (just like Excel does).
4. You want this process to be as automatic as possible.
Here is what I would do:
1. Take one file and its corresponding copybook.
2. Manually write the sort step as explained in paragraph "Deconstruct and reconstruct CSV records"
After this stage, I have a step looking like this:
| Code: |
//*==============================================
//* REFORMAT CSV TO FIXED COLUMNS
//*==============================================
//STEP010 EXEC PGM=SORT
//SYSOUT DD SYSOUT=*
//SORTIN DD *
AAAA|BBB||||||CCCC||||||DDDD
//SORTOUT DD SYSOUT=*
//SYSIN DD *
OPTION COPY
INREC PARSE=(%01=(ENDBEFR=C'|',FIXLEN=8),
%02=(ENDBEFR=C'|',FIXLEN=12),
%03=(ENDBEFR=C'|',FIXLEN=2),
%04=(ENDBEFR=C'|',FIXLEN=4),
%05=(ENDBEFR=C'|',FIXLEN=8),
%06=(ENDBEFR=C'|',FIXLEN=10),
%07=(ENDBEFR=C'|',FIXLEN=6),
%08=(ENDBEFR=C'|',FIXLEN=8),
%09=(ENDBEFR=C'|',FIXLEN=3),
%10=(ENDBEFR=C'|',FIXLEN=12),
%11=(ENDBEFR=C'|',FIXLEN=6),
%12=(ENDBEFR=C'|',FIXLEN=4),
%13=(ADDPOS=1,FIXLEN=4)),
BUILD=(%01,C'|',
%02,C'|',
%03,C'|',
%04,C'|',
%05,C'|',
%06,C'|',
%07,C'|',
%08,C'|',
%09,C'|',
%10,C'|',
%11,C'|',
%12,C'|',
%13,C'|') |
and my output looks like this:
| Code: |
| AAAA |BBB | | | | | |CCCC | | | | |DDDD| |
I used random values for the FIXLEN= parameter. You should manually enter the values found in the copybook.
I think you have already met the paragraph "Create DFSORT Symbols from COBOL Copybook" from the same book, so:
3. Copy the JCL and the REXX program.
4. Make necessary modifications so it runs on your computer.
5. Run it with the copybook you have choosen.
The output will look like this:
| Code: |
CUSTOMER-GROUP,1,8,CH
CG-NAME,9,12,CH
CG-COUNT,21,2,CH
CG-DATE,23,4,CH
CG-TIME,27,8,CH
and so on... |
If everything is OK, the length (2nd number on each line) should be the same as the one found in the FIXLEN= parameter.
6. Modify the REXX program so that the output is
| Code: |
%01=(ENDBEFR=C'|',FIXLEN=8),
%02=(ENDBEFR=C'|',FIXLEN=12), |
instead of
| Code: |
CUSTOMER-GROUP,1,8,CH
CG-NAME,9,12,CH |
That should not be too hard...
7. Rerun your job again. Output should be now:
| Code: |
%01=(ENDBEFR=C'|',FIXLEN=8),
%02=(ENDBEFR=C'|',FIXLEN=12),
%03=(ENDBEFR=C'|',FIXLEN=2),
%04=(ENDBEFR=C'|',FIXLEN=4),
%05=(ENDBEFR=C'|',FIXLEN=8),
%06=(ENDBEFR=C'|',FIXLEN=10),
%07=(ENDBEFR=C'|',FIXLEN=6),
%08=(ENDBEFR=C'|',FIXLEN=8),
%09=(ENDBEFR=C'|',FIXLEN=3),
%10=(ENDBEFR=C'|',FIXLEN=12),
%11=(ENDBEFR=C'|',FIXLEN=6),
%12=(ENDBEFR=C'|',FIXLEN=4),
%13=(ENDBEFR=C'|',FIXLEN=4), |
8. Add some more code in the REXX program in order to deal with the 1st line (starting with INREC PARSE)
and the last one (ADDPOS instead of ENDBEFR, extra parenthesis).
9. Test again and again.
10. Repeat the write loop to generate the second part of your output. Generate now:
| Code: |
%01,C'|',
%02,C'|',
%03,C'|', |
11. Modify the REXX program again to include the BUILD=( and the last parenthesis.
If you survive this, you will have the INREC PARSE and BUILD that will be tailored according to the copybook.
The whole job runs in 3 steps:
1. the cobol compile of the copybook,
2. the rexx program used to generate the INREC statement,
3. the sort processing the CSV file
The final output will be a nicely formatted list.
Before submitting the job, you will have to modify the COPY statement in the dummy cobol program and the file name in SORTIN of step 3.
I usually get paid for this kind of job. I will contact you before my next travel to your beautiful country  |
|
| Back to top |
|
|
Rohit Umarjikar
Global Moderator
Joined: 21 Sep 2010
Posts: 3109
Location: NYC,USA
|
|
|
|
| Quote: |
| 5 minutes or more , that depends on the size of the file |
I thought one would only pick up bad ones only to troubleshoot and download them instead of the whole massive data set.
Second , when you transfer this pipe delimited data set to (from notepad) to CSV/Excel, don't that go to each cell when you say so ? I am sure it goes, and then take your copybook and transpose is horizontally as first line (header) to the same CSV/Excel, then you will have the mapping that you want with copybook name and data. I assume its readable data set and not a encrypted pipe delimited. To automate further , You can set up a FTP job to get into notepad and then a macro that pick up this notepad and put it into Excel/CSV and same goes for header.
Repeat the above process just once for other files and then you are done. We used to do this and this don't really consume time at all at least in our case. |
|
| Back to top |
|
|
|
|
  |
All times are GMT + 6 Hours |
|
