I have a requirement to change a few records based on the input file but wanted to dynamically create them via sort. The sort card would be used in the next aka 2nd step.
SORT FIELDS=COPY
OUTFIL REMOVECC,
HEADER1=(C' SORT FIELDS=COPY',
/,C' INREC IFTHEN=(WHEN=INIT,OVERLAY=(1:1,1)),'),
BUILD=(8X,C'IFTHEN=(WHEN=(26,3,CH,EQ,',
C'''020'',AND,60,6,CH,EQ,C''',1,6,C'''),',/,16X,
C'OVERLAY=(60:C''',8,5,C''')),'),
TRAILER1=(C' IFTHEN=(WHEN=NONE,OVERLAY=(1:1,1))',
/,C' END ')
END
Code:
********************************* TOP OF DATA **
SORT FIELDS=COPY
INREC IFTHEN=(WHEN=INIT,OVERLAY=(1:1,1)),
IFTHEN=(WHEN=(26,3,CH,EQ,'020',AND,60,6,CH,EQ,C'D09301'),
OVERLAY=(60:C'09301')),
IFTHEN=(WHEN=(26,3,CH,EQ,'020',AND,60,6,CH,EQ,C'D05401'),
OVERLAY=(60:C'05401')),
IFTHEN=(WHEN=(26,3,CH,EQ,'020',AND,60,6,CH,EQ,C'A19001'),
OVERLAY=(60:C'19001')),
IFTHEN=(WHEN=NONE,OVERLAY=(1:1,1))
END
******************************** BOTTOM OF DATA **
In the real life, a task like this one usually requires replacement of thousands to millions of different values to some replacing values. Such as, let's say, "substitution by a dictionary".
In this situation it becomes impossible to dynamically generate SORT statements with thousands or millions of parameters. Generation of the statements is possible, but the generated statements cannot be executed.
For such situation JOIN option of SORT utility is used, which matches possible values in the "master file" with corresponding values from the "dictionary file", and replaces found values as defined. This is "training number 2" to code a different solution for the initial task.
Joerg.Findeisen, as usual your suggestion worked perfectly.
Thanks again!
Sergeyken, I truly appreciate the alternate option. I thought about this but since the sort card would be less than 500, I didn't worry about maxing out the sort statements. Furthermore, I modified your solution for my purposes as I was going from a 6-bytes to 5-bytes (see below) and the remaining 1-byte made the field incorrect:
A really working solution, but with some disadvantages:
1) overcomplicated logic (see my first example above),
2) duplicated input of the same input data is required,
3) physically limited maximum number of allowed substitutions,
4) two-step processing is required, which is not necessary for this task (see my second example above).
Simple tasks need to be done by simple methods. This is my opinion.
I felt it's best to put the new requirement in different columns to further differentiate this. Again, I can do this in two passes but wanted to know if one pass is possible.
I felt it's best to put the new requirement in different columns to further differentiate this. Again, I can do this in two passes but wanted to know if one pass is possible.
You can easily modify my example with JOIN master and dictionary files, to achieve this "changed requirement".
But now I'll not give you a ready-to-copy-and-paste answer. Only use it as a hint.
A non-obvious problem with "new requirement" is, different size and position of the field to be matched (5 or 6 chars).
One way to resolve it - use master file preprocessing (as //JNF1CNTL DD) to create intermediate field of max size 6 chars, to be matched in following JOIN operation.
All fields listed in FIELDS= parameter must match simultaneously, in a single record.
Frankly, this way also would work, but only for specific test data - when all unused matching fields in all records are filled with blanks. AFAIU, in the real life this does not happen...
sergeyken wrote:
Wrong solution.
Did you try to test it?
All fields listed in FIELDS= parameter must match simultaneously, in a single record.
Yes, I tested this and it worked for my test data but I'll test it next w/real live data.
I'll keep you posted.
but only for specific test data - when all unused matching fields in all records are filled with blanks. AFAIU, in the real life this does not happen...
//*====================================================================
//SORTSELJ EXEC PGM=SORT
//*
//SYSOUT DD SYSOUT=*
//*
//SYMNAMES DD *
Dict_Match0,15,3,CH dictionary match field 1
Dict_MatchX,1,6,CH dictionary match field 2/3
Dict_Repl5,8,5,CH 5-chars replacement
Dict_Repl6,=,6,CH 6-chars replacement
*
Mast_Match0,26,3,CH master match field 1
Mast_Match6,60,6,CH master match field 2
Mast_Match5,40,5,CH master match field 3
*
Mast_Record,1,80,CH
Join_Ind,*,1,CH joining indicator: 'B', '1', '2'
Join_Repl5,*,5,CH replacement for 5-chars field
Join_Repl6,=,6,CH replacement for 6-cahrs field
Join_Match6,*,6,CH full 6-chars matching field
Join_Match5,=,5,CH 5-chars matching field
Join_Ext6,*,1,CH extention of 5-chars to 6-chars
//*
//DICT DD *
D09301 09301 020
D05401 05401 020
A19001 19001 020
12345 99999 010
67890 88888 010
13579 77777 010
//*-+----1----+----2----+----3----+----4----+----5----+----6----+----7
//MASTER DD *
020 XXXXX D09301
021 XXXXX D09301
020 XXXXX D09302
021 XXXXX D09302
020 XXXXX D05401
021 XXXXX D05401
020 XXXXX D05402
021 XXXXX D05402
020 XXXXX A19001
021 XXXXX A19001
020 XXXXX A19002
021 XXXXX A19002
010 12345 XXXXXX
010 67890 XXXXXX
011 67890 XXXXXX
010 13579 XXXXXX
//*-+----1----+----2----+----3----+----4----+----5----+----6----+----7
//*
//SORTOUT DD SYSOUT=*
//*....................................................................
//JNF1CNTL DD *
* prepare common match field for both 5 and 6 characters
INREC IFTHEN=(WHEN=(Mast_Match0,EQ,C'020'),
OVERLAY=(Join_Match6:Mast_Match6)),
IFTHEN=(WHEN=(Mast_Match0,EQ,C'010'),
OVERLAY=(Join_Match5:Mast_Match5,
Join_Ext6:X)),
IFTHEN=(WHEN=NONE,
OVERLAY=(Join_Match6:6X))
//*....................................................................
//SYSIN DD *
JOINKEYS F1=MASTER,
FIELDS=(Mast_Match0,A, fields to look for
Join_Match6,A) extended 6-char match
JOINKEYS F2=DICT,
FIELDS=(Dict_Match0,A, values to be replaced
Dict_MatchX,A) extended 6-char match
JOIN UNPAIRED,F1
REFORMAT FIELDS=(F1:Mast_Record, full master record
?, match indicator
F2:Dict_Repl6) value to replace, 5 or 6 chars
SORT FIELDS=COPY
OUTREC IFTHEN=(WHEN=(Join_Ind,EQ,C'B', match found?
AND,Mast_Match0,EQ,C'020'), especially marked?
OVERLAY=(Mast_Match6:Join_Repl6)), replace 6-char field
IFTHEN=(WHEN=(Join_Ind,EQ,C'B', match found?
AND,Mast_Match0,EQ,C'010'), especially marked?
OVERLAY=(Mast_Match5:Join_Repl5)) replace 5-char field
P.S.
By default, during JOIN the utility usually re-orders input records to improve join operation by using sorted data.
If that is not desired, the SORT code can be slightly modified, to restore the original order of records.
And yes, the data is already sorted, so I added the parms SORTED and NOSEQCK to the job, to aid in performance.
In general case, the fields to be replaced are rarely pre-sorted on input. That's why the order of output records may be often changed by SORT.
If the order of output data needs to be retained by business requirement, the more common solution can be like this:
Code:
//*====================================================================
//SORTSELJ EXEC PGM=SORT
//*
//SYSOUT DD SYSOUT=*
//*....................................................................
//SYMNAMES DD *
Dict_Match0,15,3,CH dictionary match field 1
Dict_MatchX,1,6,CH dictionary match field 2/3
Dict_Repl5,8,5,CH 5-chars replacement
Dict_Repl6,=,6,CH 6-chars replacement
*
Mast_Match0,26,3,CH master match field 1
Use5,C'010' marker to replace 5 chars
Use6,C'020' marker to replace 6 chars
Mast_Match6,60,6,CH master match field 2
Mast_Match5,40,5,CH master match field 3
*
Mast_Record,1,80,CH
Join_Ind,*,1,CH joining indicator: 'B', '1', '2'
Join_Repl5,*,5,CH replacement for 5-char field
Join_Repl6,=,6,CH replacement for 6-char field
Join_Seq,*,8,ZD initial record sequence number
Join_Match6,*,6,CH full 6-char matching field
Join_Match5,=,5,CH 5-char matching field
Join_Ext6,*,1,CH extention of 5-char to 6-char
//*....................................................................
//DICT DD *
D09301 09301 020
D05401 05401 020
A19001 19001 020
12345 99999 010
67890 88888 010
13579 77777 010
//*-+----1----+----2----+----3----+----4----+----5----+----6----+----7
//MASTER DD *
020 XXXXX D09301
021 XXXXX D09301
020 XXXXX D09302
021 XXXXX D09302
020 XXXXX D05401
021 XXXXX D05401
020 XXXXX D05402
021 XXXXX D05402
020 XXXXX A19001
021 XXXXX A19001
020 XXXXX A19002
021 XXXXX A19002
010 12345 XXXXXX
010 67890 XXXXXX
011 67890 XXXXXX
010 13579 XXXXXX
//*-+----1----+----2----+----3----+----4----+----5----+----6----+----7
//*
//SORTOUT DD SYSOUT=*
//*....................................................................
//JNF1CNTL DD *
* prepare common match field for both 5 and 6 characters
INREC IFTHEN=(WHEN=INIT,
OVERLAY=(Join_Seq:SEQNUM,8,ZD)), re-number input
IFTHEN=(WHEN=(Mast_Match0,EQ,Use6),
OVERLAY=(Join_Match6:Mast_Match6)), save 6-char value
IFTHEN=(WHEN=(Mast_Match0,EQ,Use5),
OVERLAY=(Join_Match5:Mast_Match5, save 5-char value
Join_Ext6:X)), as 6-char one
IFTHEN=(WHEN=NONE,
OVERLAY=(Join_Match6:6X)) clean-up field
//*....................................................................
//SYSIN DD *
JOINKEYS F1=MASTER,
FIELDS=(Mast_Match0,A, fields to look for
Join_Match6,A) extended 6-char match
JOINKEYS F2=DICT,
FIELDS=(Dict_Match0,A, values to be replaced
Dict_MatchX,A) extended 6-char match
JOIN UNPAIRED,F1
REFORMAT FIELDS=(F1:Mast_Record, full master record
?, match indicator
F2:Dict_Repl6, value to replace, 5 or 6 chars
F1:Join_Seq) initial record number
INREC IFTHEN=(WHEN=(Join_Ind,EQ,C'B', match found?
AND,Mast_Match0,EQ,Use6), especially marked?
OVERLAY=(Mast_Match6:Join_Repl6)), replace 6-char field
IFTHEN=(WHEN=(Join_Ind,EQ,C'B', match found?
AND,Mast_Match0,EQ,Use5), especially marked?
OVERLAY=(Mast_Match5:Join_Repl5)) replace 5-char field
