Joined: 07 Feb 2009 Posts: 1306 Location: Vilnius, Lithuania
I'm currently using this
Code:
* Merge the LW file coming out of LIFT
*
* These sort commands can handle both the old, it's left unchanged, as
* well as the new, records are merged, LW output file.
***********************************************************************
OPTION COPY
INREC IFTHEN=(WHEN=(1,1,CH,EQ,C'1',OR, ASA
114,1,CH,EQ,C'+',OR, Old eol
114,1,CH,EQ,C'|'), Old eol
OVERLAY=(122:C' ')),
IFTHEN=(WHEN=NONE,
OVERLAY=(122:SEQNUM,8,ZD,122:122,8,ZD,MOD,+2,M11,LENGTH=1))
*
OUTREC IFTHEN=(WHEN=GROUP,BEGIN=(122,1,CH,EQ,C'1'),
PUSH=(123:1,121))
*
OUTFIL FTOV,VLTRIM=C' ',
OMIT=(122,1,CH,EQ,C'1'),
IFTHEN=(WHEN=(122,1,CH,EQ,C'0'),BUILD=(123,121,1,121)),
IFTHEN=(WHEN=(122,1,CH,EQ,C' '),BUILD=(1,121))
which works like a charm.
However, the input file is about to change, and as we're still not willing to update the regression testing framework to cater for files other than FB(121), the above has to change for additional input records, which means another ifthen-when group. I guess it's not a problem to change the current overlay to a fixed '1' followed by the current seqnum logic (Correct me if I'm wrong!) and add another with a fixed '2' plus a mod +3 sequence number, the new records need to be assembled from three records. (Yes, sigh...)
However, can I use multiple "PUSH"es? The manual tells me
Quote:
You can use the following in PUSH:
c: Specifies the output position (column) to be overlaid. If you do not
specify c: for the first item, it defaults to 1:. If you do not specify c:
for any other item, it starts after the previous item. You can specify
items in any order and overlap output columns. c can be 1 to
32752.
If you specify an item that extends the output record beyond the
end of the input record, the record length is automatically
increased to that length, and blanks are filled in on the left as
needed. For variable-length records, the RDW length is increased
to correspond to the larger record length after all of the items are
processed. Missing bytes in specified input fields are replaced with
blanks so the padded fields can be processed.
p,m
Specifies a field in the first input record of each group to be
propagated to every record of the group. p specifies the starting
position of the field in the input record and m specifies its length.
A field must not extend beyond position 32752.
ID=n
Specifies a ZD identifier of length n is to be added to every record
of each group. The identifier starts at 1 for the first group and is
incremented by 1 for each subsequent group. n can be 1 to 15.
SEQ=n
Specifies a ZD sequence number of length n is to be added to
every record of each group. The sequence number starts at 1 for
the first record of each group and is incremented by 1 for each
subsequent record of the group. n can be 1 to 15.
Emphasis added, which seems that I can only merge data from the first record?
Joined: 07 Feb 2009 Posts: 1306 Location: Vilnius, Lithuania
The fundamental problem is that you cannot push more than the first record of a group.
I'm now trying to cook something up that would allow me to use the same sort control cards for two consecutive executions of SORT, where in the first pass the first two records of both tables are merged, and in the second pass only the remaining two records of the second table are merged, and using a step-by-step approach, dumping the output of executing the various SORT control statements on my input dataset to SYSOUT, I'm five-nines certain that this is indeed possible.
If a different GROUP then another PUSH should be allowed, any error?. To let quite follow the problem share us the sample input, rules and expected output and someone may come up with relevant suggestions or all together different solutions.
type '3' records are to be merged per 3 - there are always a multiple of 3, detection:
Code:
IFTHEN=(WHEN=NONE,
and as the format of the second record of type '2' is equal to the format of the second record of type '3', the IFTHEN=(WHEN statements have to be in this order. The last two conditions for the type '1' are used to process legacy data from a time that all output consisted of unsplit records, which still occurs when running the reqression test framework.
In:
Code:
1
2a
2b
2c
2d
2e
2f
1
3a
3b
3c
3d
3e
3f
3g
3h
3i
Out:
Code:
1
2a2b
2c2d
1
3a3b3c
3d3e3f
3g3h3i
Feel free to see if you can find a solution (just replacing the three "IFTHEN=(WHEN" with tests for '1', '2', and '3').
I'm pretty close to something that uses the same set of sort control statements for the two jobsteps I need (I'm open for a one jobstep solution using ICETOOL), which is probably more exotic than using the SUM FIELDS=() solution on Stackoverflow,
A bit tricky, but "it works" (the popular slogan here )
Restriction: only "full" line groups for each type are handled properly.
Code:
1
2a
2b
2c
2d
2e
2f
1
3a
3b
3c
3d
3e
3f
3g
3h
3i
1
2a
2b
2c
2d
2e
2f
Code:
INREC IFTHEN=(WHEN=GROUP,
KEYBEGIN=(1,1),
PUSH=(81:1,1,
82:SEQ=8)),
IFTHEN=(WHEN=(1,1,ZD,GT,+0),
OVERLAY=(82:(82,8,ZD, type '1': only 1
SUB,+1), type '2': 1-2-1-2-1-2-...
MOD,81,1,ZD, type '3': 1-2-3-1-2-3-...
ADD,+1,EDIT=(T)))
SORT FIELDS=COPY
OUTREC IFTHEN=(WHEN=GROUP,
BEGIN=(82,1,CH,EQ,C'3'),
PUSH=(94:1,2)),
IFTHEN=(WHEN=GROUP,
BEGIN=(82,1,CH,EQ,C'2'),
PUSH=(92:1,2)),
IFTHEN=(WHEN=GROUP,
BEGIN=(82,1,CH,EQ,C'1'),
PUSH=(90:1,2))
OUTFIL INCLUDE=(81,1,CH,EQ,82,1,CH), - only full group
IFTHEN=(WHEN=(81,1,CH,EQ,C'1'),
BUILD=(90,2,4X)),
IFTHEN=(WHEN=(81,1,CH,EQ,C'2'),
BUILD=(90,4,2X)),
IFTHEN=(WHEN=(81,1,CH,EQ,C'3'),
BUILD=(90,6))
END
Code:
********************************* TOP OF DATA *********
1
2a2b
2c2d
2e2f
1
3a3b3c
3d3e3f
3g3h3i
1
2a2b
2c2d
2e2f
******************************** BOTTOM OF DATA *******
Joined: 07 Feb 2009 Posts: 1306 Location: Vilnius, Lithuania
That's great, but I probably should never have given out plastic data that's way to easy to handle. Your
Code:
IFTHEN=(WHEN=(1,1,ZD,GT,+0)
relies on the '1', '2', '3', whereas the real data are rather more complex, as you can see in an earlier post, and column 1 can contain the following characters, 0-9, ' ', '.', '-' and 'V', and a more pertinent issue is that I'm running z/OS 1.10 (yes, that one...) with includes a very back-level version of DFSORT ('ICE201I F').
I thought I managed to get a solution, but it seems DFSORT parses the whole of
which is OK for the first pass when the input file is FB, but falls over on the second pass, when the input file is VB, and this even though the last four "IFTHEN"s are never ever matched in that case.
So I'm stuck with either two sets of sort control statements, or a modified version of the code on Stackoverflow, splitting up the summation over a large set of fields.
And to answer the inevitable "Why not produce long records?"
A1: The regression testing framework expects FB(121) files, and
A2: The output of SuperC LINE compare is limited to a maximum of 176 characters.
And the just as inevitable "How do you look at the output when it's folded?"
Output is stored in datasets of the format 'PRINO.*.T001010' (T011T020, T021030, etc) and when viewing a member containing weekly data, the viewer automagically merges the data, there's currently only a single table containing weekly data in the 'TxxxWPyy' members, displaying it as "PRINO.*.T00101O(TxxxWPyy)", deleting this dataset upon END of the VIEW session.
Using your single-pass solution would be ideal, but I guess it's not going to be.
Joined: 07 Feb 2009 Posts: 1306 Location: Vilnius, Lithuania
This is puzzling, cut and paste the data that I cut and pasted here back to z/OS, and it works, run it with the full output and it fails...
I'll investigate by "stepping" through the various "IFTHEN" statements (by putting successive "END" statements in) and look at what's happening with intermediate output.
Joined: 07 Feb 2009 Posts: 1306 Location: Vilnius, Lithuania
OK, mea culpa, mea maxima culpa!
Turns out that I was running with a newer version of the program, where the layout of the second table has been changed widening two columns (# & KM) by one character to cater for larger values.
The data with which I was testing was saved from the current "production" version of the program, and that's what I posted. Changing the 42 in the third IFTHEN to 44 delivers the correct result, I'm still tracing step by step to see what's actually going on at each control word, in case (not unlikely) that there's going to be a third table with months in the first column.
Sorry for posting the out-of-date data, but many thanks for showing that a one-pass solution is possible! ;)
* Whatever 0120
* --------------------------------------------------------------------
* Test column 9 for '+' or '|' OR column 41 for '+' or '|'
* - copy 365 ('1') to 364
* - add seqno to 367, for two records
* T-1 / P-1 1121
* T-1 / P-2 1122
* --------------------------------------------------------------------
* Test column 44 for '+' or '|'
* - copy 366 ('2') to 364
* - add seqno to 367, for three records
When looking at the output, the columns used in the second and third IFTHEN statements (9+41 and 44) are used to select the tables, and not used anywhere else.
Yet, if I change the 44 to 80 (see the line marked with !!) (in order to avoid clashing with "old data"), and 80 is just as unique as 44, only table 2 appears in the output, and the little bit of hair that I still have is rapidly disappearing.
Why the flipping 'ell doesn't using 80 instead of 44 work???
And why use two columns for the first compare, I've not changed that (yet), but it's just as puzzling...
The issue with column 80 vs 44 looks really weird...
For debug purposes, try to run without INCLUDE in the OUTFIL statement, and add current ‘0123’ combinations from position 364 to output lines?
One more trick is possible: if markers 1/2/3 were used to verify the line types (e.g. initial OVERLAY=(364:C’1231’...) rather than C’0120’ ), it could simplify several compare operations, as in OUTFIL:
INCLUDE=(364,1,CH,EQ,367,1,CH) - is enough
WHEN=(364,1,ZD,EQ,+1) - is enough
Etc.
)
