IBM Mainframe Forum Index
 
Log In
 
IBM Mainframe Forum Index Mainframe: Search IBM Mainframe Forum: FAQ Register
 

Merging more than two records


IBM Mainframe Forums -> DFSORT/ICETOOL
Post new topic   Reply to topic
View previous topic :: View next topic  
Author Message
prino
Warnings : 1

Senior Member


Joined: 07 Feb 2009
Posts: 1238
Location: Vilnius, Lithuania

PostPosted: Tue Jan 12, 2021 9:36 pm
Reply with quote

I'm currently using this
Code:
* Merge the LW file coming out of LIFT
*
* These sort commands can handle both the old, it's left unchanged, as
* well as the new, records are merged, LW output file.
***********************************************************************
 OPTION COPY
 INREC  IFTHEN=(WHEN=(1,1,CH,EQ,C'1',OR,                 ASA
                    114,1,CH,EQ,C'+',OR,                 Old eol
                    114,1,CH,EQ,C'|'),                   Old eol
       OVERLAY=(122:C' ')),
        IFTHEN=(WHEN=NONE,
       OVERLAY=(122:SEQNUM,8,ZD,122:122,8,ZD,MOD,+2,M11,LENGTH=1))
*
 OUTREC IFTHEN=(WHEN=GROUP,BEGIN=(122,1,CH,EQ,C'1'),
          PUSH=(123:1,121))
*
 OUTFIL FTOV,VLTRIM=C' ',
          OMIT=(122,1,CH,EQ,C'1'),
        IFTHEN=(WHEN=(122,1,CH,EQ,C'0'),BUILD=(123,121,1,121)),
        IFTHEN=(WHEN=(122,1,CH,EQ,C' '),BUILD=(1,121))


which works like a charm.

However, the input file is about to change, and as we're still not willing to update the regression testing framework to cater for files other than FB(121), the above has to change for additional input records, which means another ifthen-when group. I guess it's not a problem to change the current overlay to a fixed '1' followed by the current seqnum logic (Correct me if I'm wrong!) and add another with a fixed '2' plus a mod +3 sequence number, the new records need to be assembled from three records. (Yes, sigh...)

However, can I use multiple "PUSH"es? The manual tells me

Quote:
You can use the following in PUSH:
c: Specifies the output position (column) to be overlaid. If you do not
specify c: for the first item, it defaults to 1:. If you do not specify c:
for any other item, it starts after the previous item. You can specify
items in any order and overlap output columns. c can be 1 to
32752.
If you specify an item that extends the output record beyond the
end of the input record, the record length is automatically
increased to that length, and blanks are filled in on the left as
needed. For variable-length records, the RDW length is increased
to correspond to the larger record length after all of the items are
processed. Missing bytes in specified input fields are replaced with
blanks so the padded fields can be processed.
p,m
Specifies a field in the first input record of each group to be
propagated to every record of the group.
p specifies the starting
position of the field in the input record and m specifies its length.
A field must not extend beyond position 32752.
ID=n
Specifies a ZD identifier of length n is to be added to every record
of each group. The identifier starts at 1 for the first group and is
incremented by 1 for each subsequent group. n can be 1 to 15.
SEQ=n
Specifies a ZD sequence number of length n is to be added to
every record of each group. The sequence number starts at 1 for
the first record of each group and is incremented by 1 for each
subsequent record of the group. n can be 1 to 15.


Emphasis added, which seems that I can only merge data from the first record?

Or do I have to resort to something more exotic, like Mainframes JCL Record transposing using SORT.
Back to top
View user's profile Send private message
sergeyken

Active Member


Joined: 29 Apr 2008
Posts: 926
Location: Maryland

PostPosted: Tue Jan 12, 2021 11:44 pm
Reply with quote

Maybe, try to play with SEQNUM,...,RESTART=(x,y) parameters?

P.S.
It would be helpful to take a look at a sample of actual records...
Reverse engineering is boring...
Back to top
View user's profile Send private message
prino
Warnings : 1

Senior Member


Joined: 07 Feb 2009
Posts: 1238
Location: Vilnius, Lithuania

PostPosted: Wed Jan 13, 2021 2:05 am
Reply with quote

The fundamental problem is that you cannot push more than the first record of a group.

I'm now trying to cook something up that would allow me to use the same sort control cards for two consecutive executions of SORT, where in the first pass the first two records of both tables are merged, and in the second pass only the remaining two records of the second table are merged, and using a step-by-step approach, dumping the output of executing the various SORT control statements on my input dataset to SYSOUT, I'm five-nines certain that this is indeed possible.

If I succeed, I'll post the results here.
Back to top
View user's profile Send private message
Rohit Umarjikar

Global Moderator


Joined: 21 Sep 2010
Posts: 2571
Location: NYC,USA

PostPosted: Wed Jan 13, 2021 2:55 am
Reply with quote

If a different GROUP then another PUSH should be allowed, any error?. To let quite follow the problem share us the sample input, rules and expected output and someone may come up with relevant suggestions or all together different solutions.
Back to top
View user's profile Send private message
prino
Warnings : 1

Senior Member


Joined: 07 Feb 2009
Posts: 1238
Location: Vilnius, Lithuania

PostPosted: Wed Jan 13, 2021 5:28 pm
Reply with quote

OK, here's some plasic data, all real records are FB(121), output format is VB

  • type '1' records are stand-alone, separator lines, detection:
    Code:
    IFTHEN=(WHEN=(1,2,CH,EQ,C'1 ',OR,
                  1,23,CH,EQ,C'                       ',OR,
                114,1,CH,EQ,C'+',OR,
                114,1,CH,EQ,C'|'),

  • type '2' records are to be merged per 2 - there are always a multiple of 2, detection:
    Code:
    IFTHEN=(WHEN=(9,1,CH,EQ,C'+',OR,
                  9,1,CH,EQ,C'|',OR,
                 16,1,CH,EQ,C'+',OR,
                 16,1,CH,EQ,C'|'),

  • type '3' records are to be merged per 3 - there are always a multiple of 3, detection:
    Code:
    IFTHEN=(WHEN=NONE,

and as the format of the second record of type '2' is equal to the format of the second record of type '3', the IFTHEN=(WHEN statements have to be in this order. The last two conditions for the type '1' are used to process legacy data from a time that all output consisted of unsplit records, which still occurs when running the reqression test framework.

In:
Code:
1
2a
2b
2c
2d
2e
2f
1
3a
3b
3c
3d
3e
3f
3g
3h
3i

Out:
Code:
1
2a2b
2c2d
1
3a3b3c
3d3e3f
3g3h3i


Feel free to see if you can find a solution (just replacing the three "IFTHEN=(WHEN" with tests for '1', '2', and '3').

I'm pretty close to something that uses the same set of sort control statements for the two jobsteps I need (I'm open for a one jobstep solution using ICETOOL), which is probably more exotic than using the SUM FIELDS=() solution on Stackoverflow, icon_smile.gif
Back to top
View user's profile Send private message
prino
Warnings : 1

Senior Member


Joined: 07 Feb 2009
Posts: 1238
Location: Vilnius, Lithuania

PostPosted: Wed Jan 13, 2021 5:58 pm
Reply with quote

Here is an extract of the data, remember it needs to be saved in a FB(121) dataset, leading blanks are relevant, so take care when using Cut & Paste!

Code:
1                                                                                                                       
 +------+-------------------------------+-------------------------------+-------------------------------+----------------
---------------+-------------------------------+-------------------------------+-------------------------------+         
 |      | Monday                        | Tuesday                       | Wednesday                     | Thursday       
               | Friday                        | Saturday                      | Sunday                        |         
 |      +----+---------+--------+-------+----+---------+--------+-------+----+---------+--------+-------+----+---------+-
-------+-------+----+---------+--------+-------+----+---------+--------+-------+----+---------+--------+-------+         
 | Year | #D |      KM |   Time |    V= | #D |      KM |   Time |    V= | #D |      KM |   Time |    V= | #D |      KM |
  Time |    V= | #D |      KM |   Time |    V= | #D |      KM |   Time |    V= | #D |      KM |   Time |    V= |         
 +------+----+---------+--------+-------+----+---------+--------+-------+----+---------+--------+-------+----+---------+-
-------+-------+----+---------+--------+-------+----+---------+--------+-------+----+---------+--------+-------+         
 | 1980 |  4 |  1049.2 |  12:23 |  84.7 |  2 |   355.0 |   4:02 |  88.0 |  2 |   260.0 |   2:50 |  91.8 |  1 |    77.0 |
  0:54 |  85.6 |  2 |   366.0 |   4:31 |  81.0 |  1 |   128.0 |   1:58 |  65.1 |  1 |   746.0 |   7:06 | 105.1 |         
 | 1982 |  4 |  1229.4 |  14:14 |  86.4 |  1 |   303.1 |   3:26 |  88.3 |  4 |  1281.8 |  16:14 |  79.0 |  4 |  2316.2 |
 28:23 |  81.6 |  3 |  1596.8 |  17:52 |  89.4 |  2 |   682.1 |   9:39 |  70.7 |  2 |   492.2 |   5:41 |  86.6 |         
 | 1983 |  6 |  3653.6 |  38:30 |  94.9 |  5 |  3291.0 |  38:19 |  85.9 |  4 |  1089.5 |  13:08 |  83.0 |  4 |  1675.4 |
 22:02 |  76.0 |  5 |  2029.3 |  23:36 |  86.0 |  1 |   282.8 |   3:25 |  82.8 |    |         |        |       |         
 | 1985 |  4 |  2237.9 |  25:37 |  87.4 |  3 |  1719.7 |  18:22 |  93.6 |  2 |  2267.3 |  22:17 | 101.7 |  2 |  1169.3 |
 11:30 | 101.7 |    |         |        |       |  1 |   652.3 |   8:46 |  74.4 |  1 |   483.4 |   7:02 |  68.7 |         
 +------+----+---------+--------+-------+----+---------+--------+-------+----+---------+--------+-------+----+---------+-
-------+-------+----+---------+--------+-------+----+---------+--------+-------+----+---------+--------+-------+         
                                                                                                                         
 +-----+---------------------------------+---------------------------------+---------------------------------+-----------
----------------------+---------------------------------+---------------------------------+------------------------------
---+---------------------------------+                                                                                   
 |     | Monday                          | Tuesday                         | Wednesday                       | Thursday 
                      | Friday                          | Saturday                        | Sunday                       
   | Total                           |                                                                                   
 |     +------+---------+--------+-------+------+---------+--------+-------+------+---------+--------+-------+------+----
-----+--------+-------+------+---------+--------+-------+------+---------+--------+-------+------+---------+--------+----
---+------+---------+--------+-------+                                                                                   
 | Day |    # |      KM |   Time |    V= |    # |      KM |   Time |    V= |    # |      KM |   Time |    V= |    # |   
  KM |   Time |    V= |    # |      KM |   Time |    V= |    # |      KM |   Time |    V= |    # |      KM |   Time |   
V= |    # |      KM |   Time |    V= |                                                                                   
 +-----+------+---------+--------+-------+------+---------+--------+-------+------+---------+--------+-------+------+----
-----+--------+-------+------+---------+--------+-------+------+---------+--------+-------+------+---------+--------+----
---+------+---------+--------+-------+                                                                                   
 |   1 |   27 |  4462.0 |  42:47 | 104.3 |   28 |  5466.2 |  53:48 | 101.6 |   61 |  7389.2 |  67:51 | 108.9 |   35 |  50
04.6 |  44:29 | 112.5 |   59 |  7552.0 |  73:40 | 102.5 |   16 |  1049.5 |  10:15 | 102.4 |   20 |  2878.4 |  26:51 | 107
.2 |  246 | 33801.9 | 319:41 | 105.7 |                                                                                   
 |   2 |   27 |  3233.6 |  32:30 |  99.5 |   20 |  3169.4 |  32:22 |  97.9 |   37 |  3110.3 |  30:32 | 101.9 |   18 |  51
45.6 |  53:06 |  96.9 |   15 |  3385.8 |  33:01 | 102.5 |   36 |  4217.1 |  42:17 |  99.7 |    7 |   872.1 |   7:31 | 116
.0 |  160 | 23133.9 | 231:19 | 100.0 |                                                                                   
 +-----+------+---------+--------+-------+------+---------+--------+-------+------+---------+--------+-------+------+----
-----+--------+-------+------+---------+--------+-------+------+---------+--------+-------+------+---------+--------+----
---+------+---------+--------+-------+                                                                                   
 | Tot |  971 | 19197.7 | 119:08 | 106.5 | 1009 | 37611.1 | 306:19 | 105.3 |  871 | 19047.8 | 109:28 | 107.3 |  673 | 938
83.3 | 915:33 | 102.5 |  650 | 77116.4 | 758:42 | 101.6 |  321 | 38841.0 | 377:36 | 102.9 |  293 | 44228.8 | 407:06 | 108
.6 | 4788 | 29926.1 | 993:52 | 105.1 |                                                                                   
 +-----+------+---------+--------+-------+------+---------+--------+-------+------+---------+--------+-------+------+----
-----+--------+-------+------+---------+--------+-------+------+---------+--------+-------+------+---------+--------+----
---+------+---------+--------+-------+                                                                                   
Back to top
View user's profile Send private message
sergeyken

Active Member


Joined: 29 Apr 2008
Posts: 926
Location: Maryland

PostPosted: Wed Jan 13, 2021 11:27 pm
Reply with quote

A bit tricky, but "it works" (the popular slogan here icon_biggrin.gif )
Restriction: only "full" line groups for each type are handled properly.

Code:
1
2a
2b
2c
2d
2e
2f
1
3a
3b
3c
3d
3e
3f
3g
3h
3i
1
2a
2b
2c
2d
2e
2f

Code:
 INREC IFTHEN=(WHEN=GROUP,
               KEYBEGIN=(1,1),
               PUSH=(81:1,1,
                     82:SEQ=8)),
       IFTHEN=(WHEN=(1,1,ZD,GT,+0),
               OVERLAY=(82:(82,8,ZD,   type '1': only 1
                            SUB,+1),   type '2': 1-2-1-2-1-2-...
                        MOD,81,1,ZD,   type '3': 1-2-3-1-2-3-...
                        ADD,+1,EDIT=(T)))
 SORT FIELDS=COPY
 OUTREC IFTHEN=(WHEN=GROUP,
                BEGIN=(82,1,CH,EQ,C'3'),
                PUSH=(94:1,2)),
        IFTHEN=(WHEN=GROUP,
                BEGIN=(82,1,CH,EQ,C'2'),
                PUSH=(92:1,2)),
        IFTHEN=(WHEN=GROUP,
                BEGIN=(82,1,CH,EQ,C'1'),
                PUSH=(90:1,2))
 OUTFIL INCLUDE=(81,1,CH,EQ,82,1,CH),   - only full group
        IFTHEN=(WHEN=(81,1,CH,EQ,C'1'),
                BUILD=(90,2,4X)),
        IFTHEN=(WHEN=(81,1,CH,EQ,C'2'),
                BUILD=(90,4,2X)),
        IFTHEN=(WHEN=(81,1,CH,EQ,C'3'),
                BUILD=(90,6))
 END

Code:
********************************* TOP OF DATA *********
1
2a2b
2c2d
2e2f
1
3a3b3c
3d3e3f
3g3h3i
1
2a2b
2c2d
2e2f
******************************** BOTTOM OF DATA *******
Back to top
View user's profile Send private message
Joerg.Findeisen

Active User


Joined: 15 Aug 2015
Posts: 387
Location: Bamberg, Germany

PostPosted: Thu Jan 14, 2021 12:21 am
Reply with quote

Code:
1
2a
2b
2c
2d
2e
2f
1
3a
3b
3c
3d
3e
3f
3g
3h
3i


More steps but same result:
Code:
  INREC IFTHEN=(WHEN=INIT,OVERLAY=(35:SEQNUM,4,ZD)),                 
    IFTHEN=(WHEN=(1,1,CH,EQ,C'1'),                                   
      OVERLAY=(29:+0,M11,LENGTH=1),HIT=NEXT),                         
*                                                                     
    IFTHEN=(WHEN=(1,1,CH,EQ,C'2'),                                   
      OVERLAY=(29:SEQNUM,1,ZD,START=0),HIT=NEXT),                     
    IFTHEN=(WHEN=(1,1,CH,EQ,C'2'),                                   
      OVERLAY=(29:29,1,ZD,MOD,+2,M11,LENGTH=1),HIT=NEXT),             
*                                                                     
    IFTHEN=(WHEN=(1,1,CH,EQ,C'3'),                                   
      OVERLAY=(29:SEQNUM,1,ZD,START=0),HIT=NEXT),                     
    IFTHEN=(WHEN=(1,1,CH,EQ,C'3'),                                   
      OVERLAY=(29:29,1,ZD,MOD,+3,M11,LENGTH=1),HIT=NEXT),             
*                                                                     
    IFTHEN=(WHEN=(29,1,ZD,EQ,+0),OVERLAY=(3:4Z),HIT=NEXT),           
    IFTHEN=(WHEN=(29,1,ZD,EQ,+1),OVERLAY=(3:1,2,1:2Z,5:2Z),HIT=NEXT),
    IFTHEN=(WHEN=(29,1,ZD,EQ,+2),OVERLAY=(5:1,2,1:4Z),HIT=NEXT),     
*                                                                     
    IFTHEN=(WHEN=ANY,OVERLAY=(35:35,4,ZD,SUB,29,1,ZD,M11,LENGTH=4))   
*                                                                     
    SORT FIELDS=(35,4,ZD,A)                                           
    SUM FIELDS=(1,2,BI,3,2,BI,5,2,BI)                                 
*                                                                     
    OUTFIL FNAMES=(SORTOUT),                                         
      REMOVECC,                                                       
      BUILD=(1,6)                                                     
  END


However the SUM is not really practical for the real data.
Back to top
View user's profile Send private message
prino
Warnings : 1

Senior Member


Joined: 07 Feb 2009
Posts: 1238
Location: Vilnius, Lithuania

PostPosted: Thu Jan 14, 2021 12:53 am
Reply with quote

That's great, but I probably should never have given out plastic data that's way to easy to handle. Your
Code:
IFTHEN=(WHEN=(1,1,ZD,GT,+0)
relies on the '1', '2', '3', whereas the real data are rather more complex, as you can see in an earlier post, and column 1 can contain the following characters, 0-9, ' ', '.', '-' and 'V', and a more pertinent issue is that I'm running z/OS 1.10 (yes, that one...) with includes a very back-level version of DFSORT ('ICE201I F').

I thought I managed to get a solution, but it seems DFSORT parses the whole of

Code:
OUTFIL FTOV,VLTRIM=C' ',
         OMIT=(254,1,CH,EQ,C'1',OR,247,4,CH,EQ,C'Y001'),
       IFTHEN=(WHEN=(247,1,CH,EQ,C'R'),BUILD=(1,237)),
       IFTHEN=(WHEN=(247,1,CH,EQ,C'Y'),
                     BUILD=(1,4,255,242,5,54)),
       IFTHEN=(WHEN=(248,2,CH,EQ,C'X0'),BUILD=(5,121,243:C'Q')),
       IFTHEN=(WHEN=(249,1,CH,EQ,C'1'),
                     BUILD=(255,121,5,121,243:C'Q')),
       IFTHEN=(WHEN=(250,1,CH,EQ,C'2'),
                     BUILD=(255,121,5,121,243:C'X')),
       IFTHEN=(WHEN=(248,1,CH,EQ,C'X'),BUILD=(5,121,243:C'X'))
which is OK for the first pass when the input file is FB, but falls over on the second pass, when the input file is VB, and this even though the last four "IFTHEN"s are never ever matched in that case.

So I'm stuck with either two sets of sort control statements, or a modified version of the code on Stackoverflow, splitting up the summation over a large set of fields.

And to answer the inevitable "Why not produce long records?"

A1: The regression testing framework expects FB(121) files, and
A2: The output of SuperC LINE compare is limited to a maximum of 176 characters.

And the just as inevitable "How do you look at the output when it's folded?"

Output is stored in datasets of the format 'PRINO.*.T001010' (T011T020, T021030, etc) and when viewing a member containing weekly data, the viewer automagically merges the data, there's currently only a single table containing weekly data in the 'TxxxWPyy' members, displaying it as "PRINO.*.T00101O(TxxxWPyy)", deleting this dataset upon END of the VIEW session.

Using your single-pass solution would be ideal, but I guess it's not going to be.
Back to top
View user's profile Send private message
Joerg.Findeisen

Active User


Joined: 15 Aug 2015
Posts: 387
Location: Bamberg, Germany

PostPosted: Thu Jan 14, 2021 4:42 am
Reply with quote

@Prino: Is this what you want to see as VB Output from the provided folded data?

Code:
//REFORMAT EXEC PGM=ICEMAN                                           
//SORTIN   DD DISP=SHR,DSN=&SYSUID..FB121                           
//SYSOUT   DD SYSOUT=*                                               
//SORTOUT  DD SYSOUT=*                                               
//SYSIN    DD *                                                     
  OPTION COPY                                                       
  INREC IFTHEN=(WHEN=INIT,OVERLAY=(364:+120,ZD,LENGTH=4)),           
    IFTHEN=(WHEN=GROUP,BEGIN=(9,1,SS,EQ,C'+|',OR,41,1,SS,EQ,C'+|'), 
      PUSH=(364:365,1,367:SEQ=1),RECORDS=2),                         
    IFTHEN=(WHEN=GROUP,BEGIN=(42,1,SS,EQ,C'+|'),                     
      PUSH=(364:366,1,367:SEQ=1),RECORDS=3),                         
    IFTHEN=(WHEN=GROUP,BEGIN=(367,1,ZD,EQ,+1),                       
      PUSH=(122:1,121)),                                             
    IFTHEN=(WHEN=GROUP,BEGIN=(367,1,ZD,EQ,+2),                       
      PUSH=(243:1,121))                                             
  OUTFIL FNAMES=(SORTOUT),                                           
    INCLUDE=((364,1,ZD,EQ,+0),OR,                                   
             (364,1,ZD,EQ,+1,AND,367,1,ZD,EQ,+2),OR,                 
             (364,1,ZD,EQ,+2,AND,367,1,ZD,EQ,+3)),                   
    FTOV,VLTRIM=C' ',                                               
    IFTHEN=(WHEN=(364,1,ZD,EQ,+0),BUILD=(1,121)),                   
    IFTHEN=(WHEN=(364,1,ZD,EQ,+1,AND,367,1,ZD,EQ,+2),               
      BUILD=(122,121,1,121)),                                       
    IFTHEN=(WHEN=(364,1,ZD,EQ,+2,AND,367,1,ZD,EQ,+3),               
      BUILD=(122,121,243,121,1,121))                                 
  END                                                               
/*
Back to top
View user's profile Send private message
prino
Warnings : 1

Senior Member


Joined: 07 Feb 2009
Posts: 1238
Location: Vilnius, Lithuania

PostPosted: Thu Jan 14, 2021 2:28 pm
Reply with quote

Joerg.Findeisen wrote:
@Prino: Is this what you want to see as VB Output from the provided folded data?
Yes, and sadly, no: The first table comes out OK when run with your sort control statements, the second is still completely folded. icon_sad.gif
Back to top
View user's profile Send private message
Joerg.Findeisen

Active User


Joined: 15 Aug 2015
Posts: 387
Location: Bamberg, Germany

PostPosted: Thu Jan 14, 2021 2:38 pm
Reply with quote

I have successfully tried with this data from you. So maybe c&p somewhere, but I had checked a couple of times. icon_neutral.gif

Organisation . . . : PS
Rec.format . . . . : FB
Rec.Length . . . . : 121

prino wrote:
Code:
1                                                                                                                       
 +------+-------------------------------+-------------------------------+-------------------------------+----------------
---------------+-------------------------------+-------------------------------+-------------------------------+         
 |      | Monday                        | Tuesday                       | Wednesday                     | Thursday       
               | Friday                        | Saturday                      | Sunday                        |         
 |      +----+---------+--------+-------+----+---------+--------+-------+----+---------+--------+-------+----+---------+-
-------+-------+----+---------+--------+-------+----+---------+--------+-------+----+---------+--------+-------+         
 | Year | #D |      KM |   Time |    V= | #D |      KM |   Time |    V= | #D |      KM |   Time |    V= | #D |      KM |
  Time |    V= | #D |      KM |   Time |    V= | #D |      KM |   Time |    V= | #D |      KM |   Time |    V= |         
 +------+----+---------+--------+-------+----+---------+--------+-------+----+---------+--------+-------+----+---------+-
-------+-------+----+---------+--------+-------+----+---------+--------+-------+----+---------+--------+-------+         
 | 1980 |  4 |  1049.2 |  12:23 |  84.7 |  2 |   355.0 |   4:02 |  88.0 |  2 |   260.0 |   2:50 |  91.8 |  1 |    77.0 |
  0:54 |  85.6 |  2 |   366.0 |   4:31 |  81.0 |  1 |   128.0 |   1:58 |  65.1 |  1 |   746.0 |   7:06 | 105.1 |         
 | 1982 |  4 |  1229.4 |  14:14 |  86.4 |  1 |   303.1 |   3:26 |  88.3 |  4 |  1281.8 |  16:14 |  79.0 |  4 |  2316.2 |
 28:23 |  81.6 |  3 |  1596.8 |  17:52 |  89.4 |  2 |   682.1 |   9:39 |  70.7 |  2 |   492.2 |   5:41 |  86.6 |         
 | 1983 |  6 |  3653.6 |  38:30 |  94.9 |  5 |  3291.0 |  38:19 |  85.9 |  4 |  1089.5 |  13:08 |  83.0 |  4 |  1675.4 |
 22:02 |  76.0 |  5 |  2029.3 |  23:36 |  86.0 |  1 |   282.8 |   3:25 |  82.8 |    |         |        |       |         
 | 1985 |  4 |  2237.9 |  25:37 |  87.4 |  3 |  1719.7 |  18:22 |  93.6 |  2 |  2267.3 |  22:17 | 101.7 |  2 |  1169.3 |
 11:30 | 101.7 |    |         |        |       |  1 |   652.3 |   8:46 |  74.4 |  1 |   483.4 |   7:02 |  68.7 |         
 +------+----+---------+--------+-------+----+---------+--------+-------+----+---------+--------+-------+----+---------+-
-------+-------+----+---------+--------+-------+----+---------+--------+-------+----+---------+--------+-------+         
                                                                                                                         
 +-----+---------------------------------+---------------------------------+---------------------------------+-----------
----------------------+---------------------------------+---------------------------------+------------------------------
---+---------------------------------+                                                                                   
 |     | Monday                          | Tuesday                         | Wednesday                       | Thursday 
                      | Friday                          | Saturday                        | Sunday                       
   | Total                           |                                                                                   
 |     +------+---------+--------+-------+------+---------+--------+-------+------+---------+--------+-------+------+----
-----+--------+-------+------+---------+--------+-------+------+---------+--------+-------+------+---------+--------+----
---+------+---------+--------+-------+                                                                                   
 | Day |    # |      KM |   Time |    V= |    # |      KM |   Time |    V= |    # |      KM |   Time |    V= |    # |   
  KM |   Time |    V= |    # |      KM |   Time |    V= |    # |      KM |   Time |    V= |    # |      KM |   Time |   
V= |    # |      KM |   Time |    V= |                                                                                   
 +-----+------+---------+--------+-------+------+---------+--------+-------+------+---------+--------+-------+------+----
-----+--------+-------+------+---------+--------+-------+------+---------+--------+-------+------+---------+--------+----
---+------+---------+--------+-------+                                                                                   
 |   1 |   27 |  4462.0 |  42:47 | 104.3 |   28 |  5466.2 |  53:48 | 101.6 |   61 |  7389.2 |  67:51 | 108.9 |   35 |  50
04.6 |  44:29 | 112.5 |   59 |  7552.0 |  73:40 | 102.5 |   16 |  1049.5 |  10:15 | 102.4 |   20 |  2878.4 |  26:51 | 107
.2 |  246 | 33801.9 | 319:41 | 105.7 |                                                                                   
 |   2 |   27 |  3233.6 |  32:30 |  99.5 |   20 |  3169.4 |  32:22 |  97.9 |   37 |  3110.3 |  30:32 | 101.9 |   18 |  51
45.6 |  53:06 |  96.9 |   15 |  3385.8 |  33:01 | 102.5 |   36 |  4217.1 |  42:17 |  99.7 |    7 |   872.1 |   7:31 | 116
.0 |  160 | 23133.9 | 231:19 | 100.0 |                                                                                   
 +-----+------+---------+--------+-------+------+---------+--------+-------+------+---------+--------+-------+------+----
-----+--------+-------+------+---------+--------+-------+------+---------+--------+-------+------+---------+--------+----
---+------+---------+--------+-------+                                                                                   
 | Tot |  971 | 19197.7 | 119:08 | 106.5 | 1009 | 37611.1 | 306:19 | 105.3 |  871 | 19047.8 | 109:28 | 107.3 |  673 | 938
83.3 | 915:33 | 102.5 |  650 | 77116.4 | 758:42 | 101.6 |  321 | 38841.0 | 377:36 | 102.9 |  293 | 44228.8 | 407:06 | 108
.6 | 4788 | 29926.1 | 993:52 | 105.1 |                                                                                   
 +-----+------+---------+--------+-------+------+---------+--------+-------+------+---------+--------+-------+------+----
-----+--------+-------+------+---------+--------+-------+------+---------+--------+-------+------+---------+--------+----
---+------+---------+--------+-------+                                                                                   
Back to top
View user's profile Send private message
prino
Warnings : 1

Senior Member


Joined: 07 Feb 2009
Posts: 1238
Location: Vilnius, Lithuania

PostPosted: Thu Jan 14, 2021 3:45 pm
Reply with quote

This is puzzling, cut and paste the data that I cut and pasted here back to z/OS, and it works, run it with the full output and it fails...

I'll investigate by "stepping" through the various "IFTHEN" statements (by putting successive "END" statements in) and look at what's happening with intermediate output.
Back to top
View user's profile Send private message
prino
Warnings : 1

Senior Member


Joined: 07 Feb 2009
Posts: 1238
Location: Vilnius, Lithuania

PostPosted: Thu Jan 14, 2021 6:17 pm
Reply with quote

OK, mea culpa, mea maxima culpa!

Turns out that I was running with a newer version of the program, where the layout of the second table has been changed widening two columns (# & KM) by one character to cater for larger values.

The data with which I was testing was saved from the current "production" version of the program, and that's what I posted. Changing the 42 in the third IFTHEN to 44 delivers the correct result, I'm still tracing step by step to see what's actually going on at each control word, in case (not unlikely) that there's going to be a third table with months in the first column.

Sorry for posting the out-of-date data, but many thanks for showing that a one-pass solution is possible! ;)
Back to top
View user's profile Send private message
prino
Warnings : 1

Senior Member


Joined: 07 Feb 2009
Posts: 1238
Location: Vilnius, Lithuania

PostPosted: Fri Jan 15, 2021 2:16 am
Reply with quote

Going crazy, can someone give me a clue?

Input file:
Code:

VIEW       PRINO.@@121.TEXT                                                                                 Columns 00001 00121
Command ===>                                                                                                   Scroll ===> CSR
****** ***************************************************** Top of Data *******************************************************
000001 1
000002  +------+-------------------------------+-------------------------------+-------------------------------+----------------
000003 ---------------+-------------------------------+-------------------------------+-------------------------------+
000004  |      | Monday                        | Tuesday                       | Wednesday                     | Thursday
000005                | Friday                        | Saturday                      | Sunday                        |
000006  |      +----+---------+--------+-------+----+---------+--------+-------+----+---------+--------+-------+----+---------+-
000007 -------+-------+----+---------+--------+-------+----+---------+--------+-------+----+---------+--------+-------+
000008  | Year | #D |      KM |   Time |    V= | #D |      KM |   Time |    V= | #D |      KM |   Time |    V= | #D |      KM |
000009   Time |    V= | #D |      KM |   Time |    V= | #D |      KM |   Time |    V= | #D |      KM |   Time |    V= |
000010  +------+----+---------+--------+-------+----+---------+--------+-------+----+---------+--------+-------+----+---------+-
000011 -------+-------+----+---------+--------+-------+----+---------+--------+-------+----+---------+--------+-------+
000012  | 1980 |  4 |  1049.2 |  12:23 |  84.7 |  2 |   355.0 |   4:02 |  88.0 |  2 |   260.0 |   2:50 |  91.8 |  1 |    77.0 |
000013   0:54 |  85.6 |  2 |   366.0 |   4:31 |  81.0 |  1 |   128.0 |   1:58 |  65.1 |  1 |   746.0 |   7:06 | 105.1 |
000014  +------+----+---------+--------+-------+----+---------+--------+-------+----+---------+--------+-------+----+---------+-
000015 -------+-------+----+---------+--------+-------+----+---------+--------+-------+----+---------+--------+-------+
000016 1
000017  +-----+-----------------------------------+-----------------------------------+-----------------------------------+-----
000018 ------------------------------+-----------------------------------+-----------------------------------+------------------
000019 -----------------+-----------------------------------+
000020  |     | Monday                            | Tuesday                           | Wednesday                         | Thur
000021 sday                          | Friday                            | Saturday                          | Sunday
000022                  | Total                             |
000023  |     +------+----------+---------+-------+------+----------+---------+-------+------+----------+---------+-------+-----
000024 -+----------+---------+-------+------+----------+---------+-------+------+----------+---------+-------+------+----------+
000025 ---------+-------+------+----------+---------+-------+
000026  | Day |    # |       KM |    Time |    V= |    # |       KM |    Time |    V= |    # |       KM |    Time |    V= |    #
000027  |       KM |    Time |    V= |    # |       KM |    Time |    V= |    # |       KM |    Time |    V= |    # |       KM |
000028     Time |    V= |    # |       KM |    Time |    V= |
000029  +-----+------+----------+---------+-------+------+----------+---------+-------+------+----------+---------+-------+-----
000030 -+----------+---------+-------+------+----------+---------+-------+------+----------+---------+-------+------+----------+
000031 ---------+-------+------+----------+---------+-------+
000032  |   1 |   27 |   4462.0 |   42:47 | 104.3 |   28 |   5466.2 |   53:48 | 101.6 |   61 |   7389.2 |   67:51 | 108.9 |   35
000033  |   5004.6 |   44:29 | 112.5 |   59 |   7552.0 |   73:40 | 102.5 |   16 |   1049.5 |   10:15 | 102.4 |   20 |   2878.4 |
000034    26:51 | 107.2 |  246 |  33801.9 |  319:41 | 105.7 |
000035  +-----+------+----------+---------+-------+------+----------+---------+-------+------+----------+---------+-------+-----
000036 -+----------+---------+-------+------+----------+---------+-------+------+----------+---------+-------+------+----------+
000037 ---------+-------+------+----------+---------+-------+
000038  | Tot |  971 | 119197.7 | 1119:08 | 106.5 | 1009 | 137611.1 | 1306:19 | 105.3 |  871 | 119047.8 | 1109:28 | 107.3 |  673
000039  |  93883.3 |  915:33 | 102.5 |  650 |  77116.4 |  758:42 | 101.6 |  321 |  38841.0 |  377:36 | 102.9 |  293 |  44228.8 |
000040   407:06 | 108.6 | 4788 | 629926.1 | 5993:52 | 105.1 |
000041  +-----+------+----------+---------+-------+------+----------+---------+-------+------+----------+---------+-------+-----
000042 -+----------+---------+-------+------+----------+---------+-------+------+----------+---------+-------+------+----------+
000043 ---------+-------+------+----------+---------+-------+
****** **************************************************** Bottom of Data *****************************************************


Used JCL:
Code:

//PRINOSOR JOB (PRINO),
//             'Test LW merge',
//             CLASS=A,
//             MSGCLASS=H,
//             MSGLEVEL=(2,0),
//             NOTIFY=&SYSUID
//*********************************************************************
//MERGELW EXEC PGM=SORT
//*
//SYSOUT    DD SYSOUT=*
//*
//SORTIN    DD DSN=&SYSUID..@@121.TEXT,
//             DISP=SHR
//*
//SORTOUT   DD SYSOUT=*
//*
//SYSIN     DD *
* Merge the LW file
*
* See: https://ibmmainframes.com/viewtopic.php?t=68044

  OPTION COPY
* --------------------------------------------------------------------
* Put '0120' in columns 364-367

  INREC IFTHEN=(WHEN=INIT,OVERLAY=(364:+120,ZD,LENGTH=4)),

* Whatever                       0120
* --------------------------------------------------------------------
* Test column 9 for '+' or '|' OR column 41 for '+' or '|'
* - copy 365 ('1') to 364
* - add seqno to 367, for two records

    IFTHEN=(WHEN=GROUP,BEGIN=(9,1,SS,EQ,C'+|',OR,41,1,SS,EQ,C'+|'),
      PUSH=(364:365,1,367:SEQ=1),RECORDS=2),

* T-1 / P-1                      1121
* T-1 / P-2                      1122
* --------------------------------------------------------------------
* Test column 44 for '+' or '|'
* - copy 366 ('2') to 364
* - add seqno to 367, for three records

*!! CHANGE 44 below to 80 and things go wrong!!

    IFTHEN=(WHEN=GROUP,BEGIN=(44,1,SS,EQ,C'+|'),
      PUSH=(364:366,1,367:SEQ=1),RECORDS=3),

* T-2 / P-1                      2121
* T-2 / P-2                      2122
* T-2 / P-3                      2123
* --------------------------------------------------------------------
* Test column column 367 for '1'
* - add columns 1-121 at column 122

    IFTHEN=(WHEN=GROUP,BEGIN=(367,1,ZD,EQ,+1),
      PUSH=(122:1,121)),

* Whatever                       0120
* T-1 / P-1 T-1 / P-1            1121
* T-1 / P-2 T-1 / P-1            1122
* Whatever  T-1 / P-1            0120
* T-2 / P-1 T-2 / P-1            2121
* T-2 / P-2 T-2 / P-1            2122
* T-2 / P-3 T-2 / P-1            2123
* Whatever  T-2 / P-1            0120
* --------------------------------------------------------------------
* Test column column 367 for '2'
* - add columns 1-121 at column 243

    IFTHEN=(WHEN=GROUP,BEGIN=(367,1,ZD,EQ,+2),
      PUSH=(243:1,121))

* Whatever                       0120  x 0...: 1,121
* T-1 / P-1 T-1 / P-1            1121
* T-1 / P-2 T-1 / P-1 T-1 / P-2  1122
* t-1 / P-1 t-1 / P-1 T-1 / P-2  1121  x 1..1: 122,1+1,121
* t-1 / P-2 t-1 / P-1 t-1 / P-1  1122
* Whatever  t-1 / P-1 t-1 / P-1  0120  x 0...: 1,121
* T-2 / P-1 T-2 / P-1 t-1 / P-1  2121
* T-2 / P-2 T-2 / P-1 T-2 / P-2  2122
* T-2 / P-3 T-2 / P-1 T-2 / P-2  2123  x 2..3: 122,121+243,121+1,121
* --------------------------------------------------------------------

  OUTFIL FNAMES=(SORTOUT),
    INCLUDE=((364,1,ZD,EQ,+0),OR,
             (364,1,ZD,EQ,+1,AND,367,1,ZD,EQ,+2),OR,
             (364,1,ZD,EQ,+2,AND,367,1,ZD,EQ,+3)),

    FTOV,VLTRIM=C' ',

    IFTHEN=(WHEN=(364,1,ZD,EQ,+0),BUILD=(1,121)),

    IFTHEN=(WHEN=(364,1,ZD,EQ,+1,AND,367,1,ZD,EQ,+2),
      BUILD=(122,121,1,121)),

    IFTHEN=(WHEN=(364,1,ZD,EQ,+2,AND,367,1,ZD,EQ,+3),
      BUILD=(122,121,243,121,1,121))



Resulting output, truncated, but perfect:
Code:

SDSF EDIT    PRINOSOR (JOB22303) SORTOUT                                                                                                   Columns 00001 00152
Command ===>                                                                                                                                  Scroll ===> CSR
=COLS> ----+----1----+----2----+----3----+----4----+----5----+----6----+----7----+----8----+----9----+----0----+----1----+----2----+----3----+----4----+----5--
****** ********************************************************************* Top of Data **********************************************************************
000001 1
000002  +------+-------------------------------+-------------------------------+-------------------------------+-------------------------------+---------------
000003  |      | Monday                        | Tuesday                       | Wednesday                     | Thursday                      | Friday
000004  |      +----+---------+--------+-------+----+---------+--------+-------+----+---------+--------+-------+----+---------+--------+-------+----+---------+
000005  | Year | #D |      KM |   Time |    V= | #D |      KM |   Time |    V= | #D |      KM |   Time |    V= | #D |      KM |   Time |    V= | #D |      KM |
000006  +------+----+---------+--------+-------+----+---------+--------+-------+----+---------+--------+-------+----+---------+--------+-------+----+---------+
000007  | 1980 |  4 |  1049.2 |  12:23 |  84.7 |  2 |   355.0 |   4:02 |  88.0 |  2 |   260.0 |   2:50 |  91.8 |  1 |    77.0 |   0:54 |  85.6 |  2 |   366.0 |
000008  +------+----+---------+--------+-------+----+---------+--------+-------+----+---------+--------+-------+----+---------+--------+-------+----+---------+
000009 1
000010  +-----+-----------------------------------+-----------------------------------+-----------------------------------+-----------------------------------+
000011  |     | Monday                            | Tuesday                           | Wednesday                         | Thursday                          |
000012  |     +------+----------+---------+-------+------+----------+---------+-------+------+----------+---------+-------+------+----------+---------+-------+
000013  | Day |    # |       KM |    Time |    V= |    # |       KM |    Time |    V= |    # |       KM |    Time |    V= |    # |       KM |    Time |    V= |
000014  +-----+------+----------+---------+-------+------+----------+---------+-------+------+----------+---------+-------+------+----------+---------+-------+
000015  |   1 |   27 |   4462.0 |   42:47 | 104.3 |   28 |   5466.2 |   53:48 | 101.6 |   61 |   7389.2 |   67:51 | 108.9 |   35 |   5004.6 |   44:29 | 112.5 |
000016  +-----+------+----------+---------+-------+------+----------+---------+-------+------+----------+---------+-------+------+----------+---------+-------+
000017  | Tot |  971 | 119197.7 | 1119:08 | 106.5 | 1009 | 137611.1 | 1306:19 | 105.3 |  871 | 119047.8 | 1109:28 | 107.3 |  673 |  93883.3 |  915:33 | 102.5 |
000018  +-----+------+----------+---------+-------+------+----------+---------+-------+------+----------+---------+-------+------+----------+---------+-------+
****** ******************************************************************** Bottom of Data ********************************************************************


When looking at the output, the columns used in the second and third IFTHEN statements (9+41 and 44) are used to select the tables, and not used anywhere else.

Yet, if I change the 44 to 80 (see the line marked with !!) (in order to avoid clashing with "old data"), and 80 is just as unique as 44, only table 2 appears in the output, and the little bit of hair that I still have is rapidly disappearing.

Why the flipping 'ell doesn't using 80 instead of 44 work???

And why use two columns for the first compare, I've not changed that (yet), but it's just as puzzling...
Back to top
View user's profile Send private message
sergeyken

Active Member


Joined: 29 Apr 2008
Posts: 926
Location: Maryland

PostPosted: Fri Jan 15, 2021 7:05 pm
Reply with quote

The issue with column 80 vs 44 looks really weird...
For debug purposes, try to run without INCLUDE in the OUTFIL statement, and add current ‘0123’ combinations from position 364 to output lines?

One more trick is possible: if markers 1/2/3 were used to verify the line types (e.g. initial OVERLAY=(364:C’1231’...) rather than C’0120’ ), it could simplify several compare operations, as in OUTFIL:
INCLUDE=(364,1,CH,EQ,367,1,CH) - is enough
WHEN=(364,1,ZD,EQ,+1) - is enough
Etc.
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic View Bookmarks
All times are GMT + 6 Hours
Forum Index -> DFSORT/ICETOOL

 


Similar Topics
Topic Forum Replies
No new posts Matching and non matching records usi... DFSORT/ICETOOL 11
No new posts REMOVE DUPLICATE RECORDS BASED ON A S... DFSORT/ICETOOL 4
No new posts Using sort on VB records based on off... DFSORT/ICETOOL 6
No new posts Inserting records into the empty segm... IMS DB/DC 1
No new posts JCL to extract repeating records DFSORT/ICETOOL 6
Search our Forums:

Back to Top