Portal | Manuals | References | Downloads | Info | Programs | JCLs | Master the Mainframes
IBM Mainframe Computers Forums Index
 
Register
 
IBM Mainframe Computers Forums Index Mainframe: Search IBM Mainframe Forum: FAQ Memberlist Usergroups Profile Log in to check your private messages Log in
 

 

How to massage out unwanted lines
Goto page 1, 2  Next
 
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> DFSORT/ICETOOL
View previous topic :: :: View next topic  
Author Message
prino

Active Member


Joined: 07 Feb 2009
Posts: 985
Location: Oostende, Belgium

PostPosted: Tue Jan 05, 2016 6:59 pm    Post subject: How to massage out unwanted lines
Reply with quote

I need to convert some big report files to leave only heading-of-table rows, so that a post-processing program cab be fed with those for RTF-ification.

I've started with the obvious(?):

Code:

//SYSIN     DD *
 ALTSEQ CODE=(F040,F140,F240,F340,F440,
              F540,F640,F740,F840,F940,
              4B40,7A40)
 INREC FIELDS=(1,218,TRAN=ALTSEQ)
 OMIT COND=(2,2,CH,EQ,C'+-')
 SORT FIELDS=(1,218,CH,A)
 SUM  FIELDS=NONE
 END
//*

The above reduces the number of lines by about 95%, but it still leaves me a hell of a lot of lines just containing '|', which I cannot blank out as they're also contained in heading lines. Ditto for some multi-character strings like

Code:

|     -  -   |
| -    |
|    - |

In essence, I'm only interested in the lines that are enclosed between two separator (aka of the type +-----+---+-------+...) but not having kept up-to-date with SORT, I don't know how (or even if) if it possible to keep only the manually '**' marked lines from sample below:

Code:
   +------+------+---------------------------+---------------------------+---------------------------+
** |      |      | - 1.00                    | 1.00 +                    | Total                     |
** |      |  No  +------+---------+----------+------+---------+----------+------+---------+----------+
** | Mnth | wait |    # |    Time |       W= |    # |    Time |       W= |    # |    Time |       W= |
   +------+------+------+---------+----------+------+---------+----------+------+---------+----------+
   | Jan  |   26 |  105 |   43:57 |  0:25:07 |   30 |   58:49 |  1:57:38 |  135 |  102:46 |  0:45:40 |
  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  10 Line(s) not Displayed
   | Dec  |   23 |   69 |   31:37 |  0:27:30 |   31 |   58:46 |  1:53:45 |  100 |   90:23 |  0:54:14 |
   +------+------+------+---------+----------+------+---------+----------+------+---------+----------+

   +--------+----------+---------+-------+-----+-----+-----+-----+
** | V-  #R |       KM |    Time |    V= |   T |   R |   T |   R |
   +--------+----------+---------+-------+-----+-----+-----+-----+
   | -    1 |      3.0 |    0:12 |  15.0 |  11 |  41 |  11 |  41 |
  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -   8 Line(s) not Displayed
   | - 2500 | 304535.0 | 3046:56 |  99.9 |   1 |   1 | 100 |   1 |
   +--------+----------+---------+-------+-----+-----+-----+-----+

   +-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+
** | Day | Jan | Feb | Mar | Apr | May | Jun | Jul | Aug | Sep | Oct | Nov | Dec |
   +-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+
   |   1 |   - |   2 |   2 |   3 |   6 |   3 |   2 |   3 |   4 |   1 |   2 |   2 |
  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  29 Line(s) not Displayed
   |  31 |   1 |     |   2 |     |   3 |     |   3 |   2 |     |   1 |     |   - |
   +-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+
** | Use |  23 |  21 |  20 |  20 |  26 |  30 |  31 |  31 |  30 |  27 |  24 |  14 |
   +-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+

   +----------+------+----------+---------+-------+-----+-----+-----+-----+
** | Period   |   #D |       KM |    Time |    V= |   T |   D |   T |   D |
   +----------+------+----------+---------+-------+-----+-----+-----+-----+
   |  21 days |   12 |   2981.2 |   33:44 |  88.4 |   1 |   1 |   1 |  12 |
  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  24 Line(s) not Displayed
   | 360 days |   65 |  53088.4 |  476:05 | 111.5 | 144 |   1 | 164 |   4 |
   +----------+------+----------+---------+-------+-----+-----+-----+-----+

Any suggestions would be welcome
Back to top
View user's profile Send private message

Bill Woodger

DFSORT Moderator


Joined: 09 Mar 2011
Posts: 7240

PostPosted: Tue Jan 05, 2016 7:32 pm    Post subject: Reply to: How to massage out unwanted lines
Reply with quote

What release of DFSORT do you have (include the ICE201I message from any SORT step)?

RECFM? LRECL is 218?
Back to top
View user's profile Send private message
prino

Active Member


Joined: 07 Feb 2009
Posts: 985
Location: Oostende, Belgium

PostPosted: Tue Jan 05, 2016 7:50 pm    Post subject:
Reply with quote

Sorry, should have added that one straight away:
Code:
ICE201I F RECORD TYPE IS F - DATA STARTS IN POSITION 1

Yes, it's old, don't ask.

And yes, the input file is LRECL=218, but nothing extends beyond position 121.
Back to top
View user's profile Send private message
Bill Woodger

DFSORT Moderator


Joined: 09 Mar 2011
Posts: 7240

PostPosted: Tue Jan 05, 2016 8:15 pm    Post subject: Reply to: How to massage out unwanted lines
Reply with quote

Thanks. Not too bad, it has WHEN=GROUP.
Back to top
View user's profile Send private message
prino

Active Member


Joined: 07 Feb 2009
Posts: 985
Location: Oostende, Belgium

PostPosted: Tue Jan 05, 2016 8:40 pm    Post subject:
Reply with quote

OK, so I would be able to do something like

Code:
OUTREC IFTHEN=(WHEN=GROUP,
BEGIN=(2,2,CH,EQ,C'+-'),
END=(2,2,CH,EQ,C'+-'),PUSH=(215:ID=4))


and then only output the ones with an ID <> ' '?
Back to top
View user's profile Send private message
Bill Woodger

DFSORT Moderator


Joined: 09 Mar 2011
Posts: 7240

PostPosted: Tue Jan 05, 2016 8:52 pm    Post subject: Reply to: How to massage out unwanted lines
Reply with quote

That's correct. Kind of. But unfortunately the END= will kick in straight away with the first line that is the start of the group. The thing will be to be able to END it, or to not be concerned about the END.
Back to top
View user's profile Send private message
prino

Active Member


Joined: 07 Feb 2009
Posts: 985
Location: Oostende, Belgium

PostPosted: Tue Jan 05, 2016 9:09 pm    Post subject: Re: Reply to: How to massage out unwanted lines
Reply with quote

Bill Woodger wrote:
That's correct. Kind of. But unfortunately the END= will kick in straight away with the first line that is the start of the group. The thing will be to be able to END it, or to not be concerned about the END.

That's what I already found out. My problem is that some tables have the format
Code:
+----+----+...
| h1 | h2 |...
+----+----+...
|data|data|...
|data|data|...
|data|data|...
|data|data|...
+----+----+...
| f1 | f2 |...
+----+----+...

while others do not have the f1/f2 footer. I need to capture both the always present h1/h2 line, and the only sometimes present f1/f2 line, and there isn't really anything that tells me if a table has a footer.
Back to top
View user's profile Send private message
RahulG31

Active User


Joined: 20 Dec 2014
Posts: 331
Location: USA

PostPosted: Tue Jan 05, 2016 9:49 pm    Post subject: Reply to: How to massage out unwanted lines
Reply with quote

In my opinion, this can only be done if there is a way to identify the header. As per the data provided, I don't see anything to identify it.

Using mere '+-' as identification will not work as it will include blank groups and groups with data part.

A logic is first needed to identify header and then that logic can be converted to Sort statements. What is that logic?

.
Back to top
View user's profile Send private message
Bill Woodger

DFSORT Moderator


Joined: 09 Mar 2011
Posts: 7240

PostPosted: Tue Jan 05, 2016 9:55 pm    Post subject: Reply to: How to massage out unwanted lines
Reply with quote

OK :-)

Get one bit working and the spec changes...

Is it true that blank lines only appear between the different reports, ie even across pagination?
Code:

b
+-
want

+-
don't want
+-
b
+-
want

+-
don't want
+-
want when present
+-
b


And then exclude all the +- lines, and all those lines that only contain or -.
Back to top
View user's profile Send private message
Bill Woodger

DFSORT Moderator


Joined: 09 Mar 2011
Posts: 7240

PostPosted: Tue Jan 05, 2016 10:03 pm    Post subject: Reply to: How to massage out unwanted lines
Reply with quote

RahulG31,

So you need to work out how to make it identifiable.
Back to top
View user's profile Send private message
prino

Active Member


Joined: 07 Feb 2009
Posts: 985
Location: Oostende, Belgium

PostPosted: Tue Jan 05, 2016 10:15 pm    Post subject: Re: Reply to: How to massage out unwanted lines
Reply with quote

Bill Woodger wrote:
OK :-)

Get one bit working and the spec changes...

Is it true that blank lines only appear between the different reports, ie even across pagination?
Code:
b
+-
want

+-
don't want
+-
b
+-
want

+-
don't want
+-
want when present
+-
b

And then exclude all the +- lines, and all those lines that only contain or -.

Yes there are blank lines between the tables, or ASA control characters, but in both of those cases column 2 is blank. And the spec in the first post may not have been 100% clear, but the examples were.

And your comment about zapping all lines that only contain '|' (and occasionally '-') characters is correct.
Back to top
View user's profile Send private message
RahulG31

Active User


Joined: 20 Dec 2014
Posts: 331
Location: USA

PostPosted: Tue Jan 05, 2016 10:57 pm    Post subject: Reply to: How to massage out unwanted lines
Reply with quote

Try this:
Code:
//SYSIN DD *
 OMIT COND=(1,2,CH,EQ,C'+-')
 INREC IFTHEN=(WHEN=GROUP,END=(1,9,CH,EQ,C'         '),PUSH=(220:ID=4))
 OUTREC IFTHEN=(WHEN=GROUP,KEYBEGIN=(220,4),END=(3,1,CH,NE,C' '),
                   PUSH=(225:ID=4))
 OUTFIL INCLUDE=(225,4,CH,NE,C'    ')
 OPTION COPY
/*
Back to top
View user's profile Send private message
Bill Woodger

DFSORT Moderator


Joined: 09 Mar 2011
Posts: 7240

PostPosted: Tue Jan 05, 2016 11:19 pm    Post subject: Reply to: How to massage out unwanted lines
Reply with quote

RahulG31,

The headings can contain more than one line.

Here's some test-data:
Code:

+-
| WANT THE FIRST (TWO OF THEM, ONLY ONE TO APPEAR)
| 1 2 3 4 5 6 7 8 9 .
+-
| DON'T WANT
+-
                                                 
+-
| WANT THE SECOND
| AND A SECOND LINE IN HEADING
+-
| DON'T WANT
+-
| WANT WHEN PRESENT
+-
                                                 
+-
| WANT THE FIRST (TWO OF THEM, ONLY ONE TO APPEAR)
| - - |
+-


Expected output is:

Code:
| AND A SECOND LINE IN HEADING                   
| WANT THE FIRST (TWO OF THEM, ONLY ONE TO APPEAR)
| WANT THE SECOND                                 
| WANT WHEN PRESENT
Back to top
View user's profile Send private message
RahulG31

Active User


Joined: 20 Dec 2014
Posts: 331
Location: USA

PostPosted: Tue Jan 05, 2016 11:26 pm    Post subject: Reply to: How to massage out unwanted lines
Reply with quote

I know the headings can contain more than one line and I think all of them are needed (not just one). I forgot the footer thing. Here is my test data:
Code:
+----+----+...
| A1 | A2 |...
+----+----+...
|DATA|DATA|...
|DATA|DATA|...
|DATA|DATA|...
|DATA|DATA|...
+----+----+...
| F1 | F2 |...
+----+----+...

+----+----+...
| B1 | B2 |...
+----+----+...

+----+----+...
|    | XX |...
| C1 | C2 |...
+----+----+...
|DATA|DATA|...
|DATA|DATA|...
+----+----+...
 

And the result come as:
Code:
| A1 | A2 |...   0001    0001
| B1 | B2 |...   0002    0002
|    | XX |...   0003    0003
| C1 | C2 |...   0003    0003
Back to top
View user's profile Send private message
Bill Woodger

DFSORT Moderator


Joined: 09 Mar 2011
Posts: 7240

PostPosted: Tue Jan 05, 2016 11:38 pm    Post subject: Reply to: How to massage out unwanted lines
Reply with quote

For your 1,3,NE,C' ' I don't think there's a position you can rely upon for sure. There may be, but because there are lots of different reports from a massive amount of data, we can't tell.

Although ICE201I F has WHEN=GROUP, KEYBEGIN does not appear until ICE201I H.
Back to top
View user's profile Send private message
RahulG31

Active User


Joined: 20 Dec 2014
Posts: 331
Location: USA

PostPosted: Wed Jan 06, 2016 12:06 am    Post subject: Reply to: How to massage out unwanted lines
Reply with quote

Quote:
Although ICE201I F has WHEN=GROUP, KEYBEGIN does not appear until ICE201I H.
That's discouraging.

So, the idea was to make a group before the row (with all spaces) and write the first line. For multiple header lines another group is created starting with earlier group key and ending with column 3 not equal to spaces.

Since KEYBEGIN doesn't work, we need to find another way to identify the group start.

For footer, I think it would be a lot easier if the input data can be modified so that the footer is separated and looked something like this:
Code:
+----+----+...
| A1 | A2 |...
+----+----+...
|DATA|DATA|...
|DATA|DATA|...
|DATA|DATA|...
|DATA|DATA|...
+----+----+...

+----+----+...
| F1 | F2 |...
+----+----+...


Let's find out.

.
Back to top
View user's profile Send private message
RahulG31

Active User


Joined: 20 Dec 2014
Posts: 331
Location: USA

PostPosted: Wed Jan 06, 2016 1:31 am    Post subject: Reply to: How to massage out unwanted lines
Reply with quote

I was finally able to do it in 2 steps (on the lines of the logic mentioned earlier). Passing the dataset from 1st step to 2nd step.

(Both, Footer and KEYBEGIN, is taken care of)

This was done on a sample data of LRECL=15 and I have introduced a blank line as the first record (hope that's not a problem).

Step 1:
Code:
//SYSIN DD *
 INREC IFTHEN=(WHEN=GROUP,END=(1,9,CH,EQ,C'         '),PUSH=(18:ID=4)),
       IFTHEN=(WHEN=(1,2,CH,EQ,C'+-'),
                      OVERLAY=(30:SEQNUM,4,ZD,RESTART=(18,4)))
 OUTFIL IFTHEN=(WHEN=(30,4,CH,EQ,C'0003'),BUILD=(1,15,/,9X,/,C'+---')),
        IFTHEN=(WHEN=NONE,BUILD=(1,15))
 OPTION COPY
/*

Step2:
Code:
//SYSIN DD *
 OMIT COND=(1,2,CH,EQ,C'+-')
 INREC IFTHEN=(WHEN=GROUP,END=(1,9,CH,EQ,C'         '),PUSH=(18:ID=4))
 OUTREC IFTHEN=(WHEN=GROUP,BEGIN=(1,9,CH,EQ,C'         '),
                          END=(3,1,CH,NE,C' '),PUSH=(26:ID=4))
 OUTFIL INCLUDE=(26,4,CH,NE,C'    ',AND,1,9,CH,NE,C'         ')
 OPTION COPY
/*


Input:
Code:

+----+----+...
| A1 | A2 |...
+----+----+...
|DATA|DATA|...
|DATA|DATA|...
|DATA|DATA|...
|DATA|DATA|...
+----+----+...
| F1 | F2 |...
+----+----+...

+----+----+...
| B1 | B2 |...
+----+----+...
|DATA|DATA|...
+----+----+...

+----+----+...
|    | XX |...
| C1 | C2 |...
+----+----+...
|DATA|DATA|...
|DATA|DATA|...
+----+----+...
| F3 | F4 |...
+----+----+...

+----+----+...
| D1 | D2 |...
+----+----+...
|DATA|DATA|...
+----+----+...
 

Output:
Code:
| A1 | A2 |...   0002    0001
| F1 | F2 |...   0003    0002
| B1 | B2 |...   0004    0003
|    | XX |...   0006    0005
| C1 | C2 |...   0006    0005
| F3 | F4 |...   0007    0006
| D1 | D2 |...   0008    0007


.
Back to top
View user's profile Send private message
prino

Active Member


Joined: 07 Feb 2009
Posts: 985
Location: Oostende, Belgium

PostPosted: Wed Jan 06, 2016 1:50 am    Post subject: Re: Reply to: How to massage out unwanted lines
Reply with quote

Bill Woodger wrote:
For your 1,3,NE,C' ' I don't think there's a position you can rely upon for sure. There may be, but because there are lots of different reports from a massive amount of data, we can't tell.

The third column for data rows is always a blank, all data is contained within the boxes with (at least) one space between the box and the data on either side, so I should have used
Code:

   +-----+-----+...
** | H-1 | H-2 |...
** | sub | sub |...
   +-----+-----+...
   | 123 | 456 |...
   |  ab |   y |...
   | qqq | p=5 |...
   | 1a  | ??? |...
   +-----+-----+...
** | F-1 | F-2 |...
   +-----+-----+...


Bill Woodger wrote:
ICE201I F has WHEN=GROUP, KEYBEGIN does not appear until ICE201I H.

On which release of z/OS was that implemented as standard?
Back to top
View user's profile Send private message
RahulG31

Active User


Joined: 20 Dec 2014
Posts: 331
Location: USA

PostPosted: Wed Jan 06, 2016 2:01 am    Post subject: Reply to: How to massage out unwanted lines
Reply with quote

I was using the below to mark the end of the group (for multiple header lines):
Code:
END=(3,1,CH,NE,C' ')

Since I was starting with a KEYBEGIN and ending with a non-space at position 3 this was giving multiple header lines as output.

It doesn't matter whether the data rows have something in column 3 as the group would have ended before that (assuming the header line will always have data in column 3).

.
Back to top
View user's profile Send private message
Bill Woodger

DFSORT Moderator


Joined: 09 Mar 2011
Posts: 7240

PostPosted: Wed Jan 06, 2016 2:05 am    Post subject: Reply to: How to massage out unwanted lines
Reply with quote

KEYBEGIN is in z/OS 1.13, with PTFs to apply for 1.10 and 1.12.
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> DFSORT/ICETOOL All times are GMT + 6 Hours
Goto page 1, 2  Next
Page 1 of 2

 

Search our Forum:

Similar Topics
Topic Author Forum Replies Posted
No new posts syncsort: copy lines after the keyword shreya19 SYNCSORT 7 Fri Dec 02, 2016 9:47 am
No new posts Insert Lines in JCL with Rexx after a... Willy Jensen CLIST & REXX 3 Tue Aug 30, 2016 4:18 pm
No new posts DFSORT split long VB lines efficiently BridgetBrackenbury DFSORT/ICETOOL 2 Fri Feb 12, 2016 5:10 am
No new posts DFSORT VB File make 2 lines from 1 BridgetBrackenbury DFSORT/ICETOOL 2 Fri Feb 12, 2016 3:48 am
No new posts GROUP of undefined number of lines Khadhar Basha DFSORT/ICETOOL 4 Fri Feb 05, 2016 5:15 am


Facebook
Back to Top
 
Mainframe Wiki | Forum Rules | Bookmarks | Subscriptions | FAQ | Tutorials | Contact Us