Portal | Manuals | References | Downloads | Info | Programs | JCLs | Master the Mainframes
IBM Mainframe Computers Forums Index
 
Register
 
IBM Mainframe Computers Forums Index Mainframe: Search IBM Mainframe Forum: FAQ Memberlist Usergroups Profile Log in to check your private messages Log in
 

 

Drop all the records before a criteria
Goto page 1, 2  Next
 
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> DFSORT/ICETOOL
View previous topic :: :: View next topic  
Author Message
abhijit.nayak01

Active User


Joined: 22 Mar 2009
Posts: 157
Location: South Africa

PostPosted: Tue May 14, 2013 2:13 am    Post subject: Drop all the records before a criteria
Reply with quote

Hi,

I have a file which has below specification:
Code:
Organization  . . . : PS   
Record format . . . : FB   
Record length . . . : 250 
Block size  . . . . : 27750
1st extent cylinders: 1   
Secondary cylinders : 300 
Data set name type  : 


Currently the file has multiple header records which are repeating. I am manually dropping records based on below criteria.
Step 1:
Code:
 F '01' 1 Last

Step2: Dropping all the records from top to the record found from step1.

Is it possible to do this by SORT /ICETOOL SUBSET or any other option.
Back to top
View user's profile Send private message

Bill Woodger

DFSORT Moderator


Joined: 09 Mar 2011
Posts: 7226

PostPosted: Tue May 14, 2013 3:06 am    Post subject: Reply to: Drop all the records before a criteria
Reply with quote

Are the duplicate headers contiguous?

Is all the data on the headers the same? Or is just the last one correct?
Back to top
View user's profile Send private message
abhijit.nayak01

Active User


Joined: 22 Mar 2009
Posts: 157
Location: South Africa

PostPosted: Tue May 14, 2013 3:14 am    Post subject:
Reply with quote

Yes The header, detail and the trailer are repeating.Please see below:
Code:
01Header
04Detail1
04Detail2
04Detail3
98Trailer1
99Trailer
01Header
04Detail1
04Detail2
04Detail3
98Trailer1
99Trailer
01Header
04Detail1
04Detail2
04Detail3
98Trailer1
99Trailer
Back to top
View user's profile Send private message
Skolusu

Senior Member


Joined: 07 Dec 2007
Posts: 2205
Location: San Jose

PostPosted: Tue May 14, 2013 3:30 am    Post subject:
Reply with quote

abhijit.nayak01 wrote:
Yes The header, detail and the trailer are repeating.Please see below:


And the desired output is the Last 5 records? like this

Code:

01HEADER      - DROP                   
04DETAIL1     - DROP                   
04DETAIL2     - DROP                   
04DETAIL3     - DROP                   
98TRAILER1    - DROP                   
99TRAILER     - DROP                   
01HEADER      - DROP                   
04DETAIL1     - DROP                   
04DETAIL2     - DROP                   
04DETAIL3     - DROP                   
98TRAILER1    - DROP                   
99TRAILER     - DROP                   
01HEADER      - DROP                   
04DETAIL1     - WRITE TO OUTPUT ??     
04DETAIL2     - WRITE TO OUTPUT ??     
04DETAIL3     - WRITE TO OUTPUT ??     
98TRAILER1    - WRITE TO OUTPUT ??     
99TRAILER     - WRITE TO OUTPUT ??     
Back to top
View user's profile Send private message
abhijit.nayak01

Active User


Joined: 22 Mar 2009
Posts: 157
Location: South Africa

PostPosted: Tue May 14, 2013 3:38 am    Post subject:
Reply with quote

The desire output is as below:
Code:
01HEADER      - DROP                   
04DETAIL1     - DROP                   
04DETAIL2     - DROP                   
04DETAIL3     - DROP                   
98TRAILER1    - DROP                   
99TRAILER     - DROP                   
01HEADER      - DROP                   
04DETAIL1     - DROP                   
04DETAIL2     - DROP                   
04DETAIL3     - DROP                   
98TRAILER1    - DROP                   
99TRAILER     - DROP                   
01HEADER      - WRITE TO OUTPUT                   
04DETAIL1     - WRITE TO OUTPUT     
04DETAIL2     - WRITE TO OUTPUT       
04DETAIL3     - WRITE TO OUTPUT       
98TRAILER1    - WRITE TO OUTPUT     
99TRAILER     - WRITE TO OUTPUT     
Back to top
View user's profile Send private message
Skolusu

Senior Member


Joined: 07 Dec 2007
Posts: 2205
Location: San Jose

PostPosted: Tue May 14, 2013 4:00 am    Post subject: Reply to: Drop all the records before a criteria
Reply with quote

I am not going to provide you complete Job, but will give you hints about solving it.

1. Look up Joinkeys examples of using the same file with temp matching keys created. There should at least 3 topics using this trick
2. For JNF1 you would use WHEN=GROUP with begin for '01' and add a 8 byte ID at the end
3. For JNF2 you would only INCLUDE '01' records and just build the record with 8 byte seqnum starting with zero.
4. Since you are creating the temp keys to be matched, make sure you have SORTED and NOSEQCK on the matching keys.
5. Use JOIN UNPAIRED,F1,ONLY to get the desired results


Once you got the desired results post the solution here so that it helps in future
Back to top
View user's profile Send private message
abhijit.nayak01

Active User


Joined: 22 Mar 2009
Posts: 157
Location: South Africa

PostPosted: Tue May 14, 2013 6:41 pm    Post subject:
Reply with quote

Hi Skolusu,

Thanks for your idea. I managed to get the desired output. Please check the below JCL.

Code:
//SORT0001 EXEC PGM=SORT                                   
//IN1 DD DSN=P2D.P2D.DT130513,DISP=SHR   
//IN2 DD DSN=P2D.P2D.DT130513,DISP=SHR   
//SORTOUT DD DSN=P2D.P2D.OUT,DISP=OLD     
//SYSOUT   DD SYSOUT=*                                     
//JNF1CNTL DD *                                           
 INREC IFTHEN=(WHEN=GROUP,BEGIN=(01,2,CH,EQ,C'01'),       
              PUSH=(251:ID=8))                             
/*                                                         
//JNF2CNTL DD *                                           
 INREC IFTHEN=(WHEN=(01,2,CH,EQ,C'01'),                   
              BUILD=(251:SEQNUM,8,ZD,START=00000000))     
/*                                                         
//SYSIN DD *                                               
 JOINKEYS F1=IN1,FIELDS=(251,8,A)                         
 JOINKEYS F2=IN2,FIELDS=(251,8,A)                         
 JOIN UNPAIRED,F1,ONLY                                     
 OPTION COPY                                               
/*   
Back to top
View user's profile Send private message
Bill Woodger

DFSORT Moderator


Joined: 09 Mar 2011
Posts: 7226

PostPosted: Tue May 14, 2013 7:49 pm    Post subject: Reply to: Drop all the records before a criteria
Reply with quote

Well, you've got the result, but that is not always the end of it.

Have a look again at Kolusu's points 3 and 4.

In your version, you are sorting the entire file twice, with OPTION EQUALS, to get the two files into the order in which they already are.

You have records on F2 which you do not need. Although they don't do any "harm" directly, effort is still spent in processing them to be ignored.
Back to top
View user's profile Send private message
abhijit.nayak01

Active User


Joined: 22 Mar 2009
Posts: 157
Location: South Africa

PostPosted: Wed May 15, 2013 12:15 am    Post subject:
Reply with quote

Hi Bill,
Please clarify more in detail. Do I need to create a temporary file and then use it for JOIN.
Back to top
View user's profile Send private message
Bill Woodger

DFSORT Moderator


Joined: 09 Mar 2011
Posts: 7226

PostPosted: Wed May 15, 2013 3:33 am    Post subject: Re: Reply to: Drop all the records before a criteria
Reply with quote

Skolusu wrote:
[...]
3. For JNF2 you would only INCLUDE '01' records and just build the record with 8 byte seqnum starting with zero.
4. Since you are creating the temp keys to be matched, make sure you have SORTED and NOSEQCK on the matching keys.
[...]Once you got the desired results post the solution here so that it helps in future


So, in JNF2CNTL you are not using INCLUDE, you are using IFTHEN=(WHEN=(logexp. This "works", but leaves a load of unwanted and unneeded records as input to the JOINKEYS.

So, use INCLUDE, then a simple INREC BUILD. Note the "just build" from Kolusu's point. That doesn't mean "just build it at the end of the record leaving 250 blanks lying around in front of it". Your records from JNF2 only need to have the sequence number on.

Which brings us to point 4.

You are creating group IDs on JNF1, so that file is already in sequence on the key for matching. You are creating a sequence number on JNF2, so that file is already in sequence on the key for matching as well.

The way you have coded, you are sorting both files (look at your sysout) on the keys to get them into the same order. With OPTION EQUALS.

If you read Kolusu's suggestion again, you would avoid the two sorts.
Back to top
View user's profile Send private message
abhijit.nayak01

Active User


Joined: 22 Mar 2009
Posts: 157
Location: South Africa

PostPosted: Mon May 20, 2013 6:41 pm    Post subject:
Reply with quote

Hi Bill,
Thanks for the explanation and apologies for late response. Actually was on leave. Please see below and let me know whether the code is correct or not:
Code:
//JNF1CNTL DD *                                               
 INREC IFTHEN=(WHEN=GROUP,BEGIN=(01,2,CH,EQ,C'01'),           
              PUSH=(251:ID=8))                               
/*                                                           
//JNF2CNTL DD *                                               
 INCLUDE COND=(01,2,CH,EQ,C'01')                             
 INREC BUILD=(1:SEQNUM,8,ZD,START=00000000)                   
/*                                                           
//SYSIN DD *                                                 
 JOINKEYS F1=IN1,FIELDS=(251,8,A)                             
 JOINKEYS F2=IN2,FIELDS=(1,8,A),SORTED,NOSEQCK               
 JOIN UNPAIRED,F1,ONLY                                       
 OPTION COPY                                                 
/*                                                           
Back to top
View user's profile Send private message
abhijit.nayak01

Active User


Joined: 22 Mar 2009
Posts: 157
Location: South Africa

PostPosted: Mon May 20, 2013 6:43 pm    Post subject:
Reply with quote

Forgot to mention that the code worked fine.
Back to top
View user's profile Send private message
Bill Woodger

DFSORT Moderator


Joined: 09 Mar 2011
Posts: 7226

PostPosted: Mon May 20, 2013 6:50 pm    Post subject: Reply to: Drop all the records before a criteria
Reply with quote

You should put SORTED,NOSEQCK on both JOINKEYS files.

Since the code is working, I guess you have specified the LRECL for the output file in the JCL.

It is more flexible to leave off all DCB-related information from the JCL, and let DFSORT handle it.

You then add

Code:
  INREC BUILD=(1,250)


to the JOINKEYS main-task (put it after your OPTION COPY to be logical).

You then have all the information for the data in one place (the Sort Control Cards) rather than in two places (some in the Control Cards, some in the JCL).

This makes it much easier to maintain/change/understand, and much easier to copy and re-use for another requirement.
Back to top
View user's profile Send private message
abhijit.nayak01

Active User


Joined: 22 Mar 2009
Posts: 157
Location: South Africa

PostPosted: Mon May 20, 2013 9:33 pm    Post subject:
Reply with quote

Thanks Bill and Skolusu. I am using the output file with DISP=OLD which is standard here.
Meanwhile saved my 60 hours of time as it was initially suggested to do with COBOL. icon_smile.gif
Back to top
View user's profile Send private message
Bill Woodger

DFSORT Moderator


Joined: 09 Mar 2011
Posts: 7226

PostPosted: Mon May 20, 2013 9:42 pm    Post subject: Reply to: Drop all the records before a criteria
Reply with quote

I'd still put the BUILD in, it will clarify in the Control Cards that the appended data is only temporary. Otherwise it'll confuse someone, someday. Maybe you in six months time :-)

If saving yourself time, it is also important not to charge the client for unnecessary processing (like leaving off the SORTED,NOSEQCK).
Back to top
View user's profile Send private message
Skolusu

Senior Member


Joined: 07 Dec 2007
Posts: 2205
Location: San Jose

PostPosted: Mon May 20, 2013 9:46 pm    Post subject:
Reply with quote

abhijit.nayak01 wrote:
Thanks Bill and Skolusu. I am using the output file with DISP=OLD which is standard here.
Meanwhile saved my 60 hours of time as it was initially suggested to do with COBOL. icon_smile.gif


I am assuming that it is NOT the same input dataset. If it is then you are inviting another problem using the same dataset for input and output. For a copy application, the SORTIN data set should not be the same as the SORTOUT data set or any OUTFIL data set because this can cause lost or incorrect data or unpredictable results.

You did NOT code the optimal job yet. Here are the control cards that you need. Look a the control cards and then compare it to the hints I gave earlier.

Code:

//SYSIN    DD *                                                   
  OPTION COPY                                                     
  JOINKEYS F1=INA,FIELDS=(251,8,A),SORTED,NOSEQCK                 
  JOINKEYS F2=INB,FIELDS=(001,8,A),SORTED,NOSEQCK                 
  JOIN UNPAIRED,F1,ONLY                                           
  INREC BUILD=(1,250)                                             
//*                                                               
//JNF1CNTL DD *                                                   
  INREC IFTHEN=(WHEN=GROUP,BEGIN=(1,2,CH,EQ,C'01'),PUSH=(251:ID=8))
//*                                                               
//JNF2CNTL DD *                                                   
  INCLUDE COND=(1,2,CH,EQ,C'01')                                   
  INREC BUILD=(SEQNUM,8,ZD,START=0)                               
//*
Back to top
View user's profile Send private message
abhijit.nayak01

Active User


Joined: 22 Mar 2009
Posts: 157
Location: South Africa

PostPosted: Tue May 21, 2013 1:48 am    Post subject:
Reply with quote

Hi Bill/Skolusu,
The input and output are different as per my previous JCL.
Code:
//IN1 DD DSN=P2D.D130519,DISP=SHR                   
//IN2 DD DSN=P2D.D130519,DISP=SHR                   
//SORTOUT DD DSN=P2D.DT130519.OP,DISP=OLD           
//SYSOUT   DD SYSOUT=*                             
//JNF1CNTL DD *                                     
 INREC IFTHEN=(WHEN=GROUP,BEGIN=(01,2,CH,EQ,C'01'),
              PUSH=(251:ID=8))                     
/*                                                 
//JNF2CNTL DD *                                     
 INCLUDE COND=(01,2,CH,EQ,C'01')                   
 INREC BUILD=(1:SEQNUM,8,ZD,START=00000000)         
/*                                                 
//SYSIN DD *                                       
 JOINKEYS F1=IN1,FIELDS=(251,8,A),SORTED,NOSEQCK   
 JOINKEYS F2=IN2,FIELDS=(1,8,A),SORTED,NOSEQCK     
 JOIN UNPAIRED,F1,ONLY                             
 OPTION COPY                                       
/*                                                 




Now my only question is even if the output datset is old and it has lrecl=250 do we need
Code:
INREC BUILD=(1,250) 
Back to top
View user's profile Send private message
Bill Woodger

DFSORT Moderator


Joined: 09 Mar 2011
Posts: 7226

PostPosted: Tue May 21, 2013 4:07 am    Post subject: Reply to: Drop all the records before a criteria
Reply with quote

Do you "need" it in terms of the process doing what you want? No. The JCL will be providing 250 for the LRECL, so the data will be truncated to 250, and the extended ID will never have a permanent existence.

Do you "need" it in terms of a full understanding of what the aim of the Control Cards is, now, and any time anyone needs to look at them, or to copy them to carry out some other task? I'd say Yes.

I don't think it'll make any difference to the resources used (the truncation will happen with the BUILD, rather than ever-so-slightly later, that's all). It is extra typing, but not much.

For spending that extra on it, for doing the complete job, you'll save time every time someone looks at the Control Cards, copies them for another task, looks at those copied cards, or copies them again.

For me, a "no brainer".

Specifying all outputs as DISP=OLD is "interesting". What are the benefits seen to be behind that idea?
Back to top
View user's profile Send private message
Skolusu

Senior Member


Joined: 07 Dec 2007
Posts: 2205
Location: San Jose

PostPosted: Tue May 21, 2013 4:22 am    Post subject:
Reply with quote

abhijit.nayak01 wrote:
Now my only question is even if the output datset is old and it has lrecl=250 do we need
Code:
INREC BUILD=(1,250) 


You are directing your concerns in the wrong area. Earlier you are were doing 2 sorts and reading tons of unnecessary records. Instead of correcting them and learning to optimize your job, you are looking at the most mundane issue about the LRECL truncation.

You need to spend more time understanding the job or follow directions than truncation of data.

As bill pointed I would love to know the benefits of using DISP=OLD. This is not IMS checkpoint restart job, so why would any one have DISP=OLD? Lets say for some reason the joinkeys job returned an RC=16 with some kind of error, but the job doesn't abend(The default setting for DFSORT is not to abend but return an RC=16). Since you had DISP=OLD, you would get the data from earlier run? Is that really how you process the data?
Back to top
View user's profile Send private message
dick scherrer

Site Director


Joined: 23 Nov 2006
Posts: 19270
Location: Inside the Matrix

PostPosted: Tue May 21, 2013 9:33 pm    Post subject:
Reply with quote

Hello,

Possibly, there were problems in allocating output datasets in the past and this method was chosen to make sure the dataset was "there". Also, possibly the problem of getting a duplicate when the attempt to allocate is made. Poor job of housekeeping.

Far easier to do this than actually manage the dasd and the processes that allocate datasets . . .
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> DFSORT/ICETOOL All times are GMT + 6 Hours
Goto page 1, 2  Next
Page 1 of 2

 

Search our Forum:

Similar Topics
Topic Author Forum Replies Posted
No new posts Limit duplicate records in the SORT pshongal SYNCSORT 6 Mon Nov 21, 2016 12:54 pm
No new posts How to split the records using the am... vnktrrd DFSORT/ICETOOL 24 Fri Oct 28, 2016 7:33 pm
No new posts Sort records based on numeric field. Alks SYNCSORT 2 Wed Oct 19, 2016 10:14 pm
No new posts abend sort based on count records in ... anatol DFSORT/ICETOOL 5 Mon Oct 17, 2016 10:10 pm
No new posts how to split records based on specifi... Venkata Ramayya DFSORT/ICETOOL 6 Wed Sep 28, 2016 3:20 am


Facebook
Back to Top
 
Mainframe Wiki | Forum Rules | Bookmarks | Subscriptions | FAQ | Tutorials | Contact Us