Joined: 07 Dec 2007 Posts: 2205 Location: San Jose
abhijit.nayak01 wrote:
Yes The header, detail and the trailer are repeating.Please see below:
And the desired output is the Last 5 records? like this
Code:
01HEADER - DROP
04DETAIL1 - DROP
04DETAIL2 - DROP
04DETAIL3 - DROP
98TRAILER1 - DROP
99TRAILER - DROP
01HEADER - DROP
04DETAIL1 - DROP
04DETAIL2 - DROP
04DETAIL3 - DROP
98TRAILER1 - DROP
99TRAILER - DROP
01HEADER - DROP
04DETAIL1 - WRITE TO OUTPUT ??
04DETAIL2 - WRITE TO OUTPUT ??
04DETAIL3 - WRITE TO OUTPUT ??
98TRAILER1 - WRITE TO OUTPUT ??
99TRAILER - WRITE TO OUTPUT ??
Joined: 22 Mar 2009 Posts: 161 Location: South Africa
The desire output is as below:
Code:
01HEADER - DROP
04DETAIL1 - DROP
04DETAIL2 - DROP
04DETAIL3 - DROP
98TRAILER1 - DROP
99TRAILER - DROP
01HEADER - DROP
04DETAIL1 - DROP
04DETAIL2 - DROP
04DETAIL3 - DROP
98TRAILER1 - DROP
99TRAILER - DROP
01HEADER - WRITE TO OUTPUT
04DETAIL1 - WRITE TO OUTPUT
04DETAIL2 - WRITE TO OUTPUT
04DETAIL3 - WRITE TO OUTPUT
98TRAILER1 - WRITE TO OUTPUT
99TRAILER - WRITE TO OUTPUT
Joined: 07 Dec 2007 Posts: 2205 Location: San Jose
I am not going to provide you complete Job, but will give you hints about solving it.
1. Look up Joinkeys examples of using the same file with temp matching keys created. There should at least 3 topics using this trick
2. For JNF1 you would use WHEN=GROUP with begin for '01' and add a 8 byte ID at the end
3. For JNF2 you would only INCLUDE '01' records and just build the record with 8 byte seqnum starting with zero.
4. Since you are creating the temp keys to be matched, make sure you have SORTED and NOSEQCK on the matching keys.
5. Use JOIN UNPAIRED,F1,ONLY to get the desired results
Once you got the desired results post the solution here so that it helps in future
Joined: 09 Mar 2011 Posts: 7309 Location: Inside the Matrix
Skolusu wrote:
[...]
3. For JNF2 you would only INCLUDE '01' records and just build the record with 8 byte seqnum starting with zero.
4. Since you are creating the temp keys to be matched, make sure you have SORTED and NOSEQCK on the matching keys.
[...]Once you got the desired results post the solution here so that it helps in future
So, in JNF2CNTL you are not using INCLUDE, you are using IFTHEN=(WHEN=(logexp. This "works", but leaves a load of unwanted and unneeded records as input to the JOINKEYS.
So, use INCLUDE, then a simple INREC BUILD. Note the "just build" from Kolusu's point. That doesn't mean "just build it at the end of the record leaving 250 blanks lying around in front of it". Your records from JNF2 only need to have the sequence number on.
Which brings us to point 4.
You are creating group IDs on JNF1, so that file is already in sequence on the key for matching. You are creating a sequence number on JNF2, so that file is already in sequence on the key for matching as well.
The way you have coded, you are sorting both files (look at your sysout) on the keys to get them into the same order. With OPTION EQUALS.
If you read Kolusu's suggestion again, you would avoid the two sorts.
Joined: 22 Mar 2009 Posts: 161 Location: South Africa
Hi Bill,
Thanks for the explanation and apologies for late response. Actually was on leave. Please see below and let me know whether the code is correct or not:
Joined: 09 Mar 2011 Posts: 7309 Location: Inside the Matrix
You should put SORTED,NOSEQCK on both JOINKEYS files.
Since the code is working, I guess you have specified the LRECL for the output file in the JCL.
It is more flexible to leave off all DCB-related information from the JCL, and let DFSORT handle it.
You then add
Code:
INREC BUILD=(1,250)
to the JOINKEYS main-task (put it after your OPTION COPY to be logical).
You then have all the information for the data in one place (the Sort Control Cards) rather than in two places (some in the Control Cards, some in the JCL).
This makes it much easier to maintain/change/understand, and much easier to copy and re-use for another requirement.
Joined: 22 Mar 2009 Posts: 161 Location: South Africa
Thanks Bill and Skolusu. I am using the output file with DISP=OLD which is standard here.
Meanwhile saved my 60 hours of time as it was initially suggested to do with COBOL.
Joined: 09 Mar 2011 Posts: 7309 Location: Inside the Matrix
I'd still put the BUILD in, it will clarify in the Control Cards that the appended data is only temporary. Otherwise it'll confuse someone, someday. Maybe you in six months time :-)
If saving yourself time, it is also important not to charge the client for unnecessary processing (like leaving off the SORTED,NOSEQCK).
Joined: 07 Dec 2007 Posts: 2205 Location: San Jose
abhijit.nayak01 wrote:
Thanks Bill and Skolusu. I am using the output file with DISP=OLD which is standard here.
Meanwhile saved my 60 hours of time as it was initially suggested to do with COBOL.
I am assuming that it is NOT the same input dataset. If it is then you are inviting another problem using the same dataset for input and output. For a copy application, the SORTIN data set should not be the same as the SORTOUT data set or any OUTFIL data set because this can cause lost or incorrect data or unpredictable results.
You did NOT code the optimal job yet. Here are the control cards that you need. Look a the control cards and then compare it to the hints I gave earlier.
Joined: 09 Mar 2011 Posts: 7309 Location: Inside the Matrix
Do you "need" it in terms of the process doing what you want? No. The JCL will be providing 250 for the LRECL, so the data will be truncated to 250, and the extended ID will never have a permanent existence.
Do you "need" it in terms of a full understanding of what the aim of the Control Cards is, now, and any time anyone needs to look at them, or to copy them to carry out some other task? I'd say Yes.
I don't think it'll make any difference to the resources used (the truncation will happen with the BUILD, rather than ever-so-slightly later, that's all). It is extra typing, but not much.
For spending that extra on it, for doing the complete job, you'll save time every time someone looks at the Control Cards, copies them for another task, looks at those copied cards, or copies them again.
For me, a "no brainer".
Specifying all outputs as DISP=OLD is "interesting". What are the benefits seen to be behind that idea?
Joined: 07 Dec 2007 Posts: 2205 Location: San Jose
abhijit.nayak01 wrote:
Now my only question is even if the output datset is old and it has lrecl=250 do we need
Code:
INREC BUILD=(1,250)
You are directing your concerns in the wrong area. Earlier you are were doing 2 sorts and reading tons of unnecessary records. Instead of correcting them and learning to optimize your job, you are looking at the most mundane issue about the LRECL truncation.
You need to spend more time understanding the job or follow directions than truncation of data.
As bill pointed I would love to know the benefits of using DISP=OLD. This is not IMS checkpoint restart job, so why would any one have DISP=OLD? Lets say for some reason the joinkeys job returned an RC=16 with some kind of error, but the job doesn't abend(The default setting for DFSORT is not to abend but return an RC=16). Since you had DISP=OLD, you would get the data from earlier run? Is that really how you process the data?
Joined: 23 Nov 2006 Posts: 19244 Location: Inside the Matrix
Hello,
Possibly, there were problems in allocating output datasets in the past and this method was chosen to make sure the dataset was "there". Also, possibly the problem of getting a duplicate when the attempt to allocate is made. Poor job of housekeeping.
Far easier to do this than actually manage the dasd and the processes that allocate datasets . . .
Joined: 22 Mar 2009 Posts: 161 Location: South Africa
Actually from the callout prospective we use DISP=OLD as the dataset is created in one cycle job which gets refreshed everyday after the batch run. We keep it DISP=OLD because if the job abends and when we ask the operator to rerun then we should not face the JCL error for dataset already present.
After every step we check the RC and if the RC is expected then we proceed to next step else abend the job.
But for the above scenario it is my mistake while testing. The input dataset in production is a GDG version and the output dataset is a temp and then the temp dataset goes to next step for creation of report. So, when I started testing I converted it to normal dataset and did the testing and I forgot and argued for LREC=250. Sorry for that. It was my mistake for misunderstanding. If everybody agrees then we can close this topic.
Joined: 23 Nov 2006 Posts: 19244 Location: Inside the Matrix
Hello,
Quote:
when we ask the operator to rerun then we should not face the JCL error for dataset already present.
Exactly a situation that demonstrates poor process design.
Well-implemented batch jobs ALWAYS cleanup before and/or after they run. There would be NO WAY to have problems if these jobs were properly implemented.
Joined: 09 Mar 2011 Posts: 7309 Location: Inside the Matrix
I've never heard of doing it that way :-)
That's definitely a "not fixing the brakes but making the horn louder".
I don't suppose you use RLSE? Client overpaying for DASD? Or have to have secondary allocation taking on the role of primary, so overallocation during the run, no "warning" of unexpected extensions in file sizes, blah, blah.
I'd suggest getting it fixed before the client finds out. Unless it was their idea in the first place, in which case give me their contact details, please, as I've got a bridge I'd like to sell to them :-)