my question is, is there any other way to do this in sort methods...because its taking almost 3 hours to complete this job ... i know the input file size is of huge volume...but is there any way to reduce the time atleast...please help..
Joined: 23 Nov 2006 Posts: 19244 Location: Inside the Matrix
Hello,
Quote:
is there anyway to reduce the run time .
Yes, surely. . . which is why i asked
Quote:
What reason is there to copy the entire file to skip the "H" and "T" records?
This is 100million+ unnecessary reads and writes, not counting an unnecessary very large transient file to pass from step to step.
Suggest you at least accomplish discarding the headers/trailers and sorting the needed data in one step.
You didn't answer how long the main records (the data records) are. If only part of the data is needed as output, it would be wise to only sort and write out the needed fields. I suspect that with a bit of re-work, the process might be reduced by well more than 1/2.
Joined: 14 Oct 2005 Posts: 1208 Location: Bangalore,India
Hello Karthik,
Please try below sort card if
-- Put special key of '1' in 81 for data records.
INREC IFTHEN=(WHEN=INIT,OVERLAY=(81:C'1')),
-- For header record, put special key of '' in 81.
IFTHEN=(WHEN=(1,1,CH,EQ,C'H',OVERLAY=(81:C'')),
-- For trailer record, put special key of '9' in 81.
IFTHEN=(WHEN=(1,1,CH,EQ,C'T'),OVERLAY=(81:C'9'))
-- Sort by special key ('0', '1' or '9') and then regular key.
SORT FIELDS=(81,1,CH,A,14,10,CH,A)
--Remove special key.
OUTREC FIELDS=(1,80)
Here i assumed that record length is 80. please change it accordingly & see if it helps.
-- Put special key of '1' in 81 for data records.
INREC IFTHEN=(WHEN=INIT,OVERLAY=(81:C'1')),
-- For header record, put special key of '' in 81.
IFTHEN=(WHEN=(1,1,CH,EQ,C'H',OVERLAY=(81:C'')),
-- For trailer record, put special key of '9' in 81.
IFTHEN=(WHEN=(1,1,CH,EQ,C'T'),OVERLAY=(81:C'9'))
-- Sort by special key ('0', '1' or '9') and then regular key.
SORT FIELDS=(81,1,CH,A,14,10,CH,A)
--Remove special key.
OUTREC FIELDS=(1,80)
why should i use special key and all...i confused?!
Joined: 23 Nov 2006 Posts: 19244 Location: Inside the Matrix
Hello,
The suggestion from Ekta is for cases when the header and trailer are to be preserved and the detail records sorted by some key and the final output has the original header, the newly sorted detail, and the original trailer.
In your first pass, it appeared that you intended to discard the header and trailer records. . .
Joined: 17 Oct 2006 Posts: 2481 Location: @my desk
rkarthik22,
Your input LRECL is as big as 1602 and you just need 90 bytes out of it in the ouput. Hence you must be using INREC instead of OUTREC as INREC selects the required fields before sort processing. And as Dick suggested, you dont really need two passes of data to achieve this.
Joined: 23 Nov 2006 Posts: 19244 Location: Inside the Matrix
Hello,
Quote:
It tooks only 26 minutes to run this job. And i got the expected output .(my earlier JCL tooks almost 3 hours to get the output)
Suggest you learn that it is better to think thru a process rather than just shoving together some jcl and control statements. . .
Multiple passes of large files is always questionable (in some cases it might not be wrong, but it is always worth looking into). If you find your design will do this, suggest looking for alternatives immediately. If you cannot find an alternative, ask for help.
If you are a programmer, think how you would accomplish the requirement if you wrote code. Then "program" the sort or other utility to behave similarly.
