View previous topic :: View next topic
|
Author |
Message |
jerryte
Active User
Joined: 29 Oct 2010 Posts: 202 Location: Toronto, ON, Canada
|
|
|
|
Please provide me with tips and pointers on how to make large file sort run faster. I tried reading the manuals but there are some many options that I am not sure which is the best for what I need.
I have a tape dataset LRECL=1000 with about 40 million records. It is doing an ICETOOL DATASORT with the below are the sort parms:
Code: |
OPTION SIZE=E40000000
INREC BUILD=(1,321)
SORT FIELDS=(4,43,CH,A) |
The INREC is there to reduce the size of the record before the sort. The remaining 679 bytes are just spaces.
The job takes half hour to run. The messages has "INTERMEDIATE MERGE ENTERED - PERFORMANCE MAY BE DEGRADED"
The jcl has 18 SORTWK with 5000 cylinders each. JCL has REGION=128M
Any suggestions or ideas? |
|
Back to top |
|
|
Rohit Umarjikar
Global Moderator
Joined: 21 Sep 2010 Posts: 3048 Location: NYC,USA
|
|
Back to top |
|
|
dick scherrer
Moderator Emeritus
Joined: 23 Nov 2006 Posts: 19244 Location: Inside the Matrix
|
|
|
|
Hello,
If you do not get charged for cpu and I/o when testing, you might try a few experiments with different settings and see which perform better. Several of the IT depts. I've worked with are charged for testing and are thus encouraged to run small tests until testing is nearly complete and then run full volume tests.
Something that has worked in the past was to split the input into a few more manageable files, SORT them, and then MERGE for the final output. |
|
Back to top |
|
|
Bill Woodger
Moderator Emeritus
Joined: 09 Mar 2011 Posts: 7309 Location: Inside the Matrix
|
|
|
|
I suggest you run your job adding //SORTDIAG DD DUMMY to your sort step and send the complete output to dfsort@us.ibm.com |
|
Back to top |
|
|
jerryte
Active User
Joined: 29 Oct 2010 Posts: 202 Location: Toronto, ON, Canada
|
|
|
|
I sent the job results to IBM as suggested.
I also tried doing a split into 4 smaller files on dasd, running 4 sorts, and then doing a merge. The elapsed time was about half of the original. The CPU time was only a slight improvement. |
|
Back to top |
|
|
Rohit Umarjikar
Global Moderator
Joined: 21 Sep 2010 Posts: 3048 Location: NYC,USA
|
|
|
|
Can you please let us know total count of records, CPU,Elasped time used and also a job log showing all the details? |
|
Back to top |
|
|
|