IBM Mainframe Forum Index
 
Log In
 
IBM Mainframe Forum Index Mainframe: Search IBM Mainframe Forum: FAQ Register
 

Large file sort


IBM Mainframe Forums -> DFSORT/ICETOOL
Post new topic   Reply to topic
View previous topic :: View next topic  
Author Message
jerryte

Active User


Joined: 29 Oct 2010
Posts: 202
Location: Toronto, ON, Canada

PostPosted: Fri Apr 18, 2014 12:09 am
Reply with quote

Please provide me with tips and pointers on how to make large file sort run faster. I tried reading the manuals but there are some many options that I am not sure which is the best for what I need.

I have a tape dataset LRECL=1000 with about 40 million records. It is doing an ICETOOL DATASORT with the below are the sort parms:
Code:
 OPTION SIZE=E40000000
 INREC BUILD=(1,321)
 SORT FIELDS=(4,43,CH,A)

The INREC is there to reduce the size of the record before the sort. The remaining 679 bytes are just spaces.
The job takes half hour to run. The messages has "INTERMEDIATE MERGE ENTERED - PERFORMANCE MAY BE DEGRADED"
The jcl has 18 SORTWK with 5000 cylinders each. JCL has REGION=128M

Any suggestions or ideas?
Back to top
View user's profile Send private message
Rohit Umarjikar

Global Moderator


Joined: 21 Sep 2010
Posts: 3048
Location: NYC,USA

PostPosted: Fri Apr 18, 2014 12:54 am
Reply with quote

You may try to split the process or see if any DFSPARM option helps along with below link, as there could not be a single answer to this.

pic.dhe.ibm.com/infocenter/zos/v1r12/index.jsp?topic=%2Fcom.ibm.zos.r12.icea100%2Fice1ca50245.htm
Back to top
View user's profile Send private message
dick scherrer

Moderator Emeritus


Joined: 23 Nov 2006
Posts: 19244
Location: Inside the Matrix

PostPosted: Fri Apr 18, 2014 2:01 am
Reply with quote

Hello,

If you do not get charged for cpu and I/o when testing, you might try a few experiments with different settings and see which perform better. Several of the IT depts. I've worked with are charged for testing and are thus encouraged to run small tests until testing is nearly complete and then run full volume tests.

Something that has worked in the past was to split the input into a few more manageable files, SORT them, and then MERGE for the final output.
Back to top
View user's profile Send private message
Bill Woodger

Moderator Emeritus


Joined: 09 Mar 2011
Posts: 7309
Location: Inside the Matrix

PostPosted: Fri Apr 18, 2014 2:18 am
Reply with quote

I suggest you run your job adding //SORTDIAG DD DUMMY to your sort step and send the complete output to dfsort@us.ibm.com
Back to top
View user's profile Send private message
jerryte

Active User


Joined: 29 Oct 2010
Posts: 202
Location: Toronto, ON, Canada

PostPosted: Thu Apr 24, 2014 2:52 am
Reply with quote

I sent the job results to IBM as suggested.

I also tried doing a split into 4 smaller files on dasd, running 4 sorts, and then doing a merge. The elapsed time was about half of the original. The CPU time was only a slight improvement.
Back to top
View user's profile Send private message
Rohit Umarjikar

Global Moderator


Joined: 21 Sep 2010
Posts: 3048
Location: NYC,USA

PostPosted: Thu Apr 24, 2014 3:01 am
Reply with quote

Can you please let us know total count of records, CPU,Elasped time used and also a job log showing all the details?
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic View Bookmarks
All times are GMT + 6 Hours
Forum Index -> DFSORT/ICETOOL

 


Similar Topics
Topic Forum Replies
No new posts How to split large record length file... DFSORT/ICETOOL 8
No new posts Extracting Variable decimal numbers f... DFSORT/ICETOOL 17
No new posts SFTP Issue - destination file record ... All Other Mainframe Topics 2
No new posts Access to non cataloged VSAM file JCL & VSAM 18
No new posts Need help for File Aid JCL to extract... Compuware & Other Tools 23
Search our Forums:

Back to Top