Portal | Manuals | References | Downloads | Info | Programs | JCLs | Master the Mainframes
IBM Mainframe Computers Forums Index
 
Register
 
IBM Mainframe Computers Forums Index Mainframe: Search IBM Mainframe Forum: FAQ Memberlist Usergroups Profile Log in to check your private messages Log in
 

 

Large file sort

 
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> DFSORT/ICETOOL
View previous topic :: :: View next topic  
Author Message
jerryte

Active User


Joined: 29 Oct 2010
Posts: 183
Location: Toronto, ON, Canada

PostPosted: Fri Apr 18, 2014 12:09 am    Post subject: Large file sort
Reply with quote

Please provide me with tips and pointers on how to make large file sort run faster. I tried reading the manuals but there are some many options that I am not sure which is the best for what I need.

I have a tape dataset LRECL=1000 with about 40 million records. It is doing an ICETOOL DATASORT with the below are the sort parms:
Code:
 OPTION SIZE=E40000000
 INREC BUILD=(1,321)
 SORT FIELDS=(4,43,CH,A)

The INREC is there to reduce the size of the record before the sort. The remaining 679 bytes are just spaces.
The job takes half hour to run. The messages has "INTERMEDIATE MERGE ENTERED - PERFORMANCE MAY BE DEGRADED"
The jcl has 18 SORTWK with 5000 cylinders each. JCL has REGION=128M

Any suggestions or ideas?
Back to top
View user's profile Send private message

Rohit Umarjikar

Senior Member


Joined: 21 Sep 2010
Posts: 1609
Location: NY,USA

PostPosted: Fri Apr 18, 2014 12:54 am    Post subject:
Reply with quote

You may try to split the process or see if any DFSPARM option helps along with below link, as there could not be a single answer to this.

http://pic.dhe.ibm.com/infocenter/zos/v1r12/index.jsp?topic=%2Fcom.ibm.zos.r12.icea100%2Fice1ca50245.htm
Back to top
View user's profile Send private message
dick scherrer

Site Director


Joined: 23 Nov 2006
Posts: 19270
Location: Inside the Matrix

PostPosted: Fri Apr 18, 2014 2:01 am    Post subject:
Reply with quote

Hello,

If you do not get charged for cpu and I/o when testing, you might try a few experiments with different settings and see which perform better. Several of the IT depts. I've worked with are charged for testing and are thus encouraged to run small tests until testing is nearly complete and then run full volume tests.

Something that has worked in the past was to split the input into a few more manageable files, SORT them, and then MERGE for the final output.
Back to top
View user's profile Send private message
Bill Woodger

DFSORT Moderator


Joined: 09 Mar 2011
Posts: 7223

PostPosted: Fri Apr 18, 2014 2:18 am    Post subject: Reply to: Large file sort
Reply with quote

I suggest you run your job adding //SORTDIAG DD DUMMY to your sort step and send the complete output to dfsort@us.ibm.com
Back to top
View user's profile Send private message
jerryte

Active User


Joined: 29 Oct 2010
Posts: 183
Location: Toronto, ON, Canada

PostPosted: Thu Apr 24, 2014 2:52 am    Post subject: Reply to: Large file sort
Reply with quote

I sent the job results to IBM as suggested.

I also tried doing a split into 4 smaller files on dasd, running 4 sorts, and then doing a merge. The elapsed time was about half of the original. The CPU time was only a slight improvement.
Back to top
View user's profile Send private message
Rohit Umarjikar

Senior Member


Joined: 21 Sep 2010
Posts: 1609
Location: NY,USA

PostPosted: Thu Apr 24, 2014 3:01 am    Post subject:
Reply with quote

Can you please let us know total count of records, CPU,Elasped time used and also a job log showing all the details?
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> DFSORT/ICETOOL All times are GMT + 6 Hours
Page 1 of 1

 

Search our Forum:

Similar Topics
Topic Author Forum Replies Posted
No new posts Add PD field from 2nd file to PD in 1st Sushant Garje DFSORT/ICETOOL 6 Thu Dec 01, 2016 4:32 pm
No new posts File Aid to File Manager conversion murali3955 IBM Tools 4 Thu Nov 24, 2016 3:41 pm
No new posts Limit duplicate records in the SORT pshongal SYNCSORT 6 Mon Nov 21, 2016 12:54 pm
No new posts How to convert the VBM file to VB or... Sulabh Agrawal JCL & VSAM 4 Fri Nov 18, 2016 1:04 pm
No new posts CICS Roll back partially - Need to re... dwijadas CICS 4 Wed Nov 16, 2016 4:30 pm


Facebook
Back to Top
 
Mainframe Wiki | Forum Rules | Bookmarks | Subscriptions | FAQ | Tutorials | Contact Us