Portal | Manuals | References | Downloads | Info | Programs | JCLs | Mainframe wiki | Quick Ref
IBM Mainframe Computers Forums Index
 
Register
 
IBM Mainframe Computers Forums Index Mainframe: Search IBM Mainframe Forum: FAQ Memberlist Profile Log in to check your private messages Log in
 
Large file sort

 
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> DFSORT/ICETOOL
View previous topic :: :: View next topic  
Author Message
jerryte

Active User


Joined: 29 Oct 2010
Posts: 194
Location: Toronto, ON, Canada

PostPosted: Fri Apr 18, 2014 12:09 am    Post subject: Large file sort
Reply with quote

Please provide me with tips and pointers on how to make large file sort run faster. I tried reading the manuals but there are some many options that I am not sure which is the best for what I need.

I have a tape dataset LRECL=1000 with about 40 million records. It is doing an ICETOOL DATASORT with the below are the sort parms:
Code:
 OPTION SIZE=E40000000
 INREC BUILD=(1,321)
 SORT FIELDS=(4,43,CH,A)

The INREC is there to reduce the size of the record before the sort. The remaining 679 bytes are just spaces.
The job takes half hour to run. The messages has "INTERMEDIATE MERGE ENTERED - PERFORMANCE MAY BE DEGRADED"
The jcl has 18 SORTWK with 5000 cylinders each. JCL has REGION=128M

Any suggestions or ideas?
Back to top
View user's profile Send private message

Rohit Umarjikar

Senior Member


Joined: 21 Sep 2010
Posts: 1969
Location: NY,USA

PostPosted: Fri Apr 18, 2014 12:54 am    Post subject:
Reply with quote

You may try to split the process or see if any DFSPARM option helps along with below link, as there could not be a single answer to this.

http://pic.dhe.ibm.com/infocenter/zos/v1r12/index.jsp?topic=%2Fcom.ibm.zos.r12.icea100%2Fice1ca50245.htm
Back to top
View user's profile Send private message
dick scherrer

Site Director


Joined: 23 Nov 2006
Posts: 19270
Location: Inside the Matrix

PostPosted: Fri Apr 18, 2014 2:01 am    Post subject:
Reply with quote

Hello,

If you do not get charged for cpu and I/o when testing, you might try a few experiments with different settings and see which perform better. Several of the IT depts. I've worked with are charged for testing and are thus encouraged to run small tests until testing is nearly complete and then run full volume tests.

Something that has worked in the past was to split the input into a few more manageable files, SORT them, and then MERGE for the final output.
Back to top
View user's profile Send private message
Bill Woodger

DFSORT Moderator


Joined: 09 Mar 2011
Posts: 7314

PostPosted: Fri Apr 18, 2014 2:18 am    Post subject: Reply to: Large file sort
Reply with quote

I suggest you run your job adding //SORTDIAG DD DUMMY to your sort step and send the complete output to dfsort@us.ibm.com
Back to top
View user's profile Send private message
jerryte

Active User


Joined: 29 Oct 2010
Posts: 194
Location: Toronto, ON, Canada

PostPosted: Thu Apr 24, 2014 2:52 am    Post subject: Reply to: Large file sort
Reply with quote

I sent the job results to IBM as suggested.

I also tried doing a split into 4 smaller files on dasd, running 4 sorts, and then doing a merge. The elapsed time was about half of the original. The CPU time was only a slight improvement.
Back to top
View user's profile Send private message
Rohit Umarjikar

Senior Member


Joined: 21 Sep 2010
Posts: 1969
Location: NY,USA

PostPosted: Thu Apr 24, 2014 3:01 am    Post subject:
Reply with quote

Can you please let us know total count of records, CPU,Elasped time used and also a job log showing all the details?
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> DFSORT/ICETOOL All times are GMT + 6 Hours
Page 1 of 1

 

Search our Forum:

Similar Topics
Topic Author Forum Replies Posted
No new posts Join 2 File using a Join Key scorp_rahul23 DFSORT/ICETOOL 6 Tue Aug 14, 2018 7:36 am
No new posts JCL to send email along with formated... wik123 JCL & VSAM 5 Thu Aug 09, 2018 9:48 am
No new posts How can we create a flat file in JAVA... jasorn Java & MQSeries 2 Wed Aug 08, 2018 3:08 pm
No new posts Need Help on sending email by includi... wik123 JCL & VSAM 9 Mon Jul 30, 2018 2:59 pm
No new posts How to remove spaces between variable... Selvarajars DFSORT/ICETOOL 7 Fri Jul 27, 2018 3:58 pm

Facebook
Back to Top
 
Job Vacancies | Forum Rules | Bookmarks | Subscriptions | FAQ | Polls | Contact Us