Portal | Manuals | References | Downloads | Info | Programs | JCLs | Mainframe wiki | Quick Ref
IBM Mainframe Computers Forums Index
 
Register
 
IBM Mainframe Computers Forums Index Mainframe: Search IBM Mainframe Forum: FAQ Memberlist Profile Log in to check your private messages Log in
 
Large file sort

 
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> DFSORT/ICETOOL
View previous topic :: :: View next topic  
Author Message
jerryte

Active User


Joined: 29 Oct 2010
Posts: 187
Location: Toronto, ON, Canada

PostPosted: Fri Apr 18, 2014 12:09 am    Post subject: Large file sort
Reply with quote

Please provide me with tips and pointers on how to make large file sort run faster. I tried reading the manuals but there are some many options that I am not sure which is the best for what I need.

I have a tape dataset LRECL=1000 with about 40 million records. It is doing an ICETOOL DATASORT with the below are the sort parms:
Code:
 OPTION SIZE=E40000000
 INREC BUILD=(1,321)
 SORT FIELDS=(4,43,CH,A)

The INREC is there to reduce the size of the record before the sort. The remaining 679 bytes are just spaces.
The job takes half hour to run. The messages has "INTERMEDIATE MERGE ENTERED - PERFORMANCE MAY BE DEGRADED"
The jcl has 18 SORTWK with 5000 cylinders each. JCL has REGION=128M

Any suggestions or ideas?
Back to top
View user's profile Send private message

Rohit Umarjikar

Senior Member


Joined: 21 Sep 2010
Posts: 1806
Location: NY,USA

PostPosted: Fri Apr 18, 2014 12:54 am    Post subject:
Reply with quote

You may try to split the process or see if any DFSPARM option helps along with below link, as there could not be a single answer to this.

http://pic.dhe.ibm.com/infocenter/zos/v1r12/index.jsp?topic=%2Fcom.ibm.zos.r12.icea100%2Fice1ca50245.htm
Back to top
View user's profile Send private message
dick scherrer

Site Director


Joined: 23 Nov 2006
Posts: 19270
Location: Inside the Matrix

PostPosted: Fri Apr 18, 2014 2:01 am    Post subject:
Reply with quote

Hello,

If you do not get charged for cpu and I/o when testing, you might try a few experiments with different settings and see which perform better. Several of the IT depts. I've worked with are charged for testing and are thus encouraged to run small tests until testing is nearly complete and then run full volume tests.

Something that has worked in the past was to split the input into a few more manageable files, SORT them, and then MERGE for the final output.
Back to top
View user's profile Send private message
Bill Woodger

DFSORT Moderator


Joined: 09 Mar 2011
Posts: 7315

PostPosted: Fri Apr 18, 2014 2:18 am    Post subject: Reply to: Large file sort
Reply with quote

I suggest you run your job adding //SORTDIAG DD DUMMY to your sort step and send the complete output to dfsort@us.ibm.com
Back to top
View user's profile Send private message
jerryte

Active User


Joined: 29 Oct 2010
Posts: 187
Location: Toronto, ON, Canada

PostPosted: Thu Apr 24, 2014 2:52 am    Post subject: Reply to: Large file sort
Reply with quote

I sent the job results to IBM as suggested.

I also tried doing a split into 4 smaller files on dasd, running 4 sorts, and then doing a merge. The elapsed time was about half of the original. The CPU time was only a slight improvement.
Back to top
View user's profile Send private message
Rohit Umarjikar

Senior Member


Joined: 21 Sep 2010
Posts: 1806
Location: NY,USA

PostPosted: Thu Apr 24, 2014 3:01 am    Post subject:
Reply with quote

Can you please let us know total count of records, CPU,Elasped time used and also a job log showing all the details?
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> DFSORT/ICETOOL All times are GMT + 6 Hours
Page 1 of 1

 

Search our Forum:

Similar Topics
Topic Author Forum Replies Posted
No new posts sort records based on length exceeds ... maxsubrat DFSORT/ICETOOL 7 Wed Oct 04, 2017 4:48 pm
No new posts Execute JCL step based on the content... sprikitik JCL & VSAM 2 Tue Oct 03, 2017 10:03 am
This topic is locked: you cannot edit posts or make replies. PS file data should be passed as symb... d_sarlie JCL & VSAM 15 Tue Oct 03, 2017 5:18 am
No new posts JOINKEYS Paired and Unpaired Sort MFwhiz DFSORT/ICETOOL 4 Fri Sep 29, 2017 1:48 am
No new posts Garbage on output file Hervey Martinez SYNCSORT 4 Wed Sep 27, 2017 12:50 am

Facebook
Back to Top
 
Job Vacancies | Forum Rules | Bookmarks | Subscriptions | FAQ | Polls | Contact Us