Portal | Manuals | References | Downloads | Info | Programs | JCLs | Master the Mainframes
IBM Mainframe Computers Forums Index
 
Register
 
IBM Mainframe Computers Forums Index Mainframe: Search IBM Mainframe Forum: FAQ Memberlist Usergroups Profile Log in to check your private messages Log in
 

 

Sort Control card performance issue

 
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> DFSORT/ICETOOL
View previous topic :: :: View next topic  
Author Message
Rajesh1979

New User


Joined: 24 Dec 2007
Posts: 9
Location: Chennai

PostPosted: Thu Mar 12, 2009 10:22 pm    Post subject: Sort Control card performance issue
Reply with quote

Hi,

I have the following scenario. There a 2 sort steps in a job, which uses same input having approx 180000000 records. Record length of input and output files in 600. Two outputs are created out of the 2 steps.

First step is like below,

Code:

 SORT FIELDS=(1,5,CH,A,10,3,CH,A,20,6,CH,A,45,3,CH,A)
 INCLUDE COND=(45,3,CH,EQ,C'001',                     
              OR,45,3,CH,EQ,C'032',                   
              OR,45,3,CH,GE,C'201')       


45,3 is record type. '001' is a header record which can have many records with record types '032', '201' etc.

Second step is like below,

Code:

  SORT FIELDS=(1,5,CH,A,10,3,CH,A,20,6,CH,A,45,3,CH,A)     
  INCLUDE COND=(((45,3,CH,EQ,C'001'),OR,                     
               (45,3,CH,EQ,C'100',AND,592,1,CH,EQ,C'R'),OR,
               (45,3,CH,EQ,C'200',AND,582,1,CH,EQ,C'2'),OR,
               (45,3,CH,EQ,C'300',AND,595,1,CH,EQ,C'R')))   


In the above steps they used the same SORT FIELDS,

Code:

  SORT FIELDS=(1,5,CH,A,10,3,CH,A,20,6,CH,A,45,3,CH,A)     


These steps taking lot of time as the input is having huge no of records.

Please let me know if we can use any other SORT which eliminates the performance issue.
Back to top
View user's profile Send private message

Terry Heinze

JCL Moderator


Joined: 14 Jul 2008
Posts: 1249
Location: Richfield, MN, USA

PostPosted: Thu Mar 12, 2009 11:43 pm    Post subject:
Reply with quote

Define "lot of time".
Back to top
View user's profile Send private message
Douglas Wilder

Active User


Joined: 28 Nov 2006
Posts: 305
Location: Deerfield IL

PostPosted: Fri Mar 13, 2009 12:34 am    Post subject:
Reply with quote

Run 1 sort with OUTFIL to create the 2 output files in the same step.
Back to top
View user's profile Send private message
Frank Yaeger

DFSORT Moderator


Joined: 15 Feb 2005
Posts: 7130
Location: San Jose, CA

PostPosted: Fri Mar 13, 2009 2:56 am    Post subject:
Reply with quote

Quote:
These steps taking lot of time as the input is having huge no of records.

Please let me know if we can use any other SORT which eliminates the performance issue.


This is quite vague. You may or may not be able to do anything to improve performance, but you haven't given enough information for anyone to determine that.

You INCLUDE statements are reducing the number of records to be sorted, which is a good thing. But you're using two passes over the data which is a bad thing.

Ideally, you might want to use an INCLUDE statement that includes the relevant records for both output data sets (to reduce the number of records to be sorted) and two OUTFIL statements, each with an INCLUDE operand to create the output data sets from the sorted records. Like this:

Code:

//S1    EXEC  PGM=SORT
//SYSOUT    DD  SYSOUT=*
//SORTIN DD DSN=... input file
//OUT1 DD DSN=...  output file1
//OUT2 DD DSN=...  output file2
//SYSIN    DD    *
 INCLUDE COND=(45,3,SS,EQ,C'001,032,201',OR,
               (45,3,CH,EQ,C'100',AND,592,1,CH,EQ,C'R'),OR,
               (45,3,CH,EQ,C'200',AND,582,1,CH,EQ,C'2'),OR,
               (45,3,CH,EQ,C'300',AND,595,1,CH,EQ,C'R'))
  SORT FIELDS=(1,5,CH,A,10,3,CH,A,20,6,CH,A,45,3,CH,A)
  OUTFIL FNAMES=OUT1,
    INCLUDE=(45,3,SS,EQ,C'001,032,201')
  OUTFIL FNAMES=OUT2,
   INCLUDE=(((45,3,CH,EQ,C'001'),OR,
             (45,3,CH,EQ,C'100',AND,592,1,CH,EQ,C'R'),OR,
             (45,3,CH,EQ,C'200',AND,582,1,CH,EQ,C'2'),OR,
             (45,3,CH,EQ,C'300',AND,595,1,CH,EQ,C'R')))
/*


But whether that will improve performance enough to satisfy you, or whether some other tuning you could do would help more, is impossible to answer based on the information you've given.
Back to top
View user's profile Send private message
Rajesh1979

New User


Joined: 24 Dec 2007
Posts: 9
Location: Chennai

PostPosted: Fri Mar 13, 2009 4:19 pm    Post subject:
Reply with quote

Thanks Frank,

I have tried the below sort card earlier.

Code:
SORT FIELDS=(5,3,CH,A,11,3,CH,A,17,6,CH,A,35,3,CH,A)           
OUTFIL FNAMES=SORTOU1,                                         
 INCLUDE =(45,3,CH,EQ,C'001',                     
              OR,45,3,CH,EQ,C'032',                   
              OR,45,3,CH,GE,C'201')                         
OUTFIL FNAMES=SORTOU2,                                         
  INCLUDE =(((45,3,CH,EQ,C'001'),OR,                     
               (45,3,CH,EQ,C'100',AND,592,1,CH,EQ,C'R'),OR,
               (45,3,CH,EQ,C'200',AND,582,1,CH,EQ,C'2'),OR,
               (45,3,CH,EQ,C'300',AND,595,1,CH,EQ,C'R'))) 

Which also having the same issue.

Note: The input file is a concatination of 2 files,with record count nearly 180000000.
Back to top
View user's profile Send private message
dick scherrer

Site Director


Joined: 23 Nov 2006
Posts: 19270
Location: Inside the Matrix

PostPosted: Fri Mar 13, 2009 7:55 pm    Post subject:
Reply with quote

Hello,

Set up a job that uses the sort to copy all of the input to a DD DUMMY output and see how long it takes to simply read the input.

On many/most systems it will take a while to read 180 million records. . .

Knowing how long it takes to read the data may help with the expectation of how long the select & sort process might run.

Something else to check is the blksize of these files.
Back to top
View user's profile Send private message
Terry Heinze

JCL Moderator


Joined: 14 Jul 2008
Posts: 1249
Location: Richfield, MN, USA

PostPosted: Fri Mar 13, 2009 7:55 pm    Post subject:
Reply with quote

You still haven't defined "lot of time".
Back to top
View user's profile Send private message
Frank Yaeger

DFSORT Moderator


Joined: 15 Feb 2005
Posts: 7130
Location: San Jose, CA

PostPosted: Sat Mar 14, 2009 1:56 am    Post subject:
Reply with quote

Quote:
Which also having the same issue.


You haven't really demonstrated that there is an issue, or what the issue is.

And you didn't do what I suggested.

In my job, I used an INCLUDE statement to remove unneeded records so they wouldn't be sorted. You don't have that INCLUDE statement so you're sorting all of the records. The INCLUDE statement may or may not help, but you didn't try it, so you don't know.
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> DFSORT/ICETOOL All times are GMT + 6 Hours
Page 1 of 1

 

Search our Forum:

Similar Topics
Topic Author Forum Replies Posted
No new posts Innovations in Your Mainframe Sort Arun Raj PC Guides & IT News 4 Wed Mar 15, 2017 10:38 pm
No new posts SORT VSAM file with each field one by... maxsubrat DFSORT/ICETOOL 6 Tue Mar 14, 2017 1:07 pm
No new posts SORT requirement sjiraga COBOL Programming 6 Thu Mar 09, 2017 9:11 pm
No new posts SORT to append Month and YEAR in MMYY... tisamf DFSORT/ICETOOL 1 Wed Mar 08, 2017 4:46 pm
No new posts JCL sort Match and NoMatch rakeshsekar1987 SYNCSORT 2 Thu Feb 02, 2017 10:26 pm


Facebook
Back to Top
 
Mainframe Wiki | Forum Rules | Bookmarks | Subscriptions | FAQ | Tutorials | Contact Us