I have the following scenario. There a 2 sort steps in a job, which uses same input having approx 180000000 records. Record length of input and output files in 600. Two outputs are created out of the 2 steps.
First step is like below,
Code:
SORT FIELDS=(1,5,CH,A,10,3,CH,A,20,6,CH,A,45,3,CH,A)
INCLUDE COND=(45,3,CH,EQ,C'001',
OR,45,3,CH,EQ,C'032',
OR,45,3,CH,GE,C'201')
45,3 is record type. '001' is a header record which can have many records with record types '032', '201' etc.
Second step is like below,
Code:
SORT FIELDS=(1,5,CH,A,10,3,CH,A,20,6,CH,A,45,3,CH,A)
INCLUDE COND=(((45,3,CH,EQ,C'001'),OR,
(45,3,CH,EQ,C'100',AND,592,1,CH,EQ,C'R'),OR,
(45,3,CH,EQ,C'200',AND,582,1,CH,EQ,C'2'),OR,
(45,3,CH,EQ,C'300',AND,595,1,CH,EQ,C'R')))
In the above steps they used the same SORT FIELDS,
Joined: 15 Feb 2005 Posts: 7129 Location: San Jose, CA
Quote:
These steps taking lot of time as the input is having huge no of records.
Please let me know if we can use any other SORT which eliminates the performance issue.
This is quite vague. You may or may not be able to do anything to improve performance, but you haven't given enough information for anyone to determine that.
You INCLUDE statements are reducing the number of records to be sorted, which is a good thing. But you're using two passes over the data which is a bad thing.
Ideally, you might want to use an INCLUDE statement that includes the relevant records for both output data sets (to reduce the number of records to be sorted) and two OUTFIL statements, each with an INCLUDE operand to create the output data sets from the sorted records. Like this:
But whether that will improve performance enough to satisfy you, or whether some other tuning you could do would help more, is impossible to answer based on the information you've given.
Joined: 15 Feb 2005 Posts: 7129 Location: San Jose, CA
Quote:
Which also having the same issue.
You haven't really demonstrated that there is an issue, or what the issue is.
And you didn't do what I suggested.
In my job, I used an INCLUDE statement to remove unneeded records so they wouldn't be sorted. You don't have that INCLUDE statement so you're sorting all of the records. The INCLUDE statement may or may not help, but you didn't try it, so you don't know.
.jpg)
