|
View previous topic :: View next topic
|
| Author |
Message |
sandip_mainframe
Warnings : 2
New User
Joined: 20 Sep 2006
Posts: 63
Location: pune
|
|
|
|
Hi All,
I have one SORT step in my job, which is taking huge CPU time for the sorting the file. Is there any alternate way to reduce the CPU consumtion time ?
Job step is like below -
| Code: |
//PS055 EXEC PGM=SYNCSORT,PARM='EQUALS'
//SORTIN DD DSN=XX7384A.TEST.INPUT.FILE,
// DISP=SHR
//SORTOUT DD DSN=XX7384A.TEST.SORTED.OUT.FILE,
// DISP=(,CATLG,DELETE),
// UNIT=SYSDA,SPACE=(CYL,(95,95),RLSE),
// DATACLAS=HUGE
//SYSOUT DD SYSOUT=*
//SYSIN DD *
SORT FIELDS=(1,10,CH,A,11,12,CH,A,23,2,PD,D)
SUM FIELDS=NONE
/*
|
Daily we receive around 98,70,00,000 input records, is there any way to reduce CPU time so that it should reduce the job's run cost.
Please let me know if more information is required.
Thanks,
Sandip Walsinge
Pune. |
|
| Back to top |
|
 |
enrico-sorichetti
Superior Member
Joined: 14 Mar 2007
Posts: 10888
Location: italy
|
|
|
|
define huge !
is it just generic discontent or You have evidence that similar sorts for the same record count use less cpu time ?
and please use a western punctuation style for numbers 
would that be 987.000,000 ( 987 millions of records )
and since You are aware that You are using SYNCSORT why posy in the DFSORT/ICETOOL section ?
topic moved |
|
| Back to top |
|
|
sandip_mainframe
Warnings : 2
New User
Joined: 20 Sep 2006
Posts: 63
Location: pune
|
|
|
|
Hi Enrico,
I don't have any evidence, just want to is there any way to reduce the CPU time.
Sorry for the punctuation. It's 987 Millions of records.
Please suggest if there is any way.
Thanks for your reply.
Thanks
Sandip Walsinge |
|
| Back to top |
|
|
gylbharat
Active Member
.jpg)
Joined: 31 Jul 2009
Posts: 565
Location: Bangalore
|
|
|
|
Try using ELAP/CPU parameter
CPU Parm
CPU minimizes the CPU time of each sort at the expense of sort elapsed time
and I/O activity. See BALANCE, ELAP and IO. Note that these options and
CPU are all mutually exclusive.
Cannot be used with Tape Sort.
ELAP Parm
ELAP minimizes the elapsed time (wall clock time) of each sort at the
expense of CPU time nd I/O activity. See BALANCE, CPU and IO. Note that
these options and ELAP are all utually exclusive.
Cannot be used with Tape Sort. |
|
| Back to top |
|
|
Bill Woodger
Moderator Emeritus
Joined: 09 Mar 2011
Posts: 7309
Location: Inside the Matrix
|
|
|
|
You are specifying both EQUALS and SUM FIELDS=NONE. Drop at least one of them, preferrably both.
Check in your Syncsort manual, I'm sure they have information on sorting large files. I would try to get the whole key together, and such that it would be possible to sort it all ASCENDING (what is producing the 900,000,000+ records per day, do you have any influence over that?).
Is it simply reducing CPU for charging, or do you want better thoughput? |
|
| Back to top |
|
|
Anuj Dhawan
Superior Member
Joined: 22 Apr 2006
Posts: 6248
Location: Mumbai, India
|
|
|
|
| Are you sure it's too much of CPU and not elapsed time? If you're sure, possibly read about VSCORE and VSCORET. |
|
| Back to top |
|
|
Robert Sample
Global Moderator
Joined: 06 Jun 2008
Posts: 8700
Location: Dubuque, Iowa, USA
|
|
|
|
| Quote: |
| is there any way to reduce CPU time so that it should reduce the job's run cost. |
Be aware that "run cost" is not standard across sites. Some sites use chargeback schemes to recover the mainframe cost while other sites do not. If a site does not use a chargeback scheme, then any given job will use resources (CPU time, I/O, elapsed time) but the "cost" of the job in accounting terms would be ZERO and that could not be reduced.
Furhtermore, chargeback schemes can be simple or very complex -- again, depending upon the site. If your site's chargeback scheme places a relatively high value on CPU time, then you may not be able to do much to reduce the "cost" since it's going to take a certain amount of CPU time to process that many records -- and the sort products tend to be pretty optimized already, so there's not much short of completely redesigning your processing that will cut the CPU usage. |
|
| Back to top |
|
|
Marso
REXX Moderator
Joined: 13 Mar 2006
Posts: 1353
Location: Israel
|
|
|
|
As Bill said, EQUALS and SUM FIELDS have opposite meaning.
1. loose EQUALS first.
| Syncsort 1.3 Programmers Guide wrote: |
The EQUALS PARM instructs the sort/merge to preserve the order of equal-keyed records.
EQUALS will have a slight but generally significant impact on sort performance. By making
EQUALS available on an individual sort basis, SyncSort makes this programming
option available where it is needed, without imposing it on the installation’s more routine
jobs. For sort efficiency, use EQUALS only where the preservation of the input order of
equal-keyed records is important. |
2. check if you really have duplicated records. If no loose the SUM card.
3. Specifying large enough SORTWK areas on different volumes may help too.
4. You really should open the Guide and check the available options...
| Syncsort 1.3 Programmers Guide wrote: |
| A sort is considered to finely tuned when WER124I reports an overallocation factor between 1.00 and 1.50. |
|
|
| Back to top |
|
|
Bill Woodger
Moderator Emeritus
Joined: 09 Mar 2011
Posts: 7309
Location: Inside the Matrix
|
|
|
|
| Quote: |
| EQUALS will have a slight but generally significant impact on sort performance |
I like that. If the weather forecast was for "slight but generally significant falls of snow" you'd not know whether to take an umbrella or a shovel to work with you.
Taking the EQUALS off, I'm sure you'll notice the difference.
If you really have duplicates, and you really have to have the physically latest record as the representative for that key, then there is a bit of a quandry. If there is something on the record which indicates the latest one, then post-process in whichever program is reading the file afterwards. If you have nothing, look at the viability of getting something, compared to the CPU-cost saving of remval.
You specify two seperate fields in the sort card which are contiguous. Specify them as one field. You have a PD, dislocated from the rest of the key, which is sorted descending. Look at the possibility of the producer making it negative, or "subtracted" from 999 and try to get it moved adjacent to the rest of the key, so it can all be specified in one statement.
You can prepare the records like that in an extract from your file and do some comparisons, comparing CPU usage. If the recored can be made smaller, that will help, but you'll have to find out how much. If there are records that can be excluded because they are not needed (other than duplicates) then do it as early as possible.
What about any processing that follows the sort, all the way through? Anything dumb in programs using that file is going to be significantly multiplied by the sheer size of the file. You've probably got bigger savings, as a percentage, in the rest of the processing, the Sort is just a highlight because it is a heavy user of CPU.
I'm sure if you contact Syncsort they will be willing to assist you with tuning. If you search for Allissa's posts in this forum you'll find some e-mails you can enquire on. |
|
| Back to top |
|
|
sandip_mainframe
Warnings : 2
New User
Joined: 20 Sep 2006
Posts: 63
Location: pune
|
|
|
|
Hi All,
Thanks a lot for your help. I am taking help from Alissa on this as she is from Syncsort.
Bill,
Thanks for providing Alissa's contact info, I have dropped a mail to her on this issue.
Thanks,
Sandip Walsinge |
|
| Back to top |
|
|
Marso
REXX Moderator
Joined: 13 Mar 2006
Posts: 1353
Location: Israel
|
|
|
|
Oh my! I should never post a reply when I'm hurrying before going home!
EQUALS and SUM are not opposite, on the contrary!
Anyway you're in good hands now. Don't forget to tell us the outcome. |
|
| Back to top |
|
|
PeterHolland
Global Moderator
Joined: 27 Oct 2009
Posts: 2481
Location: Netherlands, Amstelveen
|
|
|
|
| Try MAXSORT |
|
| Back to top |
|
|
expat
Global Moderator
Joined: 14 Mar 2007
Posts: 8796
Location: Welsh Wales
|
|
|
|
Not sure if this would decrease the time taken, which incidentally has not been posted.
| Code: |
//SYSIN DD *
SORT FIELDS=(1,10,CH,A,11,12,CH,A,23,2,PD,D)
SUM FIELDS=NONE
/* |
Isn't this the same as
| Code: |
//SYSIN DD *
SORT FIELDS=(1,22,CH,A,23,2,PD,D)
SUM FIELDS=NONE
/* |
|
|
| Back to top |
|
|
enrico-sorichetti
Superior Member
Joined: 14 Mar 2007
Posts: 10888
Location: italy
|
|
|
|
| i remember asking and IIRC Frank replied that the impact was negligible |
|
| Back to top |
|
|
Anuj Dhawan
Superior Member
Joined: 22 Apr 2006
Posts: 6248
Location: Mumbai, India
|
|
|
|
| That won't make any CPU related difference. SORT products combine contiguous fields internally, when possible. |
|
| Back to top |
|
|
expat
Global Moderator
Joined: 14 Mar 2007
Posts: 8796
Location: Welsh Wales
|
|
|
|
OK, I'll just get my coat then  |
|
| Back to top |
|
|
gcicchet
Senior Member
Joined: 28 Jul 2006
Posts: 1702
Location: Australia
|
|
|
|
Hi,
Enrico said
Are we going to get any updates/figures on this ?
Gerry |
|
| Back to top |
|
|
|
|
