IBM Mainframe Forum Index
 
Log In
 
IBM Mainframe Forum Index Mainframe: Search IBM Mainframe Forum: FAQ Register
 

SYNCSORT step taking huge CPU time


IBM Mainframe Forums -> JCL & VSAM
Post new topic   Reply to topic
View previous topic :: View next topic  
Author Message
sandip_mainframe
Warnings : 2

New User


Joined: 20 Sep 2006
Posts: 63
Location: pune

PostPosted: Wed Feb 29, 2012 5:33 pm
Reply with quote

Hi All,

I have one SORT step in my job, which is taking huge CPU time for the sorting the file. Is there any alternate way to reduce the CPU consumtion time ?

Job step is like below -
Code:
//PS055  EXEC PGM=SYNCSORT,PARM='EQUALS'                     
//SORTIN   DD DSN=XX7384A.TEST.INPUT.FILE,     
//            DISP=SHR                                       
//SORTOUT  DD DSN=XX7384A.TEST.SORTED.OUT.FILE,       
//            DISP=(,CATLG,DELETE),                       
//            UNIT=SYSDA,SPACE=(CYL,(95,95),RLSE),           
//            DATACLAS=HUGE                                               
//SYSOUT   DD SYSOUT=*                                             
//SYSIN    DD *
  SORT FIELDS=(1,10,CH,A,11,12,CH,A,23,2,PD,D)
  SUM  FIELDS=NONE                           
/*

Daily we receive around 98,70,00,000 input records, is there any way to reduce CPU time so that it should reduce the job's run cost.

Please let me know if more information is required.

Thanks,
Sandip Walsinge
Pune.
Back to top
View user's profile Send private message
enrico-sorichetti

Superior Member


Joined: 14 Mar 2007
Posts: 10888
Location: italy

PostPosted: Wed Feb 29, 2012 5:37 pm
Reply with quote

define huge !
is it just generic discontent or You have evidence that similar sorts for the same record count use less cpu time ?

and please use a western punctuation style for numbers icon_cool.gif
would that be 987.000,000 ( 987 millions of records )

and since You are aware that You are using SYNCSORT why posy in the DFSORT/ICETOOL section ?

topic moved
Back to top
View user's profile Send private message
sandip_mainframe
Warnings : 2

New User


Joined: 20 Sep 2006
Posts: 63
Location: pune

PostPosted: Wed Feb 29, 2012 5:44 pm
Reply with quote

Hi Enrico,

I don't have any evidence, just want to is there any way to reduce the CPU time.

Sorry for the punctuation. It's 987 Millions of records.

Please suggest if there is any way.

Thanks for your reply.

Thanks
Sandip Walsinge
Back to top
View user's profile Send private message
gylbharat

Active Member


Joined: 31 Jul 2009
Posts: 565
Location: Bangalore

PostPosted: Wed Feb 29, 2012 6:12 pm
Reply with quote

Try using ELAP/CPU parameter

CPU Parm

CPU minimizes the CPU time of each sort at the expense of sort elapsed time
and I/O activity. See BALANCE, ELAP and IO. Note that these options and
CPU are all mutually exclusive.

Cannot be used with Tape Sort.

ELAP Parm

ELAP minimizes the elapsed time (wall clock time) of each sort at the
expense of CPU time nd I/O activity. See BALANCE, CPU and IO. Note that
these options and ELAP are all utually exclusive.

Cannot be used with Tape Sort.
Back to top
View user's profile Send private message
Bill Woodger

Moderator Emeritus


Joined: 09 Mar 2011
Posts: 7309
Location: Inside the Matrix

PostPosted: Wed Feb 29, 2012 7:03 pm
Reply with quote

You are specifying both EQUALS and SUM FIELDS=NONE. Drop at least one of them, preferrably both.

Check in your Syncsort manual, I'm sure they have information on sorting large files. I would try to get the whole key together, and such that it would be possible to sort it all ASCENDING (what is producing the 900,000,000+ records per day, do you have any influence over that?).

Is it simply reducing CPU for charging, or do you want better thoughput?
Back to top
View user's profile Send private message
Anuj Dhawan

Superior Member


Joined: 22 Apr 2006
Posts: 6248
Location: Mumbai, India

PostPosted: Wed Feb 29, 2012 7:26 pm
Reply with quote

Are you sure it's too much of CPU and not elapsed time? If you're sure, possibly read about VSCORE and VSCORET.
Back to top
View user's profile Send private message
Robert Sample

Global Moderator


Joined: 06 Jun 2008
Posts: 8700
Location: Dubuque, Iowa, USA

PostPosted: Wed Feb 29, 2012 8:23 pm
Reply with quote

Quote:
is there any way to reduce CPU time so that it should reduce the job's run cost.
Be aware that "run cost" is not standard across sites. Some sites use chargeback schemes to recover the mainframe cost while other sites do not. If a site does not use a chargeback scheme, then any given job will use resources (CPU time, I/O, elapsed time) but the "cost" of the job in accounting terms would be ZERO and that could not be reduced.

Furhtermore, chargeback schemes can be simple or very complex -- again, depending upon the site. If your site's chargeback scheme places a relatively high value on CPU time, then you may not be able to do much to reduce the "cost" since it's going to take a certain amount of CPU time to process that many records -- and the sort products tend to be pretty optimized already, so there's not much short of completely redesigning your processing that will cut the CPU usage.
Back to top
View user's profile Send private message
Marso

REXX Moderator


Joined: 13 Mar 2006
Posts: 1353
Location: Israel

PostPosted: Wed Feb 29, 2012 10:22 pm
Reply with quote

As Bill said, EQUALS and SUM FIELDS have opposite meaning.
1. loose EQUALS first.
Syncsort 1.3 Programmers Guide wrote:
The EQUALS PARM instructs the sort/merge to preserve the order of equal-keyed records.
EQUALS will have a slight but generally significant impact on sort performance. By making
EQUALS available on an individual sort basis, SyncSort makes this programming
option available where it is needed, without imposing it on the installation’s more routine
jobs. For sort efficiency, use EQUALS only where the preservation of the input order of
equal-keyed records is important.

2. check if you really have duplicated records. If no loose the SUM card.
3. Specifying large enough SORTWK areas on different volumes may help too.

4. You really should open the Guide and check the available options...

Syncsort 1.3 Programmers Guide wrote:
A sort is considered to finely tuned when WER124I reports an overallocation factor between 1.00 and 1.50.
Back to top
View user's profile Send private message
Bill Woodger

Moderator Emeritus


Joined: 09 Mar 2011
Posts: 7309
Location: Inside the Matrix

PostPosted: Wed Feb 29, 2012 10:55 pm
Reply with quote

Quote:
EQUALS will have a slight but generally significant impact on sort performance


I like that. If the weather forecast was for "slight but generally significant falls of snow" you'd not know whether to take an umbrella or a shovel to work with you.

Taking the EQUALS off, I'm sure you'll notice the difference.

If you really have duplicates, and you really have to have the physically latest record as the representative for that key, then there is a bit of a quandry. If there is something on the record which indicates the latest one, then post-process in whichever program is reading the file afterwards. If you have nothing, look at the viability of getting something, compared to the CPU-cost saving of remval.

You specify two seperate fields in the sort card which are contiguous. Specify them as one field. You have a PD, dislocated from the rest of the key, which is sorted descending. Look at the possibility of the producer making it negative, or "subtracted" from 999 and try to get it moved adjacent to the rest of the key, so it can all be specified in one statement.

You can prepare the records like that in an extract from your file and do some comparisons, comparing CPU usage. If the recored can be made smaller, that will help, but you'll have to find out how much. If there are records that can be excluded because they are not needed (other than duplicates) then do it as early as possible.

What about any processing that follows the sort, all the way through? Anything dumb in programs using that file is going to be significantly multiplied by the sheer size of the file. You've probably got bigger savings, as a percentage, in the rest of the processing, the Sort is just a highlight because it is a heavy user of CPU.

I'm sure if you contact Syncsort they will be willing to assist you with tuning. If you search for Allissa's posts in this forum you'll find some e-mails you can enquire on.
Back to top
View user's profile Send private message
sandip_mainframe
Warnings : 2

New User


Joined: 20 Sep 2006
Posts: 63
Location: pune

PostPosted: Thu Mar 01, 2012 11:11 am
Reply with quote

Hi All,

Thanks a lot for your help. I am taking help from Alissa on this as she is from Syncsort.

Bill,

Thanks for providing Alissa's contact info, I have dropped a mail to her on this issue.

Thanks,
Sandip Walsinge
Back to top
View user's profile Send private message
Marso

REXX Moderator


Joined: 13 Mar 2006
Posts: 1353
Location: Israel

PostPosted: Thu Mar 01, 2012 1:27 pm
Reply with quote

Oh my! I should never post a reply when I'm hurrying before going home!
EQUALS and SUM are not opposite, on the contrary!
Anyway you're in good hands now. Don't forget to tell us the outcome.
Back to top
View user's profile Send private message
PeterHolland

Global Moderator


Joined: 27 Oct 2009
Posts: 2481
Location: Netherlands, Amstelveen

PostPosted: Thu Mar 01, 2012 1:54 pm
Reply with quote

Try MAXSORT
Back to top
View user's profile Send private message
expat

Global Moderator


Joined: 14 Mar 2007
Posts: 8796
Location: Welsh Wales

PostPosted: Thu Mar 01, 2012 2:44 pm
Reply with quote

Not sure if this would decrease the time taken, which incidentally has not been posted.
Code:
//SYSIN    DD *
  SORT FIELDS=(1,10,CH,A,11,12,CH,A,23,2,PD,D)
  SUM  FIELDS=NONE                           
/*


Isn't this the same as
Code:
//SYSIN    DD *
  SORT FIELDS=(1,22,CH,A,23,2,PD,D)
  SUM  FIELDS=NONE                           
/*
Back to top
View user's profile Send private message
enrico-sorichetti

Superior Member


Joined: 14 Mar 2007
Posts: 10888
Location: italy

PostPosted: Thu Mar 01, 2012 3:00 pm
Reply with quote

i remember asking and IIRC Frank replied that the impact was negligible
Back to top
View user's profile Send private message
Anuj Dhawan

Superior Member


Joined: 22 Apr 2006
Posts: 6248
Location: Mumbai, India

PostPosted: Thu Mar 01, 2012 3:20 pm
Reply with quote

That won't make any CPU related difference. SORT products combine contiguous fields internally, when possible.
Back to top
View user's profile Send private message
expat

Global Moderator


Joined: 14 Mar 2007
Posts: 8796
Location: Welsh Wales

PostPosted: Thu Mar 01, 2012 3:21 pm
Reply with quote

OK, I'll just get my coat then icon_lol.gif
Back to top
View user's profile Send private message
gcicchet

Senior Member


Joined: 28 Jul 2006
Posts: 1702
Location: Australia

PostPosted: Fri Mar 02, 2012 3:04 am
Reply with quote

Hi,

Enrico said
Quote:
define huge !


Are we going to get any updates/figures on this ?


Gerry
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic View Bookmarks
All times are GMT + 6 Hours
Forum Index -> JCL & VSAM

 


Similar Topics
Topic Forum Replies
No new posts Syncsort help SYNCSORT 3
No new posts SYNCSORT-DB2 interface SYNCSORT 0
No new posts Compare only first records of the fil... SYNCSORT 7
No new posts To get the the current time DFSORT/ICETOOL 13
No new posts RC query -Time column CA Products 3
Search our Forums:

Back to Top