|
View previous topic :: View next topic
|
| Author |
Message |
JAYESHBALAN
New User
Joined: 07 Jun 2012
Posts: 4
Location: india
|
|
|
|
| Im using 4 tape inputs with a total of some 2.5 billion records in SORTIN. the step is using a simple inrec statement. It takes some 50 mins to run this step alone. Could someone help me reduce the runtime of this step!!! |
|
| Back to top |
|
 |
vasanthz
Global Moderator
Joined: 28 Aug 2007
Posts: 1742
Location: Tirupur, India
|
|
|
|
Hi and welcome to the forum,
Show us the JCL step and with the SORT cards that you are using. The more relevant information you provide, the more relevant suggestions you would get. |
|
| Back to top |
|
|
JAYESHBALAN
New User
Joined: 07 Jun 2012
Posts: 4
Location: india
|
|
|
|
Jcl:
| Code: |
//SCPS720F EXEC PGM=SORT
//*
//SYSIN DD DSN=DEVSVS1.CTRL.CARDS(SCPC720B),
// DISP=SHR
//*
//SORTIN DD DSN=SCPX.SCPF720C.ITM.ASR.LCT.LVL1.DLT,
// DISP=SHR
// DD DSN=SCPX.SCPF720E.ITM.ASR.LCT.LVL2.DLT,
// DISP=SHR
// DD DSN=SCPX.SCPF720B.ITM.ASR.LCT.LVL1.IRT.UPD,
// DISP=SHR
// DD DSN=SCPX.SCPF720D.ITM.ASR.LCT.LVL2.IRT.UPD,
// DISP=SHR
//*
//SORTOUT DD DSN=SCPX.SCPF720F.JOBID.CHANGE.FORJDA(+1),
// DISP=(NEW,CATLG,DELETE),
// UNIT=TAPEV,EXPDT=99000,LABEL=(1,SL)
//*
//SYSOUT DD SYSOUT=*
//SYSPRINT DD SYSOUT=*
|
Control card:
| Code: |
INREC FIELDS=(1:1,1, * INDICATOR
2:C',',
3:2,4,BI,EDIT=(TTTTTTTTT), * ASSORTMENT NUMBER
12:C',',
13:6,2,BI,EDIT=(TTTTT), * LOCATION NUMBER
18:C',',
19:8,2,BI,EDIT=(TTTTT), * ORGANIZATION ID
24:C',',
25:10,2,BI,EDIT=(TTTTT), * JOB POS CODE
30:C',',
31:12,4,BI,EDIT=(TTTTTTTTT), * PERSONNEL NUMBER
40:C',',
41:16,4,BI,EDIT=(TTTTTTTTT)) * ITEM NUMBER
SORT FIELDS=(3,9,CH,A, * ASSORTMENT NUMBER
13,5,CH,A, * LOCATION NUMBER
19,5,CH,A, * ORGANIZATION ID
25,5,CH,A, * JOB POS CODE
31,9,CH,A, * PERSONNEL NUMBER
41,9,CH,A) * ITEM NUMBER
SUM FIELDS=NONE |
|
|
| Back to top |
|
|
dbzTHEdinosauer
Global Moderator
Joined: 20 Oct 2006
Posts: 6966
Location: porcelain throne
|
|
|
|
well you have 6 binary to decimal with and edit coversions,
and you have idiotically told SORT to create 6 separate keys,
where 1 key would suffice.
you will need to provide sysout, jesmsg, and other sortmsgs for anyone to give you advice.
actually, you should bundle it all up and email it to dfsort,
so that they can tell you what you are doing
and what you could do to have it improve, if anything. |
|
| Back to top |
|
|
JAYESHBALAN
New User
Joined: 07 Jun 2012
Posts: 4
Location: india
|
|
|
|
| I have attached the sysout of the entire job. SCPS720F is my step. |
|
| Back to top |
|
|
Bill Woodger
Moderator Emeritus
Joined: 09 Mar 2011
Posts: 7309
Location: Inside the Matrix
|
|
|
|
I would think that smaller records would sort faster.
If you do it on OUTREC instead of INREC you won't be sorting squintillions of commas and, as dbz has mentioned, you could have one key (though it probably doesn't make a difference, apparently).
Sort on the whole record, then your SUM, then it'll do the OUTREC processing where you can do your rearranging.
What order are your input datasets? If they happen to be in the order you want (each of them) then you could use MERGE instead of sort (four seperate input DDs). Is the SORT also so that you can do the SUM?
Tell us the full story, or follow dbz's advice. |
|
| Back to top |
|
|
Bill Woodger
Moderator Emeritus
Joined: 09 Mar 2011
Posts: 7309
Location: Inside the Matrix
|
|
|
|
I've looked at your output.
You have Syncsort, not DFSORT, so have posted in the wrong forum (should have been JCL) and wasted people's time. |
|
| Back to top |
|
|
dick scherrer
Moderator Emeritus
Joined: 23 Nov 2006
Posts: 19244
Location: Inside the Matrix
|
|
|
|
Hello,
How much of the elapsed time is due to tape loads and unloads? Depending on how tapes are handled on your system (manual versus robot versus virtual tape), considerable wait time may happen. |
|
| Back to top |
|
|
Robert Sample
Global Moderator
Joined: 06 Jun 2008
Posts: 8696
Location: Dubuque, Iowa, USA
|
|
|
|
| Have you tried adding BUFNO to each tape DD statement? Having 30 to 50 buffers per data set will help performance, but may require additional region for the increased buffer space. |
|
| Back to top |
|
|
Bill Woodger
Moderator Emeritus
Joined: 09 Mar 2011
Posts: 7309
Location: Inside the Matrix
|
|
|
|
Your input is 19 bytes, after your inrec it is 49 bytes. Which means you have 30 bytes extra which can be moved to OUTREC.
You have SUM with FIELDS=NONE, yet out of the 2.5billion records, not a single one was duplicate, suggesting that you may not need it (check the file specification/documentation).
If you can change those, it should be a bit zippier.
You seem to have a couple of earlier JOINKEYS which get no matches. There has to be a "design" way to avoid reading 150m records to not find a match. |
|
| Back to top |
|
|
dick scherrer
Moderator Emeritus
Joined: 23 Nov 2006
Posts: 19244
Location: Inside the Matrix
|
|
|
|
Hello,
Suggest you run a test that sorts the individual files with no editing.
Once the files are sorted, MERGE (all in one step) them into the new output file and edit the results in the output file. |
|
| Back to top |
|
|
gcicchet
Senior Member
Joined: 28 Jul 2006
Posts: 1702
Location: Australia
|
|
|
|
Hi,
I'm not able to see/download your attachment.
Why don't we ban attachments.
Gerry |
|
| Back to top |
|
|
dick scherrer
Moderator Emeritus
Joined: 23 Nov 2006
Posts: 19244
Location: Inside the Matrix
|
|
|
|
Hi Gerry,
If you right-click on the attachment and Save As to a .txt file it should properly align.
d |
|
| Back to top |
|
|
Nic Clouston
Global Moderator
Joined: 10 May 2007
Posts: 2455
Location: Hampshire, UK
|
|
|
|
| I just ignore attachments. If domrone wants help they should help the helper - adding extra steps (downloading) is hindering not helping. I am sure attachments could be disabled (but maybe not) |
|
| Back to top |
|
|
enrico-sorichetti
Superior Member
Joined: 14 Mar 2007
Posts: 10873
Location: italy
|
|
|
|
and even if somebody can download/look at the attachments expecting people to wander thru 2000 lines of info is a bit too much  |
|
| Back to top |
|
|
gcicchet
Senior Member
Joined: 28 Jul 2006
Posts: 1702
Location: Australia
|
|
|
|
Hi Dick,
thanks for the suggestion but I don't get the option to Save As.
Gerry |
|
| Back to top |
|
|
Nic Clouston
Global Moderator
Joined: 10 May 2007
Posts: 2455
Location: Hampshire, UK
|
|
|
|
| I do not know when 'someone' became 'domrone'! must have been typing in the darkness of my bedroom - especially at 5 in the morning. |
|
| Back to top |
|
|
Nic Clouston
Global Moderator
Joined: 10 May 2007
Posts: 2455
Location: Hampshire, UK
|
|
|
|
| right click gives 'save image as' which, I assume, is offering to save the icon and not the document. Maybe in the olden days.... |
|
| Back to top |
|
|
Akatsukami
Global Moderator
Joined: 03 Oct 2009
Posts: 1788
Location: Bloomington, IL
|
|
|
|
| Nic Clouston wrote: |
| I do not know when 'someone' became 'domrone'! must have been typing in the darkness of my bedroom - especially at 5 in the morning. |
Perhaps an portmanteau of "don padrone"?  |
|
| Back to top |
|
|
dick scherrer
Moderator Emeritus
Joined: 23 Nov 2006
Posts: 19244
Location: Inside the Matrix
|
|
|
|
Hello,
Nope, NOT save image as. . .
| Quote: |
| If you right-click on the attachment and Save As to a .txt file it should properly align. |
And i fat-fingered here. . . 
It should be "Save Target As". . .
Apologies. . .
d |
|
| Back to top |
|
|
Nic Clouston
Global Moderator
Joined: 10 May 2007
Posts: 2455
Location: Hampshire, UK
|
|
|
|
| No 'save target as' on my system (firefox) but left click and it gives the option to run or save. |
|
| Back to top |
|
|
|
|
