Portal | Manuals | References | Downloads | Info | Programs | JCLs | Master the Mainframes
IBM Mainframe Computers Forums Index
 
Register
 
IBM Mainframe Computers Forums Index Mainframe: Search IBM Mainframe Forum: FAQ Memberlist Usergroups Profile Log in to check your private messages Log in
 

 

DFSORT taking too long time to sort

 
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> DFSORT/ICETOOL
View previous topic :: :: View next topic  
Author Message
Singaram

New User


Joined: 19 Aug 2009
Posts: 19
Location: India

PostPosted: Tue Aug 25, 2009 10:51 am    Post subject: DFSORT taking too long time to sort
Reply with quote

The following sort is taking too long time:

1. Should I change the size to MAX?
2. Should I consider increasing BUFNO for the sequential datasets
3. Should I merge all these files before issuing the sort step.

Any advice on performance will be greatly appreciated


XXSORT0005 EXEC PGM=SORT,COND=(8,LT),
XX PARM='SIZE=60M'
39 XXSYSOUT DD SYSOUT=*
40 XXSORTWK01 DD UNIT=SRTDA,SPACE=(CYL,(3000,22))
41 XXSORTWK02 DD UNIT=SRTDA,SPACE=(CYL,(3000,22))
42 XXSORTWK03 DD UNIT=SRTDA,SPACE=(CYL,(3000,22))
43 XXSORTWK04 DD UNIT=SRTDA,SPACE=(CYL,(3000,22))
44 XXSORTWK05 DD UNIT=SRTDA,SPACE=(CYL,(3000,22))
45 XXSORTWK06 DD UNIT=SRTDA,SPACE=(CYL,(3000,22))
46 XXSORTWK07 DD UNIT=SRTDA,SPACE=(CYL,(3000,22))
47 XXSORTWK08 DD UNIT=SRTDA,SPACE=(CYL,(3000,22))
48 XXSORTIN DD DSN=AMJOS.JDFDS.TRANS.EDOSCTA.DEPURADO,
XX DISP=OLD,DCB=BUFNO=12
49 XX DD DSN=HLQ.EDOSCTA.BATCH,
XX DISP=OLD,DCB=BUFNO=2
50 XX DD DSN=HLQ.MOVS.EDOCTA,
XX DISP=OLD,DCB=BUFNO=2
51 XX DD DSN=HLQ.ESTADO.CUENTA,
XX DISP=OLD,DCB=BUFNO=2
52 XX DD DSN=HLQ.EDOCTA.NPOS,
XX DISP=OLD,DCB=BUFNO=2
53 XX DD DSN=HLQ.CARGOS.EDOSCTA,
XX DISP=OLD,DCB=BUFNO=2
54 XX DD DSN=HLQ.ABONOS.EDOSCTA,
XX DISP=OLD,DCB=BUFNO=2
55 XX DD DSN=HLQ.LEYENDA.PARACHQ,
XX DISP=OLD,DCB=BUFNO=2
56 XX DD DSN=HLQ.CMN615K.LEYENDAS,
XX DISP=OLD,DCB=BUFNO=2
57 XX DD DSN=HLQ.CMN615K.LEYENFH,
XX DISP=OLD,DCB=BUFNO=2
58 XX DD DSN=HLQ.LEYENDAS.JLS080,
XX DISP=OLD,DCB=BUFNO=2
59 XX DD DSN=HLQ.LEYEN.MULCASH,
XX DISP=OLD,DCB=BUFNO=5
60 XX DD DSN=HLQ.LINEA.OPERA.TIVA,
XX DISP=OLD,DCB=BUFNO=2
61 XX DD DSN=HLQ.LEYEN.MULCASH,
XX DISP=OLD,DCB=BUFNO=2
62 XX DD DSN=HLQ.LEYENDAS.ORDENES,
XX DISP=OLD,DCB=BUFNO=2
63 XX DD DSN=HLQ.EDOCTA,
XX DISP=OLD,DCB=BUFNO=5
64 XX DD DSN=HLQ.LEYENDAS.REMANEN,
XX DISP=OLD,DCB=BUFNO=2
65 XX DD DSN=HLQ.CHEQUES,
XX DISP=OLD,DCB=BUFNO=2
66 XX DD DSN=HLQ.CARCHEQ,
XX DISP=OLD,DCB=BUFNO=2
67 XX DD DSN=HLQ.DEBITO.EDOCTA,
XX DISP=OLD,DCB=BUFNO=2
68 XX DD DSN=HLQ.LEYEN.LIDE,
XX DISP=OLD,DCB=BUFNO=2
69 XXSORTOUT DD DSN=AMJOS.JDFDS.ACUM.TRANS.EDOSCTA,
XX UNIT=PRMDA,
XX DISP=(,CATLG,DELETE),
XX SPACE=(CYL,(2500,250),RLSE),
XX DCB=(MDLDCB.W150S.G00DX,BUFNO=30)
70 XXSYSIN DD *
SORT FIELDS=(1,25,A,32,23,A),FORMAT=BI

------------------------------------------------------------------------------------

ICE143I 0 BLOCKSET SORT TECHNIQUE SELECTED
ICE000I 1 - CONTROL STATEMENTS FOR 5740-SM1, DFSORT REL 14.0 - 23:32 ON FRI JUL 31, 2009 -
SORT FIELDS=(1,25,A,32,23,A),FORMAT=BI
ICE201I 0 RECORD TYPE IS F - DATA STARTS IN POSITION 1
ICE193I 0 ICEAM1 ENVIRONMENT IN EFFECT - ICEAM1 INSTALLATION MODULE SELECTED
ICE088I 2 PMXJDFDN.PMXJDFDN.SORT0005, INPUT LRECL = 150, BLKSIZE = 27900, TYPE = FB
ICE092I 0 MAIN STORAGE = (62914560,62914560,62914560)
ICE156I 0 MAIN STORAGE ABOVE 16MB = (62523040,62523040)
ICE127I 0 OPTIONS: OVFLO=RC0 ,PAD=RC0 ,TRUNC=RC0 ,SPANINC=RC16,VLSCMP=N,SZERO=Y,RESET=Y,VSAMEMT=Y,DYNSPC=256
ICE128I 0 OPTIONS: SIZE=62914560,MAXLIM=1048576,MINLIM=450560,EQUALS=Y,LIST=Y,ERET=ABEND,MSGDDN=SYSOUT
ICE129I 0 OPTIONS: VIO=N,RESDNT=ALL ,SMF=FULL ,WRKSEC=Y,OUTSEC=Y,VERIFY=N,CHALT=N,DYNALOC=N ,ABCODE=099
ICE130I 0 OPTIONS: RESALL=0,RESINV=0,SVC=109 ,CHECK=Y,WRKREL=Y,OUTREL=Y,CKPT=N,STIMER=Y,COBEXIT=COB1
ICE131I 0 OPTIONS: TMAXLIM=4194304,ARESALL=0,ARESINV=0,OVERRGN=65536,CINV=Y,CFW=Y,DSA=0
ICE132I 0 OPTIONS: VLSHRT=N,ZDPRINT=N,IEXIT=N,TEXIT=N,LISTX=N,EFS=NONE ,EXITCK=S,PARMDDN=DFSPARM ,FSZEST=N
ICE133I 0 OPTIONS: HIPRMAX=OPTIMAL,DSPSIZE=MAX ,ODMAXBF=0,SOLRF=Y,VLLONG=N,VSAMIO=N
ICE084I 0 EXCP ACCESS METHOD USED FOR SORTOUT
ICE084I 0 EXCP ACCESS METHOD USED FOR SORTIN
ICE750I 0 DC 11353179600 TC 0 CS DSVVV KSZ 52 VSZ 52
ICE752I 0 FSZ=75687864 RC IGN=0 E AVG=156 0 WSP=15335663 C DYN=0 0
ICE090I 0 OUTPUT LRECL = 150, BLKSIZE = 27900, TYPE = FB
ICE055I 0 INSERT 0, DELETE 0
ICE054I 0 RECORDS - IN: 75686433, OUT: 75686433
ICE134I 0 NUMBER OF BYTES SORTED: 11352964950
ICE165I 0 TOTAL WORK DATA SET TRACKS ALLOCATED: 360000 , TRACKS USED: 210825
ICE180I 0 HIPERSPACE STORAGE USED = 953448K BYTES
ICE188I 0 DATA SPACE STORAGE USED = 0K BYTES
ICE751I 0 C5C6C7C8E4C9E5E7CFCED3EDE8
ICE052I 0 END OF DFSORT
Back to top
View user's profile Send private message

Escapa

Senior Member


Joined: 16 Feb 2007
Posts: 1399
Location: IL, USA

PostPosted: Tue Aug 25, 2009 11:03 am    Post subject:
Reply with quote

Quote:
The following sort is taking too long time:

How much CPU time it is taking?

And afterall it is sorting records whose number is not less.. . icon_smile.gif
Quote:
ICE054I 0 RECORDS - IN: 75686433, OUT: 75686433
Back to top
View user's profile Send private message
Singaram

New User


Joined: 19 Aug 2009
Posts: 19
Location: India

PostPosted: Tue Aug 25, 2009 11:16 am    Post subject:
Reply with quote

Its taking around 1 hour elapsed..but Im trying to see is merge and then sort going to improve
Back to top
View user's profile Send private message
Escapa

Senior Member


Joined: 16 Feb 2007
Posts: 1399
Location: IL, USA

PostPosted: Tue Aug 25, 2009 12:13 pm    Post subject:
Reply with quote

Quote:
Its taking around 1 hour elapsed


I asked CPU time...
Back to top
View user's profile Send private message
Singaram

New User


Joined: 19 Aug 2009
Posts: 19
Location: India

PostPosted: Tue Aug 25, 2009 6:56 pm    Post subject:
Reply with quote

CPU consumed is around 6 minutes
Back to top
View user's profile Send private message
dick scherrer

Site Director


Joined: 23 Nov 2006
Posts: 19270
Location: Inside the Matrix

PostPosted: Tue Aug 25, 2009 8:43 pm    Post subject:
Reply with quote

Hello,

Are some or all of the concatenated input files already in sequence? If so, i suspect merging the files would be much faster than sorting. . .

Have the blksizes of the inputs/output been checked?
Back to top
View user's profile Send private message
Singaram

New User


Joined: 19 Aug 2009
Posts: 19
Location: India

PostPosted: Tue Aug 25, 2009 9:01 pm    Post subject:
Reply with quote

Thanks dick scherrer for the reply

AMJOS.JDFDS.TRANS.EDOSCTA.DEPURADO this file alone is in sequence and it is having record count of say around 50million.

And regarding the blocksize can you please let me know what exactly you want to be checked.
Back to top
View user's profile Send private message
dick scherrer

Site Director


Joined: 23 Nov 2006
Posts: 19270
Location: Inside the Matrix

PostPosted: Tue Aug 25, 2009 9:11 pm    Post subject:
Reply with quote

Hello,

As 50 million is 2/3s of the volume, sorting the others and then merging with these should be faster.

Talk with your storage management people to make sure the blksizes are efficient for the media being used.
Back to top
View user's profile Send private message
Terry Heinze

JCL Moderator


Joined: 14 Jul 2008
Posts: 1249
Location: Richfield, MN, USA

PostPosted: Tue Aug 25, 2009 9:22 pm    Post subject:
Reply with quote

No need to modify BUFNO -- DFSORT ignores yours and uses its own anyway. SORTWRK might be unnecessary too. Frank or Kolusu can either confirm or correct that assumption. Your 27900 BLKSIZE is as good as it's going to get. 1 hour seems reasonable considering you're sorting 75M records and using 6 CPU minutes. The DFSORT developers might have some tips for you.
Back to top
View user's profile Send private message
Dave Betten

New User


Joined: 24 Jan 2006
Posts: 26

PostPosted: Tue Aug 25, 2009 10:25 pm    Post subject: Reply to: DFSORT taking too long time to sort
Reply with quote

First I'd like to point out that you are running DFSORT Release 14 which is no longer in support. So you may want to determine if you have a more current release like V1R5 or V1R10 available.

That said, you are sorting 75686433 records which add up to about 11GB and you used about 931MB of Hipespace. DFSORT does allocate its own buffers and used EXCP to read the input files.

Nothing jumps out at me from the sysout. If you have a system with more storage resources available, you could exploit more Hiperspace and reduce i/o to the work data sets. If you add //SORTDIAG DD DUMMY to the jcl, DFSORT will write out additional diagnostic messages. If you'd like to do that and send me the sysout, I'd be happy to see if I can find any additional tuning opportunity for this job. My email is betten@us.ibm.com
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> DFSORT/ICETOOL All times are GMT + 6 Hours
Page 1 of 1

 

Search our Forum:

Similar Topics
Topic Author Forum Replies Posted
No new posts How to change 'K' or 'M' use Sort vice_versa DFSORT/ICETOOL 5 Thu May 18, 2017 7:11 am
No new posts Adding big TEXT lines to each record ... bshkris SYNCSORT 4 Sat May 06, 2017 1:40 am
No new posts DSNACCOX (can it be run on 1 db/ts, t... SRICOBSAS DB2 3 Sat May 06, 2017 12:59 am
This topic is locked: you cannot edit posts or make replies. SORT trick needed bshkris SYNCSORT 6 Tue May 02, 2017 4:35 am
No new posts LISTIDR compiled date/time jerryte IBM Tools 3 Thu Apr 20, 2017 7:37 pm


Facebook
Back to Top
 
Mainframe Wiki | Forum Rules | Bookmarks | Subscriptions | FAQ | Tutorials | Contact Us