IBM Mainframe Forum Index
 
Log In
 
IBM Mainframe Forum Index Mainframe: Search IBM Mainframe Forum: FAQ Register
 

How SORT choose the type of storage(memory or work dataset)?


IBM Mainframe Forums -> DFSORT/ICETOOL
Post new topic   Reply to topic
View previous topic :: View next topic  
Author Message
pch1982cn

New User


Joined: 02 Apr 2006
Posts: 10
Location: Dalian.China

PostPosted: Sun Apr 02, 2006 6:36 pm
Reply with quote

Hello,

I have one question:

I'm using the same JCL to sort a large file, the JCL code is as below:
Code:

//STEP020 EXEC PGM=SORT,COND=(0,NE)                                 
//SYSIN    DD DSN=CMAIN.ULIB(SRTCD),DISP=SHR                         
//SYSOUT   DD SYSOUT=*                                             
//SORTIN DD DSN=TIN.FILEIN.P001,DISP=SHR       
//SORTOUT  DD DSNTIN.FILEIN.P001.SRT,           
//            DISP=(NEW,CATLG,DELETE),                             
//            DCB=(LRECL=2614,BLKSIZE=0,RECFM=FB),                 
//            DATACLAS=COMPACT,                                   
//            UNIT=(3390,59),SPACE=(CYL,(1000,1000),RLSE)


The sort card - CMAIN.ULIB(SRTCD) is as below:
Code:

 OPTION DYNALLOC=(SYSDA,16)   
  SORT FIELDS=(1318,03,CH,A) 


This is just a simple SORT step.

But I ran it in two environments ...

In one environment, SORT choose work datasets as its sort work files:

from the Joblog I could see

ICE054I 0 RECORDS - IN: 1568500, OUT: 1568500
ICE134I 0 NUMBER OF BYTES SORTED: 4100059000
ICE165I 0 TOTAL WORK DATA SET TRACKS ALLOCATED: 99225 , TRACKS USED: 75000
ICE180I 0 HIPERSPACE STORAGE USED = 0K BYTES
ICE188I 0 DATA SPACE STORAGE USED = 201884K BYTES

but in another environment, SORT choose memory to store its sort work files

the job log is

ICE054I 0 RECORDS - IN: 3361000, OUT: 3361000
ICE134I 0 NUMBER OF BYTES SORTED: 8785654000
ICE165I 0 TOTAL WORK DATA SET TRACKS ALLOCATED: 0 , TRACKS USED: 0
ICE199I 0 MEMORY OBJECT STORAGE USED = 8424M BYTES

For their job logs, the option are both same:

ICE127I 0 OPTIONS: OVFLO=RC0 ,PAD=RC0 ,TRUNC=RC0 ,SPANINC=RC16,VLSCMP=N,SZERO=Y,RESET=Y,VSAMEMT=Y,DYNSPC=256
ICE128I 0 OPTIONS: SIZE=35740326,MAXLIM=2097152,MINLIM=450560,EQUALS=Y,LIST=Y,ERET=RC16 ,MSGDDN=SYSOUT
ICE129I 0 OPTIONS: VIO=N,RESDNT=ALL ,SMF=NO ,WRKSEC=Y,OUTSEC=Y,VERIFY=N,CHALT=N,DYNALOC=(SYSDA ,016),ABCODE=MSG
ICE130I 0 OPTIONS: RESALL=8192,RESINV=0,SVC=109 ,CHECK=Y,WRKREL=Y,OUTREL=Y,CKPT=N,STIMER=Y,COBEXIT=COB2
ICE131I 0 OPTIONS: TMAXLIM=6291456,ARESALL=0,ARESINV=0,OVERRGN=65536,CINV=Y,CFW=Y,DSA=64
ICE132I 0 OPTIONS: VLSHRT=N,ZDPRINT=Y,IEXIT=N,TEXIT=N,LISTX=N,EFS=NONE ,EXITCK=S,PARMDDN=DFSPARM ,FSZEST=N
ICE133I 0 OPTIONS: HIPRMAX=OPTIMAL,DSPSIZE=MAX ,ODMAXBF=0,SOLRF=Y,VLLONG=N,VSAMIO=N,MOSIZE=MAX
ICE235I 0 OPTIONS: NULLOUT=RC0

Although the input files are not same, they are similar ... I cannot understand why SORT was choosing different type of storage ... can any factor impact it?

Please anybody can help me?
Back to top
View user's profile Send private message
Frank Yaeger

DFSORT Developer


Joined: 15 Feb 2005
Posts: 7129
Location: San Jose, CA

PostPosted: Sun Apr 02, 2006 9:36 pm
Reply with quote

DFSORT chooses dynamically among memory object sorting, hipersorting, dataspace sorting and work data sets based on many factors including what's happening in the system at the time a particular sort application is run. DFSORT may choose different options for the same job on different systems based on the available resources, or even on the same system at different times based on the current workload.

Note that in your first case, DFSORT chose a combination of dataspace sorting and work data sets, whereas in the second case DFSORT chose memory object sorting. Memory object sorting is only available with z/OS DFSORT V1R5 and above, not with DFSORT R14. You didn't show all of the messages, so I can't tell which release of DFSORT you were using for each run.
Back to top
View user's profile Send private message
pch1982cn

New User


Joined: 02 Apr 2006
Posts: 10
Location: Dalian.China

PostPosted: Mon Apr 03, 2006 8:11 am
Reply with quote

Thanks Frank!

Frank Yaeger wrote:
so I can't tell which release of DFSORT you were using for each run.


Yes, I was using V1R5.

In one environment, the SORT chose work data sets - see the ICE165I messages

1ICE143I 0 BLOCKSET SORT TECHNIQUE SELECTED
ICE250I 0 VISIT www.ibm.com/storage/dfsort FOR DFSORT PAPERS, EXAMPLES AND MORE
ICE000I 1 - CONTROL STATEMENTS FOR 5694-A01, Z/OS DFSORT V1R5 - 20:29 ON THU MAR 09, 2006 -
0 ***********************************************************************
* *
* SORT CONTROL CARD *
* *
***********************************************************************
SORT FIELDS=(1318,03,CH,A) * BY COUNTRY
OPTION DYNALLOC=(,255)
ICE201I 0 RECORD TYPE IS F - DATA STARTS IN POSITION 1
ICE751I 0 C5-BASE C6-BASE C7-BASE C8-Q83041 E4-BASE C9-BASE E5-Q95174 E6-BASE C4-Q95174 E7-Q95174
ICE193I 0 ICEAM1 ENVIRONMENT IN EFFECT - ICEAM1 INSTALLATION MODULE SELECTED
ICE088I 1 FMSFVR0 .FMSFXCL .STEP020 , INPUT LRECL = 2614, BLKSIZE = 31368, TYPE = FB
ICE093I 0 MAIN STORAGE = (MAX,13660448,13660448)
ICE156I 0 MAIN STORAGE ABOVE 16MB = (11563296,11563296)
ICE127I 0 OPTIONS: OVFLO=RC0 ,PAD=RC0 ,TRUNC=RC0 ,SPANINC=RC16,VLSCMP=N,SZERO=Y,RESET=Y,VSAMEMT=Y,DYNSPC=256
ICE128I 0 OPTIONS: SIZE=13660448,MAXLIM=2097152,MINLIM=450560,EQUALS=Y,LIST=Y,ERET=RC16 ,MSGDDN=SYSOUT
ICE129I 0 OPTIONS: VIO=N,RESDNT=ALL ,SMF=NO ,WRKSEC=Y,OUTSEC=Y,VERIFY=N,CHALT=N,DYNALOC=(SYSDA ,255),ABCODE=MSG
ICE130I 0 OPTIONS: RESALL=8192,RESINV=0,SVC=109 ,CHECK=Y,WRKREL=Y,OUTREL=Y,CKPT=N,STIMER=Y,COBEXIT=COB2
ICE131I 0 OPTIONS: TMAXLIM=6291456,ARESALL=0,ARESINV=0,OVERRGN=65536,CINV=Y,CFW=Y,DSA=64
ICE132I 0 OPTIONS: VLSHRT=N,ZDPRINT=Y,IEXIT=N,TEXIT=N,LISTX=N,EFS=NONE ,EXITCK=S,PARMDDN=DFSPARM ,FSZEST=N
ICE133I 0 OPTIONS: HIPRMAX=OPTIMAL,DSPSIZE=MAX ,ODMAXBF=0,SOLRF=Y,VLLONG=N,VSAMIO=N,MOSIZE=MAX
ICE235I 0 OPTIONS: NULLOUT=RC0
ICE084I 0 BSAM ACCESS METHOD USED FOR SORTOUT
ICE084I 0 BSAM ACCESS METHOD USED FOR SORTIN
ICE750I 0 DC 1283474000 TC 0 CS DSVVV KSZ 7 VSZ 7
ICE752I 0 FSZ=491000 RC IGN=0 E AVG=2620 0 WSP=1670839 C DYN=13662 53216
ICE751I 1 DE-Q82816 D5-Q84357 D3-Q95174 D7-Q91626 E8-Q95174
ICE090I 0 OUTPUT LRECL = 2614, BLKSIZE = 31368, TYPE = FB
ICE055I 0 INSERT 0, DELETE 0
ICE054I 0 RECORDS - IN: 491000, OUT: 491000
ICE134I 0 NUMBER OF BYTES SORTED: 1283474000
ICE165I 0 TOTAL WORK DATA SET TRACKS ALLOCATED: 25230 , TRACKS USED: 25230
ICE199I 0 MEMORY OBJECT STORAGE USED = 0M BYTES
ICE180I 0 HIPERSPACE STORAGE USED = 65416K BYTES
ICE188I 0 DATA SPACE STORAGE USED = 0K BYTES
ICE052I 0 END OF DFSORT

BUT in another environment, the SORT uses only memory - ie ICE199I

1ICE143I 0 BLOCKSET SORT TECHNIQUE SELECTED
ICE250I 0 VISIT www.ibm.com/storage/dfsort FOR DFSORT PAPERS, EXAMPLES AND MORE
ICE000I 1 - CONTROL STATEMENTS FOR 5694-A01, Z/OS DFSORT V1R5 - 12:07 ON SAT JAN 07, 2006 -
0 ***********************************************************************
* *
* SORT CONTROL CARD *
* *
***********************************************************************
OPTION DYNALLOC=(SYSDA,16)
SORT FIELDS=(1318,03,CH,A) * BY COUNTRY
ICE201I 0 RECORD TYPE IS F - DATA STARTS IN POSITION 1
ICE751I 0 C5-Q95214 C6-Q95214 C7-Q96146 C8-Q95214 E4-Q95214 C9-BASE E5-Q95214 E6-BASE E7-K01296
ICE193I 0 ICEAM1 ENVIRONMENT IN EFFECT - ICEAM1 INSTALLATION MODULE SELECTED
ICE088I 5 FMSFVR0 .FMSFXCL .STEP020 , INPUT LRECL = 2614, BLKSIZE = 31368, TYPE = FB
ICE093I 0 MAIN STORAGE = (MAX,35740326,35740326)
ICE156I 0 MAIN STORAGE ABOVE 16MB = (35392702,35392702)
ICE127I 0 OPTIONS: OVFLO=RC0 ,PAD=RC0 ,TRUNC=RC0 ,SPANINC=RC16,VLSCMP=N,SZERO=Y,RESET=Y,VSAMEMT=Y,DYNSPC=256
ICE128I 0 OPTIONS: SIZE=35740326,MAXLIM=2097152,MINLIM=450560,EQUALS=Y,LIST=Y,ERET=RC16 ,MSGDDN=SYSOUT
ICE129I 0 OPTIONS: VIO=N,RESDNT=ALL ,SMF=NO ,WRKSEC=Y,OUTSEC=Y,VERIFY=N,CHALT=N,DYNALOC=(SYSDA ,016),ABCODE=MSG
ICE130I 0 OPTIONS: RESALL=8192,RESINV=0,SVC=109 ,CHECK=Y,WRKREL=Y,OUTREL=Y,CKPT=N,STIMER=Y,COBEXIT=COB2
ICE131I 0 OPTIONS: TMAXLIM=6291456,ARESALL=0,ARESINV=0,OVERRGN=65536,CINV=Y,CFW=Y,DSA=64
ICE132I 0 OPTIONS: VLSHRT=N,ZDPRINT=Y,IEXIT=N,TEXIT=N,LISTX=N,EFS=NONE ,EXITCK=S,PARMDDN=DFSPARM ,FSZEST=N
ICE133I 0 OPTIONS: HIPRMAX=OPTIMAL,DSPSIZE=MAX ,ODMAXBF=0,SOLRF=Y,VLLONG=N,VSAMIO=N,MOSIZE=MAX
ICE235I 0 OPTIONS: NULLOUT=RC0
ICE084I 0 BSAM ACCESS METHOD USED FOR SORTOUT
ICE084I 0 BSAM ACCESS METHOD USED FOR SORTIN
ICE750I 0 DC 8785654000 TC 0 CS DSVVV KSZ 7 VSZ 7
ICE752I 0 FSZ=3361000 RC IGN=0 E AVG=2620 0 WSP=11437247 C DYN=0 0
ICE751I 1 BA-BASE BB-K01728 BC-Q95174 E8-Q95214
ICE090I 0 OUTPUT LRECL = 2614, BLKSIZE = 31368, TYPE = FB
ICE080I 0 IN MAIN STORAGE SORT
ICE055I 0 INSERT 0, DELETE 0
ICE054I 0 RECORDS - IN: 3361000, OUT: 3361000
ICE134I 0 NUMBER OF BYTES SORTED: 8785654000
ICE165I 0 TOTAL WORK DATA SET TRACKS ALLOCATED: 0 , TRACKS USED: 0
ICE199I 0 MEMORY OBJECT STORAGE USED = 8424M BYTES
ICE180I 0 HIPERSPACE STORAGE USED = 0K BYTES
ICE188I 0 DATA SPACE STORAGE USED = 0K BYTES
ICE052I 0 END OF DFSORT

I'm not sure what caused this ...

Could you please clarify? Thanks ...
Back to top
View user's profile Send private message
Frank Yaeger

DFSORT Developer


Joined: 15 Feb 2005
Posts: 7129
Location: San Jose, CA

PostPosted: Mon Apr 03, 2006 9:01 pm
Reply with quote

I already did clarify when I said:

Quote:
DFSORT may choose different options for the same job on different systems based on the available resources, or even on the same system at different times based on the current workload.


If you had used:

//SORTDIAG DD DUMMY

in the job that didn't use Memory Object Sorting, DFSORT would have produced diagnostic messages which might have allowed me to tell you why it didn't use it. But without those diagnostic messages, I can't say.

If you can add the SORTDIAG DD and recreate the situation, send me all of the DFSORT messages offline (yaeger@us.ibm.com) and I'll take a look.

Quote:
In one environment, the SORT chose work data sets - see the ICE165I messages


Well, actually, for one run DFSORT chose hipersorting and work data sets, and for the other it chose memory object sorting.
Back to top
View user's profile Send private message
pch1982cn

New User


Joined: 02 Apr 2006
Posts: 10
Location: Dalian.China

PostPosted: Mon Apr 03, 2006 10:10 pm
Reply with quote

Ok .. I see .. thanks!

But is there a way for DFSORT V1R5 that make it don't do Memory object sorting? i.e. force V1R5 only choose data sets as work files like what R14 do?
Back to top
View user's profile Send private message
Frank Yaeger

DFSORT Developer


Joined: 15 Feb 2005
Posts: 7129
Location: San Jose, CA

PostPosted: Mon Apr 03, 2006 10:26 pm
Reply with quote

Again, R14 can choose hipersorting, dataspace sorting and/or work data sets, e.g. Hipersorting can be used alone or with work data sets. V1R5 adds memory object sorting alone or with work data sets.

You can turn off memory object sorting (V1R5 only) with the MOSIZE=0 option.

You can turn off data space sorting with the DSPSIZE=0 option.

You can turn off hipersorting with the HIPRMAX=0 option.
Back to top
View user's profile Send private message
pch1982cn

New User


Joined: 02 Apr 2006
Posts: 10
Location: Dalian.China

PostPosted: Tue Apr 04, 2006 6:45 am
Reply with quote

Excellent! Thanks Frank!
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic View Bookmarks
All times are GMT + 6 Hours
Forum Index -> DFSORT/ICETOOL

 


Similar Topics
Topic Forum Replies
No new posts Need to set RC4 through JCL SORT DFSORT/ICETOOL 5
No new posts How to split large record length file... DFSORT/ICETOOL 10
No new posts isfline didnt work in rexx at z/OS ve... CLIST & REXX 7
No new posts FINDREP - Only first record from give... DFSORT/ICETOOL 3
No new posts JCL sort card - get first day and las... JCL & VSAM 9
Search our Forums:

Back to Top