Portal | Manuals | References | Downloads | Info | Programs | JCLs | Master the Mainframes
IBM Mainframe Computers Forums Index
 
Register
 
IBM Mainframe Computers Forums Index Mainframe: Search IBM Mainframe Forum: FAQ Memberlist Usergroups Profile Log in to check your private messages Log in
 

 

ICETOOL space issues
Goto page 1, 2  Next
 
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> DFSORT/ICETOOL
View previous topic :: :: View next topic  
Author Message
umanaga

New User


Joined: 09 Apr 2007
Posts: 33
Location: India

PostPosted: Tue Jul 10, 2012 2:47 pm    Post subject: ICETOOL space issues
Reply with quote

hello, i am using icetool to split up different record types from input file using different control card.

First two control statements were executed successfully..but third is having issues.

Pasated below sort messages. this also has control cards used in sort

Any suggestion to better code sort card.

Code:

 ICE250I 0 VISIT http://www.ibm.com/storage/dfsort FOR DFSORT PAPERS, EXAMPLES AND MORE
 ICE000I 0 - CONTROL STATEMENTS FOR 5694-A01, Z/OS DFSORT 0           OPTION VLSHRT
            OUTFIL FILES=T1,
                   INCLUDE=(5,2,CH,EQ,C'AC'),           
                   OUTREC=(5,673),VTOF,
                   REMOVECC,
                   TRAILER1=(1:5,11,                   
                            12:'999999999999',         
                            24:COUNT=(M11,LENGTH=9),   
                            33:TOTAL=(269,02,ZD,EDIT=(STTTTTTTTTTT),SIGNS=(+,-)))
            SORT FIELDS=(5,673,BI,A)                   
            SUM FIELDS=NONE
 ICE146I 0 END OF STATEMENTS FROM CTL3CNTL - PARAMETER LIST STATEMENTS FOLLOW
           DEBUG NOABEND,ESTAE
           OPTION MSGDDN=DFSMSG,LIST,MSGPRT=ALL,RESINV=0,SORTDD=CTL3,SORTIN=SORTIN*
                          ,DYNALLOC
 ICE201I H RECORD TYPE IS V - DATA STARTS IN POSITION 5
 ICE751I 0 C5-K90025 C6-K90025 C7-K54603 C8-K62201 E9-K60823 C9-BASE   E5-K62201 E6-K51707 C4-K62201 E7-K62201
 ICE193I 0 ICEAM2 INVOCATION ENVIRONMENT IN EFFECT - ICEAM2 ENVIRONMENT SELECTED
 ICE088I 0 T9AVV68X.S010    .        , INPUT LRECL = 677, BLKSIZE = 32760, TYPE = VB
 ICE093I 0 MAIN STORAGE = (MAX,45574462,45545790)
 ICE156I 0 MAIN STORAGE ABOVE 16MB = (45488430,45488430)
 ICE127I 0 OPTIONS: OVFLO=RC0 ,PAD=RC0 ,TRUNC=RC0 ,SPANINC=RC16,VLSCMP=N,SZERO=Y,RESET=Y,VSAMEMT=Y,DYNSPC=256
 ICE128I 0 OPTIONS: SIZE=45574462,MAXLIM=1048576,MINLIM=450560,EQUALS=Y,LIST=Y,ERET=RC16 ,MSGDDN=DFSMSG
 ICE129I 0 OPTIONS: VIO=N,RESDNT=ALL ,SMF=FULL ,WRKSEC=Y,OUTSEC=Y,VERIFY=N,CHALT=N,DYNALOC=(SYSDA   ,004),ABCODE=MSG
 ICE130I 0 OPTIONS: RESALL=4096,RESINV=0,SVC=109 ,CHECK=Y,WRKREL=Y,OUTREL=Y,CKPT=N,COBEXIT=COB2
 ICE131I 0 OPTIONS: TMAXLIM=6291456,ARESALL=0,ARESINV=0,OVERRGN=16384,CINV=Y,CFW=Y,DSA=64
 ICE132I 0 OPTIONS: VLSHRT=Y,ZDPRINT=Y,IEXIT=N,TEXIT=N,LISTX=N,EFS=NONE    ,EXITCK=S,PARMDDN=DFSPARM ,FSZEST=N
 ICE133I 0 OPTIONS: HIPRMAX=OPTIMAL,DSPSIZE=MAX ,ODMAXBF=2097152,SOLRF=Y,VLLONG=N,VSAMIO=N,MOSIZE=MAX
 ICE235I 0 OPTIONS: NULLOUT=RC0
 ICE084I 0 BSAM ACCESS METHOD USED FOR SORTIN
 ICE750I 0 DC 14318446788 TC 0 CS DSVRR KSZ 677 VSZ 677
 ICE752I 0 FSZ=14318446788 BC  IGN=0 E  AVG=680 0  WSP=37414488 C  DYN=593355 53216
 ICE046A 0 SORT CAPACITY EXCEEDED - RECORD COUNT 51296147
 ICE253I 0 RECORDS SORTED - PROCESSED: 51296147, EXPECTED: 42362268
 ICE098I 0 AVERAGE RECORD LENGTH - PROCESSED: 239, EXPECTED: 338
 ICE753I 1 FWK=(3,593355) SWK=(0,0) TWK=(0,0) RWK=(1,84433) TOTAL=(4,677788) BLK=53216
 ICE751I 1 D8-BASE   D4-K59451 C4-K62201 C4-K62201 E8-K61438
 ICE052I 0 END OF DFSORT
Back to top
View user's profile Send private message

rakesha.hg

Active User


Joined: 21 Mar 2008
Posts: 161
Location: bangalore

PostPosted: Tue Jul 10, 2012 3:00 pm    Post subject: Reply to: ICETOOL space issues
Reply with quote

how about adding some sort work files ??
Back to top
View user's profile Send private message
Gnanas N

Active Member


Joined: 06 Sep 2007
Posts: 785
Location: Chennai, India

PostPosted: Tue Jul 10, 2012 3:05 pm    Post subject:
Reply with quote

Try increasing REGION size for the ICETOOL step/job.

IIRC, it's recommended not to code SORTWORK data sets; let DFSORT handle the space allocations as needed.
Back to top
View user's profile Send private message
umanaga

New User


Joined: 09 Apr 2007
Posts: 33
Location: India

PostPosted: Tue Jul 10, 2012 3:10 pm    Post subject:
Reply with quote

Work areas 1 to 60 were tried, still no success

so checking if any issue with sort card
Back to top
View user's profile Send private message
umanaga

New User


Joined: 09 Apr 2007
Posts: 33
Location: India

PostPosted: Tue Jul 10, 2012 3:24 pm    Post subject:
Reply with quote

For AA record
Code:
          OUTFIL FILES=T1,
                   INCLUDE=(5,2,CH,EQ,C'AA'),             
                   OUTREC=(5,408),VTOF,
                   REMOVECC,
                   TRAILER1=(1:5,11,                     
                            12:'999999999999',           
                            24:COUNT=(M11,LENGTH=9),     
                            33:TOTAL=(389,2,ZD,EDIT=(TTTTTTTTT)))
            SORT FIELDS=(5,408,BI,A)                     
            SUM FIELDS=NONE


For AB record
Code:
OUTFIL FILES=T1,
                INCLUDE=(5,2,CH,EQ,C'AB'),             
                OUTREC=(5,116),VTOF,
                REMOVECC,
                TRAILER1=(1:5,11,                     
                         12:'999999999999',           
                         24:COUNT=(M11,LENGTH=9),     
                         33:TOTAL=(68,11,ZD,EDIT=(STTTTTTTTT.TT),SIGNS=(+,-)),
                         46:TOTAL=(79,11,ZD,EDIT=(STTTTTTTTT.TT),SIGNS=(+,-)),
                         59:TOTAL=(90,11,ZD,EDIT=(STTTTTTTTT.TT),SIGNS=(+,-)))
            SORT FIELDS=(05,43,BI,A,                   
                         53,66,BI,A)                   
            SUM FIELDS=NONE


AC record..having space issue
Code:
OUTFIL FILES=T1,
                   INCLUDE=(5,2,CH,EQ,C'AC'),           
                   OUTREC=(5,673),VTOF,
                   REMOVECC,
                   TRAILER1=(1:5,11,                   
                            12:'999999999999',         
                            24:COUNT=(M11,LENGTH=9),   
                            33:TOTAL=(269,02,ZD,EDIT=(STTTTTTTTTTT),SIGNS=(+,-)))
            SORT FIELDS=(5,673,BI,A)                   
            SUM FIELDS=NONE
Back to top
View user's profile Send private message
Bill Woodger

DFSORT Moderator


Joined: 09 Mar 2011
Posts: 7236

PostPosted: Tue Jul 10, 2012 3:43 pm    Post subject: Reply to: ICETOOL space issues
Reply with quote

Is that all the cards? Are you using ICETOOL like your subject says, or not?

You are using INCLUDE on OUTFIL. That means you are sorting things which you do not need.

Use a plain INCLUDE/OMIT so that records not required are excluded before the sort.

The main difference in the control cards is the huge key one the final one. I'm not really sure how that works with the shorter records - is that why you have VLSHRT?

I think using INCLUDE/OMIT correctly will improve things greatly.

You do realise that just putting the cards in that order does not cause them to "execute" in that order?
Back to top
View user's profile Send private message
umanaga

New User


Joined: 09 Apr 2007
Posts: 33
Location: India

PostPosted: Tue Jul 10, 2012 3:53 pm    Post subject:
Reply with quote

Yes Bill, job is run using ICETOOL.

Sorry, I am not very sure why VLSHRT was coded initially.

Are you saying to put include outside as below..

I am going to try this..suggest me if im wrong
Code:

                   INCLUDE=(5,2,CH,EQ,C'AC')
              OUTFIL FILES=T1,
                   OUTREC=(5,673),VTOF,
                   REMOVECC,
                   TRAILER1=(1:5,11,                   
                            12:'999999999999',         
                            24:COUNT=(M11,LENGTH=9),   
                            33:TOTAL=(269,02,ZD,EDIT=(STTTTTTTTTTT),SIGNS=(+,-)))
            SORT FIELDS=(5,673,BI,A)                   
            SUM FIELDS=NONE
Back to top
View user's profile Send private message
Bill Woodger

DFSORT Moderator


Joined: 09 Mar 2011
Posts: 7236

PostPosted: Tue Jul 10, 2012 3:58 pm    Post subject:
Reply with quote

umanaga wrote:
Yes Bill, job is run using ICETOOL.

Sorry, I am not very sure why VLSHRT was coded initially.

Are you saying to put include outside as below..

I am going to try this..suggest me if im wrong


Basically, yes, but I'd put the OUTFIL etc in a more logical place. It will make no difference to processing.
Code:
            INCLUDE=(5,2,CH,EQ,C'AC')
            SORT FIELDS=(5,673,BI,A)                   
            SUM FIELDS=NONE
              OUTFIL FILES=T1,
                   OUTREC=(5,673),VTOF,
                   REMOVECC,
                   TRAILER1=(1:5,11,                   
                            12:'999999999999',         
                            24:COUNT=(M11,LENGTH=9),   
                            33:TOTAL=(269,02,ZD,EDIT=(STTTTTTTTTTT),SIGNS=(+,-)))


Judging from your average record-length, you'll get some good improvements in performance from from this...

EDIT: Also if you can post the sysout from one of the previous steps and the ICETOOL control cards for all steps.
Back to top
View user's profile Send private message
umanaga

New User


Joined: 09 Apr 2007
Posts: 33
Location: India

PostPosted: Tue Jul 10, 2012 4:44 pm    Post subject:
Reply with quote

I have tried with Just INCLUDE outside as in my code..it worked. As you suggested, for better results, i'll try yours.


SYSOUT messages for AA record

Code:

ICE146I 0 END OF STATEMENTS FROM CTL1CNTL - PARAMETER LIST STATEMENTS FOLLOW
           DEBUG NOABEND,ESTAE
           OPTION MSGDDN=DFSMSG,LIST,MSGPRT=ALL,RESINV=0,SORTDD=CTL1,SORTIN=SORTIN*
                          ,DYNALLOC
 ICE201I H RECORD TYPE IS V - DATA STARTS IN POSITION 5
 ICE751I 0 C5-K90025 C6-K90025 C7-K54603 C8-K62201 E9-K60823 C9-BASE   E5-K62201 E6-K51707 C4-K62201 E7-K62201
 ICE193I 0 ICEAM2 INVOCATION ENVIRONMENT IN EFFECT - ICEAM2 ENVIRONMENT SELECTED
 ICE088I 0 T9AVV68X.S010    .        , INPUT LRECL = 677, BLKSIZE = 32760, TYPE = VB
 ICE093I 0 MAIN STORAGE = (MAX,45574462,45545790)
 ICE156I 0 MAIN STORAGE ABOVE 16MB = (45488430,45488430)
 ICE127I 0 OPTIONS: OVFLO=RC0 ,PAD=RC0 ,TRUNC=RC0 ,SPANINC=RC16,VLSCMP=N,SZERO=Y,RESET=Y,VSAMEMT=Y,DYNSPC=256
 ICE128I 0 OPTIONS: SIZE=45574462,MAXLIM=1048576,MINLIM=450560,EQUALS=Y,LIST=Y,ERET=RC16 ,MSGDDN=DFSMSG
 ICE129I 0 OPTIONS: VIO=N,RESDNT=ALL ,SMF=FULL ,WRKSEC=Y,OUTSEC=Y,VERIFY=N,CHALT=N,DYNALOC=(SYSDA   ,004),ABCODE=MSG
 ICE130I 0 OPTIONS: RESALL=4096,RESINV=0,SVC=109 ,CHECK=Y,WRKREL=Y,OUTREL=Y,CKPT=N,COBEXIT=COB2
 ICE131I 0 OPTIONS: TMAXLIM=6291456,ARESALL=0,ARESINV=0,OVERRGN=16384,CINV=Y,CFW=Y,DSA=64
 ICE132I 0 OPTIONS: VLSHRT=Y,ZDPRINT=Y,IEXIT=N,TEXIT=N,LISTX=N,EFS=NONE    ,EXITCK=S,PARMDDN=DFSPARM ,FSZEST=N
 ICE133I 0 OPTIONS: HIPRMAX=OPTIMAL,DSPSIZE=MAX ,ODMAXBF=2097152,SOLRF=Y,VLLONG=N,VSAMIO=N,MOSIZE=MAX
 ICE235I 0 OPTIONS: NULLOUT=RC0
 ICE084I 0 BSAM ACCESS METHOD USED FOR SORTIN
 ICE750I 0 DC 14318446788 TC 0 CS DSVRR KSZ 412 VSZ 412
 ICE752I 0 FSZ=14318446788 BC  IGN=0 E  AVG=546 0  WSP=30041633 C  DYN=459741 53216
 ICE231I 0 STORAGE USED FOR OUTFIL : BELOW 16M = 27648, ABOVE 16M = 2142208
 ICE210I 0 CTL1OFT1 : BSAM USED, LRECL = 408, BLKSIZE = 32640, TYPE = FB
 ICE751I 1 D8-BASE   D4-K59451 EA-K57947 CB-K64631 F1-K38900 E8-K61438
 ICE055I 0 INSERT 0, DELETE 9084
 ICE054I 0 RECORDS - IN: 59366929, OUT: 59357845
 ICE227I 0 CTL1OFT1 : DELETED = 56498831, REPORT = 1, DATA = 2859014
 ICE228I 0 CTL1OFT1 : TOTAL IN = 59357845, TOTAL OUT = 2859015
 ICE134I 0 NUMBER OF BYTES SORTED: 14318446788
 ICE253I 0 RECORDS SORTED - PROCESSED: 59366929, EXPECTED: 42362268
 ICE098I 0 AVERAGE RECORD LENGTH - PROCESSED: 241, EXPECTED: 338
 ICE165I 0 TOTAL WORK DATA SET TRACKS ALLOCATED: 395385 , TRACKS USED: 380100
 ICE199I 0 MEMORY OBJECT STORAGE USED = 0M BYTES
 ICE180I 0 HIPERSPACE STORAGE USED = 6340252K BYTES
 ICE188I 0 DATA SPACE STORAGE USED = 0K BYTES
 ICE052I 0 END OF DFSORT


SYSOUT messages for AB record

Code:

ICE146I 0 END OF STATEMENTS FROM CTL2CNTL - PARAMETER LIST STATEMENTS FOLLOW
           DEBUG NOABEND,ESTAE
           OPTION MSGDDN=DFSMSG,LIST,MSGPRT=ALL,RESINV=0,SORTDD=CTL2,SORTIN=SORTIN*
                          ,DYNALLOC
 ICE201I H RECORD TYPE IS V - DATA STARTS IN POSITION 5
 ICE751I 0 C5-K90025 C6-K90025 C7-K54603 C8-K62201 E9-K60823 C9-BASE   E5-K62201 E6-K51707 C4-K62201 E7-K62201
 ICE193I 0 ICEAM2 INVOCATION ENVIRONMENT IN EFFECT - ICEAM2 ENVIRONMENT SELECTED
 ICE088I 0 T9AVV68X.S010    .        , INPUT LRECL = 677, BLKSIZE = 32760, TYPE = VB
 ICE093I 0 MAIN STORAGE = (MAX,45574462,45545790)
 ICE156I 0 MAIN STORAGE ABOVE 16MB = (45488430,45488430)
 ICE127I 0 OPTIONS: OVFLO=RC0 ,PAD=RC0 ,TRUNC=RC0 ,SPANINC=RC16,VLSCMP=N,SZERO=Y,RESET=Y,VSAMEMT=Y,DYNSPC=256
 ICE128I 0 OPTIONS: SIZE=45574462,MAXLIM=1048576,MINLIM=450560,EQUALS=Y,LIST=Y,ERET=RC16 ,MSGDDN=DFSMSG
 ICE129I 0 OPTIONS: VIO=N,RESDNT=ALL ,SMF=FULL ,WRKSEC=Y,OUTSEC=Y,VERIFY=N,CHALT=N,DYNALOC=(SYSDA   ,004),ABCODE=MSG
 ICE130I 0 OPTIONS: RESALL=4096,RESINV=0,SVC=109 ,CHECK=Y,WRKREL=Y,OUTREL=Y,CKPT=N,COBEXIT=COB2
 ICE131I 0 OPTIONS: TMAXLIM=6291456,ARESALL=0,ARESINV=0,OVERRGN=16384,CINV=Y,CFW=Y,DSA=64
 ICE132I 0 OPTIONS: VLSHRT=N,ZDPRINT=Y,IEXIT=N,TEXIT=N,LISTX=N,EFS=NONE    ,EXITCK=S,PARMDDN=DFSPARM ,FSZEST=N
 ICE133I 0 OPTIONS: HIPRMAX=OPTIMAL,DSPSIZE=MAX ,ODMAXBF=2097152,SOLRF=Y,VLLONG=N,VSAMIO=N,MOSIZE=MAX
 ICE235I 0 OPTIONS: NULLOUT=RC0
 ICE084I 0 BSAM ACCESS METHOD USED FOR SORTIN
 ICE750I 0 DC 14318446788 TC 0 CS DSVUU KSZ 113 VSZ 113
 ICE752I 0 FSZ=14318446788 BC  IGN=0 E  AVG=339 0  WSP=18652223 C  DYN=234288 53216
 ICE231I 0 STORAGE USED FOR OUTFIL : BELOW 16M = 27648, ABOVE 16M = 2081792
 ICE210I 0 CTL2OFT1 : BSAM USED, LRECL = 116, BLKSIZE = 32712, TYPE = FB
 ICE751I 1 D8-BASE   D4-K59451 EA-K57947 CB-K64631 F1-K38900 E8-K61438
 ICE055I 0 INSERT 0, DELETE 1890027
 ICE054I 0 RECORDS - IN: 59366929, OUT: 57476902
 ICE227I 0 CTL2OFT1 : DELETED = 57228011, REPORT = 1, DATA = 248891
 ICE228I 0 CTL2OFT1 : TOTAL IN = 57476902, TOTAL OUT = 248892
 ICE134I 0 NUMBER OF BYTES SORTED: 14318446788
 ICE253I 0 RECORDS SORTED - PROCESSED: 59366929, EXPECTED: 42362268
 ICE098I 0 AVERAGE RECORD LENGTH - PROCESSED: 241, EXPECTED: 338
 ICE165I 0 TOTAL WORK DATA SET TRACKS ALLOCATED: 234315 , TRACKS USED: 150810
 ICE199I 0 MEMORY OBJECT STORAGE USED = 0M BYTES
 ICE180I 0 HIPERSPACE STORAGE USED = 6733228K BYTES
 ICE188I 0 DATA SPACE STORAGE USED = 0K BYTES
 ICE052I 0 END OF DFSORT
1ICE200I 0 IDENTIFIER FROM CALLING PROGRAM IS 0003
 ICE143I 0 BLOCKSET     SORT  TECHNIQUE SELECTED



HIPERSPACE STORAGE USED ----> is this the DASD used for that sort.
Back to top
View user's profile Send private message
Bill Woodger

DFSORT Moderator


Joined: 09 Mar 2011
Posts: 7236

PostPosted: Tue Jul 10, 2012 6:36 pm    Post subject: Reply to: ICETOOL space issues
Reply with quote

umanaga,

You really need to understand the order that things operate in in DFSORT. Kolusu posts a link from time-to-time for others in your situation. Removing the INCLUDE from the OUTFIL and using the plain one is going to save you a lot of processing. Pity I can't charge you for the savings...

My reordering of your statements does not affect the way they run. The change is in the INCLUDE happening before the SORT and SUM so that only the required records are sorted/deduped.

Seperately, you are reading from tape. Three times. How about doing one pass of the tape and extracting the raw data you need to seperate files with OUTFIL and then process those in your steps.

Still haven't seen your ICETOOL. If you show the full set of control cards, there may be other suggestions.
Back to top
View user's profile Send private message
Skolusu

Senior Member


Joined: 07 Dec 2007
Posts: 2205
Location: San Jose

PostPosted: Tue Jul 10, 2012 9:41 pm    Post subject: Reply to: ICETOOL space issues
Reply with quote

Umanaga ,

You are sorting almost 60 million records and you are doing multiple passes for the same which is quite unnecessary. Run this JCL instead of all those 3 passes.
Code:

//STEP0100 EXEC PGM=SORT                                   
//SYSOUT   DD SYSOUT=*                                     
//SORTIN   DD DISP=SHR,DSN=Your Input VB File
//AA       DD DSN=Your Output AA File,
//            DISP=(NEW,CATLG,DELETE),
//            SPACE=(CYL,(X,Y),RLSE)
//*
//AB       DD DSN=Your Output AB File,
//            DISP=(NEW,CATLG,DELETE),
//            SPACE=(CYL,(X,Y),RLSE)
//*
//AC       DD DSN=Your Output AC File,
//            DISP=(NEW,CATLG,DELETE),
//            SPACE=(CYL,(X,Y),RLSE)
//*
//SYSIN    DD *                                             
  OPTION AVGRLEN=239,DYNALLOC=(SYSDA,8)
  INCLUDE COND=(5,2,SS,EQ,C'AA,AB,AC')                         
  SORT FIELDS=(5,673,BI,A)                                     
  SUM FIELDS=NONE                                             

  OUTFIL FNAMES=AA,INCLUDE=(5,2,CH,EQ,C'AA'),             
  VTOF,BUILD=(5,673),REMOVECC,                                 
  TRAILER1=(5,11,12'9',COUNT=(M11,LENGTH=9),                   
           TOTAL=(269,02,ZD,EDIT=(STTTTTTTTTTT),SIGNS=(+,-))) 

  OUTFIL FNAMES=AB,INCLUDE=(5,2,CH,EQ,C'AB'),                   
  VTOF,BUILD=(5,673),REMOVECC,                                 
  TRAILER1=(5,11,12'9',COUNT=(M11,LENGTH=9),                   
           TOTAL=(269,02,ZD,EDIT=(STTTTTTTTTTT),SIGNS=(+,-))) 

  OUTFIL FNAMES=AC,INCLUDE=(5,2,CH,EQ,C'AC'),                 
  VTOF,BUILD=(5,673),REMOVECC,                                 
  TRAILER1=(5,11,12'9',COUNT=(M11,LENGTH=9),                   
           TOTAL=(269,02,ZD,EDIT=(STTTTTTTTTTT),SIGNS=(+,-))) 

//*
Back to top
View user's profile Send private message
Bill Woodger

DFSORT Moderator


Joined: 09 Mar 2011
Posts: 7236

PostPosted: Wed Jul 11, 2012 3:30 am    Post subject: Reply to: ICETOOL space issues
Reply with quote

umanaga,

First, note how Kolusu coded to avoid the space problem. Where did he get the information? From your Sysout for the AC.

You were three times reading close to 60m records. Sorting them. De-duping them. Then excluding those you don't want.

If you:

Use OPTION COPY/SORT FIELDS=COPY to process the input and split it into the three types of data you want, ignoring the rest; Use OUTFIL INCLUDE/OMIT, which will not be a problem as you are not sorting;
Change each "step" to process the new specific input file, you'll no longer need the INCLUDE/OMIT as the input will only contain the correct records.

With this approach I'd guess you'd be sorting and summing fewer than 5m records instead of 180m.

That should be a littler zippier and you will not have any problems with space.

It might no longer make sense to clump it all into one ICETOOL step, but no-one says it made a great deal of sense in the first place.
Back to top
View user's profile Send private message
umanaga

New User


Joined: 09 Apr 2007
Posts: 33
Location: India

PostPosted: Thu Jul 12, 2012 4:18 pm    Post subject:
Reply with quote

skolusu, thanks much for the efforts in getting this coded. but not clear on one..

My AA, AB, AC records are of different length in input file. So in sort creating AA file only for its record length. But your sort job all o/p files are 673 length.

Bill, here provide the full jcl

Code:
//A222     EXEC PGM=ICETOOL
//SORTIN   DD  INPUT FILE                             
//TOOLIN   DD  *                                     
   SORT FROM(SORTIN) USING(CTL1)                     
   SORT FROM(SORTIN) USING(CTL2)                     
   SORT FROM(SORTIN) USING(CTL3)                     
   SORT FROM(SORTIN) USING(CTL4)                     
   SORT FROM(SORTIN) USING(CTL5)                     
   SORT FROM(SORTIN) USING(CTL6)                     
   SORT FROM(SORTIN) USING(CTL7)                     
//CTL1CNTL DD  *                                     
 OPTION VLSHRT                                       
 OUTFIL FILES=T1,                                     
        INCLUDE=(5,2,CH,EQ,C'AA'),                   
        OUTREC=(5,408),VTOF,                         
        REMOVECC,                                     
        TRAILER1=(1:5,11,                             
                 12:'999999999999',                   
                 24:COUNT=(M11,LENGTH=9),             
                 33:TOTAL=(389,2,ZD,EDIT=(TTTTTTTTT)))
 SORT FIELDS=(5,408,BI,A)                             
 SUM FIELDS=NONE                                     
//CTL2CNTL DD  *                                                   
 OUTFIL FILES=T1,                                                 
     INCLUDE=(5,2,CH,EQ,C'AB'),                                   
     OUTREC=(5,116),VTOF,                                         
     REMOVECC,                                                     
     TRAILER1=(1:5,11,                                             
              12:'999999999999',                                   
              24:COUNT=(M11,LENGTH=9),                             
              33:TOTAL=(68,11,ZD,EDIT=(STTTTTTTTT.TT),SIGNS=(+,-)),
 SORT FIELDS=(05,43,BI,A,                                         
              53,66,BI,A)                                         
 SUM FIELDS=NONE                                                   
//CTL3CNTL DD  *                                                   
 OPTION VLSHRT                                                     
 OUTFIL FILES=T1,                                                 
        INCLUDE=(5,2,CH,EQ,C'AC'),                                 
        OUTREC=(5,673),VTOF,                                       
        REMOVECC,                                                 
        TRAILER1=(1:5,11,                                         
                 12:'999999999999',                               
                 24:COUNT=(M11,LENGTH=9),                             
                 33:TOTAL=(269,02,ZD,EDIT=(STTTTTTTTTTT),SIGNS=(+,-)))
 SORT FIELDS=(5,673,BI,A)                                             
 SUM FIELDS=NONE                                                     
//CTL4CNTL DD  *                                                     
 OPTION VLSHRT                                                       
 OUTFIL FILES=T1,                                                     
        INCLUDE=(5,2,CH,EQ,C'AD'),                                   
        OUTREC=(5,305),VTOF,                                         
        REMOVECC,                                                     
        TRAILER1=(1:5,11,                                             
                 12:'999999999999',                                   
                 24:COUNT=(M11,LENGTH=9),                             
                 33:TOTAL=(92,03,ZD,EDIT=(STTTTTTTTT),SIGNS=(+,-)))   
 SORT FIELDS=(5,305,BI,A)                                             
 SUM FIELDS=NONE                                                     
//CTL5CNTL DD  *                                                     
 OPTION VLSHRT                                                       
 OUTFIL FILES=T1,                                                     
     INCLUDE=(5,2,CH,EQ,C'AE'),                                       
     OUTREC=(5,219),VTOF,                                             
     REMOVECC,                                                       
     TRAILER1=(1:5,11,                                             
              12:'999999999999',                                   
              24:COUNT=(M11,LENGTH=9),                             
              33:TOTAL=(76,11,ZD,EDIT=(STTTTTTTTT.TT),SIGNS=(+,-)),
 SORT FIELDS=(5,219,BI,A)                                           
 SUM FIELDS=NONE                                                   
//CTL6CNTL DD  *                                                   
 OPTION VLSHRT                                                     
 OUTFIL FILES=T1,                                                   
     INCLUDE=(5,2,CH,EQ,C'AF'),                                     
     OUTREC=(5,219),VTOF,                                           
     REMOVECC,                                                     
     TRAILER1=(1:5,11,                                             
              12:'999999999999',                                   
              24:COUNT=(M11,LENGTH=9),                             
              33:TOTAL=(76,11,ZD,EDIT=(STTTTTTTTT.TT),SIGNS=(+,-)),
 SORT FIELDS=(5,219,BI,A)                                           
 SUM FIELDS=NONE                                                   
//CTL7CNTL DD  *                                                   
 OPTION VLSHRT                                                     
 OUTFIL FILES=T1,                                                   
     INCLUDE=(5,2,CH,EQ,C'AG'),                                     
     OUTREC=(5,219),VTOF,                                           
     REMOVECC,                                                     
     TRAILER1=(1:5,11,                                             
              12:'999999999999',                                   
              24:COUNT=(M11,LENGTH=9),                             
              33:TOTAL=(76,11,ZD,EDIT=(STTTTTTTTT.TT),SIGNS=(+,-)),
 SORT FIELDS=(5,219,BI,A)                                           
 SUM FIELDS=NONE                                                   
//CTL1OFT1 DD  AA file                                                   
//CTL2OFT1 DD  AB file                                                   
//CTL3OFT1 DD  AC file                                                 
//CTL4OFT1 DD  AD file                                                 
//CTL5OFT1 DD  AE file                                                 
//CTL6OFT1 DD  AF file                                                 
//CTL7OFT1 DD  AG file           
//SORTMSG  DD  SYSOUT=*       
//SYSOUT   DD  SYSOUT=*       
//SYSUDUMP DD  SYSOUT=Y       
//*                           
//TOOLMSG  DD  SYSOUT=*       
//DFSMSG   DD  SYSOUT=*       
Back to top
View user's profile Send private message
Bill Woodger

DFSORT Moderator


Joined: 09 Mar 2011
Posts: 7236

PostPosted: Thu Jul 12, 2012 6:34 pm    Post subject: Reply to: ICETOOL space issues
Reply with quote

If you are reading the file six times, to my mind there is even more reason to do something along the lines of my last post. Read the file once. Use six OUTFILs. Process the six different files.

Are you saying you are happy with how you have it?
Back to top
View user's profile Send private message
umanaga

New User


Joined: 09 Apr 2007
Posts: 33
Location: India

PostPosted: Thu Jul 12, 2012 9:24 pm    Post subject:
Reply with quote

Bill,
With the code in my post i am still facing space issues. As mentioned by you due to reading file 7 times (not 6 i believe).

how to code OUTFIL for different file lengths..

Edited to reflect posts removed to try to keep thread manageable.
Back to top
View user's profile Send private message
dick scherrer

Site Director


Joined: 23 Nov 2006
Posts: 19270
Location: Inside the Matrix

PostPosted: Thu Jul 12, 2012 10:11 pm    Post subject:
Reply with quote

Hello,

If you write multiple output files, you would assign different SPACE parameters in the JCL - depending on the expected volume of that file.

You would build records of the appropriate length for each output file.
Back to top
View user's profile Send private message
Bill Woodger

DFSORT Moderator


Joined: 09 Mar 2011
Posts: 7236

PostPosted: Fri Jul 13, 2012 12:45 am    Post subject: Reply to: ICETOOL space issues
Reply with quote

OUTREC=(5,219)

This is where you are setting the length of the output records.

You seem to have gone back to your previous set of control cards.

You need to INCLUDE/OMIT first for the record-type you want in each step. Then you will only sort and dedupe records of the the correct type.

If you follow my earlier advice your save reading close to 300m records.
Back to top
View user's profile Send private message
Skolusu

Senior Member


Joined: 07 Dec 2007
Posts: 2205
Location: San Jose

PostPosted: Fri Jul 13, 2012 3:10 am    Post subject: Reply to: ICETOOL space issues
Reply with quote

umanaga,

Ideally I would strip off the records from the tape file and then perform individual sorts. You can club AE, AF, AG sorts as they are sorting on Similar fields and lengths.

Since you have 1 sort that does a sort on the entire 673 bytes I coded up 1 step process to handle all codes. I padded up the other fields with binary zeroes.

Code:

//STEP0100 EXEC PGM=SORT                       
//SYSOUT   DD SYSOUT=*                         
//SORTIN   DD DSN=Your Input VB tape file,DISP=SHR 
//*
//AA       DD DSN=Your Output AA file,
//            DISP=(NEW,CATLG,DELETE),
//            SPACE=(CYL,(X,Y),RLSE)
//*
//AB       DD DSN=Your Output AB file,
//            DISP=(NEW,CATLG,DELETE),
//            SPACE=(CYL,(X,Y),RLSE)
//*
//AC       DD DSN=Your Output AC file,
//            DISP=(NEW,CATLG,DELETE),
//            SPACE=(CYL,(X,Y),RLSE)
//*
//AD       DD DSN=Your Output AD file,
//            DISP=(NEW,CATLG,DELETE),
//            SPACE=(CYL,(X,Y),RLSE)
//*
//AE       DD DSN=Your Output AE file,
//            DISP=(NEW,CATLG,DELETE),
//            SPACE=(CYL,(X,Y),RLSE)
//*
//AF       DD DSN=Your Output AF file,
//            DISP=(NEW,CATLG,DELETE),
//            SPACE=(CYL,(X,Y),RLSE)   
//*
//AG       DD DSN=Your Output AG file,
//            DISP=(NEW,CATLG,DELETE),
//            SPACE=(CYL,(X,Y),RLSE)
//*
//SYSIN    DD *                                                 
  INCLUDE COND=(5,2,SS,EQ,C'AA,AB,AC,AD,AE,AF,AG')               
                                                                 
  INREC IFTHEN=(WHEN=(5,2,CH,EQ,C'AA'),BUILD=(1,4,5,408,265Z)), 
  IFTHEN=(WHEN=(5,2,CH,EQ,C'AB'),BUILD=(1,4,5,43,53,66,564Z)),   
  IFTHEN=(WHEN=(5,2,CH,EQ,C'AC'),BUILD=(1,4,5,673)),             
  IFTHEN=(WHEN=(5,2,CH,EQ,C'AD'),BUILD=(1,4,5,305,372Z)),       
  IFTHEN=(WHEN=(5,2,SS,EQ,C'AE,AF,AG'),BUILD=(1,4,5,219,458Z))   
                                                                 
  SORT FIELDS=(5,673,BI,A)                                       
  SUM FIELDS=NONE                                               
                                                                 
  OUTFIL FNAMES=AA,INCLUDE=(5,2,CH,EQ,C'AA'),                       
  VTOF,BUILD=(5,408),REMOVECC,                                       
  TRAILER1=(01:05,11,                                               
            12:12'9',                                               
            24:COUNT=(M11,LENGTH=9),                                 
            33:TOTAL=(389,2,ZD,EDIT=(TTTTTTTTT)))                   
                                                                     
  OUTFIL FNAMES=AB,INCLUDE=(5,2,CH,EQ,C'AB'),                       
  VTOF,BUILD=(5,116),REMOVECC,                                       
  TRAILER1=(5,11,                                                   
            12:12'9',                                               
            24:COUNT=(M11,LENGTH=9),                                 
            33:TOTAL=(68,11,ZD,EDIT=(STTTTTTTTT.TT),SIGNS=(+,-)))   
                                                                     
  OUTFIL FNAMES=AC,INCLUDE=(5,2,CH,EQ,C'AC'),                       
  VTOF,BUILD=(5,673),REMOVECC,                                       
  TRAILER1=(5,11,                                                   
            12:12'9',                                               
            24:COUNT=(M11,LENGTH=9),                                 
            33:TOTAL=(269,02,ZD,EDIT=(STTTTTTTTTTT),SIGNS=(+,-)))   
                                                                     
  OUTFIL FNAMES=AD,INCLUDE=(5,2,CH,EQ,C'AD'),                       
  VTOF,BUILD=(5,305),REMOVECC,                                       
  TRAILER1=(5,11,                                                   
            12:12'9',                                               
            24:COUNT=(M11,LENGTH=9),                                 
            33:TOTAL=(92,03,ZD,EDIT=(STTTTTTTTT),SIGNS=(+,-)))       
                                                                     
  OUTFIL FNAMES=AE,INCLUDE=(5,2,CH,EQ,C'AE'),                       
  VTOF,BUILD=(5,219),REMOVECC,                                       
  TRAILER1=(5,11,                                                   
            12:12'9',                                               
            24:COUNT=(M11,LENGTH=9),                                 
            33:TOTAL=(76,11,ZD,EDIT=(STTTTTTTTT.TT),SIGNS=(+,-)))   
                                                                     
  OUTFIL FNAMES=AF,INCLUDE=(5,2,CH,EQ,C'AF'),                       
  VTOF,BUILD=(5,219),REMOVECC,                                     
  TRAILER1=(5,11,                                                   
            12:12'9',                                               
            24:COUNT=(M11,LENGTH=9),                               
            33:TOTAL=(76,11,ZD,EDIT=(STTTTTTTTT.TT),SIGNS=(+,-)))   
                                                                   
  OUTFIL FNAMES=AG,INCLUDE=(5,2,CH,EQ,C'AG'),                       
  VTOF,BUILD=(5,219),REMOVECC,                                     
  TRAILER1=(5,11,                                                   
            12:12'9',                                               
            24:COUNT=(M11,LENGTH=9),                               
            33:TOTAL=(76,11,ZD,EDIT=(STTTTTTTTT.TT),SIGNS=(+,-)))   
//*
Back to top
View user's profile Send private message
Bill Woodger

DFSORT Moderator


Joined: 09 Mar 2011
Posts: 7236

PostPosted: Fri Jul 13, 2012 5:22 am    Post subject: Reply to: ICETOOL space issues
Reply with quote

Just finally to clear up the VLSHRT. You "needed" this originally as you were sending all the records through each sort. All those many 223-byte records were going into the sorts with larger sort keys, so would have failed without the VLSHRT.

OK, nearly finally. Note Kolusu's preference for the "strip and sort seperately". All the records on your file that are of those record-types are now being extended to 677 bytes. If those record-types represent the entire file or a large proportion of it, then you may get the space problem again. Remember that Kolusu showed you how to deal with that if you want to go with the "one step" solution.

In you ICETOOL step, with multiple sorts, you would always be exposed to a failure in one of the sorts (space for instance) requiring that the earlier sorts would be run again, or that the control cards would have to be changed for the re-run.

In a multi-step solution, you don't have that problem. More JCL to code and test, but more easy to manage.

Judging from your average record lengths, the multi-step solution I'd expect to run faster. On top of that, once the extract is done, the seperate steps can be in seperate jobs, all of which can be "kicked off" by the extract, and run in parallel, rather than serially as in your original single job.

Either way, you'll have a lot of "tape drive time" freed-up as well.
Back to top
View user's profile Send private message
umanaga

New User


Joined: 09 Apr 2007
Posts: 33
Location: India

PostPosted: Mon Jul 16, 2012 12:22 pm    Post subject:
Reply with quote

Bill, Thanks for mentioning on VLSHRT.

skolusu, if i know all records in my input file are AA, AB, AC, AD, AE, AF, AG can i ignore the INCLUDE at begining..as it unnessarily tries to filter on them.

Also on AB record, 48-52 column data is ignored from input file. can i get this data to o/p file without having it part of SUM fields.


Code:
 OUTFIL FILES=T1,                                                 
     INCLUDE=(5,2,CH,EQ,C'AB'),                                   
     OUTREC=(5,116),VTOF,                                         
     REMOVECC,                                                     
     TRAILER1=(1:5,11,                                             
              12:'999999999999',                                   
              24:COUNT=(M11,LENGTH=9),                             
              33:TOTAL=(68,11,ZD,EDIT=(STTTTTTTTT.TT),SIGNS=(+,-)),
 SORT FIELDS=(05,43,BI,A,                                         
              53,66,BI,A)     
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> DFSORT/ICETOOL All times are GMT + 6 Hours
Goto page 1, 2  Next
Page 1 of 2

 

Search our Forum:

Similar Topics
Topic Author Forum Replies Posted
No new posts ODPP(Optim Data privacy Provider) Iss... Rama kishore IBM Tools 1 Mon Nov 07, 2016 5:46 pm
No new posts How to delete second instance from Fl... Gunapala CN DFSORT/ICETOOL 6 Tue Oct 18, 2016 11:42 pm
No new posts ICETOOL/SYNCSORT - SYSPRINT output nartcr DFSORT/ICETOOL 17 Wed Oct 05, 2016 10:46 pm
No new posts Icetool or Joinkeys example to compar... ishant chauhan DFSORT/ICETOOL 21 Sat Aug 20, 2016 2:40 am
No new posts Faccing issues while creating a GDG V... sravindra_s JCL & VSAM 5 Thu Jun 23, 2016 11:26 am


Facebook
Back to Top
 
Mainframe Wiki | Forum Rules | Bookmarks | Subscriptions | FAQ | Tutorials | Contact Us