IBM Mainframe Forum Index
 
Log In
 
IBM Mainframe Forum Index Mainframe: Search IBM Mainframe Forum: FAQ Register
 

ICETOOL - Unable to process more than certain num of recs
IBM Mainframe Forums -> DFSORT/ICETOOL
Post new topic   Reply to topic Quick References
View previous topic :: View next topic  
Author Message
Div Grad

New User


Joined: 08 Apr 2005
Posts: 45
PostPosted: Tue Dec 02, 2014 12:52 am
 Reply with quote
I'm having a problem where it appears ICETOOL can successfully handle a small file but not a large one. The issue does not seem to be the data itself because I can take an input file of a few hundred lines that is processed successfully, then double the size of the file by repeating every row, and then the resulting file runs ok also. I can repeat this process and eventually the sort stops working.

Here is my input parm:

Code:

000001    DISPLAY FROM(SORTIN) LIST(SORTOUT) BLANK -                           
000002    TITLE('TRACKING REPORT OF UNRESOLVED DUPLICATES') DATE PAGE -       
000003    HEADER('DUPLED KEY') ON(1,25,CH) -                                   
000004    HEADER('UNRESOLVED SINCE') ON(27,8,ZD,E'9999/99/99')             



Here is my submitting JCL:

Code:

//STEP150  EXEC PGM=ICETOOL,REGION=4096K                               
//TOOLMSG  DD DSN=XXXXXXX.YYYY140D.ICETOOL.TOOLMSG1,                   
//            DISP=(NEW,CATLG,CATLG),                                 
//            UNIT=SYSDA,                                             
//            SPACE=(CYL,(10,1),RLSE),                                 
//            DCB=(RECFM=FBA,LRECL=121,BLKSIZE=0)                     
//DFSMSG   DD DSN=XXXXXXX.YYYY140D.ICETOOL.DFSMSG1,                   
//            DISP=(NEW,CATLG,CATLG),                                 
//            UNIT=SYSDA,                                             
//            SPACE=(CYL,(10,1),RLSE),                                 
//            DCB=(RECFM=FBA,LRECL=121,BLKSIZE=0)                     
//SORTIN   DD DISP=SHR,DSN=XXXXXXX.YYYY140D.INPUT1                     
//SORTOUT  DD DSN=XXXXXXX.YYYY140D.ICETOOL.TEST1,                     
//            DISP=(NEW,CATLG,CATLG),                                 
//            UNIT=SYSDA,                                             
//            SPACE=(CYL,(10,1),RLSE),                                 
//            DCB=(RECFM=FBA,LRECL=121,BLKSIZE=0)                     
//TOOLIN   DD DISP=SHR,DSN=XXXXXXX.DEVL.PARM(YYYY1404)   


Here's an example of the input data

Code:
27500DL1007    010        20141118               
27500DL1007    020        20141118               
27500DL1007    030        20141118               
27500DL1007    040        20141118               
27500DL1007    050        20141118               
27500DL1007    060        20141118               
27500DL1007    070        20141118               
27500DL1007    080        20141118               
27500DL1007    090        20141118               
27500DL1007    100        20141118               
27500DL1007    110        20141118               
27500DL1007    120        20141118               
27500DL1007    130        20141118               
27500DL1007    140        20141118               
27500DL1007    150        20141118               
27500DL1007    160        20141118               
27500DL1007    170        20141118               
27500DL1007    180        20141118     



Here's the TOOLMSG output:



Code:
1ICE600I 0 DFSORT ICETOOL UTILITY RUN STARTED

 ICE650I 0 VISIT http://www.ibm.com/storage/dfsort FOR ICETOOL PAPERS, EXAMPLES AND MORE

 ICE632I 0 SOURCE FOR ICETOOL STATEMENTS:  TOOLIN


 ICE630I 0 MODE IN EFFECT:  STOP

              DISPLAY FROM(SORTIN) LIST(SORTOUT) BLANK -
              TITLE('TRACKING REPORT OF UNRESOLVED DUPLICATES') DATE PAGE -
              HEADER('DUPLED KEY') ON(1,25,CH) -
              HEADER('UNRESOLVED SINCE') ON(27,8,ZD,E'9999/99/99')
 ICE643I 0 WIDTH OF REPORT IS 0121 BYTES
 ICE627I 0 DFSORT CALL 0001 FOR COPY FROM SORTIN   TO E35 EXIT COMPLETED
 ICE603I 0 INFORMATION PRINTED IN SORTOUT  DATA SET
 ICE628I 0 RECORD COUNT:  000000000003936
 ICE602I 0 OPERATION RETURN CODE:  00


 ICE601I 0 DFSORT ICETOOL UTILITY RUN ENDED - RETURN CODE:  00


and here's the DFS Message output:

Code:
1ICE600I 0 DFSORT ICETOOL UTILITY RUN STARTED

 ICE650I 0 VISIT http://www.ibm.com/storage/dfsort FOR ICETOOL PAPERS, EXAMPLES AND MORE

 ICE632I 0 SOURCE FOR ICETOOL STATEMENTS:  TOOLIN


 ICE630I 0 MODE IN EFFECT:  STOP

              DISPLAY FROM(SORTIN) LIST(SORTOUT) BLANK -
              TITLE('TRACKING REPORT OF UNRESOLVED DUPLICATES') DATE PAGE -
              HEADER('DUPLED KEY') ON(1,25,CH) -
              HEADER('UNRESOLVED SINCE') ON(27,8,ZD,E'9999/99/99')
 ICE643I 0 WIDTH OF REPORT IS 0121 BYTES
 ICE627I 0 DFSORT CALL 0001 FOR COPY FROM SORTIN   TO E35 EXIT COMPLETED
 ICE603I 0 INFORMATION PRINTED IN SORTOUT  DATA SET
 ICE628I 0 RECORD COUNT:  000000000003936
 ICE602I 0 OPERATION RETURN CODE:  00


 ICE601I 0 DFSORT ICETOOL UTILITY RUN ENDED - RETURN CODE:  00



Any thoughts on what the issue is here? The answer may be painfully obvious to some here but to me it is just... painful.

As you can see I'm not processing a giant file here, it is only about 4,000 lines. But the output is totally empty, as opposed to the nice result I get when the input file is smaller:

Code:
TRACKING REPORT OF UNRESOLVED DUPLICATES        12/01/14        - 1 -
                                                                     
DUPLED KEY                  UNRESOLVED SINCE                         
-------------------------   ----------------                         
27500DL1007    010                2014/11/18                         
27500DL1007    020                2014/11/18                         
27500DL1007    030                2014/11/18                         
27500DL1007    040                2014/11/18                         
27500DL1007    050                2014/11/18                         
27500DL1007    060                2014/11/18                         
27500DL1007    070                2014/11/18                         



My guess is that some parm is causing an upper limit on number of recs to process, or size of input file to process, but I do not know which one.
Back to top
View user's profile Send private message
vasanthz

Global Moderator


Joined: 28 Aug 2007
Posts: 1742
Location: Tirupur, India
PostPosted: Tue Dec 02, 2014 1:26 am
 Reply with quote
Hello,
Your TOOLMSG & DFS MSG both seem to be the same. Could you correct.

Regards,
Back to top
View user's profile Send private message
Div Grad

New User


Joined: 08 Apr 2005
Posts: 45
PostPosted: Tue Dec 02, 2014 1:58 am
 Reply with quote
Sorry about that, here is the DFS message:




Code:

1ICE200I 0 IDENTIFIER FROM CALLING PROGRAM IS 0001
 ICE201I H RECORD TYPE IS F - DATA STARTS IN POSITION 1
 ICE751I 0 C5-I12416 C6-K90026 C7-K94453 C8-K94453 E9-K60824 C9-BASE   E5-K80744 E7-I12416
 ICE143I 0 BLOCKSET     COPY  TECHNIQUE SELECTED
 ICE250I 0 VISIT http://www.ibm.com/storage/dfsort FOR DFSORT PAPERS, EXAMPLES AND MORE
 ICE000I 0 - CONTROL STATEMENTS FOR 5694-A01, Z/OS DFSORT V1R12 - 10:06 ON MON DEC 01, 2014 -
0          DEBUG NOABEND,ESTAE
           OPTION MSGDDN=DFSMSG,LIST,MSGPRT=ALL,RESINV=0,SORTIN=SORTIN,COPY,NOCHEC*
                          K
           INREC FIELDS=(1,4,1,25,27,8)
           MODS E35=(ICE35DU,12288)
 ICE193I 0 ICEAM2 INVOCATION ENVIRONMENT IN EFFECT - ICEAM2 ENVIRONMENT SELECTED
 ICE088I 0 RJM140D .STEP150 .        , INPUT LRECL = 34, BLKSIZE = 27982, TYPE = FB
 ICE093I 0 MAIN STORAGE = (MAX,6291456,6291456)
 ICE156I 0 MAIN STORAGE ABOVE 16MB = (5259264,5242880)
 ICE127I 0 OPTIONS: OVFLO=RC0 ,PAD=RC0 ,TRUNC=RC0 ,SPANINC=RC16,VLSCMP=N,SZERO=Y,RESET=Y,VSAMEMT=Y,DYNSPC=256
 ICE128I 0 OPTIONS: SIZE=6291456,MAXLIM=1048576,MINLIM=450560,EQUALS=N,LIST=Y,ERET=RC16 ,MSGDDN=DFSMSG
 ICE129I 0 OPTIONS: VIO=N,RESDNT=ALL ,SMF=NO   ,WRKSEC=Y,OUTSEC=Y,VERIFY=N,CHALT=N,DYNALOC=N             ,ABCODE=MSG
 ICE130I 0 OPTIONS: RESALL=4096,RESINV=0,SVC=109 ,CHECK=N,WRKREL=Y,OUTREL=Y,CKPT=N,COBEXIT=COB2
 ICE131I 0 OPTIONS: TMAXLIM=6291456,ARESALL=0,ARESINV=0,OVERRGN=16384,CINV=Y,CFW=Y,DSA=0
 ICE132I 0 OPTIONS: VLSHRT=N,ZDPRINT=Y,IEXIT=N,TEXIT=N,LISTX=N,EFS=NONE    ,EXITCK=S,PARMDDN=DFSPARM ,FSZEST=N
 ICE133I 0 OPTIONS: HIPRMAX=OPTIMAL,DSPSIZE=MAX ,ODMAXBF=0,SOLRF=Y,VLLONG=N,VSAMIO=N,MOSIZE=MAX
 ICE235I 0 OPTIONS: NULLOUT=RC0
 ICE236I 0 OPTIONS: DYNAPCT=10 ,MOWRK=Y
 ICE084I 0 EXCP ACCESS METHOD USED FOR SORTOUT
 ICE084I 0 EXCP ACCESS METHOD USED FOR SORTIN
 ICE751I 1 EF-BASE   F0-K91597 E8-I12416
 ICE090I 0 OUTPUT LRECL = 121, BLKSIZE = 27951, TYPE = FBA
 ICE171I 0 SORTOUT LRECL OF 121 IS DIFFERENT FROM SORTIN(NN) LRECL OF 34 - RC=0
 ICE055I 0 INSERT 0, DELETE 3936
 ICE054I 0 RECORDS - IN: 3936, OUT: 0
 ICE173I 0 NO RECORDS FOR THE SORTOUT DATA SET - RC=0
 ICE052I 0 END OF DFSORT
Back to top
View user's profile Send private message
Bill Woodger

Moderator Emeritus


Joined: 09 Mar 2011
Posts: 7309
Location: Inside the Matrix
PostPosted: Tue Dec 02, 2014 5:22 am
 Reply with quote
It's a bad idea for you to define SORTIN and SORTOUT of your own. Change that.
Back to top
View user's profile Send private message
enrico-sorichetti

Superior Member


Joined: 14 Mar 2007
Posts: 10873
Location: italy
PostPosted: Tue Dec 02, 2014 4:03 pm
 Reply with quote
works for me for 48000 lines

Code:

 ****** ***************************** Top of Data ******************************
 000001 //ENRICO1 JOB NOTIFY=&SYSUID,
 000002 //             MSGLEVEL=(1,1),CLASS=A,MSGCLASS=X
 000003 //*
 000004 //ICE     EXEC PGM=ICETOOL
 000005 //SYSPRINT  DD SYSOUT=*
 000006 //TOOLMSG   DD SYSOUT=*
 000007 //DFSMSG    DD SYSOUT=*
 000008 //SORTIN    DD *
 - - -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 48000 Line(s) not Displayed
 048009 //SORTOUT   DD SYSOUT=*
 048010 //TOOLIN    DD *
 048011   DISPLAY FROM(SORTIN) LIST(SORTOUT) BLANK -
 048012           TITLE('TRACKING REPORT OF UNRESOLVED DUPLICATES') DATE PAGE -
 048013           HEADER('DUPLED KEY') ON(1,25,CH) -
 048014           HEADER('UNRESOLVED SINCE') ON(27,8,ZD,E'9999/99/99')
 ****** **************************** Bottom of Data ****************************


Code:


********************************* TOP OF DATA **********************************
TRACKING REPORT OF UNRESOLVED DUPLICATES        12/02/14        - 1 -

DUPLED KEY                  UNRESOLVED SINCE
-------------------------   ----------------
27500DL1007    010                2014/11/18
27500DL1007    010                2014/11/18
27500DL1007    010                2014/11/18

... ... ...
... ... ...
... ... ...

TRACKING REPORT OF UNRESOLVED DUPLICATES        12/02/14        - 889 -

DUPLED KEY                  UNRESOLVED SINCE
-------------------------   ----------------
27500DL1007    010                2014/11/18
27500DL1007    010                2014/11/18
27500DL1007    010                2014/11/18

... ... ...
... ... ...
... ... ...

******************************** BOTTOM OF DATA ********************************
Back to top
View user's profile Send private message
Div Grad

New User


Joined: 08 Apr 2005
Posts: 45
PostPosted: Tue Dec 02, 2014 9:48 pm
 Reply with quote
Bill Woodger wins the prize! Thanks for the information and quick reply!

Someone else reviewing the issue turned this up more or less in parallel with my posting my query here:

Quote:

Title: z/OS V1R13.0 DFSORT Application Programming Guide
Document Number: SC26-7523-07

7.2.1 JCL Restrictions

You should avoid using ddnames reserved for ICETOOL and DFSORT in
ICETOOL operands (FROM, TO, LIST, DISCARD, WRITE ). In general, you
should also avoid supplying DD statements with ddnames reserved for
DFSORT when using ICETOOL because doing so can cause unpredictable
results. Specifically:

...

- SYSIN, SORTCNTL, SORTIN, SORTOUT, SORTINnn, and xxxxINnn (where xxxx
is specified in a USING operand) should not be used as ddnames in
ICETOOL operators nor supplied as DD statements.
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic View Bookmarks
All times are GMT + 6 Hours
Forum Index -> DFSORT/ICETOOL

 

Similar Topics
Topic Forum Replies
No new posts Shift left VB record without x00 endi... DFSORT/ICETOOL 11
No new posts how to calculate SUM value for VB fil... DFSORT/ICETOOL 1
No new posts how to calculate SUM for VB file usin... JCL & VSAM 1
No new posts Null values are considered in Total c... DFSORT/ICETOOL 6
No new posts ICETOOL to Read records SMF CEF it is... DFSORT/ICETOOL 4
Search our Forums:

Back to Top
IBMMainframes.com is not an official and/or affiliated with IBM® in anyway
Board Rules | FAQ | Downloads | Wiki | SiteMap | Contact Us