Portal | Manuals | References | Downloads | Info | Programs | JCLs | Master the Mainframes
IBM Mainframe Computers Forums Index
 
Register
 
IBM Mainframe Computers Forums Index Mainframe: Search IBM Mainframe Forum: FAQ Memberlist Usergroups Profile Log in to check your private messages Log in
 

 

ICETOOL Performance

 
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> DFSORT/ICETOOL
View previous topic :: :: View next topic  
Author Message
charanmsrit

New User


Joined: 25 Oct 2007
Posts: 80
Location: Australia

PostPosted: Thu Feb 23, 2012 6:48 am    Post subject: ICETOOL Performance
Reply with quote

Hi,

i guess there is some performance issue in my ICETOOL run. i am just running OCCUR operator to count the unique records in my file.

The records in the input file looks like below. FB and RECL=55. There are around 15 million records

...SASADSDD....CCA00000000000000924875669AAANNNN A ....
...TEWTEWTT....CCA00000000000000125485454BBBYNNNNN ....
...CBCXBCXB....CCA00000000000000654654654CCCYNNNNN ....
..<DFGHFGGF....CCA00000000000000872448524DDDYNNN N ....

my job is still running with elapsed time already 47 mins. strangely, EXCP count is almost 3.5 million already

EXCP Count CPU Time Elapsed
---------- ---------- -----------
3,431,256 19.8 47.04

OCCUR FROM(DD123) LIST(DD124) -
LINES(999) -
TITLE('NUMBER OF CUSTOMERS FOR EACH ACCNT') -
HEADER('ACCNT') ON(27,15,CH) -
HEADER('NO OF RELATED CUSTOMERS') ON(VALCNT,N15)
ICE643I 0 WIDTH OF REPORT IS 0121 BYTES
ICE200I 0 IDENTIFIER FROM CALLING PROGRAM IS 0001
ICE805I 0 JOBNAME: GTS758DB , STEPNAME: STEP020
ICE802I 0 BLOCKSET TECHNIQUE IN CONTROL
ICE201I H RECORD TYPE IS F - DATA STARTS IN POSITION 1
ICE751I 0 C5-K62149 C6-K90026 C7-K58148 C8-K90026 E9-K60824 C9-BASE E5-K62149
ICE143I 0 BLOCKSET SORT TECHNIQUE SELECTED


ICE193I 0 ICEAM2 INVOCATION ENVIRONMENT IN EFFECT - ICEAM2 ENVIRONMENT SELECTED
ICE252I 1 PARMLIB OPTIONS WERE MERGED WITH INSTALLATION MODULE DEFAULTS
ICE088I 1 XXX1118DB.STEP020 . , INPUT LRECL = 55, BLKSIZE = 27995, TYPE =
ICE093I 0 MAIN STORAGE = (MAX,16777216,16777216)
ICE156I 0 MAIN STORAGE ABOVE 16MB = (16736240,16719856)
ICE127I 0 OPTIONS: OVFLO=RC0 ,PAD=RC0 ,TRUNC=RC0 ,SPANINC=RC16,VLSCMP=N,SZERO=Y
ICE128I 0 OPTIONS: SIZE=16777216,MAXLIM=1048576,MINLIM=450560,EQUALS=N,LIST=Y,E
ICE129I 0 OPTIONS: VIO=N,RESDNT=ALL ,SMF=FULL ,WRKSEC=Y,OUTSEC=Y,VERIFY=N,CHALT=N,DYNALOC=(SYSDA ,008),ABCODE=MSG

ICE130I 0 OPTIONS: RESALL=12288,RESINV=0,SVC=109 ,CHECK=N,WRKREL=Y,OUTREL=Y,CKP
ICE131I 0 OPTIONS: TMAXLIM=16777216,ARESALL=0,ARESINV=0,OVERRGN=16384,CINV=Y,CF
ICE132I 0 OPTIONS: VLSHRT=N,ZDPRINT=Y,IEXIT=N,TEXIT=N,LISTX=N,EFS=NONE ,EXIT
ICE133I 0 OPTIONS: HIPRMAX=OPTIMAL,DSPSIZE=32 ,ODMAXBF=0,SOLRF=Y,VLLONG=N,VSAM
ICE235I 0 OPTIONS: NULLOUT=RC0
ICE236I 0 OPTIONS: DYNAPCT=25 ,MOWRK=N
ICE084I 0 EXCP ACCESS METHOD USED FOR DD123
ICE750I 0 DC 835510775 TC 0 CS DSVNN KSZ 15 VSZ 15
ICE887I 0 CSES 0,0,0 ES 0,0,0
ICE886I 0 SYS 0 TSTG 0 FS 0 INIT 0 MAX 0 LEN 0
ICE752I 0 FSZ=15191105 RC IGN=0 E AVG=28 0 WSP=552458 C DYN=0 0

Could someone advise if this is normal?
Back to top
View user's profile Send private message

Frank Yaeger

DFSORT Moderator


Joined: 15 Feb 2005
Posts: 7130
Location: San Jose, CA

PostPosted: Thu Feb 23, 2012 7:23 am    Post subject:
Reply with quote

It's not clear what you're using for //DD123 and //DD124. Please show your complete JCL for the ICETOOL job.

The number of unique accounts (= number of output records) could make a big difference too. DFSORT uses EXCP for the input file, but has to use BSAM for the output file. BSAM counts EXCPs in a funny way which makes the counts look really large.

Do you know how many unique accounts you have?
Back to top
View user's profile Send private message
charanmsrit

New User


Joined: 25 Oct 2007
Posts: 80
Location: Australia

PostPosted: Thu Feb 23, 2012 7:30 am    Post subject:
Reply with quote

Hi Frank,

Here is the complete JCL

//AAA111DB JOB (AAAAAA,AAAAAAAA),MSGCLASS=V,CLASS=E,NOTIFY=&SYSUID, *
// REGION=0M
//*MAIN SYSTEM=XXXXXXX
//*--------------------------------------------------------------------
//* -
//*--------------------------------------------------------------------
//*STEP000 : DELETE OUTPUT FILES -
//* -
//*--------------------------------------------------------------------
//STEP000 EXEC PGM=IDCAMS
//SYSPRINT DD SYSOUT=*
//SYSOUT DD SYSOUT=*
//SYSIN DD *
DELETE XXXXXX.SHARED.XXXX.AAA111.OUTPUT2 PURGE
SET MAXCC=0
/*
//*--------------------------------------------------------------------
//STEP020 EXEC PGM=ICETOOL,COND=(0,NE)
//TOOLMSG DD SYSOUT=*
//DFSMSG DD SYSOUT=*
//DD123 DD DISP=SHR,DSN=XXXXXX.SHARED.XXXX.AAA111.OUTPUT1
//DD124 DD DISP=(NEW,CATLG,CATLG),
// DSN=XXXXXX.SHARED.XXXX.AAA111.OUTPUT2,
// SPACE=(CYL,(200,250),RLSE)
//TOOLIN DD *
OCCUR FROM(DD123) LIST(DD124) -
LINES(999) -
TITLE('NUMBER OF CUSTOMERS FOR EACH ACCNT') -
HEADER('ACCNT') ON(27,15,CH) -
HEADER('NO OF RELATED CUSTOMERS') ON(VALCNT,N15)
/*
//
Back to top
View user's profile Send private message
gcicchet

Senior Member


Joined: 28 Jul 2006
Posts: 1703
Location: Australia

PostPosted: Thu Feb 23, 2012 7:41 am    Post subject:
Reply with quote

Hi,

try adding this
Code:
//            DCB=(RECFM=FBA,LRECL=121) 


I think the output file DD124 it's defaulting to LRECL=121 BLKSIZE=121


Gerry
Back to top
View user's profile Send private message
charanmsrit

New User


Joined: 25 Oct 2007
Posts: 80
Location: Australia

PostPosted: Thu Feb 23, 2012 7:54 am    Post subject:
Reply with quote

sorry Frank, forgot to answer your second question.

i am expecting anywhere between 6 million to 10 million unique records.

i cancelled the job after it crossed 5 million EXCP's with elapsed time 75 minutes. i reran with DISP=(NEW,CATLG,CATLG) for output file //DD124. cancelled after elapsed time 10 minutes. i got nearly 750000 records in the output. have i got something wrong in the SYSIN?

from looking at the Dataset informtaion of output file, i see that the file has been allocated with
Dsorg Recfm Lrecl Blksz
--------------------------
PS FBA 121 121

my understanding was that the DFSORT will allocate the best BLKSIZE. why is it 121? is it the cause for such high EXCP's
Back to top
View user's profile Send private message
charanmsrit

New User


Joined: 25 Oct 2007
Posts: 80
Location: Australia

PostPosted: Thu Feb 23, 2012 8:13 am    Post subject:
Reply with quote

Thanks Gerry,

i changed my JCL with DCB for output file as

// LRECL=121,
// BLKSIZE=0,
// DSORG=PS,
// RECFM=FB,
// SPACE=(CYL,(200,250),RLSE)

It ran in less than 2 mins
-Total CPU Time=00:00:10.64 Elapsed Time=00:01:45.80

i always thought that DFSORT will allocate the optimal LRECL and BLKSIZE for performance. Why is this an exception?
Back to top
View user's profile Send private message
Frank Yaeger

DFSORT Moderator


Joined: 15 Feb 2005
Posts: 7130
Location: San Jose, CA

PostPosted: Thu Feb 23, 2012 9:10 am    Post subject: Reply to: ICETOOL Performance
Reply with quote

For OCCUR, the default is to use the SDBMSG installation option. The default is NO, so you won't get optimal blocking unless you override that. You can do that with the LISTSDB option:

OCCUR ... LISTSDB

which overrides the default with SDBMSG=YES (SDB blocksize used).

BLKSIZE=LRECL=121 would certainly have a bad effect on the number of EXCPs.

Note that for regular output data sets, DFSORT automatically uses the SDB blocksize, but LIST data sets (for DISPLAY, OCCUR) are an exception.

You could ask your System Programmers to change the site default to SDBMSG=YES.
Back to top
View user's profile Send private message
charanmsrit

New User


Joined: 25 Oct 2007
Posts: 80
Location: Australia

PostPosted: Thu Feb 23, 2012 10:13 am    Post subject:
Reply with quote

Thanks for the clarification Frank. I have now verified the defaults in my Shop and found that SDBMSG is set to NO. I will check with the system programmers if they have any reason to keep it so.
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> DFSORT/ICETOOL All times are GMT + 6 Hours
Page 1 of 1

 

Search our Forum:

Similar Topics
Topic Author Forum Replies Posted
No new posts Using ICETOOL, how we can seperate th... bshkris SYNCSORT 5 Tue May 09, 2017 8:33 pm
No new posts Group elements using ICETOOL Learncoholic DFSORT/ICETOOL 2 Fri Apr 14, 2017 11:01 am
No new posts Converting decimal to numeric using I... Balaryan DFSORT/ICETOOL 4 Thu Feb 02, 2017 11:27 pm
No new posts BC, BCR, BRC, BRCL performance steve-myers PL/I & Assembler 0 Fri Dec 23, 2016 7:44 am
No new posts PL/I code tuning/Performance improvement Virendra Shambharkar PL/I & Assembler 4 Mon Dec 05, 2016 11:57 am


Facebook
Back to Top
 
Mainframe Wiki | Forum Rules | Bookmarks | Subscriptions | FAQ | Tutorials | Contact Us