IBM Mainframe Forum Index
 
Log In
 
IBM Mainframe Forum Index Mainframe: Search IBM Mainframe Forum: FAQ Register
 

ICETOOL Performance


IBM Mainframe Forums -> DFSORT/ICETOOL
Post new topic   Reply to topic
View previous topic :: View next topic  
Author Message
charanmsrit

New User


Joined: 25 Oct 2007
Posts: 81
Location: Australia

PostPosted: Thu Feb 23, 2012 6:48 am
Reply with quote

Hi,

i guess there is some performance issue in my ICETOOL run. i am just running OCCUR operator to count the unique records in my file.

The records in the input file looks like below. FB and RECL=55. There are around 15 million records

...SASADSDD....CCA00000000000000924875669AAANNNN A ....
...TEWTEWTT....CCA00000000000000125485454BBBYNNNNN ....
...CBCXBCXB....CCA00000000000000654654654CCCYNNNNN ....
..<DFGHFGGF....CCA00000000000000872448524DDDYNNN N ....

my job is still running with elapsed time already 47 mins. strangely, EXCP count is almost 3.5 million already

EXCP Count CPU Time Elapsed
---------- ---------- -----------
3,431,256 19.8 47.04

OCCUR FROM(DD123) LIST(DD124) -
LINES(999) -
TITLE('NUMBER OF CUSTOMERS FOR EACH ACCNT') -
HEADER('ACCNT') ON(27,15,CH) -
HEADER('NO OF RELATED CUSTOMERS') ON(VALCNT,N15)
ICE643I 0 WIDTH OF REPORT IS 0121 BYTES
ICE200I 0 IDENTIFIER FROM CALLING PROGRAM IS 0001
ICE805I 0 JOBNAME: GTS758DB , STEPNAME: STEP020
ICE802I 0 BLOCKSET TECHNIQUE IN CONTROL
ICE201I H RECORD TYPE IS F - DATA STARTS IN POSITION 1
ICE751I 0 C5-K62149 C6-K90026 C7-K58148 C8-K90026 E9-K60824 C9-BASE E5-K62149
ICE143I 0 BLOCKSET SORT TECHNIQUE SELECTED


ICE193I 0 ICEAM2 INVOCATION ENVIRONMENT IN EFFECT - ICEAM2 ENVIRONMENT SELECTED
ICE252I 1 PARMLIB OPTIONS WERE MERGED WITH INSTALLATION MODULE DEFAULTS
ICE088I 1 XXX1118DB.STEP020 . , INPUT LRECL = 55, BLKSIZE = 27995, TYPE =
ICE093I 0 MAIN STORAGE = (MAX,16777216,16777216)
ICE156I 0 MAIN STORAGE ABOVE 16MB = (16736240,16719856)
ICE127I 0 OPTIONS: OVFLO=RC0 ,PAD=RC0 ,TRUNC=RC0 ,SPANINC=RC16,VLSCMP=N,SZERO=Y
ICE128I 0 OPTIONS: SIZE=16777216,MAXLIM=1048576,MINLIM=450560,EQUALS=N,LIST=Y,E
ICE129I 0 OPTIONS: VIO=N,RESDNT=ALL ,SMF=FULL ,WRKSEC=Y,OUTSEC=Y,VERIFY=N,CHALT=N,DYNALOC=(SYSDA ,008),ABCODE=MSG

ICE130I 0 OPTIONS: RESALL=12288,RESINV=0,SVC=109 ,CHECK=N,WRKREL=Y,OUTREL=Y,CKP
ICE131I 0 OPTIONS: TMAXLIM=16777216,ARESALL=0,ARESINV=0,OVERRGN=16384,CINV=Y,CF
ICE132I 0 OPTIONS: VLSHRT=N,ZDPRINT=Y,IEXIT=N,TEXIT=N,LISTX=N,EFS=NONE ,EXIT
ICE133I 0 OPTIONS: HIPRMAX=OPTIMAL,DSPSIZE=32 ,ODMAXBF=0,SOLRF=Y,VLLONG=N,VSAM
ICE235I 0 OPTIONS: NULLOUT=RC0
ICE236I 0 OPTIONS: DYNAPCT=25 ,MOWRK=N
ICE084I 0 EXCP ACCESS METHOD USED FOR DD123
ICE750I 0 DC 835510775 TC 0 CS DSVNN KSZ 15 VSZ 15
ICE887I 0 CSES 0,0,0 ES 0,0,0
ICE886I 0 SYS 0 TSTG 0 FS 0 INIT 0 MAX 0 LEN 0
ICE752I 0 FSZ=15191105 RC IGN=0 E AVG=28 0 WSP=552458 C DYN=0 0

Could someone advise if this is normal?
Back to top
View user's profile Send private message
Frank Yaeger

DFSORT Developer


Joined: 15 Feb 2005
Posts: 7129
Location: San Jose, CA

PostPosted: Thu Feb 23, 2012 7:23 am
Reply with quote

It's not clear what you're using for //DD123 and //DD124. Please show your complete JCL for the ICETOOL job.

The number of unique accounts (= number of output records) could make a big difference too. DFSORT uses EXCP for the input file, but has to use BSAM for the output file. BSAM counts EXCPs in a funny way which makes the counts look really large.

Do you know how many unique accounts you have?
Back to top
View user's profile Send private message
charanmsrit

New User


Joined: 25 Oct 2007
Posts: 81
Location: Australia

PostPosted: Thu Feb 23, 2012 7:30 am
Reply with quote

Hi Frank,

Here is the complete JCL

//AAA111DB JOB (AAAAAA,AAAAAAAA),MSGCLASS=V,CLASS=E,NOTIFY=&SYSUID, *
// REGION=0M
//*MAIN SYSTEM=XXXXXXX
//*--------------------------------------------------------------------
//* -
//*--------------------------------------------------------------------
//*STEP000 : DELETE OUTPUT FILES -
//* -
//*--------------------------------------------------------------------
//STEP000 EXEC PGM=IDCAMS
//SYSPRINT DD SYSOUT=*
//SYSOUT DD SYSOUT=*
//SYSIN DD *
DELETE XXXXXX.SHARED.XXXX.AAA111.OUTPUT2 PURGE
SET MAXCC=0
/*
//*--------------------------------------------------------------------
//STEP020 EXEC PGM=ICETOOL,COND=(0,NE)
//TOOLMSG DD SYSOUT=*
//DFSMSG DD SYSOUT=*
//DD123 DD DISP=SHR,DSN=XXXXXX.SHARED.XXXX.AAA111.OUTPUT1
//DD124 DD DISP=(NEW,CATLG,CATLG),
// DSN=XXXXXX.SHARED.XXXX.AAA111.OUTPUT2,
// SPACE=(CYL,(200,250),RLSE)
//TOOLIN DD *
OCCUR FROM(DD123) LIST(DD124) -
LINES(999) -
TITLE('NUMBER OF CUSTOMERS FOR EACH ACCNT') -
HEADER('ACCNT') ON(27,15,CH) -
HEADER('NO OF RELATED CUSTOMERS') ON(VALCNT,N15)
/*
//
Back to top
View user's profile Send private message
gcicchet

Senior Member


Joined: 28 Jul 2006
Posts: 1702
Location: Australia

PostPosted: Thu Feb 23, 2012 7:41 am
Reply with quote

Hi,

try adding this
Code:
//            DCB=(RECFM=FBA,LRECL=121) 


I think the output file DD124 it's defaulting to LRECL=121 BLKSIZE=121


Gerry
Back to top
View user's profile Send private message
charanmsrit

New User


Joined: 25 Oct 2007
Posts: 81
Location: Australia

PostPosted: Thu Feb 23, 2012 7:54 am
Reply with quote

sorry Frank, forgot to answer your second question.

i am expecting anywhere between 6 million to 10 million unique records.

i cancelled the job after it crossed 5 million EXCP's with elapsed time 75 minutes. i reran with DISP=(NEW,CATLG,CATLG) for output file //DD124. cancelled after elapsed time 10 minutes. i got nearly 750000 records in the output. have i got something wrong in the SYSIN?

from looking at the Dataset informtaion of output file, i see that the file has been allocated with
Dsorg Recfm Lrecl Blksz
--------------------------
PS FBA 121 121

my understanding was that the DFSORT will allocate the best BLKSIZE. why is it 121? is it the cause for such high EXCP's
Back to top
View user's profile Send private message
charanmsrit

New User


Joined: 25 Oct 2007
Posts: 81
Location: Australia

PostPosted: Thu Feb 23, 2012 8:13 am
Reply with quote

Thanks Gerry,

i changed my JCL with DCB for output file as

// LRECL=121,
// BLKSIZE=0,
// DSORG=PS,
// RECFM=FB,
// SPACE=(CYL,(200,250),RLSE)

It ran in less than 2 mins
-Total CPU Time=00:00:10.64 Elapsed Time=00:01:45.80

i always thought that DFSORT will allocate the optimal LRECL and BLKSIZE for performance. Why is this an exception?
Back to top
View user's profile Send private message
Frank Yaeger

DFSORT Developer


Joined: 15 Feb 2005
Posts: 7129
Location: San Jose, CA

PostPosted: Thu Feb 23, 2012 9:10 am
Reply with quote

For OCCUR, the default is to use the SDBMSG installation option. The default is NO, so you won't get optimal blocking unless you override that. You can do that with the LISTSDB option:

OCCUR ... LISTSDB

which overrides the default with SDBMSG=YES (SDB blocksize used).

BLKSIZE=LRECL=121 would certainly have a bad effect on the number of EXCPs.

Note that for regular output data sets, DFSORT automatically uses the SDB blocksize, but LIST data sets (for DISPLAY, OCCUR) are an exception.

You could ask your System Programmers to change the site default to SDBMSG=YES.
Back to top
View user's profile Send private message
charanmsrit

New User


Joined: 25 Oct 2007
Posts: 81
Location: Australia

PostPosted: Thu Feb 23, 2012 10:13 am
Reply with quote

Thanks for the clarification Frank. I have now verified the defaults in my Shop and found that SDBMSG is set to NO. I will check with the system programmers if they have any reason to keep it so.
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic View Bookmarks
All times are GMT + 6 Hours
Forum Index -> DFSORT/ICETOOL

 


Similar Topics
Topic Forum Replies
No new posts Shift left VB record without x00 endi... DFSORT/ICETOOL 11
No new posts how to calculate SUM value for VB fil... DFSORT/ICETOOL 1
No new posts how to calculate SUM for VB file usin... JCL & VSAM 1
No new posts exploiting Z16 performance PL/I & Assembler 2
No new posts Null values are considered in Total c... DFSORT/ICETOOL 6
Search our Forums:

Back to Top