i guess there is some performance issue in my ICETOOL run. i am just running OCCUR operator to count the unique records in my file.
The records in the input file looks like below. FB and RECL=55. There are around 15 million records
...SASADSDD....CCA00000000000000924875669AAANNNN A ....
...TEWTEWTT....CCA00000000000000125485454BBBYNNNNN ....
...CBCXBCXB....CCA00000000000000654654654CCCYNNNNN ....
..<DFGHFGGF....CCA00000000000000872448524DDDYNNN N ....
my job is still running with elapsed time already 47 mins. strangely, EXCP count is almost 3.5 million already
EXCP Count CPU Time Elapsed
---------- ---------- -----------
3,431,256 19.8 47.04
OCCUR FROM(DD123) LIST(DD124) -
LINES(999) -
TITLE('NUMBER OF CUSTOMERS FOR EACH ACCNT') -
HEADER('ACCNT') ON(27,15,CH) -
HEADER('NO OF RELATED CUSTOMERS') ON(VALCNT,N15)
ICE643I 0 WIDTH OF REPORT IS 0121 BYTES
ICE200I 0 IDENTIFIER FROM CALLING PROGRAM IS 0001
ICE805I 0 JOBNAME: GTS758DB , STEPNAME: STEP020
ICE802I 0 BLOCKSET TECHNIQUE IN CONTROL
ICE201I H RECORD TYPE IS F - DATA STARTS IN POSITION 1
ICE751I 0 C5-K62149 C6-K90026 C7-K58148 C8-K90026 E9-K60824 C9-BASE E5-K62149
ICE143I 0 BLOCKSET SORT TECHNIQUE SELECTED
Joined: 15 Feb 2005 Posts: 7129 Location: San Jose, CA
It's not clear what you're using for //DD123 and //DD124. Please show your complete JCL for the ICETOOL job.
The number of unique accounts (= number of output records) could make a big difference too. DFSORT uses EXCP for the input file, but has to use BSAM for the output file. BSAM counts EXCPs in a funny way which makes the counts look really large.
sorry Frank, forgot to answer your second question.
i am expecting anywhere between 6 million to 10 million unique records.
i cancelled the job after it crossed 5 million EXCP's with elapsed time 75 minutes. i reran with DISP=(NEW,CATLG,CATLG) for output file //DD124. cancelled after elapsed time 10 minutes. i got nearly 750000 records in the output. have i got something wrong in the SYSIN?
from looking at the Dataset informtaion of output file, i see that the file has been allocated with
Dsorg Recfm Lrecl Blksz
--------------------------
PS FBA 121 121
my understanding was that the DFSORT will allocate the best BLKSIZE. why is it 121? is it the cause for such high EXCP's
Joined: 15 Feb 2005 Posts: 7129 Location: San Jose, CA
For OCCUR, the default is to use the SDBMSG installation option. The default is NO, so you won't get optimal blocking unless you override that. You can do that with the LISTSDB option:
OCCUR ... LISTSDB
which overrides the default with SDBMSG=YES (SDB blocksize used).
BLKSIZE=LRECL=121 would certainly have a bad effect on the number of EXCPs.
Note that for regular output data sets, DFSORT automatically uses the SDB blocksize, but LIST data sets (for DISPLAY, OCCUR) are an exception.
You could ask your System Programmers to change the site default to SDBMSG=YES.
Thanks for the clarification Frank. I have now verified the defaults in my Shop and found that SDBMSG is set to NO. I will check with the system programmers if they have any reason to keep it so.