Inconsistent Results with ICETOOL

SJW · New User Joined: 04 Oct 2005 Posts: 2

I have a production job which until recently performed as expected, using ICETOOL to compare today's file with yesterday's file, and write out a file of today's records which have changed from yesterday (other than changes to the date and a file identifier). Within the last couple of weeks, the results have been inconsistent, as sometimes the change file contains only the changed records as expected, while sometimes it contains basically all of the new file records, whether they have changed or not.

The TOOLIN is as follows:

SORT FROM(RAWFILE) TO(NEWFILE) USING(CTL1)

COPY FROM(OLDFILEI) TO(OLDFILE) USING(CTL2)

SELECT FROM(CONCT) ON(LOAD_DATA1) ON(LOAD_DATA2) ALLDUPS -
DISCARD(CHGFILEO) -
USING(CTL3)

CTL1 is as follows:

SUM FIELDS=NONE
SORT FIELDS=(FDW_COLLATERAL_ID,A,
ASOF_DT,A,
CURRENCY_CD,A)

CTL2 is as follows:

OUTREC FIELDS=(1:ALL_DATA,OLDFLAG)

CTL3 is as follows:

OUTFIL FNAMES=CHGFILEO,
INCLUDE=(RECORD_TYPE,EQ,NEWFLAG),
OUTREC=(1:ALL_DATA)

SYMNAMES is as follows:

*
* FIELD NAMES
*
BUSINESS_UNIT,1,5,CH
FDW_COLLATERAL_ID,*,20,CH
ASOF_DT,*,10,CH
CURRENCY_CD,*,3,CH
*
* CONSTANTS
*
NEWFLAG,'N'
OLDFLAG,'O'
*
*
ALL_DATA,1,286,CH
*
*
LOAD_DATA1,1,25,CH
LOAD_DATE,*,10,CH
LOAD_DATA2,*,245,CH
PROCESS_INSTANCE,*,6,PD
RECORD_TYPE,*,1,CH

The JCL looks like this...

//RAWFILE DD DSN=A
//NEWFILE DD DSN=B
//OLDFILEI DD DSN=C
//OLDFILE DD DSN=&&OLD
//CONCT DD DSN=*.NEWFILE
// DD DSN=*.OLDFILE
//CHGFILEO DD DSN=D

I recently ran this ICETOOL step in test within 15 minutes of each other, getting results from the third step which looked like this

Run 1 (as expected):

ICE055I 0 INSERT 83422, DELETE 83422
ICE054I 0 RECORDS - IN: 165429, OUT: 165429
ICE227I 0 CHGFILEO : DELETED = 164705, REPORT = 0, DATA = 724
ICE228I 0 CHGFILEO : TOTAL IN = 165429, TOTAL OUT = 724

Run 2 (everything):

ICE055I 0 INSERT 165253, DELETE 165253
ICE054I 0 RECORDS - IN: 165429, OUT: 165429
ICE227I 0 CHGFILEO : DELETED = 82874, REPORT = 0, DATA = 82555
ICE228I 0 CHGFILEO : TOTAL IN = 165429, TOTAL OUT = 82555

I have noticed that when the job works as expected, I get a ICE752I message with DYN = x y, where x and y are two numbers, but when it doesn't work as expected, x and y are both zero.

We have other jobs which use the same basic code, with different input files, and they all seem to be working fine.

Scott Winhold.

Frank Yaeger · Posted: Tue Oct 04, 2005 2:33 am

This might be caused by the system restriction related to concatenation that I've mentioned before on this board that's documented in the second bullet at:

publibz.boulder.ibm.com/cgi-bin/bookmgr_OS390/BOOKS/ICE1CA10/1.8.3.1?SHELF=&DT=20050222160456&CASE=

I'd suggest avoiding using concatenation by making these changes which will avoid that system restriction:

SJW · New User Joined: 04 Oct 2005 Posts: 2

Frank,

I tried the job with changes as you suggested, but am still having inconsistent results. With the test files I am using sometimes I get 724 records in CHGFILEO (which is the number I expect), and sometimes I get 82,555 records in CHGFILEO. This is running the exact same JCL with the exact same input files.

The only other thing I have noticed is that for runs where I get 82,555 records in CHGFILEO, I also get the following message displayed:

ICE080I 0 IN MAIN STORAGE SORT

This message does not appear in the runs when I get 724 records.

Frank Yaeger · Posted: Wed Oct 05, 2005 8:23 pm

In order to analize this kind of thing, I'd need to see exactly you're doing and what results you're getting. This is NOT the kind of thing we can do on a helpboard. If you like, add the following to your job:

//SORTDIAG DD DUMMY

and send the JES, TOOLMSG and DFSMSG output for a "good" run and a "bad" run to me offline (yaeger@us.ibm.com) and I'll take a look. Please put "DFSORT" somewhere in your Subject line to catch my attention.