huge data is getting missed out while writing on DASD

priyankassharma · New User Joined: 29 May 2007 Posts: 16 Location: mumbai

Hi,

Could somebody please help me knowing the reason that considerable no of records would be missed out while writing to a dasd file .The missing records belongs to one block which is been dropped due to some reason.
The records were read and processed in the application, seems that its an environment issue.

Thanks!

William Thompson · Posted: Thu Aug 16, 2007 4:21 pm

Take a look at the EREP information.

priyankassharma · New User Joined: 29 May 2007 Posts: 16 Location: mumbai

Could you please suggest where can i get that information i.e. EREP information.

Thanks!

dbzTHEdinosauer · Posted: Thu Aug 16, 2007 5:51 pm

Start with this link and then pick a reference guide and user manual, download them to your PC and do a little reading. It will take you a little time, the first time, but then you have added a valuable tool to your knowledge skills.

dbzTHEdinosauer · Posted: Thu Aug 16, 2007 6:13 pm

priyankassharma · New User Joined: 29 May 2007 Posts: 16 Location: mumbai

Thanks a lot .. will go thru it...
I believe it's going to help!

Thanks!

aru_bec · New User Joined: 23 Dec 2005 Posts: 19 Location: Bangalore

This link may help you...

http://publibz.boulder.ibm.com/cgi-bin/bookmgr_OS390/BOOKS/ICE1CA10/1.8.3.1?SHELF=&DT=20050222160456&CASE=

I am copying the a part of the content here for ur quick reference -

Be aware that, in some cases, if a DD statement specifies a data set
for output that is extended to a second or subsequent volume, and
another DD statement within the same step requests the same data set,
only the records on the first volume will be read, and incorrect
output will result.

priyankassharma · New User Joined: 29 May 2007 Posts: 16 Location: mumbai

Thanks for the info ! But the case mentioned above does not fit the scenario that I have.

Have checked the JES output the first thing but didn't get any evitable info
rmation from there.

The block missing is from middle of the file.There are some 57 k records that have been missed out.

have tried generating EREp reports but its didnt run as the access was denied for SYS1.LOGREC dataset.Tried copying the SYS1.LOGREC dataset to test qualifier for running the job, this time the job ran but TOURIST has rendered information that the dataset does not have valid attributes.

Thanks!

dick scherrer · Posted: Mon Aug 20, 2007 11:29 pm

Hello,

How many records does the input file contain?

As was asked before, how are you sure that the records were processed when the file was read? How do you know they were actually written to the output file?

How many records are in the output file?

What does the JES output from the problem job show? What (if any) diagnostic information was provided?

Can a sort or some other program successfully read the file that is missing the 57k records?

Has this been re-run since the problem run?

If you explain the process that "lost" the records as well as the jcl for the problem job, we may be able to offer suggestions.

priyankassharma · New User Joined: 29 May 2007 Posts: 16 Location: mumbai

Hi ,

Please find below the answers to the queries you raised :
Hello,

How many records does the input file contain?
Input file contains 1146557 records.

As was asked before, how are you sure that the records were processed when the file was read? How do you know they were actually written to the output file?

We have a counter inthe code which would increment the count by one on reading each records, moreover we cam to know that the data is missed out by rerunning the job again.
On comparing both the outputs we noticed that first 54 K records were written, then 57 K records were skipped and then 567 K records were written.
No pattern was found in those 57 K missed out records i.e the records were not skipped cause of any logic inside the code.
How many records are in the output file?
621,653 in the file with missed out file and 678,739 in the run which gave the correct output.Moreover in the sysout the no records for both the runs have always been the same but it was missed out while writing to o/p in the first run.

What does the JES output from the problem job show? What (if any) diagnostic information was provided?
When comparing JESYMSG in JHS outputs we see that the DA MSS counts for the outputs are different ex:3608 in first run and 9608 in second run.
We didn't notice anything other than this going wrong.

Can a sort or some other program successfully read the file that is missing the 57k records?
We had a successful run thereafter with the same data and no modifications.
Has this been re-run since the problem run?
Yes and we got expected output i.e the same statistics that SYSOUT rendered through the counts for the records read and written to o/p file.
If you explain the process that "lost" the records as well as the jcl for the problem job, we may be able to offer suggestions.

The file in question is an o/p file with the following definition:
OUTFILE1 DD DSN=&STAT.QW.QW456.OUTFIL1,
DISP=(NEW,CATLG,DELETE),
DCB=AMX.MODLDSCB,
RECFM=VB,
LRECL=10756,
BLKSIZE=0,
BUFNO=20,
UNIT=&SYSDA,
DATACLAS=&DCHUGE
The same O/p file is been used in the next SORT step as input.For the erroneous run this step didnt run as the job abended because of the missed out records under the audit procedure.

Thanks!

dick scherrer · Posted: Wed Aug 22, 2007 5:50 am

Hello,

Quite a mystery. . . .

There is no possibility that a different version of the code ran for the problem run?

Have the SMF records for that run been looked at? If there were temporary or permanent i/o errors, they shou;d be recorded. If there were permanent i/o errors, though, you should have seen them without looking at SMF.

You might check the "volume" where the "bad" output file was created to see if there have any errors reported against that part of the dasd.

It sounds like it failed only that one time and the failure cannot be repeated. Unless some "footprint" is dicsovered, i'm not sure what else you can do to diagnose.

My guess is that somewhere there was an issue with the code, but i don't know how to verify this.

We're here if more info is discovered.

William Thompson · Posted: Wed Aug 22, 2007 6:29 am

priyankassharma · New User Joined: 29 May 2007 Posts: 16 Location: mumbai

Hi,

The whole purpose of doing the analysis is to find a permanent fix such that one or the other thing could be done to avoid this problem in future which had a major impact this time itself.

Am still looking in for some clues somewhere and been thankful to all you guys for providing your valuable inputs.

Was just wondering if there is ome way that I as an application programmer can generate EREP report, as mentioned before have tried doing it but didnt get any success.

Do you have any suggestions over that part also.

All I have looked into is JES , do not know much about SMF .Will be doing analysis in that regard and get back to you if find anything.

Thanks Again!

dick scherrer · Posted: Wed Aug 22, 2007 7:46 pm

You're welcome

Good Luck.

We're here when there is new info and/or questions.

jasorn · Active User Joined: 12 Jul 2006 Posts: 191 Location: USA

I suppose it could be hardware but my money on a logic issue. Usually when I've encountered similar issues in the wild the cause has been either an issue with the code itself or maybe some table that is used to determine how the program runs or passed parms, program not compiled, etc. It doesn't look like the step of having sort or something try to read the 'bad' output file was performed.

And I've NEVER seen counters get incremented improperly

I'll count this one as, "someone made a silly mistake and would rather it not get out".

dbzTHEdinosauer · Posted: Thu Aug 23, 2007 7:05 pm

hardware problems do not usually pick-on one program. If the OP had a hardware problem, sure-as-shoot'n, someone else did also.