IBM Mainframe Forum Index
 
Log In
 
IBM Mainframe Forum Index Mainframe: Search IBM Mainframe Forum: FAQ Register
 

Error during Tape file to Tape file Copy


IBM Mainframe Forums -> JCL & VSAM
Post new topic   Reply to topic
View previous topic :: View next topic  
Author Message
LeonPanokarren

New User


Joined: 17 Jan 2007
Posts: 6
Location: USA

PostPosted: Thu Jan 18, 2007 4:00 am
Reply with quote

Hi,

I am getting an error when I try copying a concatenation of 3 Tape Datasets to another Tape Dataset.

Here is the JCL -

//TIBNRCPY JOB (Account Code),
// 'COPY IBNR IP',
// CLASS=R,
// MSGCLASS=T,NOTIFY=&SYSUID
//* $ACFJ219 ACF2 ACTIVE EIS
//** COPY THE IBNR INPUT FILE INTO A TEST GDG
//CPYIBNR EXEC PGM=IDCAMS
//SYSOUT DD SYSOUT=($,,STAN)
//SYSPRINT DD SYSOUT=($,,STAN)
//INDD DD DSN=TEST.N3F00.CREMIDAA.INF.M0000,DISP=SHR
// DD DSN=TEST.N3F00.CREMIDAA.INF.M0001,DISP=SHR
// DD DSN=TEST.N3F00.CREMIDAA.INF.M0002,DISP=SHR
//OUTDD DD DSN=TEST.N3F00.CREMIDAA(+1),
// DISP=(NEW,CATLG,DELETE),VOL=(,,,20),
// UNIT=T3480,LABEL=RETPD=010,
// DCB=(SYS3.DSCB,RECFM=FB,LRECL=1000,BLKSIZE=0)
//SYSIN DD *
REPRO INFILE(INDD) OUTFILE(OUTDD)
/*

This task fails after around an hour and a half, with the following error -

IDC3302I ACTION ERROR ON TEST.N3F00.CREMIDAA.G0001V00
IDC3309I ** RECORD X'F0F0F0C7E4' NOT WRITTEN. LENGTH INVALID

IDC3302I ACTION ERROR ON TEST.N3F00.CREMIDAA.G0001V00
IDC3309I ** RECORD X'F0F0F0C7D2' NOT WRITTEN. LENGTH INVALID

IDC3302I ACTION ERROR ON TEST.N3F00.CREMIDAA.G0001V00
IDC3309I ** RECORD X'F0F0F0C7D2' NOT WRITTEN. LENGTH INVALID

IDC3302I ACTION ERROR ON TEST.N3F00.CREMIDAA.G0001V00
IDC3309I ** RECORD X'F0F0F0C7D2' NOT WRITTEN. LENGTH INVALID
IDC31467I MAXIMUM ERROR LIMIT REACHED.

IDC0005I NUMBER OF RECORDS PROCESSED WAS 1595555

IDC3003I FUNCTION TERMINATED. CONDITION CODE IS 12

IDC0002I IDCAMS PROCESSING COMPLETE. MAXIMUM CONDITION CODE WAS 12

Useful Information -
1595555 records are the number of records in the First Input Dataset.

I am almost positive that the Record-Length phrase in the error message doesnt have anything to do with the Record Length of the 2nd or the 3rd Input Tape Dataset. If that was to be true, the Concatenation should have thrown an error (I could be wrong!). The following JCL I used to create the Input Tape Datasets should make it clear that the Datasets are of the correct LRECL -

//N3FCRIM2 EXEC PGM=IEBGENER
//SYSPRINT DD SYSOUT=($,,STAN)
//SYSOUT DD SYSOUT=($,,STAN)
//SYSIN DD DUMMY
//SYSUT1 DD DUMMY,DCB=(RECFM=FB,LRECL=1000,BLKSIZE=1000)
//SYSUT2 DD DSN=TEST.N3F00.CREMIDAA.INF.M0001,
// DISP=(,CATLG,DELETE),VOL=(,,,20),
// UNIT=T3480,LABEL=RETPD=0095,
// DCB=(RECFM=FB,LRECL=1000)
//*


Besides, even if we get this to work somehow, the amount of time it takes now (1.5 hours is UNACCEPTABLE). The same process completes in our Production environment in 18 minutes, except that in Production we have NO DD CONCATENATION. There is just one Input Tape Dataset.

The Concatentation together has around 7.4 Million records.
Gurus please rescue!!! This is REAL URGENT!!!!!

Many thanks in advance!
Back to top
View user's profile Send private message
dick scherrer

Moderator Emeritus


Joined: 23 Nov 2006
Posts: 19243
Location: Inside the Matrix

PostPosted: Thu Jan 18, 2007 4:14 am
Reply with quote

Hello,

Check the dcb info for all 3 files.

Even if the lrecl is the same, blksize could be what is causing the error.

Make sure the first file in the concatenation is the file with the largest blksize. If you need the files in the order you've specified, then create an empty dasd dataset with a blksize equal to or larger than the largest of the 3 files you need to input.
Back to top
View user's profile Send private message
dick scherrer

Moderator Emeritus


Joined: 23 Nov 2006
Posts: 19243
Location: Inside the Matrix

PostPosted: Thu Jan 18, 2007 4:18 am
Reply with quote

Hello,

Also, was there a reason to combine the files with IDCAMS rather than IEBGENER or SORT? SORT will usually copy files faster than either IDCAMS or SORT.
Back to top
View user's profile Send private message
LeonPanokarren

New User


Joined: 17 Jan 2007
Posts: 6
Location: USA

PostPosted: Thu Jan 18, 2007 4:35 am
Reply with quote

First up,

Thanks a lot for the quick reply .. that was real fast!!!

I am in the process of reallocating & repopulating these files with all the 7.4 Million records. Just this time, I am going to make sure I have the Block sizes of all them set to 1000.

Since this is going to take some time, I will get back in some time.

Thanks again! You will hear from me regardless of whether this solves the problem.

Leon
Back to top
View user's profile Send private message
dick scherrer

Moderator Emeritus


Joined: 23 Nov 2006
Posts: 19243
Location: Inside the Matrix

PostPosted: Thu Jan 18, 2007 4:41 am
Reply with quote

Hi Leon,

Sounds like a plan icon_smile.gif

If you'd like to touch base later, i'll be back online this evening for a while. Probably around 7 or so Central time.

Good Luck!

Oh, if you can, i'd really recommend using a MUCH larger blocksize. Almost all of the elapsed time is the to the number of physical reads and writes.
Back to top
View user's profile Send private message
Frank Yaeger

DFSORT Developer


Joined: 15 Feb 2005
Posts: 7129
Location: San Jose, CA

PostPosted: Thu Jan 18, 2007 4:44 am
Reply with quote

Leon,

You might want to try DFSORT as Dick suggested. Just replace the IDCAMS step with:

Code:

//CPYIBNR EXEC PGM=ICEMAN
//SYSOUT DD SYSOUT=($,,STAN)
//INDD DD DSN=TEST.N3F00.CREMIDAA.INF.M0000,DISP=SHR
// DD DSN=TEST.N3F00.CREMIDAA.INF.M0001,DISP=SHR
// DD DSN=TEST.N3F00.CREMIDAA.INF.M0002,DISP=SHR
//OUTDD DD DSN=TEST.N3F00.CREMIDAA(+1),
// DISP=(NEW,CATLG,DELETE),VOL=(,,,20),
// UNIT=T3480,LABEL=RETPD=010,
// DCB=(SYS3.DSCB,RECFM=FB,LRECL=1000,BLKSIZE=0)
//DFSPARM DD *
   OPTION COPY,SORTIN=INDD,SORTOUT=OUTDD
/*


Or you could do the same thing with DFSORT's ICETOOL like this:

Code:

//CPYIBNR EXEC PGM=ICETOOL
//TOOLMSG DD SYSOUT=($,,STAN)
//DFSMSG DD SYSOUT=($,,STAN)
//INDD DD DSN=TEST.N3F00.CREMIDAA.INF.M0000,DISP=SHR
// DD DSN=TEST.N3F00.CREMIDAA.INF.M0001,DISP=SHR
// DD DSN=TEST.N3F00.CREMIDAA.INF.M0002,DISP=SHR
//OUTDD DD DSN=TEST.N3F00.CREMIDAA(+1),
// DISP=(NEW,CATLG,DELETE),VOL=(,,,20),
// UNIT=T3480,LABEL=RETPD=010,
// DCB=(SYS3.DSCB,RECFM=FB,LRECL=1000,BLKSIZE=0)
//TOOLIN DD *
COPY FROM(INDD) TO(OUTDD)
/*


Quote:
i'd really recommend using a MUCH larger blocksize


Dick is right and this is very important! 32000 would be a good BLKSIZE in this case. 1000 would be a very inefficient BLKSIZE.
Back to top
View user's profile Send private message
LeonPanokarren

New User


Joined: 17 Jan 2007
Posts: 6
Location: USA

PostPosted: Thu Jan 18, 2007 8:48 am
Reply with quote

Thanks a zillion to both of you!

Frank per your suggestion I ran the following JCL -
//TIBNRCPY JOB (Acct Programer)
// 'COPY IBNR IP',
// CLASS=R,
//*
//CPYIBNR EXEC PGM=ICETOOL
//TOOLMSG DD SYSOUT=($,,STAN)
//DFSMSG DD SYSOUT=($,,STAN)
//INDD DD DSN=TEST.N3F00.CREMIDAA.INF.M0000,DISP=OLD,
// DCB=(RECFM=FB,LRECL=1000,BLKSIZE=32000)
// DD DSN=TEST.N3F00.CREMIDAA.INF.M0001,DISP=SHR
// DD DSN=TEST.N3F00.CREMIDAA.INF.M0002,DISP=SHR
//OUTDD DD DSN=TEST.N3F00.CREMIDAA(+1),
// DISP=(NEW,CATLG,DELETE),VOL=(,,,20),
// UNIT=T3480,LABEL=RETPD=010,
// DCB=(SYS3.DSCB,RECFM=FB,LRECL=1000,BLKSIZE=0)
//TOOLIN DD *
COPY FROM(INDD) TO(OUTDD)
/*


I know I was sounding as if I am going to delete and reallocate and repopulate the 3 files with all the data, but turns out that it is going to take too much time. Meanwhile, I thought of giving this a try.

This Job continues to take a lot of time.

What could be wrong? It has been 25 minutes and there are no signs of this Job finishing.

Just so you know, I tried running the Job without the DCB on the First Input Dataset used for concatenation. It failed with a User ABEND code U016 saying that INDD had different RECFM than OUTDD viz., INDD had RECFM "U". That comes as REAL SURPRISE knowing the JCL I used to create the Tape Dataset.

Please help! This thing is driving me crazy!!!



Leon
Back to top
View user's profile Send private message
dick scherrer

Moderator Emeritus


Joined: 23 Nov 2006
Posts: 19243
Location: Inside the Matrix

PostPosted: Thu Jan 18, 2007 9:19 am
Reply with quote

Hi Leon,

If you specified the 32k blksize for your INPUT data, it will not improve the performance as the actual file(s) are blocked lower. The "undefined" format does look strange, but see below.

Do you still have the sysout form the job(s) that created the 3 files you want to use as input? If you can post them, that may help.

If these are all "tape" files, i'd hope the tape management system has information about the files. Please look at (or ask the folks who do tape support to help) the information about the 3 input files. This is just to make sure they really do have the expected dcb info.

Did this file

Code:
//OUTDD DD DSN=TEST.N3F00.CREMIDAA(+1),


already exist as a gdg in your system? If it did not, you will first have to define the gdg to the system. This may well be what caused the "undefined". When you specified the "SYS3.DSCB" and the definition was not there, that may well have yielded recfm=u.

For tonight, i'd suggest re-running the job with a totally different output dataset name - and not one that is the high level name of the other datasets and not a new gdg. I believe the jcl used was Frank's example - not meant to literally be the jcl to submit. This output file WILL bebefit from the larger blksize.

Good luck and be in touch.
Back to top
View user's profile Send private message
LeonPanokarren

New User


Joined: 17 Jan 2007
Posts: 6
Location: USA

PostPosted: Thu Jan 18, 2007 9:45 am
Reply with quote

Thanks Dick!

Here's the Mainframe SYSOUT for the 2 Tape Input files -

For File INF.M000

DATA SET UTILITY - GENERATE
WER164B 10,844K BYTES OF VIRTUAL STORAGE AVAILABLE, MAX REQUESTED,
WER164B 64K BYTES RESERVE REQUESTED, 242,104 BYTES USED
WER146B 20K BYTES OF EMERGENCY SPACE ALLOCATED
WER108I SYSUT1 : RECFM=FB ; LRECL= 1000; BLKSIZE= 1000
WER110I SYSUT2 : RECFM=FB ; LRECL= 1000; BLKSIZE= 32000
WER211B SYNCSMF CALLED BY SYNCSORT; RC=0000
WER449I SYNCSORT GLOBAL DSM SUBSYSTEM ACTIVE
WER416B BSAM WAS USED FOR SYSUT1
WER416B BSAM WAS USED FOR SYSUT2
WER054I RCD IN 0, OUT 0
WER169I RELEASE 1.1D BATCH 0414 TPF LEVEL 3
WER052I END SYNCSORT - TNF4IBCM,STEP01,N3FCRIM2,DIAG=8C00,CA82,A08E,6C6C,C472,
SYNCSORT COPY FEATURE CALLED - RC = 0

For File INF.M0001
DATA SET UTILITY - GENERATE
WER164B 10,844K BYTES OF VIRTUAL STORAGE AVAILABLE, MAX REQUESTED,
WER164B 64K BYTES RESERVE REQUESTED, 242,104 BYTES USED
WER146B 20K BYTES OF EMERGENCY SPACE ALLOCATED
WER108I SYSUT1 : RECFM=FB ; LRECL= 1000; BLKSIZE= 1000
WER110I SYSUT2 : RECFM=FB ; LRECL= 1000; BLKSIZE= 32000
WER211B SYNCSMF CALLED BY SYNCSORT; RC=0000
WER449I SYNCSORT GLOBAL DSM SUBSYSTEM ACTIVE
WER416B BSAM WAS USED FOR SYSUT1
WER416B BSAM WAS USED FOR SYSUT2
WER054I RCD IN 0, OUT 0
WER169I RELEASE 1.1D BATCH 0414 TPF LEVEL 3
WER052I END SYNCSORT - TNF4IBCM,STEP02,N3FCRIM2,DIAG=8600,C004,AA82,E866,CE7E,
SYNCSORT COPY FEATURE CALLED - RC = 0

For INF.M0002

DATA SET UTILITY - GENERATE
WER164B 10,844K BYTES OF VIRTUAL STORAGE AVAILABLE, MAX REQUESTED,
WER164B 64K BYTES RESERVE REQUESTED, 242,104 BYTES USED
WER146B 20K BYTES OF EMERGENCY SPACE ALLOCATED
WER108I SYSUT1 : RECFM=FB ; LRECL= 1000; BLKSIZE= 1000
WER110I SYSUT2 : RECFM=FB ; LRECL= 1000; BLKSIZE= 32000
WER211B SYNCSMF CALLED BY SYNCSORT; RC=0000
WER449I SYNCSORT GLOBAL DSM SUBSYSTEM ACTIVE
WER416B BSAM WAS USED FOR SYSUT1
WER416B BSAM WAS USED FOR SYSUT2
WER054I RCD IN 0, OUT 0
WER169I RELEASE 1.1D BATCH 0414 TPF LEVEL 3
WER052I END SYNCSORT - TNF4IBCM,STEP03,N3FCRIM2,DIAG=A400,628E,888A,60C4,EC76,
SYNCSORT COPY FEATURE CALLED - RC = 0

I am currently working on your suggestion.

Thanks!
Leon
Back to top
View user's profile Send private message
dick scherrer

Moderator Emeritus


Joined: 23 Nov 2006
Posts: 19243
Location: Inside the Matrix

PostPosted: Thu Jan 18, 2007 10:04 am
Reply with quote

Hi Leon,

The copy jobs look good. The new output files should have the 32k blksize (per the output you posted).

However. . . . The output also shows that you are running SYNCSORT - not the IBM sort (DFSORT). The ICETOOL is part of DFSORT, not SYNCSORT.

It also appears that your site has implemented the "trick" that when a straight copy by "IEBGENER" is detected, the sort is executed instead for better performance.

If you concatenate the 3 "new" output/input files and run another step, you should get the single file you are looking for. . . icon_biggrin.gif
Back to top
View user's profile Send private message
LeonPanokarren

New User


Joined: 17 Jan 2007
Posts: 6
Location: USA

PostPosted: Thu Jan 18, 2007 10:26 am
Reply with quote

I finally have gotten into deleting - reallocating and repopulating the Input Tape Datasets with data - 7.4 Million Records.

I noted one difference in my Utility that populates these 3 Datasets with data against it's Production Counterpart. I have changed my Test Utility so that that's taken care of. I am waiting to see how this helps.

The Block Size of all three files is now 32000 as seen in the Dataset allocation Job.

I will let you know how I make out. I am just hoping it works.

Leon
Back to top
View user's profile Send private message
dick scherrer

Moderator Emeritus


Joined: 23 Nov 2006
Posts: 19243
Location: Inside the Matrix

PostPosted: Thu Jan 18, 2007 10:30 am
Reply with quote

Hi Leon,

Sounds like the movement is forward icon_biggrin.gif

Good luck and i'll check back in the a.m.
Back to top
View user's profile Send private message
LeonPanokarren

New User


Joined: 17 Jan 2007
Posts: 6
Location: USA

PostPosted: Thu Jan 18, 2007 11:52 pm
Reply with quote

Hi,

This finally got resolved. I know you think I must be crazy when I say this, but it looks like it was in the difference I had in the Utitity that fills data in the Datasets that caused the problem. I hadn't put the Tape Dataset name in quotes the last time when I ran the uploads. This was different from Production where in the Utility had quotes in it for the Dataset name.

Wonder how this could have caused the dataset already allocated to have a RECFM of FB to become U?

Spare me my ignorance!

Dick, Frank, I really appreciate all your help in this regard!!

Leon
Back to top
View user's profile Send private message
dick scherrer

Moderator Emeritus


Joined: 23 Nov 2006
Posts: 19243
Location: Inside the Matrix

PostPosted: Fri Jan 19, 2007 1:01 am
Reply with quote

Hi Leon,

One thing that happens with some processes that receive a dsn "without quotes" is that "your id" is placed before the rest of the value you specify.

Go into tso and allocate a dataset via 3.2 - specify the dsn as MYTEST then hit enter. Notice that there is a node added before MYTEST on the Allocate New Data Set screen. PF3 out to prevent actual allocation. Now do this again but specify 'MYTEST' and notice that only MYTEST is in the dsn. This is probably what happened in your case.

Good to hear that it is working icon_biggrin.gif
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic View Bookmarks
All times are GMT + 6 Hours
Forum Index -> JCL & VSAM

 


Similar Topics
Topic Forum Replies
No new posts IEW2315E 1369 Link error All Other Mainframe Topics 3
No new posts Unable to interpret a hex value to De... COBOL Programming 7
No new posts Help on IMS -API URL migrating from P... IMS DB/DC 1
No new posts Copy only TEXT or String from a record SYNCSORT 4
No new posts JES datasets IO Error ABENDS & Debugging 3
Search our Forums:

Back to Top