IBM Mainframe Forum Index
 
Log In
 
IBM Mainframe Forum Index Mainframe: Search IBM Mainframe Forum: FAQ Register
 

RECORD OUT OF SEQUENCE error on VSAM KSDS file..


IBM Mainframe Forums -> JCL & VSAM
Post new topic   Reply to topic
View previous topic :: View next topic  
Author Message
chetanambi

New User


Joined: 21 Jan 2012
Posts: 58
Location: India

PostPosted: Mon Jul 16, 2012 3:00 pm
Reply with quote

Hi All,

We have VSAM KSDS file defined with below attributes:-

Code:

DELETE XXXXX.XX.XXXX.XXXXXXX -                       
    PURGE -                                             
    CLUSTER                                             
DEFINE CLUSTER (NAME(XXXXX.XX.XXXX.XXXXXXX) -         
       VOLUMES(* * * * *) -                               
       FREESPACE(20,20) -                               
       UNIQUE -                                         
       INDEXED -                                       
       CYLINDERS(30,10) -                               
       KEYS(14 1) -                                     
       RECORDSIZE(106 106) -                           
       SHAREOPTIONS(3 3))   -                           
    DATA (  -                                           
      NAME(XXXXX.XX.XXXX.XXXXXXX.DATA) CISZ(4096) ) -
    INDEX (  -                                         
      NAME(XXXXX.XX.XXXX.XXXXXXX.INDEX) CISZ(2048) ) 


Production job is abending with below information while copying data from this VSAM fle to flat file using IDCAMS. I am thinking that this might be related to duplicate records in the VSAM file and may be thats the reason we might be getting RECORD OUT OF SEQUENCE error message. But VSAM KSDS file should not have duplicate record. rite?. How is this possible? I am confused.

Error message from SYSPRINT:--
Code:

IDCAMS  SYSTEM SERVICES                                           TIME: 04:03:38
                                                                               
  REPRO     INFILE(DD1) -                                                       
            OUTFILE(DD2) -                                                     
            REPLACE                                                             
IDC3302I  ACTION ERROR ON XXXXX.XX.XXXX.XXXXXXX                               
IDC3314I **RECORD OUT OF SEQUENCE - KEY FOLLOWS:                               
000000  F0F0F0F0 F0F5F1F4 F6F2F3F0 F9F0                                         
                                                                               
IDC3302I  ACTION ERROR ON XXXXX.XX.XXXX.XXXXXXX                               
IDC3314I **RECORD OUT OF SEQUENCE - KEY FOLLOWS:                               
000000  F0F0F0F0 F0F5F1F4 F3F8F2F3 F1F7                                         
                                                                               
IDC0005I NUMBER OF RECORDS PROCESSED WAS 199784                                 
IDC0001I FUNCTION COMPLETED, HIGHEST CONDITION CODE WAS 8                       
                                                                               
IDC0002I IDCAMS PROCESSING COMPLETE. MAXIMUM CONDITION CODE WAS 8               


Please note that file is already in sorted order.

As a temparory fix we are sorting the DATA component file by removing duplicate record and using this o/p file for further processing. But my question how duplicate records are getting inserted.

Note: This VSAM file is being used in 2 different CICS regions. But even in that case VSAM will handle duplicate condition as per my understanding or is this the reason for the issue?

Please advice..

Regards,
Chetan
Back to top
View user's profile Send private message
expat

Global Moderator


Joined: 14 Mar 2007
Posts: 8797
Location: Welsh Wales

PostPosted: Mon Jul 16, 2012 3:02 pm
Reply with quote

KSDS = unique key, therefor any duplicate keys cause the problem.
Back to top
View user's profile Send private message
PeterHolland

Global Moderator


Joined: 27 Oct 2009
Posts: 2481
Location: Netherlands, Amstelveen

PostPosted: Mon Jul 16, 2012 3:04 pm
Reply with quote

F0F0F0F0 F0F5F1F4 F6F2F3F0 F9F0
0000 0514 6230 90
F0F0F0F0 F0F5F1F4 F3F8F2F3 F1F7
0000 0514 3823 17


For me that is out of sequence.
Back to top
View user's profile Send private message
Robert Sample

Global Moderator


Joined: 06 Jun 2008
Posts: 8696
Location: Dubuque, Iowa, USA

PostPosted: Mon Jul 16, 2012 3:08 pm
Reply with quote

Quote:
I am thinking that this might be related to duplicate records in the VSAM file
This is your first problem -- stop thinking and read the error message. It says the data is out of sequence, which merely means the data being presented is not in ascending sequence by the VSAM key. Nothing about that message indicates or implies that there is duplicate data -- just data out of sequence.

And I think if you really closely study the job, you will find that this error message is coming on the load of the VSAM file, not the unload. There would be no reason to use REPLACE on an unload of a VSAM file.
Back to top
View user's profile Send private message
expat

Global Moderator


Joined: 14 Mar 2007
Posts: 8797
Location: Welsh Wales

PostPosted: Mon Jul 16, 2012 3:20 pm
Reply with quote

Possibly a misconception regarding the definition of the KSDS key, using offset against actual icon_confused.gif
Back to top
View user's profile Send private message
Anuj Dhawan

Superior Member


Joined: 22 Apr 2006
Posts: 6250
Location: Mumbai, India

PostPosted: Mon Jul 16, 2012 3:24 pm
Reply with quote

Your interpretation does not seem correct. This
Quote:
Production job is abending with below information while copying data from this VSAM fle to flat file using IDCAMS.
won't give you the error you show. For QSAM file, in this context, key-sequence does not even matter. Are you sure it's not for Load step, as Robert has also indicated.
Back to top
View user's profile Send private message
chetanambi

New User


Joined: 21 Jan 2012
Posts: 58
Location: India

PostPosted: Mon Jul 16, 2012 4:11 pm
Reply with quote

Peter,
There are number of records between those 2 records which you have mentioned. How it will create out of sequence only for these 2 records? What about other records in between?

Anuj / Robert,
I am also getting duplicate error message sometimes. So got confused while posting. I will post about duplicate detail as well later. As of now i am concerned about out of sequence error.

This is unload step where I am copying from VSAM to flat file using IDCAMS. Please see the below step which is giving abend while unloading:-
Code:

PS010    EXEC  PGM=ADUT515,PARM=IDCAMS   
SYSPRINT DD  SYSOUT=&CLSJ                                           
DD1      DD  DSN=XXXXX.XX.XXXX.XXXXXXX,DISP=SHR                   
DD2      DD  DSN=XXXXX.CA.XXXXXX.UNLOAD(+1),                   
             DISP=(NEW,CATLG,DELETE),                               
             UNIT=STOR,SPACE=(TRK,(1800,250),RLSE),                 
             DCB=(B0.DSCB,RECFM=FB,LRECL=106,BLKSIZE=1060)         
SYSIN    DD  *
 REPRO     INFILE(DD1) -
           OUTFILE(DD2) -
          REPLACE       

My question: How to avoid this error abend. As I have told in previous message as a temparory fix we sorting using data component which is not good programming.

Please let me know if you need any information.
Back to top
View user's profile Send private message
Bill Woodger

Moderator Emeritus


Joined: 09 Mar 2011
Posts: 7309
Location: Inside the Matrix

PostPosted: Mon Jul 16, 2012 5:25 pm
Reply with quote

Read what others have written to you in reply.

You cannot get that message by REPROing any VSAM file to a "flat"/QSAM/PS file,

Look at the error message.

Now, show the messages output from the step you have having a problem with. Unless your "wrapper" program is doing something, the step is NOT abending.

Why would you feel that the presence of intervening records would argue that the data you have shown is not out of sequence anyway.

I think you need to start again. Forget everything you "know" and look at outputs to understand what is happening. You certainly do know "know" which command is failing,
Back to top
View user's profile Send private message
chetanambi

New User


Joined: 21 Jan 2012
Posts: 58
Location: India

PostPosted: Mon Jul 16, 2012 6:41 pm
Reply with quote

Bill,

I am really getting that error message by REPROing VSAM to flat file. The IDCAMS step which I have shown above is from a production job itself. Sometimes job will run without any issues. and is abending only when there are out of sequence records. Could you please tell me where exactly I am wrong.

Expat,
what is misconception you are referring? I didnt understand.

Regards,
Chetan
Back to top
View user's profile Send private message
Robert Sample

Global Moderator


Joined: 06 Jun 2008
Posts: 8696
Location: Dubuque, Iowa, USA

PostPosted: Mon Jul 16, 2012 7:15 pm
Reply with quote

Are you running on a z/OS machine or something else like an AS/400?
Back to top
View user's profile Send private message
chetanambi

New User


Joined: 21 Jan 2012
Posts: 58
Location: India

PostPosted: Mon Jul 16, 2012 8:03 pm
Reply with quote

yes.. Its z/OS machine only..
Back to top
View user's profile Send private message
dick scherrer

Moderator Emeritus


Joined: 23 Nov 2006
Posts: 19244
Location: Inside the Matrix

PostPosted: Mon Jul 16, 2012 9:16 pm
Reply with quote

Hello,

Suggest you start over as your posts are confusing at best. . .

Why a REPLACE when creating an output sequential file?

Why mention duplicates, when the error is an out of sequence?

Suggest you use your sort product and copy the vasm data to a sequential file. Then look at this sequential file for the keys identified in the error. Look at the records immediately before and after the problem records. . .

Post what you find.
Back to top
View user's profile Send private message
expat

Global Moderator


Joined: 14 Mar 2007
Posts: 8797
Location: Welsh Wales

PostPosted: Mon Jul 16, 2012 9:21 pm
Reply with quote

It seems that you are not using NATIVE IDCAMS, what product are you actually using here where you need to specify IDCAMS as a parm value.
Back to top
View user's profile Send private message
vasanthkumarhb

Active User


Joined: 06 Sep 2007
Posts: 275
Location: Bang,iflex

PostPosted: Tue Jul 17, 2012 2:07 am
Reply with quote

I assume there may be one or morel steps involved in your job. This is 100% due to VSAM key is not in ascending order.

do a test as below

1. load VSAM file with sorted sequential file .
2. Then used IDCAMS repro VSAM file to any GDG dataset or flat dataset needed.

Also check your key should be KEY(14,0). Please note that unloading VSAM file to any flat file doesn't give sequence error or duplication error.
Back to top
View user's profile Send private message
chetanambi

New User


Joined: 21 Jan 2012
Posts: 58
Location: India

PostPosted: Tue Jul 17, 2012 11:33 am
Reply with quote

expat,
Even with NATIVE IDCAMS also I am getting same error. That ADUT515 is just a utility which will use IDCAMS.

Yesterday, when I was getting out of sequence, I have sorted the file using data component and loaded that data to VSAM file using delete define command. Today same job abended with duplicate record error. Please see below:-
Code:

IDCAMS  SYSTEM SERVICES                                           TIME: 00:52:20
                                                                               
 REPRO INFILE(INFILE) OUTFILE(OUTFILE)                                         
IDC3302I  ACTION ERROR ON XXXXX.XX.XXXX.XXXXXXX                               
IDC3351I ** VSAM I/O RETURN CODE IS 156 - RPLFDBWD = X'2A08009C'               
IDC3302I  ACTION ERROR ON XXXXX.XX.XXXX.XXXXXXX                               
IDC3308I ** DUPLICATE RECORD - KEY FOLLOWS:                                     
000000  F0F0F0F0 F0F5F1F5 F6F1F2F4 F9F8                                         
                                                                               
IDC31467I MAXIMUM ERROR LIMIT REACHED.                                         
IDC0005I NUMBER OF RECORDS PROCESSED WAS 202177                                 
IDC3003I FUNCTION TERMINATED. CONDITION CODE IS 12                             
                                                                               
IDC0002I IDCAMS PROCESSING COMPLETE. MAXIMUM CONDITION CODE WAS 12             

I tried to view the data in the VSAM file after the record 00000515612498 which is causing the issue. It is displaying data upto that issue record and if i hit 'FP8' to see next records I am getting 'LOGICAL VSAM ERROR' on top right corner.
I have referred IBM manuals on VSAM I/O RETURN CODE IS 156 and RPLFDBWD = X'2A08009C'. Its referring to INVALID CONTROL INTERVAL and they listed below posible reasons-->
1.A key is not greater than the previous key.
2.A key is not in the current control interval.
3.A spanned record RDF is encountered.
4.A freespace pointer is invalid.
5.The number of records does not match a group RDF record count.

I could not figure out which exactly is causing the issue. Please guide me.

And about out of sequence error:- When l get out of sequence error next time I will try without REPLACE option. But I am 100% sure that I am getting out of sequence error while copying from VSAM to flat file. I agree with you guys that we should not use REPLACE option while copying from VSAM to flat file (referred manuals on this).

Regards,
Chetan
Back to top
View user's profile Send private message
expat

Global Moderator


Joined: 14 Mar 2007
Posts: 8797
Location: Welsh Wales

PostPosted: Tue Jul 17, 2012 11:35 am
Reply with quote

When you sort the file to remove duplicates, please show the sort card used in that job
Back to top
View user's profile Send private message
chetanambi

New User


Joined: 21 Jan 2012
Posts: 58
Location: India

PostPosted: Tue Jul 17, 2012 11:39 am
Reply with quote

Vasanth,

Yes.. there are more steps in my job. This is what it doing:-
Step1: Copy VSAM fiel to flat file using REPRO.
Step2: Process the flat file and create o/p file out of it.
Step 3: Delete and Define the VSAM.
Step 4: Load data from o/p of step 2 to VSAM file using REPRO (with REPLACE option).

Regards,
Chetan
Back to top
View user's profile Send private message
PeterHolland

Global Moderator


Joined: 27 Oct 2009
Posts: 2481
Location: Netherlands, Amstelveen

PostPosted: Tue Jul 17, 2012 12:16 pm
Reply with quote

Quote:

I have referred IBM manuals on VSAM I/O RETURN CODE IS 156 and RPLFDBWD = X'2A08009C'. Its referring to INVALID CONTROL INTERVAL and they listed below posible reasons-->
1.A key is not greater than the previous key.
2.A key is not in the current control interval.
3.A spanned record RDF is encountered.
4.A freespace pointer is invalid.
5.The number of records does not match a group RDF record count.


Do a VERIFY and try again.
Back to top
View user's profile Send private message
chetanambi

New User


Joined: 21 Jan 2012
Posts: 58
Location: India

PostPosted: Tue Jul 17, 2012 1:17 pm
Reply with quote

1. I have used data component to copy to flat file from VSAM (without REPLACE option):
This I have already posted in previous posts.

2. Using SYNCSORT removed duplicates from above file
Code:

SYSIN :                 
 SORT FIELDS=(2,14,CH,A)
 SUM FIELDS=NONE,XSUM   


3. Then loaded file without duplicates to VSAM

Peter,
File is already used by production job. I am trying to close file in production and do VERIFY. Hope it will not cause any problem. I will get back to you soon.

Thanks,
Chetan
Back to top
View user's profile Send private message
chetanambi

New User


Joined: 21 Jan 2012
Posts: 58
Location: India

PostPosted: Tue Jul 17, 2012 2:48 pm
Reply with quote

Peter,

Even VERIFY is throwing same duplicate error message. Please see below:-
Code:

IDCAMS  SYSTEM SERVICES                                           TIME: 05:07:16
                                                                               
 VERIFY DATASET(XXXXX.XX.XXXX.XXXXXXX)                                       
IDC3302I  ACTION ERROR ON XXXXX.XX.XXXX.XXXXXXX                               
IDC3351I ** VSAM I/O RETURN CODE IS 156 - RPLFDBWD = X'D708009C'               
IDC3003I FUNCTION TERMINATED. CONDITION CODE IS 12                             
                                                                               
IDC0002I IDCAMS PROCESSING COMPLETE. MAXIMUM CONDITION CODE WAS 12             

Please suggest me..

Thanks,
Chetan
Back to top
View user's profile Send private message
chetanambi

New User


Joined: 21 Jan 2012
Posts: 58
Location: India

PostPosted: Tue Jul 17, 2012 2:55 pm
Reply with quote

Please see the below is LISTCAT o/p as well. File was not being used when taking LISTCAT output:
Code:

IDCAMS  SYSTEM SERVICES                                           TIME: 05:20:32
                                                                               
 LISTCAT -                                                                     
  ENTRIES (XXXXX.XX.XXXX.XXXXXXX) -                                           
  CLUSTER -                                                                     
  ALL                                                                           
CLUSTER ------- XXXXX.XX.XXXX.XXXXXXX                                         
     IN-CAT --- CATALOG.VVCATPW                                                 
     HISTORY                                                                   
       DATASET-OWNER-----(NULL)     CREATION--------2012.195                   
       RELEASE----------------2     EXPIRATION------0000.000                   
     SMSDATA                                                                   
       STORAGECLASS ----OLCACHE     MANAGEMENTCLASS----OLSTD                   
       DATACLASS ---------OLSTD     LBACKUP ---0000.000.0000                   
       CA-RECLAIM---------(YES)                                                 
       EATTR-------------(NULL)                                                 
       BWO STATUS------00000000     BWO TIMESTAMP---00000 00:00:00.0           
       BWO---------------(NULL)                                                 
     RLSDATA                                                                   
       LOG ----------------(NULL)   RECOVERY REQUIRED --(NO)     FRLOG ---------
       VSAM QUIESCED -------(NO)    RLS IN USE ---------(NO)                   
       LOGSTREAMID-----------------------------(NULL)                           
       RECOVERY TIMESTAMP LOCAL-----X'0000000000000000'                         
       RECOVERY TIMESTAMP GMT-------X'0000000000000000'                         
     PROTECTION-PSWD-----(NULL)     RACF----------------(NO)                   
     ASSOCIATIONS                                                               
       DATA-----XXXXX.XX.XXXX.XXXXXXX.DATA                                   
       INDEX----XXXXX.XX.XXXX.XXXXXXX.INDEX                                   
IDCAMS  SYSTEM SERVICES                                           TIME: 05:20:32
         THE NUMBER OF ENTRIES PROCESSED WAS:                                   
                   AIX -------------------0                                     
                   ALIAS -----------------0                                     
                   CLUSTER ---------------1                                     
                   DATA ------------------0                                     
                   GDG -------------------0                                     
                   INDEX -----------------0                                     
                   NONVSAM ---------------0                                     
                   PAGESPACE -------------0                                     
                   PATH ------------------0                                     
                   SPACE -----------------0                                     
                   USERCATALOG -----------0                                     
                   TAPELIBRARY -----------0                                     
                   TAPEVOLUME ------------0                                     
                   TOTAL -----------------1                                     
         THE NUMBER OF PROTECTED ENTRIES SUPPRESSED WAS 0                       
IDC0001I FUNCTION COMPLETED, HIGHEST CONDITION CODE WAS 0                       
                                                                               
IDC0002I IDCAMS PROCESSING COMPLETE. MAXIMUM CONDITION CODE WAS 0                 
Back to top
View user's profile Send private message
PeterHolland

Global Moderator


Joined: 27 Oct 2009
Posts: 2481
Location: Netherlands, Amstelveen

PostPosted: Tue Jul 17, 2012 3:01 pm
Reply with quote

chetanambi wrote:
Peter,

Even VERIFY is throwing same duplicate error message. Please see below:-
Code:

IDCAMS  SYSTEM SERVICES                                           TIME: 05:07:16
                                                                               
 VERIFY DATASET(XXXXX.XX.XXXX.XXXXXXX)                                       
IDC3302I  ACTION ERROR ON XXXXX.XX.XXXX.XXXXXXX                               
IDC3351I ** VSAM I/O RETURN CODE IS 156 - RPLFDBWD = X'D708009C'               
IDC3003I FUNCTION TERMINATED. CONDITION CODE IS 12                             
                                                                               
IDC0002I IDCAMS PROCESSING COMPLETE. MAXIMUM CONDITION CODE WAS 12             

Please suggest me..

Thanks,
Chetan


This is another error (see the RPLFDBWD) and is described in "VSAM Demystified"
Back to top
View user's profile Send private message
chetanambi

New User


Joined: 21 Jan 2012
Posts: 58
Location: India

PostPosted: Tue Jul 17, 2012 3:54 pm
Reply with quote

Peter,

I have looked into "VSAM Demystified" and if you see my previous messages I am doing the same 4 steps they have mentioned to correct corrupted index. But we are getting same abend occurs after few days or weeks or months.
 For recovering the index:
– Use the REPRO Command to copy just the data component of the KSDS
to a sequential data set. Specify the data component name (not the
Cluster Name) in the REPRO INFILE parameter.
– Sort the sequential data set by key.
– DELETE and reDEFINE the damaged Cluster.
– REPRO from the Sorted Sequential File to the newly defined Cluster.
Record Management will rebuild the Index Component.

But I have noticed one thing in step 1 that it is copying upto the record which is causing the issue. Thru LISTCAT i found that its having 211310 records. But step 1 is copying only 202317 (and still thows return code 156).

My question is VSAM KSDS should not have out of sequence or duplicate records. rite? Why is this happening? How to avoid this?

Thanks,
Chetan
Back to top
View user's profile Send private message
PeterHolland

Global Moderator


Joined: 27 Oct 2009
Posts: 2481
Location: Netherlands, Amstelveen

PostPosted: Tue Jul 17, 2012 4:03 pm
Reply with quote

Quote:
But we are getting same abend occurs after few days or weeks or months.


Well there must be somewhere a program that is f...ing your KSDS. So i think that you have to check your KSDS after every time it is used.
Back to top
View user's profile Send private message
Bill Woodger

Moderator Emeritus


Joined: 09 Mar 2011
Posts: 7309
Location: Inside the Matrix

PostPosted: Tue Jul 17, 2012 4:36 pm
Reply with quote

I think Peter is spot on.

I've never seen anything like that, but something is trashing your KSDS such that the index is obtaining records out-of-sequence, at least on serial processing.

This explains how REPRO to a QSAM file gets your message, athough I never realised you could get that message from a sequence error on the input dataset.

Obviously you have a serious problem which may involve loss of data. As Peter suggests, check the KSDS after each completed use to find which particular program/s might be to blame and then get into that/those and find out what is going on.

Anything with Assembler accessing the KSDS? Especially if doing anything "clever"?

Just for marking something off, have you tried IDCAMS with EXEC PGM=IDCAMS rather than the "wrapper"?

VERIFY seems to spot the error, so you could try that, although I think I'd go for the REPRO as you know for certain it catches "something".

Good luck. Let us know the resolution please, as it increases our knowledge and may help someone with a similar problem in the future.

If you have more questions, someone'll be here.

Get your support/techies onto this immediately, make sure your boss is aware, there could be very serious implications. Review fault-reporting system for things which might relate to missing data or wrong data.
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic View Bookmarks
All times are GMT + 6 Hours
Forum Index -> JCL & VSAM Goto page 1, 2, 3  Next

 


Similar Topics
Topic Forum Replies
No new posts How to split large record length file... DFSORT/ICETOOL 9
No new posts Error to read log with rexx CLIST & REXX 11
No new posts Extracting Variable decimal numbers f... DFSORT/ICETOOL 17
No new posts Error when install DB2 DB2 2
No new posts SFTP Issue - destination file record ... All Other Mainframe Topics 2
Search our Forums:

Back to Top