IBM Mainframe Forum Index
 
Log In
 
IBM Mainframe Forum Index Mainframe: Search IBM Mainframe Forum: FAQ Register
 

Reg data loss if input and output dataset is same in SORT


IBM Mainframe Forums -> DFSORT/ICETOOL
Post new topic   Reply to topic
View previous topic :: View next topic  
Author Message
ramsankar rajkumar

New User


Joined: 07 Jun 2011
Posts: 18
Location: Chennai, India

PostPosted: Thu Nov 10, 2011 4:26 pm
Reply with quote

Hi All,
What are all the possibilities will lead to Data loss, if input and output datasets are same in sort.

input file :
0000000

I just want to add one to the seven digit number each time when this run. this file will have only one record.

When job abends for some reason, will data loss occur since the same file is used for INPUT and OUTPUT.

Code:

//STEP0040 EXEC PGM=SORT,REGION=0M,COND=(0,NE)           
//SYSOUT   DD  SYSOUT=*                                   
//SORTIN   DD DSN=&HLQ..CONTROL,DISP=SHR       
//SORTOUT  DD DSN=&HLQ..CONTROL,DISP=SHR       
//SYSIN    DD *
 SORT FIELDS=COPY                                           
 OUTREC  BUILD=(1,7,ZD,ADD,+1,ZD,TO=ZD,LENGTH=7)
/*
 
Back to top
View user's profile Send private message
expat

Global Moderator


Joined: 14 Mar 2007
Posts: 8797
Location: Welsh Wales

PostPosted: Thu Nov 10, 2011 4:53 pm
Reply with quote

Using the same file for input and output in the same step is pure idiocy.
Back to top
View user's profile Send private message
Bill Woodger

Moderator Emeritus


Joined: 09 Mar 2011
Posts: 7309
Location: Inside the Matrix

PostPosted: Thu Nov 10, 2011 5:03 pm
Reply with quote

You are losing data every time it runs, whether or not it abends.

Don't believe me? Run it again. Did you get the expected results? No? Because you lost your input data on the first run.

So, if you ever want to re-run part of your system which uses that file, you first have to think "do I need to do that control-file update, or not?". Crummy.

Have a look at this recent topic.
Back to top
View user's profile Send private message
ramsankar rajkumar

New User


Joined: 07 Jun 2011
Posts: 18
Location: Chennai, India

PostPosted: Thu Nov 10, 2011 5:43 pm
Reply with quote

Yes Bill.

That is expected result. Each time when it runs, it should be incremented by one.

When it abends, control number shouldn't get update. That is my requirement.
Back to top
View user's profile Send private message
Bill Woodger

Moderator Emeritus


Joined: 09 Mar 2011
Posts: 7309
Location: Inside the Matrix

PostPosted: Thu Nov 10, 2011 5:54 pm
Reply with quote

Nope, you've missed the point.


Code:
Input data : 0000006
Output data: 0000007

Input data : 0000007
Output data: 0000008


That is what you want for two runs on Day 7 and Day 8.

However, immediately after running the job, you have lost your input data.

Try it. Pretend you are at the start of day 7. Run the job. Run it again. Now what day is it?

To make the job re-runnable, you have to back-up the dataset before running the job, and you need a restore for when you are doing a re-run.

If you want to run the risk of accidental submission/selecting the wrong file for a re-run, I don't see why you are worried about an abend in the step. It is a one-record file, what do you think can go wrong with it? Space problem? S0C7?

OK, yes you can have the job fail. My point is not that you can't, but if you are worred about that why are you not more worried about destroying your input file? Much more likely to be problematic than the unaided chance of a failure in the job.
Back to top
View user's profile Send private message
ramsankar rajkumar

New User


Joined: 07 Jun 2011
Posts: 18
Location: Chennai, India

PostPosted: Thu Nov 10, 2011 6:00 pm
Reply with quote

Thanks Bill.

Actually i believe nothing wrong will happen to this step. But i have to Convince others.
Back to top
View user's profile Send private message
enrico-sorichetti

Superior Member


Joined: 14 Mar 2007
Posts: 10872
Location: italy

PostPosted: Thu Nov 10, 2011 6:05 pm
Reply with quote

Quote:
Actually i believe nothing wrong will happen to this step. But i have to Convince others.


even if SORT allows using the same dataset for input and output
it still a wrong practice and the DFSORT developers have may times advised not to do it

so all You want to do is to convince other to do something, by any good practice, wrong
Back to top
View user's profile Send private message
Anuj Dhawan

Superior Member


Joined: 22 Apr 2006
Posts: 6250
Location: Mumbai, India

PostPosted: Thu Nov 10, 2011 7:41 pm
Reply with quote

I, too, don't advocate using same file as input and output; but then just yesterday one of the other co-worker was beating around the same bush (-sh*t) and I said use (OLD,KEEP,KEEP) for SORTOUT in the hope of not loosing the DSN but then it was a very poor suggestion.

If I remember correctly, DFSORT does NOT support the use of the same data set for SORTIN and SORTOUT for a COPY operation, which is what your SORT application is doing. For a COPY operation, DFSORT opens the SORTIN and SORTOUT data sets in parallel. It reads a record from the data set and then writes that record to the same data set. Obviously, this can result in messing up the data set. It may work sometimes, but it is NOT guaranteed to work. You're plotting a time-bomb, when time it not right - it gets risky.

By the way, your SORT statements are not correct, you don't need the red text ZD:
Quote:
OUTREC BUILD=(1,7,ZD,ADD,+1,ZD,TO=ZD,LENGTH=7)
Back to top
View user's profile Send private message
Bill Woodger

Moderator Emeritus


Joined: 09 Mar 2011
Posts: 7309
Location: Inside the Matrix

PostPosted: Thu Nov 10, 2011 8:05 pm
Reply with quote

And the REGION=0M? What is that for?

I've never heard of any "control file" that is just bodged-about with a sort or any other utility. For any type of run-control file, you need a connection to the previous day, and you need a business date. You need control information updated when the schedule is "cut", and you need to verify that the whole lot hangs together logically - meaning also that the program maintaining the control file has to know about working days and holidays, as they apply to your system.

If you are looking for a conceptual shift to make this work, just call it a NOCONTROL file. It'll avoid any false sense of security, and you can mess around with it as you like.

I imagine you'll change your COPY so that it will do an actual sort, just bodge your way past that one. Remind the OPS in the run instructions to "NUMBER OFF" when they have to edit the dataset. Wouldn't that be fun?
Back to top
View user's profile Send private message
enrico-sorichetti

Superior Member


Joined: 14 Mar 2007
Posts: 10872
Location: italy

PostPosted: Thu Nov 10, 2011 8:15 pm
Reply with quote

even if
Quote:
General Coding Notes

A SORTINnn data set should not be the same as the SORTOUT data set or any OUTFIL data set because this can cause lost or incorrect data or unpredictable results.


I checked with a silly sort snippet
Code:
 ****** ***************************** Top of Data ******************************
 000001 //ENRICO1  JOB NOTIFY=&SYSUID,                                         
 000002 //             MSGLEVEL=(1,1),CLASS=A,MSGCLASS=H                       
 000003 //*                                                                     
 000004 //ICE     EXEC PGM=SORT                                                 
 000005 //SYSOUT    DD SYSOUT=*                                                 
 000006 //SYSPRINT  DD SYSOUT=*                                                 
 000007 //SORTIN    DD DISP=SHR,DSN=ENRICO.TEST.PS1                             
 000008 //SORTOUT   DD DISP=SHR,DSN=ENRICO.TEST.PS1                             
 000009 //SYSIN     DD *                                                       
 000010   SORT FIELDS=COPY                                                     
 ****** **************************** Bottom of Data ****************************
and unfortunately ( my opinion ) sort did not complain

topic is going to be locked pretty soon to prevent further stupidity icon_cool.gif
Back to top
View user's profile Send private message
Akatsukami

Global Moderator


Joined: 03 Oct 2009
Posts: 1788
Location: Bloomington, IL

PostPosted: Thu Nov 10, 2011 8:25 pm
Reply with quote

It is probably overkill in so trivial a case, but is this not exactly the sort of situation for which generation data sets were invented?
Back to top
View user's profile Send private message
dick scherrer

Moderator Emeritus


Joined: 23 Nov 2006
Posts: 19244
Location: Inside the Matrix

PostPosted: Thu Nov 10, 2011 9:14 pm
Reply with quote

Hello,

Quote:
Actually i believe nothing wrong will happen to this step.
Beliefs do not count for much. . .

Actually, well-managed organizations will not permit what you want to do because it is such a bad idea.

As suggested, this is
Quote:
exactly the sort of situation for which generation data sets were invented
Back to top
View user's profile Send private message
Frank Yaeger

DFSORT Developer


Joined: 15 Feb 2005
Posts: 7129
Location: San Jose, CA

PostPosted: Fri Nov 11, 2011 2:21 am
Reply with quote

From the DFSORT Application Programming Guide:

Quote:
For a copy application, the SORTIN data set should not be the same as the SORTOUT data set or any OUTFIL data set because this can cause lost or incorrect data or unpredictable results.

For a sort application, the SORTIN data set should not be the same as any SORTWKdd data set because this can cause lost or incorrect data or unpredictable results. The SORTIN data set can be the same as the SORTOUT data set or an OUTFIL data set, but this situation can lead to the loss of the data set if the sort application does not end successfully.


So you should NOT use the same input and output data set for a COPY application.

You can use the same input and output data set for a SORT application, but it's usually NOT a good idea.
Back to top
View user's profile Send private message
ramsankar rajkumar

New User


Joined: 07 Jun 2011
Posts: 18
Location: Chennai, India

PostPosted: Fri Nov 11, 2011 2:36 pm
Reply with quote

Thanks all.

I will keep GDG file to avoid risk.
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic View Bookmarks
All times are GMT + 6 Hours
Forum Index -> DFSORT/ICETOOL

 


Similar Topics
Topic Forum Replies
No new posts How to split large record length file... DFSORT/ICETOOL 10
No new posts Extracting Variable decimal numbers f... DFSORT/ICETOOL 17
No new posts Sortjoin and Search for a String and ... DFSORT/ICETOOL 1
No new posts FINDREP - Only first record from give... DFSORT/ICETOOL 3
No new posts Data set Rec-Cnt and Byte-Cnt Testing & Performance 2
Search our Forums:

Back to Top