IBM Mainframe Forum Index
 
Log In
 
IBM Mainframe Forum Index Mainframe: Search IBM Mainframe Forum: FAQ Register
 

duplicate removal


IBM Mainframe Forums -> SYNCSORT
Post new topic   Reply to topic
View previous topic :: View next topic  
Author Message
wiprov

New User


Joined: 13 Feb 2008
Posts: 15
Location: Chennai

PostPosted: Fri Sep 26, 2014 11:26 pm
Reply with quote

Hi,

My i/p file has header and detail records as below:

Input File:
Four set of records and first byte - '1' indicates header.
First byte - '5' or '6' indicates detail records.

1HDR1
5detail1
6detail1

1HDR3
5detail3
6detail3

1HDR2
5detail2
6detail2

1HDR3
5detail3
6detail3

Requirement:
If you could notice, 1HDR3 set of records is getting repeated and the duplicate occurrence of that, needs to be removed.

Expected o/p File:

1HDR1
5detail1
6detail1

1HDR3
5detail3
6detail3

1HDR2
5detail2
6detail2

Could you please let me know, is this viable to crack through DFSORT.
Kindly let me know, if you need any other details from my end. Thanks
Back to top
View user's profile Send private message
Rohit Umarjikar

Global Moderator


Joined: 21 Sep 2010
Posts: 2588
Location: NYC,USA

PostPosted: Sat Sep 27, 2014 2:45 am
Reply with quote

Add a SEQNUM at the end of each record, remove duplicates and the sort it back using the SEQNUM.
Back to top
View user's profile Send private message
Bill Woodger

Moderator Emeritus


Joined: 09 Mar 2011
Posts: 7311
Location: Inside the Matrix

PostPosted: Sat Sep 27, 2014 2:54 am
Reply with quote

What is the RECFM and LRECL?

Do you consider it duplicate if the header is duplicate, or need the data be duplicate as well?
Back to top
View user's profile Send private message
wiprov

New User


Joined: 13 Feb 2008
Posts: 15
Location: Chennai

PostPosted: Sat Sep 27, 2014 9:33 am
Reply with quote

RECFM=FB and LRECL=10.


if the header is duplicate, then we can drop the entire set of records.
In our example, second set of 1HDR3 can contain different set of records.
Back to top
View user's profile Send private message
Rohit Umarjikar

Global Moderator


Joined: 21 Sep 2010
Posts: 2588
Location: NYC,USA

PostPosted: Thu Oct 02, 2014 8:18 pm
Reply with quote

While Bill takes a look on this further, Did you give a try on what I have suggested earlier and if you need a sample how to do it then it is verymuch available on Google or on this forum?
Back to top
View user's profile Send private message
Bill Woodger

Moderator Emeritus


Joined: 09 Mar 2011
Posts: 7311
Location: Inside the Matrix

PostPosted: Thu Oct 02, 2014 8:53 pm
Reply with quote

Well, wiprov took it off to another site. Turns out he has SyncSORT as well.
Back to top
View user's profile Send private message
wiprov

New User


Joined: 13 Feb 2008
Posts: 15
Location: Chennai

PostPosted: Thu Oct 02, 2014 10:03 pm
Reply with quote

Hi Rohit,

Sorry for the late response. Adding a sequence number could be a problem, if the same detail records exists in another header (which is not duplicate). We might miss those records in that case...... icon_sad.gif
Back to top
View user's profile Send private message
Rohit Umarjikar

Global Moderator


Joined: 21 Sep 2010
Posts: 2588
Location: NYC,USA

PostPosted: Fri Oct 03, 2014 6:00 am
Reply with quote

I might be missing something,but can you please explain with an example for this statement?

Quote:
if the same detail records exists in another header (which is not duplicate). We might miss those records in that case......


As I guess same set of detail records and header would retain from one of the duplicate that found, like in your example still below sets of records would come.
Code:
HDR3
5detail3
6detail3
Back to top
View user's profile Send private message
Bill Woodger

Moderator Emeritus


Joined: 09 Mar 2011
Posts: 7311
Location: Inside the Matrix

PostPosted: Fri Oct 03, 2014 10:54 am
Reply with quote

Quote:
if the header is duplicate, then we can drop the entire set of records.
In our example, second set of 1HDR3 can contain different set of records.
Back to top
View user's profile Send private message
Rohit Umarjikar

Global Moderator


Joined: 21 Sep 2010
Posts: 2588
Location: NYC,USA

PostPosted: Thu Oct 09, 2014 5:23 am
Reply with quote

Right, I looked today closey and yes the sequence number option wouldn't fit here..

wiprov, have you got any solution or you still open to the options?
Back to top
View user's profile Send private message
saiprasadh

Active User


Joined: 20 Sep 2006
Posts: 154
Location: US

PostPosted: Fri Oct 10, 2014 11:39 am
Reply with quote

Wiprov,

I used below mentioned sort card to obtain the result.

Code:
 SORT FIELDS=COPY                                           
 INREC  IFTHEN=(WHEN=INIT,                                 
        BUILD=(1:1,10,81:20C'0')),                         
        IFTHEN=(WHEN=GROUP,BEGIN=(1,4,CH,EQ,C'1HDR'),       
        PUSH=(81:ID=10)),                                   
        IFTHEN=(WHEN=GROUP,BEGIN=(1,5,CH,EQ,C'1HDR3'),     
        END=(1,4,CH,EQ,C'1HDR',AND,1,5,CH,NE,C'1HDR3'),     
        PUSH=(91:ID=10))                                   
 OUTFIL FNAMES=SORTOUT,                                     
 INCLUDE=((2,3,CH,EQ,C'DET',AND,91,10,ZD,EQ,0),OR,         
          (1,4,CH,EQ,C'1HDR',AND,1,5,CH,NE,C'1HDR3'),OR,   
          (1,5,CH,EQ,C'1HDR3',AND,91,10,ZD,EQ,1),OR,       
          (2,3,CH,EQ,C'DET',AND,91,10,ZD,EQ,1)),           
 OUTREC=(1:1,10) 


Input:
Code:
1HDR1       
5DETAIL1     
6DETAIL1     
1HDR3       
5DETAIL3     
6DETAIL3     
5DETAIL3     
6DETAIL3     
1HDR3       
5DETAIL3     
6DETAIL3     
1HDR2       
5DETAIL2     
6DETAIL2     
1HDR3       
5DETAIL3     
6DETAIL3


Output:

Code:
1HDR1       
5DETAIL1     
6DETAIL1     
1HDR3       
5DETAIL3     
6DETAIL3     
5DETAIL3     
6DETAIL3     
1HDR2       
5DETAIL2     
6DETAIL2 
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic View Bookmarks
All times are GMT + 6 Hours
Forum Index -> SYNCSORT

 


Similar Topics
Topic Forum Replies
No new posts REMOVE DUPLICATE RECORDS BASED ON A S... DFSORT/ICETOOL 4
No new posts Need to read duplicate rows from tabl... DB2 3
No new posts Compare duplicate records in same file DFSORT/ICETOOL 3
No new posts How to count duplicate key records co... SYNCSORT 0
This topic is locked: you cannot edit posts or make replies. Removing duplicate record based on th... DFSORT/ICETOOL 4
Search our Forums:

Back to Top