Portal | Manuals | References | Downloads | Info | Programs | JCLs | Master the Mainframes
IBM Mainframe Computers Forums Index
 
Register
 
IBM Mainframe Computers Forums Index Mainframe: Search IBM Mainframe Forum: FAQ Memberlist Usergroups Profile Log in to check your private messages Log in
 

 

duplicate removal

 
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> SYNCSORT
View previous topic :: :: View next topic  
Author Message
wiprov

New User


Joined: 13 Feb 2008
Posts: 15
Location: Chennai

PostPosted: Fri Sep 26, 2014 11:26 pm    Post subject: duplicate removal
Reply with quote

Hi,

My i/p file has header and detail records as below:

Input File:
Four set of records and first byte - '1' indicates header.
First byte - '5' or '6' indicates detail records.

1HDR1
5detail1
6detail1

1HDR3
5detail3
6detail3

1HDR2
5detail2
6detail2

1HDR3
5detail3
6detail3

Requirement:
If you could notice, 1HDR3 set of records is getting repeated and the duplicate occurrence of that, needs to be removed.

Expected o/p File:

1HDR1
5detail1
6detail1

1HDR3
5detail3
6detail3

1HDR2
5detail2
6detail2

Could you please let me know, is this viable to crack through DFSORT.
Kindly let me know, if you need any other details from my end. Thanks
Back to top
View user's profile Send private message

Rohit Umarjikar

Senior Member


Joined: 21 Sep 2010
Posts: 1609
Location: NY,USA

PostPosted: Sat Sep 27, 2014 2:45 am    Post subject:
Reply with quote

Add a SEQNUM at the end of each record, remove duplicates and the sort it back using the SEQNUM.
Back to top
View user's profile Send private message
Bill Woodger

DFSORT Moderator


Joined: 09 Mar 2011
Posts: 7223

PostPosted: Sat Sep 27, 2014 2:54 am    Post subject: Reply to: duplicate removal
Reply with quote

What is the RECFM and LRECL?

Do you consider it duplicate if the header is duplicate, or need the data be duplicate as well?
Back to top
View user's profile Send private message
wiprov

New User


Joined: 13 Feb 2008
Posts: 15
Location: Chennai

PostPosted: Sat Sep 27, 2014 9:33 am    Post subject:
Reply with quote

RECFM=FB and LRECL=10.


if the header is duplicate, then we can drop the entire set of records.
In our example, second set of 1HDR3 can contain different set of records.
Back to top
View user's profile Send private message
Rohit Umarjikar

Senior Member


Joined: 21 Sep 2010
Posts: 1609
Location: NY,USA

PostPosted: Thu Oct 02, 2014 8:18 pm    Post subject:
Reply with quote

While Bill takes a look on this further, Did you give a try on what I have suggested earlier and if you need a sample how to do it then it is verymuch available on Google or on this forum?
Back to top
View user's profile Send private message
Bill Woodger

DFSORT Moderator


Joined: 09 Mar 2011
Posts: 7223

PostPosted: Thu Oct 02, 2014 8:53 pm    Post subject: Reply to: duplicate removal
Reply with quote

Well, wiprov took it off to another site. Turns out he has SyncSORT as well.
Back to top
View user's profile Send private message
wiprov

New User


Joined: 13 Feb 2008
Posts: 15
Location: Chennai

PostPosted: Thu Oct 02, 2014 10:03 pm    Post subject: Reply to: duplicate removal
Reply with quote

Hi Rohit,

Sorry for the late response. Adding a sequence number could be a problem, if the same detail records exists in another header (which is not duplicate). We might miss those records in that case...... icon_sad.gif
Back to top
View user's profile Send private message
Rohit Umarjikar

Senior Member


Joined: 21 Sep 2010
Posts: 1609
Location: NY,USA

PostPosted: Fri Oct 03, 2014 6:00 am    Post subject:
Reply with quote

I might be missing something,but can you please explain with an example for this statement?

Quote:
if the same detail records exists in another header (which is not duplicate). We might miss those records in that case......


As I guess same set of detail records and header would retain from one of the duplicate that found, like in your example still below sets of records would come.
Code:
HDR3
5detail3
6detail3
Back to top
View user's profile Send private message
Bill Woodger

DFSORT Moderator


Joined: 09 Mar 2011
Posts: 7223

PostPosted: Fri Oct 03, 2014 10:54 am    Post subject: Reply to: duplicate removal
Reply with quote

Quote:
if the header is duplicate, then we can drop the entire set of records.
In our example, second set of 1HDR3 can contain different set of records.
Back to top
View user's profile Send private message
Rohit Umarjikar

Senior Member


Joined: 21 Sep 2010
Posts: 1609
Location: NY,USA

PostPosted: Thu Oct 09, 2014 5:23 am    Post subject:
Reply with quote

Right, I looked today closey and yes the sequence number option wouldn't fit here..

wiprov, have you got any solution or you still open to the options?
Back to top
View user's profile Send private message
saiprasadh

Active User


Joined: 20 Sep 2006
Posts: 154
Location: US

PostPosted: Fri Oct 10, 2014 11:39 am    Post subject:
Reply with quote

Wiprov,

I used below mentioned sort card to obtain the result.

Code:
 SORT FIELDS=COPY                                           
 INREC  IFTHEN=(WHEN=INIT,                                 
        BUILD=(1:1,10,81:20C'0')),                         
        IFTHEN=(WHEN=GROUP,BEGIN=(1,4,CH,EQ,C'1HDR'),       
        PUSH=(81:ID=10)),                                   
        IFTHEN=(WHEN=GROUP,BEGIN=(1,5,CH,EQ,C'1HDR3'),     
        END=(1,4,CH,EQ,C'1HDR',AND,1,5,CH,NE,C'1HDR3'),     
        PUSH=(91:ID=10))                                   
 OUTFIL FNAMES=SORTOUT,                                     
 INCLUDE=((2,3,CH,EQ,C'DET',AND,91,10,ZD,EQ,0),OR,         
          (1,4,CH,EQ,C'1HDR',AND,1,5,CH,NE,C'1HDR3'),OR,   
          (1,5,CH,EQ,C'1HDR3',AND,91,10,ZD,EQ,1),OR,       
          (2,3,CH,EQ,C'DET',AND,91,10,ZD,EQ,1)),           
 OUTREC=(1:1,10) 


Input:
Code:
1HDR1       
5DETAIL1     
6DETAIL1     
1HDR3       
5DETAIL3     
6DETAIL3     
5DETAIL3     
6DETAIL3     
1HDR3       
5DETAIL3     
6DETAIL3     
1HDR2       
5DETAIL2     
6DETAIL2     
1HDR3       
5DETAIL3     
6DETAIL3


Output:

Code:
1HDR1       
5DETAIL1     
6DETAIL1     
1HDR3       
5DETAIL3     
6DETAIL3     
5DETAIL3     
6DETAIL3     
1HDR2       
5DETAIL2     
6DETAIL2 
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> SYNCSORT All times are GMT + 6 Hours
Page 1 of 1

 

Search our Forum:

Similar Topics
Topic Author Forum Replies Posted
No new posts Limit duplicate records in the SORT pshongal SYNCSORT 6 Mon Nov 21, 2016 12:54 pm
No new posts how to find the duplicate list RAVIGUPTA1990 DB2 6 Wed Mar 30, 2016 12:11 am
No new posts Duplicate component install allowed i... kratos86 CA Products 2 Fri Oct 03, 2014 12:08 am
No new posts Need to sum the values when a duplica... pavanakar DFSORT/ICETOOL 1 Thu Aug 21, 2014 1:25 pm
No new posts How to eliminate duplicate records - ... Rohan saldanha DFSORT/ICETOOL 7 Tue Jul 15, 2014 7:35 pm


Facebook
Back to Top
 
Mainframe Wiki | Forum Rules | Bookmarks | Subscriptions | FAQ | Tutorials | Contact Us