IBM Mainframe Forum Index
 
Log In
 
IBM Mainframe Forum Index Mainframe: Search IBM Mainframe Forum: FAQ Register
 

duplicate removal
IBM Mainframe Forums -> SYNCSORT
Post new topic   Reply to topic Quick References
View previous topic :: View next topic  
Author Message
wiprov

New User


Joined: 13 Feb 2008
Posts: 15
Location: Chennai
PostPosted: Fri Sep 26, 2014 11:26 pm
 Reply with quote
Hi,

My i/p file has header and detail records as below:

Input File:
Four set of records and first byte - '1' indicates header.
First byte - '5' or '6' indicates detail records.

1HDR1
5detail1
6detail1

1HDR3
5detail3
6detail3

1HDR2
5detail2
6detail2

1HDR3
5detail3
6detail3

Requirement:
If you could notice, 1HDR3 set of records is getting repeated and the duplicate occurrence of that, needs to be removed.

Expected o/p File:

1HDR1
5detail1
6detail1

1HDR3
5detail3
6detail3

1HDR2
5detail2
6detail2

Could you please let me know, is this viable to crack through DFSORT.
Kindly let me know, if you need any other details from my end. Thanks
Back to top
View user's profile Send private message
Rohit Umarjikar

Global Moderator


Joined: 21 Sep 2010
Posts: 3053
Location: NYC,USA
PostPosted: Sat Sep 27, 2014 2:45 am
 Reply with quote
Add a SEQNUM at the end of each record, remove duplicates and the sort it back using the SEQNUM.
Back to top
View user's profile Send private message
Bill Woodger

Moderator Emeritus


Joined: 09 Mar 2011
Posts: 7309
Location: Inside the Matrix
PostPosted: Sat Sep 27, 2014 2:54 am
 Reply with quote
What is the RECFM and LRECL?

Do you consider it duplicate if the header is duplicate, or need the data be duplicate as well?
Back to top
View user's profile Send private message
wiprov

New User


Joined: 13 Feb 2008
Posts: 15
Location: Chennai
PostPosted: Sat Sep 27, 2014 9:33 am
 Reply with quote
RECFM=FB and LRECL=10.


if the header is duplicate, then we can drop the entire set of records.
In our example, second set of 1HDR3 can contain different set of records.
Back to top
View user's profile Send private message
Rohit Umarjikar

Global Moderator


Joined: 21 Sep 2010
Posts: 3053
Location: NYC,USA
PostPosted: Thu Oct 02, 2014 8:18 pm
 Reply with quote
While Bill takes a look on this further, Did you give a try on what I have suggested earlier and if you need a sample how to do it then it is verymuch available on Google or on this forum?
Back to top
View user's profile Send private message
Bill Woodger

Moderator Emeritus


Joined: 09 Mar 2011
Posts: 7309
Location: Inside the Matrix
PostPosted: Thu Oct 02, 2014 8:53 pm
 Reply with quote
Well, wiprov took it off to another site. Turns out he has SyncSORT as well.
Back to top
View user's profile Send private message
wiprov

New User


Joined: 13 Feb 2008
Posts: 15
Location: Chennai
PostPosted: Thu Oct 02, 2014 10:03 pm
 Reply with quote
Hi Rohit,

Sorry for the late response. Adding a sequence number could be a problem, if the same detail records exists in another header (which is not duplicate). We might miss those records in that case...... icon_sad.gif
Back to top
View user's profile Send private message
Rohit Umarjikar

Global Moderator


Joined: 21 Sep 2010
Posts: 3053
Location: NYC,USA
PostPosted: Fri Oct 03, 2014 6:00 am
 Reply with quote
I might be missing something,but can you please explain with an example for this statement?

Quote:
if the same detail records exists in another header (which is not duplicate). We might miss those records in that case......


As I guess same set of detail records and header would retain from one of the duplicate that found, like in your example still below sets of records would come.
Code:
HDR3
5detail3
6detail3
Back to top
View user's profile Send private message
Bill Woodger

Moderator Emeritus


Joined: 09 Mar 2011
Posts: 7309
Location: Inside the Matrix
PostPosted: Fri Oct 03, 2014 10:54 am
 Reply with quote
Quote:
if the header is duplicate, then we can drop the entire set of records.
In our example, second set of 1HDR3 can contain different set of records.
Back to top
View user's profile Send private message
Rohit Umarjikar

Global Moderator


Joined: 21 Sep 2010
Posts: 3053
Location: NYC,USA
PostPosted: Thu Oct 09, 2014 5:23 am
 Reply with quote
Right, I looked today closey and yes the sequence number option wouldn't fit here..

wiprov, have you got any solution or you still open to the options?
Back to top
View user's profile Send private message
saiprasadh

Active User


Joined: 20 Sep 2006
Posts: 154
Location: US
PostPosted: Fri Oct 10, 2014 11:39 am
 Reply with quote
Wiprov,

I used below mentioned sort card to obtain the result.

Code:
 SORT FIELDS=COPY                                           
 INREC  IFTHEN=(WHEN=INIT,                                 
        BUILD=(1:1,10,81:20C'0')),                         
        IFTHEN=(WHEN=GROUP,BEGIN=(1,4,CH,EQ,C'1HDR'),       
        PUSH=(81:ID=10)),                                   
        IFTHEN=(WHEN=GROUP,BEGIN=(1,5,CH,EQ,C'1HDR3'),     
        END=(1,4,CH,EQ,C'1HDR',AND,1,5,CH,NE,C'1HDR3'),     
        PUSH=(91:ID=10))                                   
 OUTFIL FNAMES=SORTOUT,                                     
 INCLUDE=((2,3,CH,EQ,C'DET',AND,91,10,ZD,EQ,0),OR,         
          (1,4,CH,EQ,C'1HDR',AND,1,5,CH,NE,C'1HDR3'),OR,   
          (1,5,CH,EQ,C'1HDR3',AND,91,10,ZD,EQ,1),OR,       
          (2,3,CH,EQ,C'DET',AND,91,10,ZD,EQ,1)),           
 OUTREC=(1:1,10) 


Input:
Code:
1HDR1       
5DETAIL1     
6DETAIL1     
1HDR3       
5DETAIL3     
6DETAIL3     
5DETAIL3     
6DETAIL3     
1HDR3       
5DETAIL3     
6DETAIL3     
1HDR2       
5DETAIL2     
6DETAIL2     
1HDR3       
5DETAIL3     
6DETAIL3


Output:

Code:
1HDR1       
5DETAIL1     
6DETAIL1     
1HDR3       
5DETAIL3     
6DETAIL3     
5DETAIL3     
6DETAIL3     
1HDR2       
5DETAIL2     
6DETAIL2 
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic View Bookmarks
All times are GMT + 6 Hours
Forum Index -> SYNCSORT

 

Similar Topics
Topic Forum Replies
No new posts Duplicate transid's declared using CEDA CICS 3
No new posts Duplicate several members of/in one l... JCL & VSAM 7
No new posts Newbie Stuck on "Duplicate Datas... TSO/ISPF 5
No new posts Duplicate data in PUT CONTAINER using... CICS 4
No new posts Removal of date value records DFSORT/ICETOOL 12
Search our Forums:

Back to Top
IBMMainframes.com is not an official and/or affiliated with IBM® in anyway
Board Rules | FAQ | Downloads | Wiki | SiteMap | Contact Us