Portal | Manuals | References | Downloads | Info | Programs | JCLs | Master the Mainframes
IBM Mainframe Computers Forums Index
 
Register
 
IBM Mainframe Computers Forums Index Mainframe: Search IBM Mainframe Forum: FAQ Memberlist Usergroups Profile Log in to check your private messages Log in
 

 

Stripping off only first occurence of duplicate record

 
Post new topic   This topic is locked: you cannot edit posts or make replies.    IBMMAINFRAMES.com Support Forums -> All Other Mainframe Topics
View previous topic :: :: View next topic  
Author Message
Jay Villaverde

New User


Joined: 08 Mar 2014
Posts: 26
Location: USA

PostPosted: Tue Mar 18, 2014 12:38 am    Post subject: Stripping off only first occurence of duplicate record
Reply with quote

Hi. I have a file that has duplicate records only and the requestor wants to remove only the first occurence and leave all the other duplicates. So if there are 3 duplicate records, remove 1 and leave 2.

Can this be done using SYNCSORT? If so, what commands would I need for that?

By the way, my file only has dupes so really just need to drop one dupe records and leave others. Doesn't neccesarily have to be the first one.

Thanks
Back to top
View user's profile Send private message

Rohit Umarjikar

Senior Member


Joined: 21 Sep 2010
Posts: 1609
Location: NY,USA

PostPosted: Tue Mar 18, 2014 1:06 am    Post subject:
Reply with quote

May be you can use the below approach,

1) Take the input file and using Syncsort add numbers at the every last like 1,2,3 and so on for every unique chance of the record
2) And finally add a condition to remove the record which has last number added euqals to 1.
3) By this you will remove first entry of every duplicate.

E.g.

1) As per #1

Code:
AAAA1234.340001
AAAA1234.340002
AAAA1234.340003
BBBB1234.340001
BBBB1234.340002
BBBB1234.340003
BBBB1234.340004


Out put as per #2 above


Code:
AAAA1234.34
AAAA1234.34
BBBB1234.34
BBBB1234.34
BBBB1234.34


However to help you by other experts you need to provide all the necessary details and sample input data otherwise none can be helpful.
Back to top
View user's profile Send private message
Jay Villaverde

New User


Joined: 08 Mar 2014
Posts: 26
Location: USA

PostPosted: Tue Mar 18, 2014 1:12 am    Post subject:
Reply with quote

That would work well. How do I add such a counter at the end as in your first example?

My data looks like this. Some have just 2 dupes, some have more than 2:

Code:
0000000024343090800074433902
0000000024343090800074433902
0000000024351661261958120101
0000000024351661261958120101
0000000024352050300074377903
0000000024352050300074377903
0000000024352050300074377903
Back to top
View user's profile Send private message
Jay Villaverde

New User


Joined: 08 Mar 2014
Posts: 26
Location: USA

PostPosted: Tue Mar 18, 2014 1:37 am    Post subject:
Reply with quote

Got it to work using SUM FIELDS=NONE,XSUM
Back to top
View user's profile Send private message
dick scherrer

Site Director


Joined: 23 Nov 2006
Posts: 19270
Location: Inside the Matrix

PostPosted: Tue Mar 18, 2014 8:06 pm    Post subject:
Reply with quote

Good to hear it is working - thank you for letting us know and posting your solution icon_smile.gif

d
Back to top
View user's profile Send private message
Bill Woodger

DFSORT Moderator


Joined: 09 Mar 2011
Posts: 7225

PostPosted: Tue Mar 18, 2014 8:22 pm    Post subject: Reply to: Stripping off only first occurence of duplicate re
Reply with quote

What is "working" is not what is described in the question.

This will retain one record with a duplicate key for each key. The record retained depends on EQUALS (the first) or NOEQUALS (can't predict which) and the discarded records will be written to the XSUM DD.

hailashwin has the correct approach. A sequence number with a RESTART= for the key in question, then OUTFIL OMIT=(sequencenumbersisone).

There is your manual, there are examples here.
Back to top
View user's profile Send private message
Jay Villaverde

New User


Joined: 08 Mar 2014
Posts: 26
Location: USA

PostPosted: Tue Mar 18, 2014 8:28 pm    Post subject:
Reply with quote

Sorry not following because XSUM did give me what I needed. As I stated in my question it didn't necessarily need to be the first record dropped with others kept. Just needed to drop 1 dupe and keep the rest in a separate file. XSUM achieved that for me.

Yes, the other approach would have worked as well, but XSUM was quicker and easier for me.

Regards
Back to top
View user's profile Send private message
Bill Woodger

DFSORT Moderator


Joined: 09 Mar 2011
Posts: 7225

PostPosted: Tue Mar 18, 2014 9:35 pm    Post subject: Reply to: Stripping off only first occurence of duplicate re
Reply with quote

I see now that your question says both, and doesn't say anything about needing to keep the records which have been dropped.

However, you show one instance where you have three duplicates, so two will be dropped. Doesn't fit the "only one" from any interpretation of your question.

Using SUM with XSUM are you SORTing the records? Were you SORTing them anyway?

I suppose it may take up to a minute to code differently, but you'll save many, many minutes by not having to SORT the file.

To collect together the dropped records by the other method suggested, you'd just need a second OUTFIL with SAVE. There would be no duplicate keys on that file, unlike your XSUM file.

Looking at the sample data you have shown, it is irrelevant which record is dropped, because the duplicates are identical to each other.

If you are happy with XSUM, fine, just don't pretend it satisfied what you asked.
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   This topic is locked: you cannot edit posts or make replies.    IBMMAINFRAMES.com Support Forums -> All Other Mainframe Topics All times are GMT + 6 Hours
Page 1 of 1

 

Search our Forum:

Similar Topics
Topic Author Forum Replies Posted
No new posts Limit duplicate records in the SORT pshongal SYNCSORT 6 Mon Nov 21, 2016 12:54 pm
No new posts How to update a portion of text in a ... Bill Woodger DFSORT/ICETOOL 25 Wed Nov 09, 2016 9:41 pm
No new posts sort with previous record anatol DFSORT/ICETOOL 9 Thu Oct 06, 2016 2:36 am
No new posts Get Record count in summary record fo... Atul Banke DFSORT/ICETOOL 21 Fri Sep 23, 2016 4:17 pm
No new posts Stripping and Sorting of VB file G SRINIVASA RAO SYNCSORT 5 Wed Sep 14, 2016 1:34 pm


Facebook
Back to Top
 
Mainframe Wiki | Forum Rules | Bookmarks | Subscriptions | FAQ | Tutorials | Contact Us