Using ICETOOL, how we can seperate the duplicate and non dup

bshkris · New User Joined: 21 Mar 2005 Posts: 41 Location: pune

Hello,
I need some idea how can separate dup and non dup records separately..
either by using ICETOOL or SORT.

Input file
----------
2017-05-240001000002400
2017-04-020001000002600
2017-04-020001000002600
2017-04-260001000004400
2017-05-250001000002400
2017-05-240001000002400

OutFile1:
----------
2017-05-240001000002400
2017-05-250001000002400
2017-05-240001000002400

Outfile2:
--------
2017-04-020001000002600
2017-04-020001000002600
2017-04-260001000004400

Read more: ibmmainframes.com/viewtopic.php?p=339983#339983#ixzz4gatcM4Jb

Arun Raj · Posted: Tue May 09, 2017 8:37 pm

Key position and length is important in doing this, which is missing here.

In general, the ICETOOL SELECT operator with NODUPS/ALLDUPS along with a DISCARD dd should do this.

Your recent posts indicate that you are using Syncsort and NOT DFSORT.
This has already been brought to your attention by Robert in the other topic.

Please start paying attention to the questions/suggestions made by others, and make sure you post your new sort-related topics in the Syncsort forum, going forward. If you have a totally different requirement, always start a new topic rather than continuing on a solved topic.

bshkris · New User Joined: 21 Mar 2005 Posts: 41 Location: pune

I have issue with dates, like when I have records in same month more than 1, example

case1:
2017-05-240001000002400
2017-05-250001000002400
2017-05-240001000002400

all above records should go in out put file 1.

case 2: if only two records comes in input file
2017-05-240001000002400
2017-05-240001000002400

only one rec I need to put in out put file 2

should go another file

Tried many ways, not happening.

Rohit Umarjikar · Posted: Tue May 09, 2017 10:14 pm

You should be posting it under Beginner's forum henceforth.

Arun Raj · Posted: Tue May 09, 2017 10:40 pm

bshkris,

It is not so clear (at least to me) what you are trying to do.
For the sample input shown in the original post, can you explain how you arrived at the expected output.

V S Amarendra Reddy · Active User Joined: 13 Sep 2006 Posts: 216 Location: USA

Yeah, this is bit confusing without key and real expected output details. Based on his second post 'perhaps' this is the expectation, not sure(The TS needs to confirm).

1. Group the input records by Month first.
2. Check if all the records in that 'Month group' are of same date. If yes, remove all the dups and return one(Again, which one to return??). If not of same date, copy all the input records to output.