Syncsort Merge and remove duplicates in one step!

shilpa.khaire · New User Joined: 21 Jul 2006 Posts: 39 Location: US

Hi all!

I want to merge two files on one field. And then I want to remove the duplicates from this merged file based on some other field.
e.g.

File A + File B = File C - Merge on field 1
File C - Remove duplicates based on field 2.

Is this possible in a single SORT step? Is there any other(other than 2 steps) efficient way of doing this?

Thanks!

dbzTHEdinosauer · Posted: Fri Nov 06, 2009 12:53 am

syncsort or dfsort?

shilpa.khaire · New User Joined: 21 Jul 2006 Posts: 39 Location: US

Syncsort, even Synctool can do.

Alissa Margulies · Posted: Thu Nov 12, 2009 2:49 am

shilpa.khaire · New User Joined: 21 Jul 2006 Posts: 39 Location: US

First Merge is on and the LRECL is 1500.
SORT FIELDS=(463,10,CH,D)

This is the second step which takes the output of above as an input.
SORT FIELDS=(26,21,CH,A),EQUALS
SUM FIELDS=NONE,XSUM

Op RECFM is FB and LRECL = 1500, same as input

Thanks!

Alissa Margulies · Posted: Fri Nov 13, 2009 2:13 am

I am not clear as to what your first step is doing. Are you trying to "merge" or "join" a record from File 1 with a record from File 2 to create a new record, or are you combining all records from File 1 and File 2 to create a master file?