Portal | Manuals | References | Downloads | Info | Programs | JCLs | Master the Mainframes
IBM Mainframe Computers Forums Index
 
Register
 
IBM Mainframe Computers Forums Index Mainframe: Search IBM Mainframe Forum: FAQ Memberlist Usergroups Profile Log in to check your private messages Log in
 

 

SORT to remove duplicates with Input having 2 record types

 
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> DFSORT/ICETOOL
View previous topic :: :: View next topic  
Author Message
andrearak

New User


Joined: 29 Apr 2009
Posts: 3
Location: US

PostPosted: Wed Apr 29, 2009 7:27 pm    Post subject: SORT to remove duplicates with Input having 2 record types
Reply with quote

How can I use sort on the following FB LRECL=500 file to remove duplicate records. There are 2 record types. In this file example the first four records are unique, the remaining 8 are duplicates of the first four. I need to maintain the order of the first four records and have the duplicates (last eight removed):

Currently using this, which is causing the correct recs to be written (no dups) just in ascending order, which is not desired:
SORT FIELDS=(1,500,A),FORMAT=CH
SUM FIELDS=NONE
END

Is there some command to skip every other record so that only the first four records are written in the existing order?

0000001405700100000000000TFS701F 0102010202
ADH_LOS-ANGELES_20090422_124744
0000001408900100000000000TFS701R 0102010281
ADH_LOS-ANGELES_20090427_140010
0000001405700100000000000TFS701F 0102010202
ADH_LOS-ANGELES_20090422_124744
0000001408900100000000000TFS701R 0102010281
ADH_LOS-ANGELES_20090427_140010
0000001405700100000000000TFS701F 0102010202
ADH_LOS-ANGELES_20090422_124744
0000001408900100000000000TFS701R 0102010281
ADH_LOS-ANGELES_20090427_140010
Back to top
View user's profile Send private message

Frank Yaeger

DFSORT Moderator


Joined: 15 Feb 2005
Posts: 7130
Location: San Jose, CA

PostPosted: Wed Apr 29, 2009 9:05 pm    Post subject:
Reply with quote

Here's a DFSORT/ICETOOL job that will do what you asked for:

Code:

//S1   EXEC  PGM=ICETOOL
//TOOLMSG   DD  SYSOUT=*
//DFSMSG    DD  SYSOUT=*
//IN DD DSN=...  input file (FB/500)
//T1 DD DSN=&&T1,UNIT=SYSDA,SPACE=(CYL,(5,5)),DISP=(,PASS)
//OUT DD DSN=...  output file (FB/500)
//TOOLIN DD *
SORT FROM(IN) TO(T1) USING(CTL1)
SORT FROM(T1) TO(OUT) USING(CTL2)
/*
//CTL1CNTL DD *
  OPTION EQUALS
  INREC OVERLAY=(501:SEQNUM,8,ZD)
  SORT FIELDS=(1,500,A),FORMAT=CH
  SUM FIELDS=NONE
/*
//CTL2CNTL DD *
  SORT FIELDS=(501,8,ZD,A)
  OUTREC BUILD=(1,500)
/*
Back to top
View user's profile Send private message
andrearak

New User


Joined: 29 Apr 2009
Posts: 3
Location: US

PostPosted: Wed Apr 29, 2009 9:51 pm    Post subject:
Reply with quote

Thank you. I understand what you've recommended here and it worked perfectly.

One question: Why use ZD?
Back to top
View user's profile Send private message
Frank Yaeger

DFSORT Moderator


Joined: 15 Feb 2005
Posts: 7130
Location: San Jose, CA

PostPosted: Wed Apr 29, 2009 9:57 pm    Post subject:
Reply with quote

You could use any supported numeric format for the sequence numbers - ZD, PD, BI or FS. I like to use ZD because it's readable. But any of the others would work as well. For example, instead of SEQNUM,8,ZD, you could use SEQNUM,5,PD and save some bytes (but sacrifice readability when you're debugging).
Back to top
View user's profile Send private message
Skolusu

Senior Member


Joined: 07 Dec 2007
Posts: 2205
Location: San Jose

PostPosted: Wed Apr 29, 2009 10:06 pm    Post subject:
Reply with quote

andrearak,

Here is an alternate way doing it using the new WHEN=GROUP function in one pass. You want to consider every 2 records as a single record and remove the duplicates. Using when=Group we push the first record on to the second record and sort on the full 1000 bytes and remove the duplicates.

Using OUTFIL we write out the original order once again

Code:

//STEP0100 EXEC PGM=SORT                                             
//SYSOUT   DD SYSOUT=*                                               
//SORTIN   DD DSN=your input 500 fb file,DISP=SHR                           
//SORTOUT  DD SYSOUT=*                                               
//SYSIN    DD *                                                     
  INREC IFTHEN=(WHEN=INIT,OVERLAY=(1001:SEQNUM,1,ZD,START=0,INCR=5)),
  IFTHEN=(WHEN=GROUP,RECORDS=2,PUSH=(501:1,500))                     
  SORT FIELDS=(1,1000,CH,A),EQUALS                                   
  SUM FIELDS=NONE                                                   
  OUTFIL OMIT=(1001,1,ZD,EQ,0),BUILD=(501,500,/,1,500)               
/*                                                                   
Back to top
View user's profile Send private message
andrearak

New User


Joined: 29 Apr 2009
Posts: 3
Location: US

PostPosted: Wed Apr 29, 2009 10:39 pm    Post subject:
Reply with quote

Thanks Frank for the explanation. And thanks Kolusu for the alternative solution, that's clever.
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> DFSORT/ICETOOL All times are GMT + 6 Hours
Page 1 of 1

 

Search our Forum:

Similar Topics
Topic Author Forum Replies Posted
No new posts SORT to unpack a feild shr_amar DFSORT/ICETOOL 7 Fri Jul 07, 2017 4:32 am
No new posts How to write Specific Fields from Mul... Padhu SYNCSORT 6 Thu Jul 06, 2017 10:26 am
No new posts How to generate a new unique Input fi... for1ranjith CLIST & REXX 11 Sat Jul 01, 2017 12:09 pm
No new posts Optimize sort inrec parse vishwakotin DFSORT/ICETOOL 6 Mon Jun 26, 2017 11:15 pm
No new posts Reduce CPU Times for Join Sort santoshn SYNCSORT 12 Sat Jun 10, 2017 1:40 pm


Facebook
Back to Top
 
Mainframe Wiki | Forum Rules | Bookmarks | Subscriptions | FAQ | Tutorials | Contact Us