IBM Mainframe Forum Index
 
Log In
 
IBM Mainframe Forum Index Mainframe: Search IBM Mainframe Forum: FAQ Register
 

How to remove duplicates within a group of records


IBM Mainframe Forums -> DFSORT/ICETOOL
Post new topic   Reply to topic
View previous topic :: View next topic  
Author Message
Mariraj

New User


Joined: 09 Jul 2007
Posts: 59
Location: Chennai

PostPosted: Tue Jun 07, 2011 6:08 pm
Reply with quote

The input record contains multiple group of records as shown below with a single header for each group as shown below.

Code:
01 - HEADER
01 - AAAAA
01 - AAAAA
01 - BBBBB
02 - HEADER
02 - CCCCC
02 - DDDDD


I need to remove the duplicates within a group. In the above sample, there are 3 records under 01 - HEADER followed by another group. In this first group the second record is a duplicate and has to be removed. the required output is as follows:

Code:
01 - HEADER
01 - AAAAA
01 - BBBBB
02 - HEADER
02 - CCCCC
02 - DDDDD


RECFM=FB, LRECL=80.
Back to top
View user's profile Send private message
PeterHolland

Global Moderator


Joined: 27 Oct 2009
Posts: 2481
Location: Netherlands, Amstelveen

PostPosted: Tue Jun 07, 2011 6:12 pm
Reply with quote

How about reading the manual?
Back to top
View user's profile Send private message
Escapa

Senior Member


Joined: 16 Feb 2007
Posts: 1399
Location: IL, USA

PostPosted: Tue Jun 07, 2011 6:23 pm
Reply with quote

check if this helps you..
Code:

//STEP0100 EXEC PGM=SORT                             
//SYSOUT   DD SYSOUT=*                               
//SORTIN   DD *                                     
01 - HEADER                                         
01 - AAAAA                                           
01 - AAAAA                                           
01 - BBBBB                                           
02 - HEADER                                         
02 - CCCCC                                           
02 - DDDDD                                           
//SORTOUT  DD DSN=&&TEMP,DISP=(NEW,PASS),RECFM=FB   
//SYSIN    DD *                                     
 INREC OVERLAY=(81:SEQNUM,8,ZD)                     
 SORT FIELDS=(1,2,CH,A,6,10,CH,A)                   
 SUM FIELDS=NONE                                     
//STEP0200 EXEC PGM=SORT                             
//SYSOUT   DD SYSOUT=*                               
//SORTIN   DD DSN=&&TEMP,DISP=(MOD,PASS)             
//SORTOUT  DD SYSOUT=*                               
//SYSIN    DD *                                     
 SORT FIELDS=(81,8,ZD,A)     
 OUTREC BUILD=(1,80)                         
Back to top
View user's profile Send private message
Mariraj

New User


Joined: 09 Jul 2007
Posts: 59
Location: Chennai

PostPosted: Tue Jun 07, 2011 6:27 pm
Reply with quote

Thanks for your advice Peter. I went through the manuals and various posts here based on which I tried the following SORTCARD.

Code:
INREC IFTHEN=(WHEN=GROUP,BEGIN=(6,6,CH,EQ,C'HEADER'),
  PUSH=(81:ID=8))                                     
SORT FIELDS=(81,8,ZD,A,1,10,CH,A),EQUALS             
SUM FIELDS=NONE                                       


But this sortcard sorts the header too. Though I understand why it sorts it, but I am not able to avoid it. Can someone let me know how can I avoid sorting the header of the group and retain it at the top of the group.
Back to top
View user's profile Send private message
Escapa

Senior Member


Joined: 16 Feb 2007
Posts: 1399
Location: IL, USA

PostPosted: Tue Jun 07, 2011 6:41 pm
Reply with quote

Hi Mariraj,
You got to do minor change to your sortcard...
Code:

//STEP0100 EXEC PGM=SORT                                             
//SYSOUT   DD SYSOUT=*                                               
//SORTIN   DD *                                                       
01 - HEADER                                                           
01 - AAAAA                                                           
01 - AAAAA                                                           
01 - BBBBB                                                           
02 - HEADER                                                           
02 - CCCCC                                                           
02 - DDDDD                                                           
//SORTOUT  DD SYSOUT=*                                               
//SYSIN    DD *                                                       
  INREC IFTHEN=(WHEN=INIT,OVERLAY=(89:C'2')),                         
        IFTHEN=(WHEN=GROUP,BEGIN=(6,6,CH,EQ,C'HEADER'),               
        PUSH=(81:ID=8)),                                             
        IFTHEN=(WHEN=(6,6,CH,EQ,C'HEADER'),OVERLAY=(89:C'1'))         
  SORT FIELDS=(81,8,ZD,A,89,1,ZD,A,1,10,CH,A),EQUALS                 
  SUM FIELDS=NONE                                                     
  OUTFIL BUILD=(1,80)                                                 
Back to top
View user's profile Send private message
Mariraj

New User


Joined: 09 Jul 2007
Posts: 59
Location: Chennai

PostPosted: Tue Jun 07, 2011 6:56 pm
Reply with quote

Sambhaji,

Thanks a ton for both your code snippets. Though I haven't checked the first sortcard, the second one worked perfectly. I had tried the following to get the results.

Code:
SORT FROM(SORTIN) TO(TEMPDS) USING(CTL1)                 
SORT FROM(TEMPDS) TO(SORTOUT) USING(CTL2)               
/*                                                       
//CTL1CNTL DD *                                         
    INREC IFTHEN=(WHEN=GROUP,BEGIN=(6,6,CH,EQ,C'HEADER'),
     PUSH=(81:ID=4,85:SEQ=4))                           
   SORT FIELDS=(1,10,CH,A)                               
   SUM FIELDS=NONE                                       
//*  OUTREC BUILD=(1,88)                                 
//CTL2CNTL DD *                                         
   SORT FIELDS=(81,4,ZD,A,85,4,ZD,A)                     
   OUTREC BUILD=(1,80)     


Certainly not an efficient way to sort. But nevertheless I was able to achieve what I was looking for icon_smile.gif. I will go with your second suggestion since the job is done in a single step without any intermediary temp files.
Back to top
View user's profile Send private message
Skolusu

Senior Member


Joined: 07 Dec 2007
Posts: 2205
Location: San Jose

PostPosted: Tue Jun 07, 2011 9:50 pm
Reply with quote

Mariraj,

You can use the following DFSORT JCl which will give you the desired results
Code:

//STEP0100 EXEC PGM=SORT                                               
//SYSOUT   DD SYSOUT=*                                                 
//SORTIN   DD *                                                       
01 - HEADER                                                           
01 - AAAAA                                                             
01 - AAAAA                                                             
01 - BBBBB                                                             
02 - HEADER                                                           
02 - CCCCC                                                             
02 - DDDDD                                                             
//SORTOUT  DD SYSOUT=*                                                 
//SYSIN    DD *                                                       
  INREC IFTHEN=(WHEN=GROUP,BEGIN=(6,6,CH,EQ,C'HEADER'),PUSH=(81:ID=8)),
  IFTHEN=(WHEN=(6,6,CH,NE,C'HEADER'),OVERLAY=(89:1,10))               
  SORT FIELDS=(81,18,CH,A),EQUALS                                     
  SUM FIELDS=NONE                                                     
  OUTREC BUILD=(1,80)                                                 
//*
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic View Bookmarks
All times are GMT + 6 Hours
Forum Index -> DFSORT/ICETOOL

 


Similar Topics
Topic Forum Replies
No new posts Sortjoin and Search for a String and ... DFSORT/ICETOOL 1
No new posts Compare only first records of the fil... SYNCSORT 7
No new posts Pulling a fixed number of records fro... DB2 2
No new posts Remove leading zeroes SYNCSORT 4
No new posts Join multiple records using splice DFSORT/ICETOOL 5
Search our Forums:

Back to Top