Portal | Manuals | References | Downloads | Info | Programs | JCLs | Mainframe wiki | Quick Ref
IBM Mainframe Computers Forums Index
 
Register
 
IBM Mainframe Computers Forums Index Mainframe: Search IBM Mainframe Forum: FAQ Memberlist Profile Log in to check your private messages Log in
 
How to remove duplicates within a group of records

 
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> DFSORT/ICETOOL
View previous topic :: :: View next topic  
Author Message
Mariraj

New User


Joined: 09 Jul 2007
Posts: 59
Location: Chennai

PostPosted: Tue Jun 07, 2011 6:08 pm    Post subject: How to remove duplicates within a group of records
Reply with quote

The input record contains multiple group of records as shown below with a single header for each group as shown below.

Code:
01 - HEADER
01 - AAAAA
01 - AAAAA
01 - BBBBB
02 - HEADER
02 - CCCCC
02 - DDDDD


I need to remove the duplicates within a group. In the above sample, there are 3 records under 01 - HEADER followed by another group. In this first group the second record is a duplicate and has to be removed. the required output is as follows:

Code:
01 - HEADER
01 - AAAAA
01 - BBBBB
02 - HEADER
02 - CCCCC
02 - DDDDD


RECFM=FB, LRECL=80.
Back to top
View user's profile Send private message

PeterHolland

Global Moderator


Joined: 27 Oct 2009
Posts: 2442
Location: Netherlands, Amstelveen

PostPosted: Tue Jun 07, 2011 6:12 pm    Post subject:
Reply with quote

How about reading the manual?
Back to top
View user's profile Send private message
Escapa

Senior Member


Joined: 16 Feb 2007
Posts: 1399
Location: IL, USA

PostPosted: Tue Jun 07, 2011 6:23 pm    Post subject:
Reply with quote

check if this helps you..
Code:

//STEP0100 EXEC PGM=SORT                             
//SYSOUT   DD SYSOUT=*                               
//SORTIN   DD *                                     
01 - HEADER                                         
01 - AAAAA                                           
01 - AAAAA                                           
01 - BBBBB                                           
02 - HEADER                                         
02 - CCCCC                                           
02 - DDDDD                                           
//SORTOUT  DD DSN=&&TEMP,DISP=(NEW,PASS),RECFM=FB   
//SYSIN    DD *                                     
 INREC OVERLAY=(81:SEQNUM,8,ZD)                     
 SORT FIELDS=(1,2,CH,A,6,10,CH,A)                   
 SUM FIELDS=NONE                                     
//STEP0200 EXEC PGM=SORT                             
//SYSOUT   DD SYSOUT=*                               
//SORTIN   DD DSN=&&TEMP,DISP=(MOD,PASS)             
//SORTOUT  DD SYSOUT=*                               
//SYSIN    DD *                                     
 SORT FIELDS=(81,8,ZD,A)     
 OUTREC BUILD=(1,80)                         
Back to top
View user's profile Send private message
Mariraj

New User


Joined: 09 Jul 2007
Posts: 59
Location: Chennai

PostPosted: Tue Jun 07, 2011 6:27 pm    Post subject: Reply to: How to remove duplicates within a group of records
Reply with quote

Thanks for your advice Peter. I went through the manuals and various posts here based on which I tried the following SORTCARD.

Code:
INREC IFTHEN=(WHEN=GROUP,BEGIN=(6,6,CH,EQ,C'HEADER'),
  PUSH=(81:ID=8))                                     
SORT FIELDS=(81,8,ZD,A,1,10,CH,A),EQUALS             
SUM FIELDS=NONE                                       


But this sortcard sorts the header too. Though I understand why it sorts it, but I am not able to avoid it. Can someone let me know how can I avoid sorting the header of the group and retain it at the top of the group.
Back to top
View user's profile Send private message
Escapa

Senior Member


Joined: 16 Feb 2007
Posts: 1399
Location: IL, USA

PostPosted: Tue Jun 07, 2011 6:41 pm    Post subject:
Reply with quote

Hi Mariraj,
You got to do minor change to your sortcard...
Code:

//STEP0100 EXEC PGM=SORT                                             
//SYSOUT   DD SYSOUT=*                                               
//SORTIN   DD *                                                       
01 - HEADER                                                           
01 - AAAAA                                                           
01 - AAAAA                                                           
01 - BBBBB                                                           
02 - HEADER                                                           
02 - CCCCC                                                           
02 - DDDDD                                                           
//SORTOUT  DD SYSOUT=*                                               
//SYSIN    DD *                                                       
  INREC IFTHEN=(WHEN=INIT,OVERLAY=(89:C'2')),                         
        IFTHEN=(WHEN=GROUP,BEGIN=(6,6,CH,EQ,C'HEADER'),               
        PUSH=(81:ID=8)),                                             
        IFTHEN=(WHEN=(6,6,CH,EQ,C'HEADER'),OVERLAY=(89:C'1'))         
  SORT FIELDS=(81,8,ZD,A,89,1,ZD,A,1,10,CH,A),EQUALS                 
  SUM FIELDS=NONE                                                     
  OUTFIL BUILD=(1,80)                                                 
Back to top
View user's profile Send private message
Mariraj

New User


Joined: 09 Jul 2007
Posts: 59
Location: Chennai

PostPosted: Tue Jun 07, 2011 6:56 pm    Post subject:
Reply with quote

Sambhaji,

Thanks a ton for both your code snippets. Though I haven't checked the first sortcard, the second one worked perfectly. I had tried the following to get the results.

Code:
SORT FROM(SORTIN) TO(TEMPDS) USING(CTL1)                 
SORT FROM(TEMPDS) TO(SORTOUT) USING(CTL2)               
/*                                                       
//CTL1CNTL DD *                                         
    INREC IFTHEN=(WHEN=GROUP,BEGIN=(6,6,CH,EQ,C'HEADER'),
     PUSH=(81:ID=4,85:SEQ=4))                           
   SORT FIELDS=(1,10,CH,A)                               
   SUM FIELDS=NONE                                       
//*  OUTREC BUILD=(1,88)                                 
//CTL2CNTL DD *                                         
   SORT FIELDS=(81,4,ZD,A,85,4,ZD,A)                     
   OUTREC BUILD=(1,80)     


Certainly not an efficient way to sort. But nevertheless I was able to achieve what I was looking for icon_smile.gif. I will go with your second suggestion since the job is done in a single step without any intermediary temp files.
Back to top
View user's profile Send private message
Skolusu

Senior Member


Joined: 07 Dec 2007
Posts: 2205
Location: San Jose

PostPosted: Tue Jun 07, 2011 9:50 pm    Post subject:
Reply with quote

Mariraj,

You can use the following DFSORT JCl which will give you the desired results
Code:

//STEP0100 EXEC PGM=SORT                                               
//SYSOUT   DD SYSOUT=*                                                 
//SORTIN   DD *                                                       
01 - HEADER                                                           
01 - AAAAA                                                             
01 - AAAAA                                                             
01 - BBBBB                                                             
02 - HEADER                                                           
02 - CCCCC                                                             
02 - DDDDD                                                             
//SORTOUT  DD SYSOUT=*                                                 
//SYSIN    DD *                                                       
  INREC IFTHEN=(WHEN=GROUP,BEGIN=(6,6,CH,EQ,C'HEADER'),PUSH=(81:ID=8)),
  IFTHEN=(WHEN=(6,6,CH,NE,C'HEADER'),OVERLAY=(89:1,10))               
  SORT FIELDS=(81,18,CH,A),EQUALS                                     
  SUM FIELDS=NONE                                                     
  OUTREC BUILD=(1,80)                                                 
//*
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> DFSORT/ICETOOL All times are GMT + 6 Hours
Page 1 of 1

 

Search our Forum:

Similar Topics
Topic Author Forum Replies Posted
No new posts sort records based on length exceeds ... maxsubrat DFSORT/ICETOOL 7 Wed Oct 04, 2017 4:48 pm
No new posts Addition of two records on 2 fields amar143 SYNCSORT 8 Tue Oct 03, 2017 11:14 am
No new posts column with count of rows within dist... ronald wouterson DB2 4 Sun Sep 17, 2017 9:48 pm
No new posts Remove leading spaces from numeric field rexx77 SYNCSORT 6 Wed Sep 06, 2017 2:15 am
No new posts Join records from 2 files with No Dup... Poha Eater DFSORT/ICETOOL 22 Sun Aug 27, 2017 10:35 pm

Facebook
Back to Top
 
Job Vacancies | Forum Rules | Bookmarks | Subscriptions | FAQ | Polls | Contact Us