Portal | Manuals | References | Downloads | Info | Programs | JCLs | Mainframe wiki | Quick Ref
IBM Mainframe Computers Forums Index
 
Register
 
IBM Mainframe Computers Forums Index Mainframe: Search IBM Mainframe Forum: FAQ Memberlist Profile Log in to check your private messages Log in
 
Sort utility step to eliminate duplicate records

 
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> DFSORT/ICETOOL
View previous topic :: :: View next topic  
Author Message
Nagendran

New User


Joined: 24 Jan 2007
Posts: 89
Location: USA

PostPosted: Mon Sep 24, 2007 12:46 pm    Post subject: Sort utility step to eliminate duplicate records
Reply with quote

Hi ,

I am using below step to eliminate duplicate records from the file , but the output file is having the duplicate records.

can any body help with below?

Code:

//STEP03  EXEC PGM=SORT,COND=(0,NE)               
//SORTIN  DD DSN=TEST.FILE2,               
//        DISP=SHR                                 
//SORTOUT DD DSN=TEST.FILE3,               
//        DISP=(MOD,CATLG,CATLG),                 
//        SPACE=(CYL,(300,300),RLSE),             
//        UNIT=TESTDA,                             
//        DCB=(LRECL=80,RECFM=FB,BLKSIZE=0)       
//SORTWK01 DD UNIT=SYSDA,DISP=(NEW,CATLG,CATLG),   
//         SPACE=(TRK,(30,30),RLSE)               
//SYSOUT DD SYSOUT=*                               
//SYSIN   DD *                                     
  SORT FIELDS=COPY                                 
  SUM FIELDS=NONE                                 
/*       


regards,
Nagu
Back to top
View user's profile Send private message

guptae

Moderator


Joined: 14 Oct 2005
Posts: 1187
Location: Bangalore,India

PostPosted: Mon Sep 24, 2007 12:52 pm    Post subject:
Reply with quote

Hi Nagendran,

If you want to remove duplicate record on the basis of key then please provide the follwing sort card

Code:

  SORT FIELDS=(1,80,CH,A)
  SUM FIELDS=NONE


here i assume that u wnat to remove duplicate on the bassis of thre whole record
But if u have any specific keys then u can change the record accordingly.please read the manual on sort for more clarification
Back to top
View user's profile Send private message
Nagendran

New User


Joined: 24 Jan 2007
Posts: 89
Location: USA

PostPosted: Mon Sep 24, 2007 12:57 pm    Post subject: Reply to: Sort utility
Reply with quote

hi,


I dont want to sort... I want to eliminate only the duplicates.

thanks,
nagu
Back to top
View user's profile Send private message
murmohk1

Senior Member


Joined: 29 Jun 2006
Posts: 1439
Location: Bangalore,India

PostPosted: Mon Sep 24, 2007 1:01 pm    Post subject:
Reply with quote

Nagendran,

Quote:
I dont want to sort... I want to eliminate only the duplicates.

You are supposed to sort for elimination dups.
Back to top
View user's profile Send private message
enrico-sorichetti

Global Moderator


Joined: 14 Mar 2007
Posts: 10326
Location: italy

PostPosted: Mon Sep 24, 2007 1:01 pm    Post subject: Reply to: Sort utility
Reply with quote

Quote:


I dont want to sort... I want to eliminate only the duplicates.


... by grace, will You tell us how to do it without sorting

regards

e.s
Back to top
View user's profile Send private message
guptae

Moderator


Joined: 14 Oct 2005
Posts: 1187
Location: Bangalore,India

PostPosted: Mon Sep 24, 2007 1:05 pm    Post subject:
Reply with quote

Hi There,

Use the following jcl

Code:
//STEP001  EXEC PGM=SORT
//SORTLIB  DD DSN=SYS1.SORTLIB,DISP=SHR                                 
//SORTWK01 DD SPACE=(CYL,(100)),UNIT=DISK                               
//SYSIN    DD  *
   INREC FIELDS=(1,80,SEQNUM,8,ZD)
   SORT FIELDS=(1,80,CH,A)
   SUM FIELDS=NONE
/*
//SORTIN   DD  DSN=TBH2.SUM.SORTIN.DATA,DISP=SHR
//SORTOUT  DD  DSN=TBH2.SUM.SORTOUT.DATA,
//             UNIT=DISK,SPACE=(TRK,(1,1),RLSE),DISP=(,CATLG,DELETE),
//             DCB=(LRECL=88,BLKSIZE=880,RECFM=FB)
//SYSOUT   DD  SYSOUT=*
//SYSOUT   DD SYSOUT=*                                                 
//STATOUT DD SYSOUT=*                                                   
//SYSUDUMP DD SYSOUT=*                                               
//*
//STEP002  EXEC PGM=SORT
//SORTLIB  DD DSN=SYS1.SORTLIB,DISP=SHR                                 
//SORTWK01 DD SPACE=(CYL,(100)),UNIT=DISK                               
//SYSIN    DD  *
   SORT FIELDS=(80,8,CH,A)
   OUTREC FIELDS=(1:1,80)
/*
//SORTIN   DD  DSN=TBH2.SUM.SORTOUT.DATA,DISP=SHR
//SORTOUT  DD  DSN=TBH2.SUM.SORTOUT1.DATA,
//             UNIT=DISK,SPACE=(TRK,(1,1),RLSE),DISP=(,CATLG,DELETE),
//             DCB=(LRECL=80,BLKSIZE=800,RECFM=FB)
//SYSOUT   DD  SYSOUT=*
//SYSOUT   DD SYSOUT=*                                                 
//STATOUT DD SYSOUT=*                                                   
//SYSUDUMP DD SYSOUT=*


Here i am assuming that u want to remove the duplicate on the basis of entire record .
First sort card will add an sequence number starting at 81 th byte & sort the input file on the basis of 1 to 80 th byte & remove the duplicate
Second sort will put the records in there orignal order.
Back to top
View user's profile Send private message
Frank Yaeger

DFSORT Moderator


Joined: 15 Feb 2005
Posts: 7130
Location: San Jose, CA

PostPosted: Mon Sep 24, 2007 8:51 pm    Post subject:
Reply with quote

Nagu,

SUM FIELDS=NONE requires the use of a sort key to identify the duplicate values. That sort key can be the entire record, but the sort key still must be specified to use SUM FIELDS=NONE.

If your records are not already in sorted order, but you want to remove the duplicates and keep the records in their original order, you can use a DFSORT/ICETOOL job like this:

Code:

//S1    EXEC  PGM=ICETOOL
//TOOLMSG   DD  SYSOUT=*
//DFSMSG   DD  SYSOUT=*
//IN DD *
CCCCCCCCCCCCCCCCCCCC
AAAAAAAAAAAAAAAAAAAA
DDDDDDDDDDDDDDDDDDDD
AAAAAAAAAAAAAAAAAAAA
BBBBBBBBBBBBBBBBBBBB
BBBBBBBBBBBBBBBBBBBB
CCCCCCCCCCCCCCCCCCCC
AAAAAAAAAAAAAAAAAAAA
/*
//T1 DD DSN=&&T1,UNIT=SYSDA,SPACE=(CYL,(5,5)),DISP=(,PASS)
//OUT DD SYSOUT=*
//TOOLIN   DD    *
SELECT FROM(IN) TO(T1) ON(1,80,CH) FIRST USING(CTL1)
SORT FROM(T1) TO(OUT) USING(CTL2)
/*
//CTL1CNTL DD *
  INREC OVERLAY=(81:SEQNUM,8,ZD)
//CTL2CNTL DD *
  SORT FIELDS=(81,8,ZD,A)
  OUTREC BUILD=(1,80)
/*


OUT would have:

Code:

CCCCCCCCCCCCCCCCCCCC 
AAAAAAAAAAAAAAAAAAAA 
DDDDDDDDDDDDDDDDDDDD 
BBBBBBBBBBBBBBBBBBBB 


However, if your records are already in sorted order, you can avoid sorting them by using a MERGE or a COPY.

Here's an example of a DFSORT merge job:

Code:

//S2    EXEC  PGM=ICEMAN
//SYSOUT    DD  SYSOUT=*
//SORTIN01 DD *
AAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAA
BBBBBBBBBBBBBBBBBBBB
BBBBBBBBBBBBBBBBBBBB
CCCCCCCCCCCCCCCCCCCC
CCCCCCCCCCCCCCCCCCCC
DDDDDDDDDDDDDDDDDDDD
/*
//SORTOUT DD SYSOUT=*
//SYSIN    DD    *
  OPTION EQUALS
  MERGE FIELDS=(1,80,CH,A)
  SUM FIELDS=NONE
/*


SORTOUT would have:

Code:

AAAAAAAAAAAAAAAAAAAA
BBBBBBBBBBBBBBBBBBBB
CCCCCCCCCCCCCCCCCCCC
DDDDDDDDDDDDDDDDDDDD


Here's an example of a DFSORT COPY job:

Code:

//S3    EXEC  PGM=ICEMAN
//SYSOUT    DD  SYSOUT=*
//SORTIN DD *
AAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAA
BBBBBBBBBBBBBBBBBBBB
BBBBBBBBBBBBBBBBBBBB
CCCCCCCCCCCCCCCCCCCC
CCCCCCCCCCCCCCCCCCCC
DDDDDDDDDDDDDDDDDDDD
/*
//SORTOUT DD SYSOUT=*
//SYSIN    DD    *
  OPTION COPY
  OUTFIL REMOVECC,NODETAIL,
    SECTIONS=(1,80,
      TRAILER3=(1,80))
/*


SORTOUT would have:

Code:

AAAAAAAAAAAAAAAAAAAA
BBBBBBBBBBBBBBBBBBBB
CCCCCCCCCCCCCCCCCCCC
DDDDDDDDDDDDDDDDDDDD


In the future, please show an example of your input records and expected output records so we know what you are trying to do.
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> DFSORT/ICETOOL All times are GMT + 6 Hours
Page 1 of 1

 

Search our Forum:

Similar Topics
Topic Author Forum Replies Posted
No new posts Table(Unicode(Graphic) table) loading... muralikrishnan_new DB2 0 Thu Oct 05, 2017 5:10 pm
No new posts sort records based on length exceeds ... maxsubrat DFSORT/ICETOOL 7 Wed Oct 04, 2017 4:48 pm
No new posts Addition of two records on 2 fields amar143 SYNCSORT 8 Tue Oct 03, 2017 11:14 am
No new posts Execute JCL step based on the content... sprikitik JCL & VSAM 2 Tue Oct 03, 2017 10:03 am
No new posts JOINKEYS Paired and Unpaired Sort MFwhiz DFSORT/ICETOOL 4 Fri Sep 29, 2017 1:48 am

Facebook
Back to Top
 
Job Vacancies | Forum Rules | Bookmarks | Subscriptions | FAQ | Polls | Contact Us