IBM Mainframe Forum Index
 
Log In
 
IBM Mainframe Forum Index Mainframe: Search IBM Mainframe Forum: FAQ Register
 

Syncsort - Removing duplicate based on condition
IBM Mainframe Forums -> JCL & VSAM
Post new topic   Reply to topic Quick References
View previous topic :: View next topic  
Author Message
ranjithmainframe

New User


Joined: 21 Jan 2008
Posts: 39
Location: chennai
PostPosted: Wed Jul 09, 2008 8:56 pm
 Reply with quote
Hi,

Could you please help in my following query...

I need to remove the duplicates from my input file based on particular field value..

Input records,

Code:

,BROWSE   ,APGxx.CCCCCN.CCC.FILE0703       ,Line,00000916,Col,001 080,
,Command ===>,                                                ,Scroll ===>,CSR ,
----+----1----+----2----+----3----+----4----+----5----+----6----+----7----+----8
004095667WCB EMP #2104940    01 0001800000CMANDARIN ENTERPRISES LLC      1110 S
004095667WCB EMP #2104940    02 200803241547002                   0000000000
004095667WCB EMP #2104940    02 200803241547002                   0000000000
004095667WCB EMP #2104940    02 200803241547002                   0000000000
004095667WCB EMP #2104940    03 20080630001PNLTY DATES:20050613 20050918 2008032
004095667WCB EMP #2104940    03 20080630001PNLTY DATES:20070813 20070829 2008032
004095667WCB EMP #2104940    03 20080630001PNLTY DATES:20061224 20070129 2008032
004095667WCB EMP #2104940    03 20080630001PNLTY DATES:20061224 20070129 2008032
004095667WCB EMP #2104940    03 20080630001PNLTY DATES:20061224 20070129 2008032


I need to remove the duplicates only if the 30th column value is equal to “03”.

So, my output should be as follows,

Code:

,BROWSE   ,APGxx.CCCCCN.CCC.FILE0703       ,Line,00000916,Col,001 6,Col,001 080,
,Command ===>,                                                ,Scroll ===>,CSR ,
----+----1----+----2----+----3----+----4----+----5----+----6----+----7----+----8
004095667WCB EMP #2104940    01 0001800000CMANDARIN ENTERPRISES LLC      1110 S
004095667WCB EMP #2104940    02 200803241547002                   0000000000
004095667WCB EMP #2104940    02 200803241547002                   0000000000
004095667WCB EMP #2104940    02 200803241547002                   0000000000
004095667WCB EMP #2104940    03 20080630001PNLTY DATES:20050613 20050918 2008032
004095667WCB EMP #2104940    03 20080630001PNLTY DATES:20070813 20070829 2008032
004095667WCB EMP #2104940    03 20080630001PNLTY DATES:20061224 20070129 2008032


Can I do this using external sort in JCL.

Thanks in advance,
Ranjith
Back to top
View user's profile Send private message
Frank Yaeger

DFSORT Developer


Joined: 15 Feb 2005
Posts: 7129
Location: San Jose, CA
PostPosted: Wed Jul 09, 2008 9:58 pm
 Reply with quote
Remove duplicates based on what? The entire 80-byte record? A particular fields or fields in the 80-byte record (which fields)?
Back to top
View user's profile Send private message
ranjithmainframe

New User


Joined: 21 Jan 2008
Posts: 39
Location: chennai
PostPosted: Wed Jul 09, 2008 10:30 pm
 Reply with quote
Hi,

Yes, It is required to sort based on entire 80 byte length.
Back to top
View user's profile Send private message
Frank Yaeger

DFSORT Developer


Joined: 15 Feb 2005
Posts: 7129
Location: San Jose, CA
PostPosted: Wed Jul 09, 2008 11:18 pm
 Reply with quote
Here's a DFSORT job that will do what you asked for:

Code:

//S1    EXEC  PGM=ICEMAN
//SYSOUT    DD  SYSOUT=*
//SORTIN DD DSN=...  input file (FB/80)
//SORTOUT DD DSN=...  output file (FB/80)
//SYSIN    DD    *
  OPTION COPY
  INREC IFTHEN=(WHEN=INIT,BUILD=(1:SEQNUM,8,ZD,9:1,80)),
        IFTHEN=(WHEN=(38,2,CH,EQ,C'03'),OVERLAY=(1:8C'0'))
  OUTFIL REMOVECC,NODETAIL,
    BUILD=(80X),
    SECTIONS=(1,88,
      TRAILER3=(9,80))
/*
Back to top
View user's profile Send private message
ranjithmainframe

New User


Joined: 21 Jan 2008
Posts: 39
Location: chennai
PostPosted: Thu Jul 10, 2008 7:20 am
 Reply with quote
Hello.. It is working fine..
Thanks you very much...

TO make this code generic, I have tried with an input file with different record length (300bytes) where we had same kind of issue..But I am getting error as follows and trying to resolve this... Any suggestion pls to correct this error..


OPTION COPY
INREC IFTHEN=(WHEN=INIT,BUILD=(1:SEQNUM,8,ZD,9:1,300)),
IFTHEN=(WHEN=(38,2,CH,EQ,C'03'),OVERLAY=(1:8C'0'))
OUTFIL REMOVECC,NODETAIL,
BUILD=(300X),
SECTIONS=(1,308,
*
TRAILER3=(9,300))
WER271A OUTFIL STATEMENT : NUMERIC FIELD ERROR
WER211B SYNCSMF CALLED BY SYNCSORT; RC=0000

Thanks..
Back to top
View user's profile Send private message
gcicchet

Senior Member


Joined: 28 Jul 2006
Posts: 1702
Location: Australia
PostPosted: Thu Jul 10, 2008 7:57 am
 Reply with quote
Him

your messages are SYNCSORT and not DFSORT related. Opposite side of the fence.


Gerry
Back to top
View user's profile Send private message
ranjithmainframe

New User


Joined: 21 Jan 2008
Posts: 39
Location: chennai
PostPosted: Thu Jul 10, 2008 10:14 am
 Reply with quote
Gerry,

The sysout is from ICEMAN utility only.

Thanks..
Back to top
View user's profile Send private message
Anuj Dhawan

Superior Member


Joined: 22 Apr 2006
Posts: 6250
Location: Mumbai, India
PostPosted: Thu Jul 10, 2008 11:00 am
 Reply with quote
Hello,
ranjithmainframe wrote:
The sysout is from ICEMAN utility only.

If You've coded this
Code:
//S1    EXEC  PGM=ICEMAN

that doesn't mean You used DFSORT in place of SyncSort.

PGM=SORT or PGM=ICEMAN will invoke the sort product used at your site which could be DFSORT, Syncsort or CA-SORT.
Back to top
View user's profile Send private message
ranjithmainframe

New User


Joined: 21 Jan 2008
Posts: 39
Location: chennai
PostPosted: Thu Jul 10, 2008 11:05 am
 Reply with quote
Gerry,

I understood what you are trying to say,.,.

But the JCL provided by Frank is worked fine for the file with 80 byte length. I used the same JCL modified for 300 byte file but it is throwing the mentioned error...

Please throw some more light on this...

Thanks..
Back to top
View user's profile Send private message
gcicchet

Senior Member


Joined: 28 Jul 2006
Posts: 1702
Location: Australia
PostPosted: Thu Jul 10, 2008 11:54 am
 Reply with quote
Hi,

a lot of the functions are the same for both products.

Again WER messages are SYNCSORT messages.


SECTIONS=(1,308,
*
TRAILER3=(9,300))

Is there a line between SECTIONS and TRAILER3 ?

You should post your statements as code.


Gerry
Back to top
View user's profile Send private message
ranjithmainframe

New User


Joined: 21 Jan 2008
Posts: 39
Location: chennai
PostPosted: Thu Jul 10, 2008 1:14 pm
 Reply with quote
Gerry,

There is no line between the section and the trailer parms..

Here my code,

Code:
//S1    EXEC  PGM=ICEMAN
//SORTIN DD DSN=APGRK.CCDNYWCN.NEW.INTLINK.IN,DISP=SHR
//SORTOUT DD DSN=APGRK.CCDNYWCN.NEW.INTLINK.S1,DISP=(NEW,CATLG,CATLG),
//            DCB=*.SORTIN,SPACE=(TRK,(1,3),RLSE)
//SYSOUT   DD  SYSOUT=*
//SYSIN    DD    *
  OPTION COPY
  INREC IFTHEN=(WHEN=INIT,BUILD=(1:SEQNUM,8,ZD,9:1,300)),
        IFTHEN=(WHEN=(38,2,CH,EQ,C'03'),OVERLAY=(1:8C'0'))
  OUTFIL REMOVECC,NODETAIL,
    BUILD=(300X),
    SECTIONS=(1,308,TRAILER3=(9,300))
/*

Thanks...
Back to top
View user's profile Send private message
dick scherrer

Moderator Emeritus


Joined: 23 Nov 2006
Posts: 19244
Location: Inside the Matrix
PostPosted: Thu Jul 10, 2008 8:17 pm
 Reply with quote
Hello,

In the same sysout as the WER messages will be the Syncsort release info. Please post that info as well.

There are many features that are coded the same in both products and there are others that are not.

Also, please notice that your jcl and sort control info has been "Code"ed for readability. The "Code" tag (top center of the reply panbel) will preserve indentation and uses a fixed font.
Back to top
View user's profile Send private message
Arun Raj

Moderator


Joined: 17 Oct 2006
Posts: 2481
Location: @my desk
PostPosted: Thu Jul 10, 2008 9:24 pm
 Reply with quote
Hello ranjith,

I believe, the SECTIONS parameter has an upper limit of 256 bytes.

If you are not really worried about the order of output records with '03' type , you can try the below card.

Code:
  INREC IFTHEN=(WHEN=INIT,OVERLAY=(301:SEQNUM,8,ZD)),       
        IFTHEN=(WHEN=(30,2,CH,EQ,C'03'),OVERLAY=(301:8C'0'))
  SORT FIELDS=(1,308,CH,A)                                 
  SUM FIELDS=NONE                                           
  OUTREC FIELDS=(1,300)                                     


Thanks,
Arun
Back to top
View user's profile Send private message
ranjithmainframe

New User


Joined: 21 Jan 2008
Posts: 39
Location: chennai
PostPosted: Thu Jul 10, 2008 11:49 pm
 Reply with quote
Arun,

I tried your code. It copies all the records to ouput file and there is no difference between input and output. Further, It is required not to change the order of gthe input records.

Dick,

If this is what you are asking about. Here my WER mesages in sysout.

Code:
  ********************************* TOP OF DATA **********************************
1 SYNCSORT FOR Z/OS  1.2.3.1R    U.S. PATENTS: 4210961, 5117495   (C) 2005 SYNCS
                                      DUN & BRADSTREET - NBIC-ISM    z/OS   1.7.
  PRODUCT LICENSED FOR CPU SERIAL NUMBER 5D0EC, MODEL 2064 103              LICE
  SYSIN :
    OPTION COPY
    INREC IFTHEN=(WHEN=INIT,BUILD=(1:SEQNUM,8,ZD,9:1,300)),
          IFTHEN=(WHEN=(38,2,CH,EQ,C'03'),OVERLAY=(1:8C'0'))
    OUTFIL REMOVECC,NODETAIL,
    BUILD=(300X),
    SECTIONS=(1,308,TRAILER3=(9,300))
                *
  WER271A  OUTFIL STATEMENT  : NUMERIC FIELD ERROR
  WER211B  SYNCSMF  CALLED BY SYNCSORT; RC=0000
******************************** BOTTOM OF DATA ********************************
Back to top
View user's profile Send private message
Arun Raj

Moderator


Joined: 17 Oct 2006
Posts: 2481
Location: @my desk
PostPosted: Fri Jul 11, 2008 12:22 pm
 Reply with quote
Ranjth,

I got the below results with the card which I posted.

Input
Code:
004095667WCB EMP #2104940    01 0001800000CMANDARIN ENTERPRISES LLC     
004095667WCB EMP #2104940    02 200803241547002                   000000
004095667WCB EMP #2104940    02 200803241547002                   000000
004095667WCB EMP #2104940    02 200803241547002                   000000
004095667WCB EMP #2104940    03 20080630001PNLTY DATES:20050613 2005091
004095667WCB EMP #2104940    03 20080630001PNLTY DATES:20070813 2007082
004095667WCB EMP #2104940    03 20080630001PNLTY DATES:20061224 2007012
004095667WCB EMP #2104940    03 20080630001PNLTY DATES:20061224 2007012
004095667WCB EMP #2104940    03 20080630001PNLTY DATES:20061224 2007012


Output
Code:
004095667WCB EMP #2104940    01 0001800000CMANDARIN ENTERPRISES LLC             
004095667WCB EMP #2104940    02 200803241547002                   000000       
004095667WCB EMP #2104940    02 200803241547002                   000000       
004095667WCB EMP #2104940    02 200803241547002                   000000       
004095667WCB EMP #2104940    03 20080630001PNLTY DATES:20050613 2005091         
004095667WCB EMP #2104940    03 20080630001PNLTY DATES:20061224 2007012         
004095667WCB EMP #2104940    03 20080630001PNLTY DATES:20070813 2007082         


Thanks,
Arun
Back to top
View user's profile Send private message
ranjithmainframe

New User


Joined: 21 Jan 2008
Posts: 39
Location: chennai
PostPosted: Fri Jul 11, 2008 5:56 pm
 Reply with quote
Arun,

It is great. It is working fine with your code.
Sorry about that It was my oversight tested with incorrect code previously.

Thanks everyone!!

Ranjith
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic View Bookmarks
All times are GMT + 6 Hours
Forum Index -> JCL & VSAM

 

Similar Topics
Topic Forum Replies
No new posts Duplicate transid's declared using CEDA CICS 3
No new posts Compare only first records of the fil... SYNCSORT 7
No new posts Duplicate several members of/in one l... JCL & VSAM 7
No new posts Newbie Stuck on "Duplicate Datas... TSO/ISPF 5
No new posts To search DB2 table based on Conditio... DB2 1
Search our Forums:

Back to Top
IBMMainframes.com is not an official and/or affiliated with IBM® in anyway
Board Rules | FAQ | Downloads | Wiki | SiteMap | Contact Us