Portal | Manuals | References | Downloads | Info | Programs | JCLs | Master the Mainframes
IBM Mainframe Computers Forums Index
 
Register
 
IBM Mainframe Computers Forums Index Mainframe: Search IBM Mainframe Forum: FAQ Memberlist Usergroups Profile Log in to check your private messages Log in
 

 

Syncsort - Removing duplicate based on condition

 
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> JCL & VSAM
View previous topic :: :: View next topic  
Author Message
ranjithmainframe

New User


Joined: 21 Jan 2008
Posts: 39
Location: chennai

PostPosted: Wed Jul 09, 2008 8:56 pm    Post subject: Syncsort - Removing duplicate based on condition
Reply with quote

Hi,

Could you please help in my following query...

I need to remove the duplicates from my input file based on particular field value..

Input records,

Code:

,BROWSE   ,APGxx.CCCCCN.CCC.FILE0703       ,Line,00000916,Col,001 080,
,Command ===>,                                                ,Scroll ===>,CSR ,
----+----1----+----2----+----3----+----4----+----5----+----6----+----7----+----8
004095667WCB EMP #2104940    01 0001800000CMANDARIN ENTERPRISES LLC      1110 S
004095667WCB EMP #2104940    02 200803241547002                   0000000000
004095667WCB EMP #2104940    02 200803241547002                   0000000000
004095667WCB EMP #2104940    02 200803241547002                   0000000000
004095667WCB EMP #2104940    03 20080630001PNLTY DATES:20050613 20050918 2008032
004095667WCB EMP #2104940    03 20080630001PNLTY DATES:20070813 20070829 2008032
004095667WCB EMP #2104940    03 20080630001PNLTY DATES:20061224 20070129 2008032
004095667WCB EMP #2104940    03 20080630001PNLTY DATES:20061224 20070129 2008032
004095667WCB EMP #2104940    03 20080630001PNLTY DATES:20061224 20070129 2008032


I need to remove the duplicates only if the 30th column value is equal to “03”.

So, my output should be as follows,

Code:

,BROWSE   ,APGxx.CCCCCN.CCC.FILE0703       ,Line,00000916,Col,001 6,Col,001 080,
,Command ===>,                                                ,Scroll ===>,CSR ,
----+----1----+----2----+----3----+----4----+----5----+----6----+----7----+----8
004095667WCB EMP #2104940    01 0001800000CMANDARIN ENTERPRISES LLC      1110 S
004095667WCB EMP #2104940    02 200803241547002                   0000000000
004095667WCB EMP #2104940    02 200803241547002                   0000000000
004095667WCB EMP #2104940    02 200803241547002                   0000000000
004095667WCB EMP #2104940    03 20080630001PNLTY DATES:20050613 20050918 2008032
004095667WCB EMP #2104940    03 20080630001PNLTY DATES:20070813 20070829 2008032
004095667WCB EMP #2104940    03 20080630001PNLTY DATES:20061224 20070129 2008032


Can I do this using external sort in JCL.

Thanks in advance,
Ranjith
Back to top
View user's profile Send private message

Frank Yaeger

DFSORT Moderator


Joined: 15 Feb 2005
Posts: 7130
Location: San Jose, CA

PostPosted: Wed Jul 09, 2008 9:58 pm    Post subject:
Reply with quote

Remove duplicates based on what? The entire 80-byte record? A particular fields or fields in the 80-byte record (which fields)?
Back to top
View user's profile Send private message
ranjithmainframe

New User


Joined: 21 Jan 2008
Posts: 39
Location: chennai

PostPosted: Wed Jul 09, 2008 10:30 pm    Post subject: Reply to: Removing duplicate based on condition
Reply with quote

Hi,

Yes, It is required to sort based on entire 80 byte length.
Back to top
View user's profile Send private message
Frank Yaeger

DFSORT Moderator


Joined: 15 Feb 2005
Posts: 7130
Location: San Jose, CA

PostPosted: Wed Jul 09, 2008 11:18 pm    Post subject:
Reply with quote

Here's a DFSORT job that will do what you asked for:

Code:

//S1    EXEC  PGM=ICEMAN
//SYSOUT    DD  SYSOUT=*
//SORTIN DD DSN=...  input file (FB/80)
//SORTOUT DD DSN=...  output file (FB/80)
//SYSIN    DD    *
  OPTION COPY
  INREC IFTHEN=(WHEN=INIT,BUILD=(1:SEQNUM,8,ZD,9:1,80)),
        IFTHEN=(WHEN=(38,2,CH,EQ,C'03'),OVERLAY=(1:8C'0'))
  OUTFIL REMOVECC,NODETAIL,
    BUILD=(80X),
    SECTIONS=(1,88,
      TRAILER3=(9,80))
/*
Back to top
View user's profile Send private message
ranjithmainframe

New User


Joined: 21 Jan 2008
Posts: 39
Location: chennai

PostPosted: Thu Jul 10, 2008 7:20 am    Post subject: Reply to: Removing duplicate based on condition
Reply with quote

Hello.. It is working fine..
Thanks you very much...

TO make this code generic, I have tried with an input file with different record length (300bytes) where we had same kind of issue..But I am getting error as follows and trying to resolve this... Any suggestion pls to correct this error..


OPTION COPY
INREC IFTHEN=(WHEN=INIT,BUILD=(1:SEQNUM,8,ZD,9:1,300)),
IFTHEN=(WHEN=(38,2,CH,EQ,C'03'),OVERLAY=(1:8C'0'))
OUTFIL REMOVECC,NODETAIL,
BUILD=(300X),
SECTIONS=(1,308,
*
TRAILER3=(9,300))
WER271A OUTFIL STATEMENT : NUMERIC FIELD ERROR
WER211B SYNCSMF CALLED BY SYNCSORT; RC=0000

Thanks..
Back to top
View user's profile Send private message
gcicchet

Senior Member


Joined: 28 Jul 2006
Posts: 1703
Location: Australia

PostPosted: Thu Jul 10, 2008 7:57 am    Post subject:
Reply with quote

Him

your messages are SYNCSORT and not DFSORT related. Opposite side of the fence.


Gerry
Back to top
View user's profile Send private message
ranjithmainframe

New User


Joined: 21 Jan 2008
Posts: 39
Location: chennai

PostPosted: Thu Jul 10, 2008 10:14 am    Post subject: Reply to: Removing duplicate based on condition
Reply with quote

Gerry,

The sysout is from ICEMAN utility only.

Thanks..
Back to top
View user's profile Send private message
Anuj Dhawan

Senior Member


Joined: 22 Apr 2006
Posts: 6258
Location: Mumbai, India

PostPosted: Thu Jul 10, 2008 11:00 am    Post subject: Re: Reply to: Removing duplicate based on condition
Reply with quote

Hello,
ranjithmainframe wrote:
The sysout is from ICEMAN utility only.

If You've coded this
Code:
//S1    EXEC  PGM=ICEMAN

that doesn't mean You used DFSORT in place of SyncSort.

PGM=SORT or PGM=ICEMAN will invoke the sort product used at your site which could be DFSORT, Syncsort or CA-SORT.
Back to top
View user's profile Send private message
ranjithmainframe

New User


Joined: 21 Jan 2008
Posts: 39
Location: chennai

PostPosted: Thu Jul 10, 2008 11:05 am    Post subject:
Reply with quote

Gerry,

I understood what you are trying to say,.,.

But the JCL provided by Frank is worked fine for the file with 80 byte length. I used the same JCL modified for 300 byte file but it is throwing the mentioned error...

Please throw some more light on this...

Thanks..
Back to top
View user's profile Send private message
gcicchet

Senior Member


Joined: 28 Jul 2006
Posts: 1703
Location: Australia

PostPosted: Thu Jul 10, 2008 11:54 am    Post subject:
Reply with quote

Hi,

a lot of the functions are the same for both products.

Again WER messages are SYNCSORT messages.


SECTIONS=(1,308,
*
TRAILER3=(9,300))

Is there a line between SECTIONS and TRAILER3 ?

You should post your statements as code.


Gerry
Back to top
View user's profile Send private message
ranjithmainframe

New User


Joined: 21 Jan 2008
Posts: 39
Location: chennai

PostPosted: Thu Jul 10, 2008 1:14 pm    Post subject: Reply to: Removing duplicate based on condition
Reply with quote

Gerry,

There is no line between the section and the trailer parms..

Here my code,

Code:
//S1    EXEC  PGM=ICEMAN
//SORTIN DD DSN=APGRK.CCDNYWCN.NEW.INTLINK.IN,DISP=SHR
//SORTOUT DD DSN=APGRK.CCDNYWCN.NEW.INTLINK.S1,DISP=(NEW,CATLG,CATLG),
//            DCB=*.SORTIN,SPACE=(TRK,(1,3),RLSE)
//SYSOUT   DD  SYSOUT=*
//SYSIN    DD    *
  OPTION COPY
  INREC IFTHEN=(WHEN=INIT,BUILD=(1:SEQNUM,8,ZD,9:1,300)),
        IFTHEN=(WHEN=(38,2,CH,EQ,C'03'),OVERLAY=(1:8C'0'))
  OUTFIL REMOVECC,NODETAIL,
    BUILD=(300X),
    SECTIONS=(1,308,TRAILER3=(9,300))
/*

Thanks...
Back to top
View user's profile Send private message
dick scherrer

Site Director


Joined: 23 Nov 2006
Posts: 19270
Location: Inside the Matrix

PostPosted: Thu Jul 10, 2008 8:17 pm    Post subject:
Reply with quote

Hello,

In the same sysout as the WER messages will be the Syncsort release info. Please post that info as well.

There are many features that are coded the same in both products and there are others that are not.

Also, please notice that your jcl and sort control info has been "Code"ed for readability. The "Code" tag (top center of the reply panbel) will preserve indentation and uses a fixed font.
Back to top
View user's profile Send private message
Arun Raj

Moderator


Joined: 17 Oct 2006
Posts: 2158
Location: @my desk

PostPosted: Thu Jul 10, 2008 9:24 pm    Post subject:
Reply with quote

Hello ranjith,

I believe, the SECTIONS parameter has an upper limit of 256 bytes.

If you are not really worried about the order of output records with '03' type , you can try the below card.

Code:
  INREC IFTHEN=(WHEN=INIT,OVERLAY=(301:SEQNUM,8,ZD)),       
        IFTHEN=(WHEN=(30,2,CH,EQ,C'03'),OVERLAY=(301:8C'0'))
  SORT FIELDS=(1,308,CH,A)                                 
  SUM FIELDS=NONE                                           
  OUTREC FIELDS=(1,300)                                     


Thanks,
Arun
Back to top
View user's profile Send private message
ranjithmainframe

New User


Joined: 21 Jan 2008
Posts: 39
Location: chennai

PostPosted: Thu Jul 10, 2008 11:49 pm    Post subject: Reply to: Syncsort - Removing duplicate based on condition
Reply with quote

Arun,

I tried your code. It copies all the records to ouput file and there is no difference between input and output. Further, It is required not to change the order of gthe input records.

Dick,

If this is what you are asking about. Here my WER mesages in sysout.

Code:
  ********************************* TOP OF DATA **********************************
1 SYNCSORT FOR Z/OS  1.2.3.1R    U.S. PATENTS: 4210961, 5117495   (C) 2005 SYNCS
                                      DUN & BRADSTREET - NBIC-ISM    z/OS   1.7.
  PRODUCT LICENSED FOR CPU SERIAL NUMBER 5D0EC, MODEL 2064 103              LICE
  SYSIN :
    OPTION COPY
    INREC IFTHEN=(WHEN=INIT,BUILD=(1:SEQNUM,8,ZD,9:1,300)),
          IFTHEN=(WHEN=(38,2,CH,EQ,C'03'),OVERLAY=(1:8C'0'))
    OUTFIL REMOVECC,NODETAIL,
    BUILD=(300X),
    SECTIONS=(1,308,TRAILER3=(9,300))
                *
  WER271A  OUTFIL STATEMENT  : NUMERIC FIELD ERROR
  WER211B  SYNCSMF  CALLED BY SYNCSORT; RC=0000
******************************** BOTTOM OF DATA ********************************
Back to top
View user's profile Send private message
Arun Raj

Moderator


Joined: 17 Oct 2006
Posts: 2158
Location: @my desk

PostPosted: Fri Jul 11, 2008 12:22 pm    Post subject:
Reply with quote

Ranjth,

I got the below results with the card which I posted.

Input
Code:
004095667WCB EMP #2104940    01 0001800000CMANDARIN ENTERPRISES LLC     
004095667WCB EMP #2104940    02 200803241547002                   000000
004095667WCB EMP #2104940    02 200803241547002                   000000
004095667WCB EMP #2104940    02 200803241547002                   000000
004095667WCB EMP #2104940    03 20080630001PNLTY DATES:20050613 2005091
004095667WCB EMP #2104940    03 20080630001PNLTY DATES:20070813 2007082
004095667WCB EMP #2104940    03 20080630001PNLTY DATES:20061224 2007012
004095667WCB EMP #2104940    03 20080630001PNLTY DATES:20061224 2007012
004095667WCB EMP #2104940    03 20080630001PNLTY DATES:20061224 2007012


Output
Code:
004095667WCB EMP #2104940    01 0001800000CMANDARIN ENTERPRISES LLC             
004095667WCB EMP #2104940    02 200803241547002                   000000       
004095667WCB EMP #2104940    02 200803241547002                   000000       
004095667WCB EMP #2104940    02 200803241547002                   000000       
004095667WCB EMP #2104940    03 20080630001PNLTY DATES:20050613 2005091         
004095667WCB EMP #2104940    03 20080630001PNLTY DATES:20061224 2007012         
004095667WCB EMP #2104940    03 20080630001PNLTY DATES:20070813 2007082         


Thanks,
Arun
Back to top
View user's profile Send private message
ranjithmainframe

New User


Joined: 21 Jan 2008
Posts: 39
Location: chennai

PostPosted: Fri Jul 11, 2008 5:56 pm    Post subject: Reply to: Syncsort - Removing duplicate based on condition
Reply with quote

Arun,

It is great. It is working fine with your code.
Sorry about that It was my oversight tested with incorrect code previously.

Thanks everyone!!

Ranjith
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> JCL & VSAM All times are GMT + 6 Hours
Page 1 of 1

 

Search our Forum:

Similar Topics
Topic Author Forum Replies Posted
No new posts syncsort: copy lines after the keyword shreya19 SYNCSORT 3 Fri Dec 02, 2016 9:47 am
No new posts Limit duplicate records in the SORT pshongal SYNCSORT 6 Mon Nov 21, 2016 12:54 pm
No new posts Syncsort - Hard coded Value in output PORYES SYNCSORT 1 Thu Nov 03, 2016 9:23 am
This topic is locked: you cannot edit posts or make replies. How to pass the previous month date i... Suganya87 DFSORT/ICETOOL 5 Mon Oct 31, 2016 4:13 pm
No new posts Sort records based on numeric field. Alks SYNCSORT 2 Wed Oct 19, 2016 10:14 pm


Facebook
Back to Top
 
Mainframe Wiki | Forum Rules | Bookmarks | Subscriptions | FAQ | Tutorials | Contact Us