Portal | Manuals | References | Downloads | Info | Programs | JCLs | Mainframe wiki | Quick Ref
IBM Mainframe Computers Forums Index
 
Register
 
IBM Mainframe Computers Forums Index Mainframe: Search IBM Mainframe Forum: FAQ Memberlist Profile Log in to check your private messages Log in
 
Removing duplicates without considering header and trailer

 
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> JCL & VSAM
View previous topic :: :: View next topic  
Author Message
Vasanthr

New User


Joined: 24 Jan 2011
Posts: 21
Location: India

PostPosted: Wed Mar 23, 2011 11:53 am    Post subject: Removing duplicates without considering header and trailer
Reply with quote

Hi

I have a file with header,data and trailer as shown below

DD-MM-YYYY
1111999-12-30-12.55.11.8333720
1111999-12-30-13.55.11.8333720
1111959-12-30-15.55.11.8353720
1111999-12-30-15.55.11.8353720
DD-MM-YYYY

I want to remove the duplicates in data in position 15-25 alone and i want the header and trailer records
The output should be like this

DD-MM-YYYY
1111999-12-30-12.55.11.8333720
1111999-12-30-13.55.11.8333720
1111959-12-30-15.55.11.8353720
DD-MM-YYYY

I want this to be done in a single step of sort.

Do let me know if this could be achieved?
Back to top
View user's profile Send private message

Frank Yaeger

DFSORT Moderator


Joined: 15 Feb 2005
Posts: 7130
Location: San Jose, CA

PostPosted: Wed Mar 23, 2011 10:27 pm    Post subject:
Reply with quote

You can use a DFSORT/ICETOOL job like the following to do what you asked for:

Code:

//S1    EXEC  PGM=ICETOOL
//TOOLMSG DD SYSOUT=*
//DFSMSG  DD SYSOUT=*
//IN DD DSN=...  input file
//OUT DD DSN=...  output file
//TOOLIN DD *
DATASORT FROM(IN) TO(OUT) HEADER TRAILER USING(CTL1)
/*
//CTL1CNTL DD *
  SORT FIELDS=(15,11,CH,A)
  SUM FIELDS=NONE
/*
Back to top
View user's profile Send private message
Vasanthr

New User


Joined: 24 Jan 2011
Posts: 21
Location: India

PostPosted: Thu Mar 24, 2011 9:23 am    Post subject: Reply to: Removing duplicates without considering header and
Reply with quote

thanks .. its working fine
Back to top
View user's profile Send private message
Vasanthr

New User


Joined: 24 Jan 2011
Posts: 21
Location: India

PostPosted: Mon Mar 28, 2011 3:52 pm    Post subject: Reply to: Removing duplicates without considering header and
Reply with quote

Hi

That sort JCL works fine for files with 1 header and 1 trailer. I have some files with more than 1 header and 1 trailer.. In such cases it doesnt work as other headers are getting eliminated.

Is there any way to get all the headers and trailers with non duplicate records ?
Sample I/p

000header-----
000header-----
000header-----
111rec-------------------
111rec-------------------
111rec-------------------
111rec-------------------
999trailer----------------
999trailer---------------
Back to top
View user's profile Send private message
prino

Senior Member


Joined: 07 Feb 2009
Posts: 1072
Location: Oostende, Belgium

PostPosted: Mon Mar 28, 2011 3:58 pm    Post subject: Re: Reply to: Removing duplicates without considering header
Reply with quote

Vasanthr wrote:
That sort JCL works fine for files with 1 header and 1 trailer. I have some files with more than 1 header and 1 trailer.. In such cases it doesnt work as other headers are getting eliminated.


Why didn't you mention this requirement in the first place????
Back to top
View user's profile Send private message
Frank Yaeger

DFSORT Moderator


Joined: 15 Feb 2005
Posts: 7130
Location: San Jose, CA

PostPosted: Mon Mar 28, 2011 10:53 pm    Post subject:
Reply with quote

Vasanth,

If you want a complete solution, you need to provide a complete requirement.

Can there be any number of headers and any number of trailers? What is the maximum number? Or are there always a specific number of headers and trailers (e.g. 3 and 2 in your example)?

How do you identify a header? How do you identify a trailer?

What is the RECFM and LRECL of your input file? What is the starting position, length and format of each relevant field?

Also, run this job and show the resulting //SYSOUT messages so we can see which product and level you have:

Code:

//S1    EXEC  PGM=SORT
//SYSOUT    DD  SYSOUT=*
//SORTIN DD *
RECORD
//SORTOUT DD DUMMY
//SYSIN    DD    *
    OPTION COPY
/*
Back to top
View user's profile Send private message
Vasanthr

New User


Joined: 24 Jan 2011
Posts: 21
Location: India

PostPosted: Tue Mar 29, 2011 2:00 pm    Post subject: Reply to: Removing duplicates without considering header and
Reply with quote

Here are the details you have asked for:

There can be any number of header and trailers.No maximum limit.

you can dentify header with "000" in the 1-3 posistion and trailer has "999" in 1-3 posistion

LRECL=80,RECFM=FB

Duplicates needs to be removed on SORT FIELDS=(15,11,CH,A)

Below is the SYSOUT for the Job you have asked to run.

Code:
SYNCSORT FOR Z/OS  1.3.2.1R    U.S. PATENTS: 4210961, 5117495  (C) 2007 SYNCSORT INC.                                                     

SYNCSORT LICENSED FOR CPU SERIAL NUMBER 40BE0, MODEL 2817 735
SYSIN :
    OPTION COPY
WER276B  SYSDIAG= 1386520, 2951853, 2951853, 1741425
WER164B  6,884K BYTES OF VIRTUAL STORAGE AVAILABLE, MAX REQUESTED,
WER164B     0 BYTES RESERVE REQUESTED, 272K BYTES USED
WER146B  32K BYTES OF EMERGENCY SPACE ALLOCATED
WER108I  SORTIN   : RECFM=FB   ; LRECL=    80; BLKSIZE=    80
WER110I  SORTOUT  : RECFM=FB   ; LRECL=    80; BLKSIZE=    80
WER410B  5,856K BYTES OF VIRTUAL STORAGE AVAILABLE ABOVE THE 16MEG LIN
WER410B     0 BYTES RESERVE REQUESTED, 156K BYTES USED
WER211B  SYNCSMF  CALLED BY SYNCSORT; RC=0000
WER449I  SYNCSORT GLOBAL DSM SUBSYSTEM ACTIVE
WER416B  BSAM WAS USED FOR SORTIN
WER416B  BSAM WAS USED FOR SORTOUT
WER054I  RCD IN          1, OUT          1
WER169I  RELEASE 1.3 BATCH 0506 TPF LEVEL 2.1
WER052I  END SYNCSORT - P89BFL1T,S1,,DIAG=AE00,6082,820E,004E,E6F2,48A
Back to top
View user's profile Send private message
Frank Yaeger

DFSORT Moderator


Joined: 15 Feb 2005
Posts: 7130
Location: San Jose, CA

PostPosted: Tue Mar 29, 2011 11:07 pm    Post subject:
Reply with quote

The messages indicate you're using Syncsort, not DFSORT. I'm a DFSORT developer. DFSORT and Syncsort are competitive products. I'm happy to answer questions on DFSORT and DFSORT's ICETOOL, but I don't answer questions on Syncsort.
Back to top
View user's profile Send private message
nareshdacha

New User


Joined: 12 Jan 2010
Posts: 66
Location: US

PostPosted: Wed Mar 30, 2011 12:28 pm    Post subject: Reply to: Removing duplicates without considering header and
Reply with quote

Hi,

If there are 3 headers and 2 trailors.. you can use the below code.....

Code:

 //TOOLIN    DD    *                                         
   DATASORT FROM(IN) TO(OUT) HEADER(3) TRAILER(2) USING(CTL1)
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> JCL & VSAM All times are GMT + 6 Hours
Page 1 of 1

 

Search our Forum:

Similar Topics
Topic Author Forum Replies Posted
No new posts Updating multiple counts in trailer!! Vignesh Sid SYNCSORT 4 Mon Nov 06, 2017 1:04 pm
No new posts Updating the Trailer count in variabl... satheshbabur DFSORT/ICETOOL 6 Wed Aug 30, 2017 9:49 pm
No new posts SORT Trailer Count - LRECL Output co... amorante DFSORT/ICETOOL 5 Tue Aug 29, 2017 8:57 pm
No new posts Join records from 2 files with No Dup... Poha Eater DFSORT/ICETOOL 22 Sun Aug 27, 2017 10:35 pm
No new posts How to add header with Date(YYMMDD) i... Rajan Moorthy DFSORT/ICETOOL 2 Thu Jul 06, 2017 11:44 pm

Facebook
Back to Top
 
Job Vacancies | Forum Rules | Bookmarks | Subscriptions | FAQ | Polls | Contact Us