Portal | Manuals | References | Downloads | Info | Programs | JCLs | Master the Mainframes
IBM Mainframe Computers Forums Index
 
Register
 
IBM Mainframe Computers Forums Index Mainframe: Search IBM Mainframe Forum: FAQ Memberlist Usergroups Profile Log in to check your private messages Log in
 

 

Remove duplicates with some exceptions!

 
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> JCL & VSAM
View previous topic :: :: View next topic  
Author Message
fredrick andrews

New User


Joined: 10 Jun 2010
Posts: 52
Location: chennai

PostPosted: Thu Nov 11, 2010 8:11 pm    Post subject: Remove duplicates with some exceptions!
Reply with quote

We have a requirement where I will have to remove the records ( the entire LRECL) that are duplicates.. However, there are some exceptions which i do not want to process. Is there a way to do this??
Back to top
View user's profile Send private message

enrico-sorichetti

Global Moderator


Joined: 14 Mar 2007
Posts: 10211
Location: italy

PostPosted: Thu Nov 11, 2010 9:07 pm    Post subject: Reply to: Remove duplicates with some exceptions!
Reply with quote

Quote:
Is there a way to do this??


most probably yes, but it would be wiser to post some more details...

the dfsort level You are using ( You can find out by running something like )
Code:
//SORTSTEP EXEC PGM=SORT
//SYSOUT   DD SYSOUT=*
//SORTIN   DD *
ABC
//SORTOUT  DD SYSOUT=*
//SYSIN    DD *
  SORT     FIELDS=COPY


input/output dcb info ( recfm, lrecl )
a sample of the input
a sample of the output
a logic description of the duplicate processing and exception criteria

the usual things that should be provided every time for any sort related issue
without having to be asked every time!
Back to top
View user's profile Send private message
fredrick andrews

New User


Joined: 10 Jun 2010
Posts: 52
Location: chennai

PostPosted: Tue Nov 16, 2010 1:13 am    Post subject:
Reply with quote

Version: SYNCSORT FOR Z/OS 1.3.2.2R


Input/output dcb info ( recfm, lrecl )

For both Input and Output datasets

Record format . . . : FB
Record length . . . : 500


A sample of the input


Input File:

Code:



HEADER-1                                 
                                       
TALA311057MIR   10201458THALADAAA  A   
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC
DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD
EEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEE
SUB-PRE-FOOTER-1
SUB-FOOTER-1
TALA311057MIR   10201458THALADAAA  B   
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC
DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD
EEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEE
PRE-FOOTER-1
FOOTER-1
HEADER-2
                                       
TALA315024MIR   10201812THALADAAA  C   
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
YYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY
ZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZ
PPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPP
MMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM
PRE-FOOTER-2
FOOTER-2
HEADER-3
                                       
TALA314231MIR   10201758THALADAAA  D   
RRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRR
SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
TTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTT
UUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUU
0VVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVV
PRE-FOOTER-3
FOOTER-3




A sample of the output


Code:



HEADER-1                                 
                                       
TALA311057MIR   10201458THALADAAA  A   
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC
DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD
EEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEE
SUB-PRE-FOOTER-1
SUB-FOOTER-1
TALA311057MIR   10201458THALADAAA  B   
PRE-FOOTER-1
FOOTER-1
HEADER-2
                                       
TALA315024MIR   10201812THALADAAA  C   
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
YYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY
ZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZ
PPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPP
MMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM
PRE-FOOTER-2
FOOTER-2
HEADER-3
                                       
TALA314231MIR   10201758THALADAAA  D   
RRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRR
SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
TTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTT
UUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUU
0VVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVV
PRE-FOOTER-3
FOOTER-3





A logic description of the duplicate processing and exception criteria





If there are more than 2 TALA records under one single HEADER,

I will have to block delete the duplicate.i.e, I'll have to leave just one instance of it and delete rest.


The FOOTER and PRE-FOOTER should not be processed for duplicates as they are all the same...

We do it manualy now.. But, We are automating it.


Any suggestions and help is appreciated!
Back to top
View user's profile Send private message
Arun Raj

Moderator


Joined: 17 Oct 2006
Posts: 2180
Location: @my desk

PostPosted: Tue Nov 16, 2010 10:12 am    Post subject:
Reply with quote

Quote:
I will have to block delete the duplicate.i.e, I'll have to leave just one instance of it and delete rest
What is the rule which defines "a block" here? Also you have "SUB-PRE-FOOTER" and "SUB-FOOTER" for the first "TALA" group but for the rest of the group it is just "PRE-FOOTER" and "FOOTER". Will this be the case always?
Quote:
The FOOTER and PRE-FOOTER should not be processed for duplicates as they are all the same
Why do you just need to keep the footers for the deleted group?
Back to top
View user's profile Send private message
fredrick andrews

New User


Joined: 10 Jun 2010
Posts: 52
Location: chennai

PostPosted: Tue Nov 16, 2010 12:02 pm    Post subject:
Reply with quote

Arun Raj wrote:
Quote:
I will have to block delete the duplicate.i.e, I'll have to leave just one instance of it and delete rest
What is the rule which defines "a block" here? Also you have "SUB-PRE-FOOTER" and "SUB-FOOTER" for the first "TALA" group but for the rest of the group it is just "PRE-FOOTER" and "FOOTER". Will this be the case always?
Quote:
The FOOTER and PRE-FOOTER should not be processed for duplicates as they are all the same
Why do you just need to keep the footers for the deleted group?


Hi, Thanks for the reply.

Sometimes, there will be more than one set of TALA records under one HEADER. That is why i put them as sub-pre-footer .

Technically, both the FOOTER, and SUB-FOOTER are the same.


Yes, for the duplicate batch to be deleted, I actually don't want the FOOTER AND PRE_FOOTER . If this can be done here, it is great.
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> JCL & VSAM All times are GMT + 6 Hours
Page 1 of 1

 

Search our Forum:

Similar Topics
Topic Author Forum Replies Posted
No new posts Removing Duplicates based on certain ... chandracdac DFSORT/ICETOOL 8 Fri Dec 09, 2016 4:40 am
No new posts Updating the counters after eliminati... PANDU1 DFSORT/ICETOOL 12 Mon Nov 21, 2016 9:47 am
No new posts Remove Special Characters from Mainfr... Rodger Zhang All Other Mainframe Topics 6 Wed Jul 06, 2016 1:12 am
No new posts Remove header only knobi SYNCSORT 5 Mon Feb 15, 2016 10:42 pm
No new posts Append to file, but also remove old r... Garry Carroll JCL & VSAM 3 Mon Jan 25, 2016 8:05 pm


Facebook
Back to Top
 
Mainframe Wiki | Forum Rules | Bookmarks | Subscriptions | FAQ | Tutorials | Contact Us