IBM Mainframe Forum Index
 
Log In
 
IBM Mainframe Forum Index Mainframe: Search IBM Mainframe Forum: FAQ Register
 

Remove duplicates with some exceptions!


IBM Mainframe Forums -> JCL & VSAM
Post new topic   Reply to topic
View previous topic :: View next topic  
Author Message
fredrick andrews

New User


Joined: 10 Jun 2010
Posts: 52
Location: chennai

PostPosted: Thu Nov 11, 2010 8:11 pm
Reply with quote

We have a requirement where I will have to remove the records ( the entire LRECL) that are duplicates.. However, there are some exceptions which i do not want to process. Is there a way to do this??
Back to top
View user's profile Send private message
enrico-sorichetti

Superior Member


Joined: 14 Mar 2007
Posts: 10872
Location: italy

PostPosted: Thu Nov 11, 2010 9:07 pm
Reply with quote

Quote:
Is there a way to do this??


most probably yes, but it would be wiser to post some more details...

the dfsort level You are using ( You can find out by running something like )
Code:
//SORTSTEP EXEC PGM=SORT
//SYSOUT   DD SYSOUT=*
//SORTIN   DD *
ABC
//SORTOUT  DD SYSOUT=*
//SYSIN    DD *
  SORT     FIELDS=COPY


input/output dcb info ( recfm, lrecl )
a sample of the input
a sample of the output
a logic description of the duplicate processing and exception criteria

the usual things that should be provided every time for any sort related issue
without having to be asked every time!
Back to top
View user's profile Send private message
fredrick andrews

New User


Joined: 10 Jun 2010
Posts: 52
Location: chennai

PostPosted: Tue Nov 16, 2010 1:13 am
Reply with quote

Version: SYNCSORT FOR Z/OS 1.3.2.2R


Input/output dcb info ( recfm, lrecl )

For both Input and Output datasets

Record format . . . : FB
Record length . . . : 500


A sample of the input


Input File:

Code:



HEADER-1                                 
                                       
TALA311057MIR   10201458THALADAAA  A   
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC
DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD
EEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEE
SUB-PRE-FOOTER-1
SUB-FOOTER-1
TALA311057MIR   10201458THALADAAA  B   
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC
DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD
EEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEE
PRE-FOOTER-1
FOOTER-1
HEADER-2
                                       
TALA315024MIR   10201812THALADAAA  C   
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
YYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY
ZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZ
PPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPP
MMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM
PRE-FOOTER-2
FOOTER-2
HEADER-3
                                       
TALA314231MIR   10201758THALADAAA  D   
RRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRR
SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
TTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTT
UUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUU
0VVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVV
PRE-FOOTER-3
FOOTER-3




A sample of the output


Code:



HEADER-1                                 
                                       
TALA311057MIR   10201458THALADAAA  A   
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC
DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD
EEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEE
SUB-PRE-FOOTER-1
SUB-FOOTER-1
TALA311057MIR   10201458THALADAAA  B   
PRE-FOOTER-1
FOOTER-1
HEADER-2
                                       
TALA315024MIR   10201812THALADAAA  C   
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
YYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY
ZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZ
PPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPP
MMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM
PRE-FOOTER-2
FOOTER-2
HEADER-3
                                       
TALA314231MIR   10201758THALADAAA  D   
RRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRR
SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
TTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTT
UUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUU
0VVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVVV
PRE-FOOTER-3
FOOTER-3





A logic description of the duplicate processing and exception criteria





If there are more than 2 TALA records under one single HEADER,

I will have to block delete the duplicate.i.e, I'll have to leave just one instance of it and delete rest.


The FOOTER and PRE-FOOTER should not be processed for duplicates as they are all the same...

We do it manualy now.. But, We are automating it.


Any suggestions and help is appreciated!
Back to top
View user's profile Send private message
Arun Raj

Moderator


Joined: 17 Oct 2006
Posts: 2481
Location: @my desk

PostPosted: Tue Nov 16, 2010 10:12 am
Reply with quote

Quote:
I will have to block delete the duplicate.i.e, I'll have to leave just one instance of it and delete rest
What is the rule which defines "a block" here? Also you have "SUB-PRE-FOOTER" and "SUB-FOOTER" for the first "TALA" group but for the rest of the group it is just "PRE-FOOTER" and "FOOTER". Will this be the case always?
Quote:
The FOOTER and PRE-FOOTER should not be processed for duplicates as they are all the same
Why do you just need to keep the footers for the deleted group?
Back to top
View user's profile Send private message
fredrick andrews

New User


Joined: 10 Jun 2010
Posts: 52
Location: chennai

PostPosted: Tue Nov 16, 2010 12:02 pm
Reply with quote

Arun Raj wrote:
Quote:
I will have to block delete the duplicate.i.e, I'll have to leave just one instance of it and delete rest
What is the rule which defines "a block" here? Also you have "SUB-PRE-FOOTER" and "SUB-FOOTER" for the first "TALA" group but for the rest of the group it is just "PRE-FOOTER" and "FOOTER". Will this be the case always?
Quote:
The FOOTER and PRE-FOOTER should not be processed for duplicates as they are all the same
Why do you just need to keep the footers for the deleted group?


Hi, Thanks for the reply.

Sometimes, there will be more than one set of TALA records under one HEADER. That is why i put them as sub-pre-footer .

Technically, both the FOOTER, and SUB-FOOTER are the same.


Yes, for the duplicate batch to be deleted, I actually don't want the FOOTER AND PRE_FOOTER . If this can be done here, it is great.
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic View Bookmarks
All times are GMT + 6 Hours
Forum Index -> JCL & VSAM

 


Similar Topics
Topic Forum Replies
No new posts Sortjoin and Search for a String and ... DFSORT/ICETOOL 1
No new posts Remove leading zeroes SYNCSORT 4
No new posts How to remove block of duplicates DFSORT/ICETOOL 8
This topic is locked: you cannot edit posts or make replies. Compare files with duplicates in one ... DFSORT/ICETOOL 11
No new posts To Remove spaces (which is in hex for... JCL & VSAM 10
Search our Forums:

Back to Top