IBM Mainframe Forum Index
 
Log In
 
IBM Mainframe Forum Index Mainframe: Search IBM Mainframe Forum: FAQ Register
 

Remove duplicates.


IBM Mainframe Forums -> JCL & VSAM
Post new topic   Reply to topic
View previous topic :: View next topic  
Author Message
arvind.m

Active User


Joined: 28 Aug 2008
Posts: 205
Location: Hyderabad

PostPosted: Wed Sep 02, 2009 4:28 pm
Reply with quote

Hi

i have the following input:

state |supp# |item# |upc |qty |status
Code:

03000|0008770|0111315|00001121539210|00024|CURR
03000|0008770|0111315|00007362100243|00024|CURR
03000|0008770|0111315|00007362100243|00036|PREV
03000|0008770|0111317|00001121530618|00024|CURR
03000|0008770|0111318|00001121533191|00024|CURR


Now i need to check if there are any duplicate with key state,supp,item,upc and remove the duplicates but i need to make sure that whenever i'm elimination dups the CURR should be in the output file (record with status "CURR" should not be eliminated). i have done the below sort card but its not working as expected. (the dup file has records with status "CURR")

Code:

//STEP022 EXEC PGM=SORT                                         
//SYSOUT   DD  SYSOUT=*                                         
//SYSPRINT DD  SYSOUT=*                                         
//SORTIN   DD  DISP=SHR,DSN=XXXX.EPQ.EXTRACT.DIFF123         
//SORTOUT  DD  DISP=(,CATLG,DELETE),DSN=XXXX.SORT.DIFF,     
//             UNIT=3390,SPACE=(TRK,(3500,1500),RLSE),         
//             LRECL=80,RECFM=FB                               
//SYSIN    DD  *                                               
  SORT FIELDS=(1,5,ZD,A,7,7,ZD,A,15,7,ZD,A,23,14,ZD,A,38,5,ZD,A,
                44,4,CH,A)                                     
//*                                                             
//STEP022 EXEC PGM=SORT                                         
//SYSOUT   DD  SYSOUT=*                                         
//SYSPRINT DD  SYSOUT=*                                         
//SORTIN   DD  DISP=SHR,DSN=XXXX.SORT.DIFF                   
//SORTOUT  DD  DISP=(,CATLG,DELETE),DSN=XXXX.SORT.GOOD,     
//             UNIT=3390,SPACE=(TRK,(3500,1500),RLSE),         
//             LRECL=80,RECFM=FB                               
//SORTXSUM DD  DISP=(,CATLG,DELETE),DSN=XXXX.DUP.GOOD,       
//             UNIT=3390,SPACE=(TRK,(3500,1500),RLSE),     
//             LRECL=80,RECFM=FB                           
//SYSIN    DD  *                                           
  SORT FIELDS=(1,5,ZD,A,7,7,ZD,A,15,7,ZD,A,23,14,ZD,A)     
  SUM FIELDS=NONE,XSUM                                     
//*                                                       



can anyone help me on this.
Back to top
View user's profile Send private message
arvind.m

Active User


Joined: 28 Aug 2008
Posts: 205
Location: Hyderabad

PostPosted: Wed Sep 02, 2009 4:34 pm
Reply with quote

In the above example, item 0111315 and upc 00007362100243 have duplicate combination of state,supp,item and upc. so the output should be just

Code:

03000|0008770|0111315|00007362100243|00036|PREV
Back to top
View user's profile Send private message
gcicchet

Senior Member


Joined: 28 Jul 2006
Posts: 1702
Location: Australia

PostPosted: Wed Sep 02, 2009 4:43 pm
Reply with quote

Hi,

try
Code:
  SORT FIELDS=(1,5,ZD,A,7,7,ZD,A,15,7,ZD,A,23,14,ZD,A),EQUALS     



Gerry
Back to top
View user's profile Send private message
arvind.m

Active User


Joined: 28 Aug 2008
Posts: 205
Location: Hyderabad

PostPosted: Wed Sep 02, 2009 4:49 pm
Reply with quote

Hi Gerry,

where do i need to add the sort card (i mean in which step).

Quote:

try Code:
SORT FIELDS=(1,5,ZD,A,7,7,ZD,A,15,7,ZD,A,23,14,ZD,A),EQUALS



But i changed the first sort card as:

Code:

SORT FIELDS=(1,5,ZD,A,7,7,ZD,A,15,7,ZD,A,23,14,ZD,A,
              44,4,CH,A)


i worked. can you confirm this.
Back to top
View user's profile Send private message
arvind.m

Active User


Joined: 28 Aug 2008
Posts: 205
Location: Hyderabad

PostPosted: Wed Sep 02, 2009 4:52 pm
Reply with quote

Hi Gerry,

i used the sort card which you mentioned in the second step. but it did not work.
Back to top
View user's profile Send private message
gcicchet

Senior Member


Joined: 28 Jul 2006
Posts: 1702
Location: Australia

PostPosted: Wed Sep 02, 2009 4:54 pm
Reply with quote

Hi Arvind,

the line I gave you belongs to the second step.

The first step has additional sort fields.


Gerry
Back to top
View user's profile Send private message
gcicchet

Senior Member


Joined: 28 Jul 2006
Posts: 1702
Location: Australia

PostPosted: Wed Sep 02, 2009 4:57 pm
Reply with quote

Hi,

what does the output from step1 look like ?

Gerry
Back to top
View user's profile Send private message
arvind.m

Active User


Joined: 28 Aug 2008
Posts: 205
Location: Hyderabad

PostPosted: Wed Sep 02, 2009 5:01 pm
Reply with quote

Gerry,

My first step will sort the input file and will make sure that the record with status "CURR" is on the top. (not taking qty into consideration)


and the second step will remove the duplicate records. Since the reocrds with status "CURR" are on the top they are not eliminated.

let me know if this makes any sense.
Back to top
View user's profile Send private message
gcicchet

Senior Member


Joined: 28 Jul 2006
Posts: 1702
Location: Australia

PostPosted: Wed Sep 02, 2009 5:07 pm
Reply with quote

Hi Arvind,

I have run the JCL without the parameter XSUM as it is not supported with the version of DFSORT I'm using and the results are correct.

What does the file from the first step look like ?

Gerry
Back to top
View user's profile Send private message
arvind.m

Active User


Joined: 28 Aug 2008
Posts: 205
Location: Hyderabad

PostPosted: Wed Sep 02, 2009 5:25 pm
Reply with quote

the above input file has only few records to test it.

new input to first step
Code:

03000|0008770|0111315|00001121539210|00024|CURR
03000|0008770|0111315|00007362100243|00024|CURR
03000|0008770|0111315|00007362100243|00036|PREV
03000|0008770|0111317|00001121530618|00024|CURR
03000|0008770|0164614|00003700019004|00004|PREV
03000|0008770|0164614|00003700019004|00006|CURR


output from first step
Code:

03000|0008770|0111315|00001121539210|00024|CURR
03000|0008770|0111315|00007362100243|00024|CURR
03000|0008770|0111315|00007362100243|00036|PREV
03000|0008770|0111317|00001121530618|00024|CURR
03000|0008770|0164614|00003700019004|00004|CURR
03000|0008770|0164614|00003700019004|00006|PREV



check the last two records.
Back to top
View user's profile Send private message
arvind.m

Active User


Joined: 28 Aug 2008
Posts: 205
Location: Hyderabad

PostPosted: Wed Sep 02, 2009 5:27 pm
Reply with quote

sorry....

output from first step
Code:


03000|0008770|0111315|00001121539210|00024|CURR
03000|0008770|0111315|00007362100243|00024|CURR
03000|0008770|0111315|00007362100243|00036|PREV
03000|0008770|0111317|00001121530618|00024|CURR
03000|0008770|0164614|00003700019004|00006|CURR
03000|0008770|0164614|00003700019004|00004|PREV
 

Back to top
View user's profile Send private message
gcicchet

Senior Member


Joined: 28 Jul 2006
Posts: 1702
Location: Australia

PostPosted: Wed Sep 02, 2009 5:43 pm
Reply with quote

Hi Armand,

this is getting very confusing.

How do the last 2 records go from 00004 PREV to 00004 CURR and 00006 CURR to 00006 PREV ?

I would use the following in step1
Code:
  SORT FIELDS=(1,5,ZD,A,7,7,ZD,A,15,7,ZD,A,23,14,ZD,A, 
                44,4,CH,A)                     


and the following in step2

Code:
  SORT FIELDS=(1,5,ZD,A,7,7,ZD,A,15,7,ZD,A,23,14,ZD,A),EQUALS   
  SUM FIELDS=NONE                                         



Gerry
Back to top
View user's profile Send private message
arvind.m

Active User


Joined: 28 Aug 2008
Posts: 205
Location: Hyderabad

PostPosted: Wed Sep 02, 2009 5:49 pm
Reply with quote

sorry Gerry...there was a typo...

correct output is

Code:

03000|0008770|0111315|00001121539210|00024|CURR
03000|0008770|0111315|00007362100243|00024|CURR
03000|0008770|0111315|00007362100243|00036|PREV
03000|0008770|0111317|00001121530618|00024|CURR
03000|0008770|0164614|00003700019004|00006|CURR
03000|0008770|0164614|00003700019004|00004|PREV
Back to top
View user's profile Send private message
gcicchet

Senior Member


Joined: 28 Jul 2006
Posts: 1702
Location: Australia

PostPosted: Wed Sep 02, 2009 5:56 pm
Reply with quote

Hi Arvind,

I think you are wrong again, the output from your first step is as below
Code:
03000|0008770|0111315|00001121539210|00024|CURR
03000|0008770|0111315|00007362100243|00024|CURR
03000|0008770|0111315|00007362100243|00036|PREV
03000|0008770|0111317|00001121530618|00024|CURR
03000|0008770|0164614|00003700019004|00004|PREV
03000|0008770|0164614|00003700019004|00006|CURR


That's why I suggested a different sort field card for step1.

Gerry
Back to top
View user's profile Send private message
arvind.m

Active User


Joined: 28 Aug 2008
Posts: 205
Location: Hyderabad

PostPosted: Wed Sep 02, 2009 6:15 pm
Reply with quote

No Gerry,
in the first sort card "qty" is not considered (5th field) while sorting.....

Code:

SORT FIELDS=(1,5,ZD,A,7,7,ZD,A,15,7,ZD,A,23,14,ZD,A, 
                44,4,CH,A)
Back to top
View user's profile Send private message
gcicchet

Senior Member


Joined: 28 Jul 2006
Posts: 1702
Location: Australia

PostPosted: Wed Sep 02, 2009 6:27 pm
Reply with quote

Hi Arvind,

I didn't realize you had changed the sort card in step1, now that we agree on the first sort card, are you still having problems when using the 2nd sort card I provided.


Gerry
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic View Bookmarks
All times are GMT + 6 Hours
Forum Index -> JCL & VSAM

 


Similar Topics
Topic Forum Replies
No new posts Sortjoin and Search for a String and ... DFSORT/ICETOOL 1
No new posts Remove leading zeroes SYNCSORT 4
No new posts How to remove block of duplicates DFSORT/ICETOOL 8
This topic is locked: you cannot edit posts or make replies. Compare files with duplicates in one ... DFSORT/ICETOOL 11
No new posts To Remove spaces (which is in hex for... JCL & VSAM 10
Search our Forums:

Back to Top