Portal | Manuals | References | Downloads | Info | Programs | JCLs | Master the Mainframes
IBM Mainframe Computers Forums Index
 
Register
 
IBM Mainframe Computers Forums Index Mainframe: Search IBM Mainframe Forum: FAQ Memberlist Usergroups Profile Log in to check your private messages Log in
 

 

Remove duplicates.

 
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> JCL & VSAM
View previous topic :: :: View next topic  
Author Message
arvind.m

Active User


Joined: 28 Aug 2008
Posts: 205
Location: Hyderabad

PostPosted: Wed Sep 02, 2009 4:28 pm    Post subject: Remove duplicates.
Reply with quote

Hi

i have the following input:

state |supp# |item# |upc |qty |status
Code:

03000|0008770|0111315|00001121539210|00024|CURR
03000|0008770|0111315|00007362100243|00024|CURR
03000|0008770|0111315|00007362100243|00036|PREV
03000|0008770|0111317|00001121530618|00024|CURR
03000|0008770|0111318|00001121533191|00024|CURR


Now i need to check if there are any duplicate with key state,supp,item,upc and remove the duplicates but i need to make sure that whenever i'm elimination dups the CURR should be in the output file (record with status "CURR" should not be eliminated). i have done the below sort card but its not working as expected. (the dup file has records with status "CURR")

Code:

//STEP022 EXEC PGM=SORT                                         
//SYSOUT   DD  SYSOUT=*                                         
//SYSPRINT DD  SYSOUT=*                                         
//SORTIN   DD  DISP=SHR,DSN=XXXX.EPQ.EXTRACT.DIFF123         
//SORTOUT  DD  DISP=(,CATLG,DELETE),DSN=XXXX.SORT.DIFF,     
//             UNIT=3390,SPACE=(TRK,(3500,1500),RLSE),         
//             LRECL=80,RECFM=FB                               
//SYSIN    DD  *                                               
  SORT FIELDS=(1,5,ZD,A,7,7,ZD,A,15,7,ZD,A,23,14,ZD,A,38,5,ZD,A,
                44,4,CH,A)                                     
//*                                                             
//STEP022 EXEC PGM=SORT                                         
//SYSOUT   DD  SYSOUT=*                                         
//SYSPRINT DD  SYSOUT=*                                         
//SORTIN   DD  DISP=SHR,DSN=XXXX.SORT.DIFF                   
//SORTOUT  DD  DISP=(,CATLG,DELETE),DSN=XXXX.SORT.GOOD,     
//             UNIT=3390,SPACE=(TRK,(3500,1500),RLSE),         
//             LRECL=80,RECFM=FB                               
//SORTXSUM DD  DISP=(,CATLG,DELETE),DSN=XXXX.DUP.GOOD,       
//             UNIT=3390,SPACE=(TRK,(3500,1500),RLSE),     
//             LRECL=80,RECFM=FB                           
//SYSIN    DD  *                                           
  SORT FIELDS=(1,5,ZD,A,7,7,ZD,A,15,7,ZD,A,23,14,ZD,A)     
  SUM FIELDS=NONE,XSUM                                     
//*                                                       



can anyone help me on this.
Back to top
View user's profile Send private message

arvind.m

Active User


Joined: 28 Aug 2008
Posts: 205
Location: Hyderabad

PostPosted: Wed Sep 02, 2009 4:34 pm    Post subject:
Reply with quote

In the above example, item 0111315 and upc 00007362100243 have duplicate combination of state,supp,item and upc. so the output should be just

Code:

03000|0008770|0111315|00007362100243|00036|PREV
Back to top
View user's profile Send private message
gcicchet

Senior Member


Joined: 28 Jul 2006
Posts: 1703
Location: Australia

PostPosted: Wed Sep 02, 2009 4:43 pm    Post subject:
Reply with quote

Hi,

try
Code:
  SORT FIELDS=(1,5,ZD,A,7,7,ZD,A,15,7,ZD,A,23,14,ZD,A),EQUALS     



Gerry
Back to top
View user's profile Send private message
arvind.m

Active User


Joined: 28 Aug 2008
Posts: 205
Location: Hyderabad

PostPosted: Wed Sep 02, 2009 4:49 pm    Post subject:
Reply with quote

Hi Gerry,

where do i need to add the sort card (i mean in which step).

Quote:

try Code:
SORT FIELDS=(1,5,ZD,A,7,7,ZD,A,15,7,ZD,A,23,14,ZD,A),EQUALS



But i changed the first sort card as:

Code:

SORT FIELDS=(1,5,ZD,A,7,7,ZD,A,15,7,ZD,A,23,14,ZD,A,
              44,4,CH,A)


i worked. can you confirm this.
Back to top
View user's profile Send private message
arvind.m

Active User


Joined: 28 Aug 2008
Posts: 205
Location: Hyderabad

PostPosted: Wed Sep 02, 2009 4:52 pm    Post subject:
Reply with quote

Hi Gerry,

i used the sort card which you mentioned in the second step. but it did not work.
Back to top
View user's profile Send private message
gcicchet

Senior Member


Joined: 28 Jul 2006
Posts: 1703
Location: Australia

PostPosted: Wed Sep 02, 2009 4:54 pm    Post subject:
Reply with quote

Hi Arvind,

the line I gave you belongs to the second step.

The first step has additional sort fields.


Gerry
Back to top
View user's profile Send private message
gcicchet

Senior Member


Joined: 28 Jul 2006
Posts: 1703
Location: Australia

PostPosted: Wed Sep 02, 2009 4:57 pm    Post subject:
Reply with quote

Hi,

what does the output from step1 look like ?

Gerry
Back to top
View user's profile Send private message
arvind.m

Active User


Joined: 28 Aug 2008
Posts: 205
Location: Hyderabad

PostPosted: Wed Sep 02, 2009 5:01 pm    Post subject:
Reply with quote

Gerry,

My first step will sort the input file and will make sure that the record with status "CURR" is on the top. (not taking qty into consideration)


and the second step will remove the duplicate records. Since the reocrds with status "CURR" are on the top they are not eliminated.

let me know if this makes any sense.
Back to top
View user's profile Send private message
gcicchet

Senior Member


Joined: 28 Jul 2006
Posts: 1703
Location: Australia

PostPosted: Wed Sep 02, 2009 5:07 pm    Post subject:
Reply with quote

Hi Arvind,

I have run the JCL without the parameter XSUM as it is not supported with the version of DFSORT I'm using and the results are correct.

What does the file from the first step look like ?

Gerry
Back to top
View user's profile Send private message
arvind.m

Active User


Joined: 28 Aug 2008
Posts: 205
Location: Hyderabad

PostPosted: Wed Sep 02, 2009 5:25 pm    Post subject:
Reply with quote

the above input file has only few records to test it.

new input to first step
Code:

03000|0008770|0111315|00001121539210|00024|CURR
03000|0008770|0111315|00007362100243|00024|CURR
03000|0008770|0111315|00007362100243|00036|PREV
03000|0008770|0111317|00001121530618|00024|CURR
03000|0008770|0164614|00003700019004|00004|PREV
03000|0008770|0164614|00003700019004|00006|CURR


output from first step
Code:

03000|0008770|0111315|00001121539210|00024|CURR
03000|0008770|0111315|00007362100243|00024|CURR
03000|0008770|0111315|00007362100243|00036|PREV
03000|0008770|0111317|00001121530618|00024|CURR
03000|0008770|0164614|00003700019004|00004|CURR
03000|0008770|0164614|00003700019004|00006|PREV



check the last two records.
Back to top
View user's profile Send private message
arvind.m

Active User


Joined: 28 Aug 2008
Posts: 205
Location: Hyderabad

PostPosted: Wed Sep 02, 2009 5:27 pm    Post subject:
Reply with quote

sorry....

output from first step
Code:


03000|0008770|0111315|00001121539210|00024|CURR
03000|0008770|0111315|00007362100243|00024|CURR
03000|0008770|0111315|00007362100243|00036|PREV
03000|0008770|0111317|00001121530618|00024|CURR
03000|0008770|0164614|00003700019004|00006|CURR
03000|0008770|0164614|00003700019004|00004|PREV
 

Back to top
View user's profile Send private message
gcicchet

Senior Member


Joined: 28 Jul 2006
Posts: 1703
Location: Australia

PostPosted: Wed Sep 02, 2009 5:43 pm    Post subject:
Reply with quote

Hi Armand,

this is getting very confusing.

How do the last 2 records go from 00004 PREV to 00004 CURR and 00006 CURR to 00006 PREV ?

I would use the following in step1
Code:
  SORT FIELDS=(1,5,ZD,A,7,7,ZD,A,15,7,ZD,A,23,14,ZD,A, 
                44,4,CH,A)                     


and the following in step2

Code:
  SORT FIELDS=(1,5,ZD,A,7,7,ZD,A,15,7,ZD,A,23,14,ZD,A),EQUALS   
  SUM FIELDS=NONE                                         



Gerry
Back to top
View user's profile Send private message
arvind.m

Active User


Joined: 28 Aug 2008
Posts: 205
Location: Hyderabad

PostPosted: Wed Sep 02, 2009 5:49 pm    Post subject:
Reply with quote

sorry Gerry...there was a typo...

correct output is

Code:

03000|0008770|0111315|00001121539210|00024|CURR
03000|0008770|0111315|00007362100243|00024|CURR
03000|0008770|0111315|00007362100243|00036|PREV
03000|0008770|0111317|00001121530618|00024|CURR
03000|0008770|0164614|00003700019004|00006|CURR
03000|0008770|0164614|00003700019004|00004|PREV
Back to top
View user's profile Send private message
gcicchet

Senior Member


Joined: 28 Jul 2006
Posts: 1703
Location: Australia

PostPosted: Wed Sep 02, 2009 5:56 pm    Post subject:
Reply with quote

Hi Arvind,

I think you are wrong again, the output from your first step is as below
Code:
03000|0008770|0111315|00001121539210|00024|CURR
03000|0008770|0111315|00007362100243|00024|CURR
03000|0008770|0111315|00007362100243|00036|PREV
03000|0008770|0111317|00001121530618|00024|CURR
03000|0008770|0164614|00003700019004|00004|PREV
03000|0008770|0164614|00003700019004|00006|CURR


That's why I suggested a different sort field card for step1.

Gerry
Back to top
View user's profile Send private message
arvind.m

Active User


Joined: 28 Aug 2008
Posts: 205
Location: Hyderabad

PostPosted: Wed Sep 02, 2009 6:15 pm    Post subject:
Reply with quote

No Gerry,
in the first sort card "qty" is not considered (5th field) while sorting.....

Code:

SORT FIELDS=(1,5,ZD,A,7,7,ZD,A,15,7,ZD,A,23,14,ZD,A, 
                44,4,CH,A)
Back to top
View user's profile Send private message
gcicchet

Senior Member


Joined: 28 Jul 2006
Posts: 1703
Location: Australia

PostPosted: Wed Sep 02, 2009 6:27 pm    Post subject:
Reply with quote

Hi Arvind,

I didn't realize you had changed the sort card in step1, now that we agree on the first sort card, are you still having problems when using the 2nd sort card I provided.


Gerry
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> JCL & VSAM All times are GMT + 6 Hours
Page 1 of 1

 

Search our Forum:

Similar Topics
Topic Author Forum Replies Posted
No new posts Updating the counters after eliminati... PANDU1 DFSORT/ICETOOL 12 Mon Nov 21, 2016 9:47 am
No new posts Remove Special Characters from Mainfr... Rodger Zhang All Other Mainframe Topics 6 Wed Jul 06, 2016 1:12 am
No new posts Remove header only knobi SYNCSORT 5 Mon Feb 15, 2016 10:42 pm
No new posts Append to file, but also remove old r... Garry Carroll JCL & VSAM 3 Mon Jan 25, 2016 8:05 pm
No new posts Remove intermediate spaces Pallavi Vishwanath SYNCSORT 1 Fri Jan 15, 2016 2:45 am


Facebook
Back to Top
 
Mainframe Wiki | Forum Rules | Bookmarks | Subscriptions | FAQ | Tutorials | Contact Us