Portal | Manuals | References | Downloads | Info | Programs | JCLs | Master the Mainframes
IBM Mainframe Computers Forums Index
 
Register
 
IBM Mainframe Computers Forums Index Mainframe: Search IBM Mainframe Forum: FAQ Memberlist Usergroups Profile Log in to check your private messages Log in
 

 

Is there any way to eliminate duplicate records from a file

 
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> DFSORT/ICETOOL
View previous topic :: :: View next topic  
Author Message
Ganesh Kalam
Warnings : 1

New User


Joined: 11 Aug 2005
Posts: 45
Location: India

PostPosted: Fri Nov 17, 2006 2:23 pm    Post subject: Is there any way to eliminate duplicate records from a file
Reply with quote

Is there any way to eliminate duplicate records from a file with out sorting it.

SUM FIELDS=NONE will eliminate duplicates but we have to use it with SORT FIELDS.

I want to keep the order of the records as is but want to eliminate duplicates.

If the input contains

3
5
4
3
5
4

the output should be


3
5
4

If any one has idea on how to do this...please help..
Back to top
View user's profile Send private message

guptae

Moderator


Joined: 14 Oct 2005
Posts: 1187
Location: Bangalore,India

PostPosted: Fri Nov 17, 2006 2:32 pm    Post subject:
Reply with quote

Hi there,

U can acheiev in two steps.

1) In first step Specify

Code:
INREC FIELDS=(1,80,SEQNUM,8,ZD)
SORT FIELDS=(1,4,CH,A)         
SUM FIELDS=NONE                   


This will add sequence number after ur record & sort the field & remove duplicate.

2) In second step Specify
Code:
SORT FIELDS=(80,8,CH,A)
OUTREC FIELDS=(1:1,80)   

This will keep the record in prev order( According to seqnumber)

Hope it will helpful.
Back to top
View user's profile Send private message
Gautam512

Active User


Joined: 05 Oct 2005
Posts: 309
Location: Vizag / US

PostPosted: Fri Nov 17, 2006 7:38 pm    Post subject: Re: Is there any way to eliminate duplicate records from a f
Reply with quote

thats a gud solution....
u can go wth it.....
Back to top
View user's profile Send private message
Frank Yaeger

DFSORT Moderator


Joined: 15 Feb 2005
Posts: 7130
Location: San Jose, CA

PostPosted: Fri Nov 17, 2006 10:00 pm    Post subject:
Reply with quote

You do need two passes, but you can do it in one DFSORT/ICETOOL step:

Code:

//S1 EXEC PGM=ICETOOL                                           
//TOOLMSG DD SYSOUT=*                                           
//DFSMSG DD SYSOUT=*                                           
//IN DD DSN=...  input file                               
//T1 DD DSN=&&T1,UNIT=SYSDA,SPACE=(CYL,(5,5)),DISP=(,PASS)     
//OUT DD DSN=...  output file                                       
//TOOLIN DD *                                                   
SELECT FROM(IN) TO(T1) ON(1,1,CH) FIRST USING(CTL1)             
SORT FROM(T1) TO(OUT) USING(CTL2)   
/*                           
//CTL1CNTL DD *                                                 
  INREC OVERLAY=(81:SEQNUM,8,ZD)                               
/*
//CTL2CNTL DD *                                                 
  SORT FIELDS=(81,8,ZD,A)                                       
  OUTREC BUILD=(1,80)                                           
/*
Back to top
View user's profile Send private message
Ganesh Kalam
Warnings : 1

New User


Joined: 11 Aug 2005
Posts: 45
Location: India

PostPosted: Sat Nov 18, 2006 10:45 pm    Post subject: Re: Is there any way to eliminate duplicate records from a f
Reply with quote

Hi Guptae/Frank,

Both the solutions are working very fine.. Thanks for your help..
Back to top
View user's profile Send private message
guptae

Moderator


Joined: 14 Oct 2005
Posts: 1187
Location: Bangalore,India

PostPosted: Sat Nov 18, 2006 10:48 pm    Post subject:
Reply with quote

Hi Ganesh,

Glad i could help
Back to top
View user's profile Send private message
Alain Benveniste

New User


Joined: 14 Feb 2005
Posts: 88

PostPosted: Sun Nov 19, 2006 12:34 am    Post subject:
Reply with quote

Ganesh,

your input shows few different keys. If it is the reality and if you know them you could that i 1 pass.

Alain
Back to top
View user's profile Send private message
Ganesh Kalam
Warnings : 1

New User


Joined: 11 Aug 2005
Posts: 45
Location: India

PostPosted: Sun Nov 19, 2006 2:03 pm    Post subject: Re: Is there any way to eliminate duplicate records from a f
Reply with quote

Hi Alain,

I dont know the key values. Just i gave the sample input to explain my problem.

If you have any solution other than that given by Guptae and Frank, please share with us.

Thanks for your help in advance.
Back to top
View user's profile Send private message
Alain Benveniste

New User


Joined: 14 Feb 2005
Posts: 88

PostPosted: Sun Nov 19, 2006 9:11 pm    Post subject:
Reply with quote

Ganesh,

Code:

I dont know the key values

I wanted you to clarify this point. But, I don't have anything else to add to the solutions given above.

Alain
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> DFSORT/ICETOOL All times are GMT + 6 Hours
Page 1 of 1

 

Search our Forum:

Similar Topics
Topic Author Forum Replies Posted
No new posts Converting Variable block file to Fix... bhavana yalavarthi DFSORT/ICETOOL 10 Tue May 16, 2017 2:20 pm
No new posts Using ICETOOL, how we can seperate th... bshkris SYNCSORT 5 Tue May 09, 2017 8:33 pm
No new posts Check if any Detail records and extra... V S Amarendra Reddy SYNCSORT 19 Mon May 08, 2017 8:54 pm
No new posts Getting error while opening a variabl... apandey1 COBOL Programming 5 Fri May 05, 2017 12:22 pm
No new posts Issue with NDM process to transmit ES... chetanambi All Other Mainframe Topics 6 Wed May 03, 2017 10:52 am


Facebook
Back to Top
 
Mainframe Wiki | Forum Rules | Bookmarks | Subscriptions | FAQ | Tutorials | Contact Us