Removing duplicates based on two positions using ICEOOL

harithb · New User Joined: 08 Jun 2009 Posts: 15 Location: Bangalore

Hi,

I have a flat file which will have a number of records. Each record has length 80.

My requirement is to remove records where duplicates can occur in positions 10-20 and 30-40. That is if two records have same value in positions 10-20 and 30-40 (remaining positions need not have same values), then those records should be eliminated.

Is this possible using icetool ? If yes, how ?

Thanks in advance.

bipinpeter · Posted: Thu Jul 02, 2009 10:05 am

Hi ,
Please try the below code in ICETOOL

harithb · New User Joined: 08 Jun 2009 Posts: 15 Location: Bangalore

Yes. I tried it and it is working. Thank you.

But I have a problem here. The input file has a header and a trailer which will always have spaces from position 15 till 80. And since they are the same in these positions (having spaces in both header and trailer) both of them are getting deleted in the output file

The trailer is identified by the first position 'Z'. The trailer should have the count of number of records in the output file and I think I can use COUNT and again write one more output file in a new step after the step having removal of duplicate records.

But how should I be handling the header record. The header record is identified by the first position which is always an 'A'.

Frank Yaeger · Posted: Thu Jul 02, 2009 8:49 pm

Please show an example of the records in your input file and what you expect for output.

Give the RECFM of the input file.