|
View previous topic :: View next topic
|
| Author |
Message |
Nitin Bhargava
New User
Joined: 22 May 2012
Posts: 32
Location: india
|
|
|
|
Hi All,
How to remove the duplicates from a file based on some condition and keeping the count of duplicate records eliminated using ICETOOL or DFSORT?
I have a file with following records:
01NITINSURRENDER10OWNER BHARGAVA
01NITINSURRENDER00INSURED BHARGAVA
01NITINSUREENDER30CO-OWNER BHAR
01ROHITACTIVE 20CO-INSUREDSINGHAI
01ROHITACTIVE 00INSURED SINGHAI
01ROHITACTIVE 10OWNER SINGH
01NIKHITERMINATE00INSURED JAIN
Here first 7 bytes are the key, next 10 bytes is for status, next 2 bytes is for TypeCode, next 10 bytes for TypeCode description and last 10 bytes for surname.
I have to first sort it in ascending order based on key and typecode. I did it using below sort.
SORT FIELDS=(1,7,CH,A,17,2,CH,A)
After this the file looks like
01NITINSURRENDER00INSURED BHARGAVA
01NITINSURRENDER10OWNER BHARGAVA
01NITINSUREENDER30CO-OWNER BHAR
01ROHITACTIVE 00INSURED SINGHAI
01ROHITACTIVE 10OWNER SINGH
01ROHITACTIVE 20CO-INSUREDSINGHAI
01NIKHITERMINATE00INSURED JAIN
Now I want to only consider TypeCode 00 and 10 (byte 17-18) and then have to remove the duplictae. For removing duplicate, condition is I have to consider full record except byte 17 to 26.
In output, even I don’t have to consider records with status as “SURRENDER” or “TERMINATE” and have to keep the count for records eliminated.
Output will look like :
01ROHITACTIVE 00INSURED SINGHAI
Whether it is possible to do all this in one step using ICETOOL or DFSORT? |
|
| Back to top |
|
 |
Gnanas N
Active Member
Joined: 06 Sep 2007
Posts: 792
Location: Chennai, India
|
|
|
|
| Quote: |
| ... have to keep the count for records eliminated. |
Where do you keep the count? Do want the count in a separate data set?
If so, please post how it should be in this case. |
|
| Back to top |
|
|
Bill Woodger
Moderator Emeritus
Joined: 09 Mar 2011
Posts: 7309
Location: Inside the Matrix
|
|
|
|
You only want type 00 and 10. You don't want those if they are "SURRENDER" or "TERMINATE". You then want to drop duplicates based on 1-16.
Somehow a count comes into it which you didn't explain.
Is that it? |
|
| Back to top |
|
|
Nitin Bhargava
New User
Joined: 22 May 2012
Posts: 32
Location: india
|
|
|
|
one correction in output:
The output will look like
01ROHITACTIVE 00INSURED SINGHAI
01ROHITACTIVE 10OWNER SINGH
For total count: I just have to populate it in a PS of 80 bytes. In this case the total record skipped are 5 so output will be:
Count: 5 |
|
| Back to top |
|
|
Bill Woodger
Moderator Emeritus
Joined: 09 Mar 2011
Posts: 7309
Location: Inside the Matrix
|
|
|
|
| I think I'd look in the Smart DFSORT Tricks book for the example of XSUM (and much more). If at the end you really find that a blind count of duplicates dropped is useful, you can just COUNT on the file created. |
|
| Back to top |
|
|
Nitin Bhargava
New User
Joined: 22 May 2012
Posts: 32
Location: india
|
|
|
|
| could we do it in 2 steps |
|
| Back to top |
|
|
Bill Woodger
Moderator Emeritus
Joined: 09 Mar 2011
Posts: 7309
Location: Inside the Matrix
|
|
|
|
| Err... Yes. On the confirmation you have given I can say almost definitely yes. |
|
| Back to top |
|
|
Skolusu
Senior Member
Joined: 07 Dec 2007
Posts: 2205
Location: San Jose
|
|
|
|
Nitin Bhargava,
| Nitin Bhargava wrote: |
Here first 7 bytes are the key, next 10 bytes is for status, next 2 bytes is for TypeCode, next 10 bytes for TypeCode description and last 10 bytes for surname.
Now I want to only consider TypeCode 00 and 10 (byte 17-18) and then have to remove the duplictae. For removing duplicate, condition is I have to consider full record except byte 17 to 26.
|
Something doesn't add up. You need to get your positions right. Based on the first part of your description about the fields Typecode + Type code description start at position 18 and the surname is at position 30. However you want to eliminate duplicates without considering byte 17 thru 26? That would split the Typecode description. |
|
| Back to top |
|
|
Nitin Bhargava
New User
Joined: 22 May 2012
Posts: 32
Location: india
|
|
|
|
| i am able to do it in 2 steps by myself....thanks to all |
|
| Back to top |
|
|
dick scherrer
Moderator Emeritus
Joined: 23 Nov 2006
Posts: 19244
Location: Inside the Matrix
|
|
|
|
Hello,
As your request was not clear and your details incorrect, it is probably best that you did get it running by yourself. . .
d |
|
| Back to top |
|
|
|
|
