|
View previous topic :: View next topic
|
| Author |
Message |
shilpa.khaire
New User
Joined: 21 Jul 2006
Posts: 39
Location: US
|
|
|
|
Hi,
I have a file which has records as:
1234 1
1234 271
1233 1
1255 1
1276 1
1276 271
I want the unique records as well as the first record out of duplicates. Expected O/P :
1234 1
1233 1
1255 1
1276 1
I am using a SORT step to SORT by the 2 fields and tthen ICETOOL .
SELECT FROM(IN) TO(OUT) ON(2,20,CH) FIRSTDUP.
But this is eliminating all the unique records. Now my O/P looks like below which not what I need:
1234 1
1276 1
Please help me at the earliest!
Thanks!
|
|
| Back to top |
|
 |
dick scherrer
Moderator Emeritus
Joined: 23 Nov 2006
Posts: 19244
Location: Inside the Matrix
|
|
|
|
Hello,
Try this:
| Code: |
SORT FIELDS=(your sort fields from the unposted sort step)
SUM FIELDS=NONE |
I believe the output of this one SORT step will be what you want. |
|
| Back to top |
|
|
Arun Raj
Moderator
Joined: 17 Oct 2006
Posts: 2481
Location: @my desk
|
|
|
|
shilpa.khaire,
How about trying FIRST instead of FIRSTDUP. |
|
| Back to top |
|
|
dick scherrer
Moderator Emeritus
Joined: 23 Nov 2006
Posts: 19244
Location: Inside the Matrix
|
|
|
|
Hi Arun,
If done in the sort step, would it not save a step and an extra read/write of the file? |
|
| Back to top |
|
|
gcicchet
Senior Member
Joined: 28 Jul 2006
Posts: 1702
Location: Australia
|
|
|
|
Hi,
I believe a seqnum will need to be added and output resorted as the the requested output is not in ascending order.
Gerry |
|
| Back to top |
|
|
gcicchet
Senior Member
Joined: 28 Jul 2006
Posts: 1702
Location: Australia
|
|
|
|
Hi Dick,
I believe we need more information, if 1234 1 covers the 2 fields, ie.
1234 is field1 and 1 is field2, and
1234 is field1 and 271 is field2, the sum fields=none will not delete any records.
Gerry |
|
| Back to top |
|
|
dick scherrer
Moderator Emeritus
Joined: 23 Nov 2006
Posts: 19244
Location: Inside the Matrix
|
|
|
|
Hi Gerry,
Good catch - i misread  |
|
| Back to top |
|
|
Arun Raj
Moderator
Joined: 17 Oct 2006
Posts: 2481
Location: @my desk
|
|
|
|
| dick scherrer wrote: |
Hi Arun,
If done in the sort step, would it not save a step and an extra read/write of the file? |
Hi Dick,
I believe we dont need to have an extra SORT while using SELECT.
If the OP is interested in retaining the input order, then I would agree with Gerry. But I dont undertand the purpose of 'sorting on 2 fields' and the expected output is
| Code: |
1234 1
1233 1
1255 1
1276 1 |
instead of
| Code: |
1233 1
1234 1
1255 1
1276 1 |
|
|
| Back to top |
|
|
|
|
