Portal | Manuals | References | Downloads | Info | Programs | JCLs | Mainframe wiki | Quick Ref
IBM Mainframe Computers Forums Index
 
Register
 
IBM Mainframe Computers Forums Index Mainframe: Search IBM Mainframe Forum: FAQ Memberlist Profile Log in to check your private messages Log in
 
How to remove duplicates from file with keeping a count

 
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> DFSORT/ICETOOL
View previous topic :: :: View next topic  
Author Message
Nitin Bhargava

New User


Joined: 22 May 2012
Posts: 32
Location: india

PostPosted: Mon Jun 25, 2012 1:13 pm    Post subject: How to remove duplicates from file with keeping a count
Reply with quote

Hi All,



How to remove the duplicates from a file based on some condition and keeping the count of duplicate records eliminated using ICETOOL or DFSORT?

I have a file with following records:



01NITINSURRENDER10OWNER BHARGAVA

01NITINSURRENDER00INSURED BHARGAVA

01NITINSUREENDER30CO-OWNER BHAR

01ROHITACTIVE 20CO-INSUREDSINGHAI

01ROHITACTIVE 00INSURED SINGHAI

01ROHITACTIVE 10OWNER SINGH

01NIKHITERMINATE00INSURED JAIN



Here first 7 bytes are the key, next 10 bytes is for status, next 2 bytes is for TypeCode, next 10 bytes for TypeCode description and last 10 bytes for surname.

I have to first sort it in ascending order based on key and typecode. I did it using below sort.



SORT FIELDS=(1,7,CH,A,17,2,CH,A)

After this the file looks like

01NITINSURRENDER00INSURED BHARGAVA

01NITINSURRENDER10OWNER BHARGAVA

01NITINSUREENDER30CO-OWNER BHAR

01ROHITACTIVE 00INSURED SINGHAI

01ROHITACTIVE 10OWNER SINGH

01ROHITACTIVE 20CO-INSUREDSINGHAI

01NIKHITERMINATE00INSURED JAIN



Now I want to only consider TypeCode 00 and 10 (byte 17-18) and then have to remove the duplictae. For removing duplicate, condition is I have to consider full record except byte 17 to 26.



In output, even I don’t have to consider records with status as “SURRENDER” or “TERMINATE” and have to keep the count for records eliminated.



Output will look like :

01ROHITACTIVE 00INSURED SINGHAI



Whether it is possible to do all this in one step using ICETOOL or DFSORT?
Back to top
View user's profile Send private message

Gnanas N

Active Member


Joined: 06 Sep 2007
Posts: 788
Location: Chennai, India

PostPosted: Mon Jun 25, 2012 1:24 pm    Post subject: Reply to: How to remove duplicates from file with keeping a
Reply with quote

Quote:
... have to keep the count for records eliminated.

Where do you keep the count? Do want the count in a separate data set?

If so, please post how it should be in this case.
Back to top
View user's profile Send private message
Bill Woodger

DFSORT Moderator


Joined: 09 Mar 2011
Posts: 7315

PostPosted: Mon Jun 25, 2012 1:24 pm    Post subject: Reply to: How to remove duplicates from file with keeping a
Reply with quote

You only want type 00 and 10. You don't want those if they are "SURRENDER" or "TERMINATE". You then want to drop duplicates based on 1-16.

Somehow a count comes into it which you didn't explain.

Is that it?
Back to top
View user's profile Send private message
Nitin Bhargava

New User


Joined: 22 May 2012
Posts: 32
Location: india

PostPosted: Mon Jun 25, 2012 2:06 pm    Post subject:
Reply with quote

one correction in output:

The output will look like
01ROHITACTIVE 00INSURED SINGHAI
01ROHITACTIVE 10OWNER SINGH

For total count: I just have to populate it in a PS of 80 bytes. In this case the total record skipped are 5 so output will be:

Count: 5
Back to top
View user's profile Send private message
Bill Woodger

DFSORT Moderator


Joined: 09 Mar 2011
Posts: 7315

PostPosted: Mon Jun 25, 2012 2:38 pm    Post subject: Reply to: How to remove duplicates from file with keeping a
Reply with quote

I think I'd look in the Smart DFSORT Tricks book for the example of XSUM (and much more). If at the end you really find that a blind count of duplicates dropped is useful, you can just COUNT on the file created.
Back to top
View user's profile Send private message
Nitin Bhargava

New User


Joined: 22 May 2012
Posts: 32
Location: india

PostPosted: Mon Jun 25, 2012 3:38 pm    Post subject: Reply to: How to remove duplicates from file with keeping a
Reply with quote

could we do it in 2 steps
Back to top
View user's profile Send private message
Bill Woodger

DFSORT Moderator


Joined: 09 Mar 2011
Posts: 7315

PostPosted: Mon Jun 25, 2012 3:47 pm    Post subject: Reply to: How to remove duplicates from file with keeping a
Reply with quote

Err... Yes. On the confirmation you have given I can say almost definitely yes.
Back to top
View user's profile Send private message
Skolusu

Senior Member


Joined: 07 Dec 2007
Posts: 2205
Location: San Jose

PostPosted: Mon Jun 25, 2012 9:26 pm    Post subject:
Reply with quote

Nitin Bhargava,

Nitin Bhargava wrote:
Here first 7 bytes are the key, next 10 bytes is for status, next 2 bytes is for TypeCode, next 10 bytes for TypeCode description and last 10 bytes for surname.

Now I want to only consider TypeCode 00 and 10 (byte 17-18) and then have to remove the duplictae. For removing duplicate, condition is I have to consider full record except byte 17 to 26.



Something doesn't add up. You need to get your positions right. Based on the first part of your description about the fields Typecode + Type code description start at position 18 and the surname is at position 30. However you want to eliminate duplicates without considering byte 17 thru 26? That would split the Typecode description.
Back to top
View user's profile Send private message
Nitin Bhargava

New User


Joined: 22 May 2012
Posts: 32
Location: india

PostPosted: Tue Jun 26, 2012 4:13 pm    Post subject:
Reply with quote

i am able to do it in 2 steps by myself....thanks to all
Back to top
View user's profile Send private message
dick scherrer

Site Director


Joined: 23 Nov 2006
Posts: 19270
Location: Inside the Matrix

PostPosted: Tue Jun 26, 2012 11:15 pm    Post subject: Reply to: How to remove duplicates from file with keeping a
Reply with quote

Hello,

As your request was not clear and your details incorrect, it is probably best that you did get it running by yourself. . .

d
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> DFSORT/ICETOOL All times are GMT + 6 Hours
Page 1 of 1

 

Search our Forum:

Similar Topics
Topic Author Forum Replies Posted
No new posts column with count of rows within dist... ronald wouterson DB2 4 Sun Sep 17, 2017 9:48 pm
No new posts Split 1 file into 10 output Files - S... Prasanth Kumar SYNCSORT 5 Sat Sep 16, 2017 12:02 am
No new posts BUILD OUTFIL based on condition other... balaji81_k DFSORT/ICETOOL 13 Fri Sep 08, 2017 11:06 pm
No new posts Remove leading spaces from numeric field rexx77 SYNCSORT 6 Wed Sep 06, 2017 2:15 am
No new posts Updating the Trailer count in variabl... satheshbabur DFSORT/ICETOOL 6 Wed Aug 30, 2017 9:49 pm

Facebook
Back to Top
 
Job Vacancies | Forum Rules | Bookmarks | Subscriptions | FAQ | Polls | Contact Us