Portal | Manuals | References | Downloads | Info | Programs | JCLs | Master the Mainframes
IBM Mainframe Computers Forums Index
 
Register
 
IBM Mainframe Computers Forums Index Mainframe: Search IBM Mainframe Forum: FAQ Memberlist Usergroups Profile Log in to check your private messages Log in
 

 

How to remove duplicates from file with keeping a count

 
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> DFSORT/ICETOOL
View previous topic :: :: View next topic  
Author Message
Nitin Bhargava

New User


Joined: 22 May 2012
Posts: 32
Location: india

PostPosted: Mon Jun 25, 2012 1:13 pm    Post subject: How to remove duplicates from file with keeping a count
Reply with quote

Hi All,



How to remove the duplicates from a file based on some condition and keeping the count of duplicate records eliminated using ICETOOL or DFSORT?

I have a file with following records:



01NITINSURRENDER10OWNER BHARGAVA

01NITINSURRENDER00INSURED BHARGAVA

01NITINSUREENDER30CO-OWNER BHAR

01ROHITACTIVE 20CO-INSUREDSINGHAI

01ROHITACTIVE 00INSURED SINGHAI

01ROHITACTIVE 10OWNER SINGH

01NIKHITERMINATE00INSURED JAIN



Here first 7 bytes are the key, next 10 bytes is for status, next 2 bytes is for TypeCode, next 10 bytes for TypeCode description and last 10 bytes for surname.

I have to first sort it in ascending order based on key and typecode. I did it using below sort.



SORT FIELDS=(1,7,CH,A,17,2,CH,A)

After this the file looks like

01NITINSURRENDER00INSURED BHARGAVA

01NITINSURRENDER10OWNER BHARGAVA

01NITINSUREENDER30CO-OWNER BHAR

01ROHITACTIVE 00INSURED SINGHAI

01ROHITACTIVE 10OWNER SINGH

01ROHITACTIVE 20CO-INSUREDSINGHAI

01NIKHITERMINATE00INSURED JAIN



Now I want to only consider TypeCode 00 and 10 (byte 17-18) and then have to remove the duplictae. For removing duplicate, condition is I have to consider full record except byte 17 to 26.



In output, even I don’t have to consider records with status as “SURRENDER” or “TERMINATE” and have to keep the count for records eliminated.



Output will look like :

01ROHITACTIVE 00INSURED SINGHAI



Whether it is possible to do all this in one step using ICETOOL or DFSORT?
Back to top
View user's profile Send private message

Gnanas N

Active Member


Joined: 06 Sep 2007
Posts: 785
Location: Chennai, India

PostPosted: Mon Jun 25, 2012 1:24 pm    Post subject: Reply to: How to remove duplicates from file with keeping a
Reply with quote

Quote:
... have to keep the count for records eliminated.

Where do you keep the count? Do want the count in a separate data set?

If so, please post how it should be in this case.
Back to top
View user's profile Send private message
Bill Woodger

DFSORT Moderator


Joined: 09 Mar 2011
Posts: 7232

PostPosted: Mon Jun 25, 2012 1:24 pm    Post subject: Reply to: How to remove duplicates from file with keeping a
Reply with quote

You only want type 00 and 10. You don't want those if they are "SURRENDER" or "TERMINATE". You then want to drop duplicates based on 1-16.

Somehow a count comes into it which you didn't explain.

Is that it?
Back to top
View user's profile Send private message
Nitin Bhargava

New User


Joined: 22 May 2012
Posts: 32
Location: india

PostPosted: Mon Jun 25, 2012 2:06 pm    Post subject:
Reply with quote

one correction in output:

The output will look like
01ROHITACTIVE 00INSURED SINGHAI
01ROHITACTIVE 10OWNER SINGH

For total count: I just have to populate it in a PS of 80 bytes. In this case the total record skipped are 5 so output will be:

Count: 5
Back to top
View user's profile Send private message
Bill Woodger

DFSORT Moderator


Joined: 09 Mar 2011
Posts: 7232

PostPosted: Mon Jun 25, 2012 2:38 pm    Post subject: Reply to: How to remove duplicates from file with keeping a
Reply with quote

I think I'd look in the Smart DFSORT Tricks book for the example of XSUM (and much more). If at the end you really find that a blind count of duplicates dropped is useful, you can just COUNT on the file created.
Back to top
View user's profile Send private message
Nitin Bhargava

New User


Joined: 22 May 2012
Posts: 32
Location: india

PostPosted: Mon Jun 25, 2012 3:38 pm    Post subject: Reply to: How to remove duplicates from file with keeping a
Reply with quote

could we do it in 2 steps
Back to top
View user's profile Send private message
Bill Woodger

DFSORT Moderator


Joined: 09 Mar 2011
Posts: 7232

PostPosted: Mon Jun 25, 2012 3:47 pm    Post subject: Reply to: How to remove duplicates from file with keeping a
Reply with quote

Err... Yes. On the confirmation you have given I can say almost definitely yes.
Back to top
View user's profile Send private message
Skolusu

Senior Member


Joined: 07 Dec 2007
Posts: 2205
Location: San Jose

PostPosted: Mon Jun 25, 2012 9:26 pm    Post subject:
Reply with quote

Nitin Bhargava,

Nitin Bhargava wrote:
Here first 7 bytes are the key, next 10 bytes is for status, next 2 bytes is for TypeCode, next 10 bytes for TypeCode description and last 10 bytes for surname.

Now I want to only consider TypeCode 00 and 10 (byte 17-18) and then have to remove the duplictae. For removing duplicate, condition is I have to consider full record except byte 17 to 26.



Something doesn't add up. You need to get your positions right. Based on the first part of your description about the fields Typecode + Type code description start at position 18 and the surname is at position 30. However you want to eliminate duplicates without considering byte 17 thru 26? That would split the Typecode description.
Back to top
View user's profile Send private message
Nitin Bhargava

New User


Joined: 22 May 2012
Posts: 32
Location: india

PostPosted: Tue Jun 26, 2012 4:13 pm    Post subject:
Reply with quote

i am able to do it in 2 steps by myself....thanks to all
Back to top
View user's profile Send private message
dick scherrer

Site Director


Joined: 23 Nov 2006
Posts: 19270
Location: Inside the Matrix

PostPosted: Tue Jun 26, 2012 11:15 pm    Post subject: Reply to: How to remove duplicates from file with keeping a
Reply with quote

Hello,

As your request was not clear and your details incorrect, it is probably best that you did get it running by yourself. . .

d
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> DFSORT/ICETOOL All times are GMT + 6 Hours
Page 1 of 1

 

Search our Forum:

Similar Topics
Topic Author Forum Replies Posted
No new posts Add PD field from 2nd file to PD in 1st Sushant Garje DFSORT/ICETOOL 6 Thu Dec 01, 2016 4:32 pm
No new posts File Aid to File Manager conversion murali3955 IBM Tools 4 Thu Nov 24, 2016 3:41 pm
No new posts Updating the counters after eliminati... PANDU1 DFSORT/ICETOOL 12 Mon Nov 21, 2016 9:47 am
No new posts How to convert the VBM file to VB or... Sulabh Agrawal JCL & VSAM 4 Fri Nov 18, 2016 1:04 pm
No new posts CICS Roll back partially - Need to re... dwijadas CICS 4 Wed Nov 16, 2016 4:30 pm


Facebook
Back to Top
 
Mainframe Wiki | Forum Rules | Bookmarks | Subscriptions | FAQ | Tutorials | Contact Us