Portal | Manuals | References | Downloads | Info | Programs | JCLs | Master the Mainframes
IBM Mainframe Computers Forums Index
 
Register
 
IBM Mainframe Computers Forums Index Mainframe: Search IBM Mainframe Forum: FAQ Memberlist Usergroups Profile Log in to check your private messages Log in
 

 

Easytrieve - Compare files - Duplicate Key
Goto page 1, 2  Next
 
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> CA Products
View previous topic :: :: View next topic  
Author Message
Michaelod
Warnings : 1

New User


Joined: 02 Sep 2008
Posts: 49
Location: Edinburgh

PostPosted: Mon Sep 24, 2012 2:03 pm    Post subject: Easytrieve - Compare files - Duplicate Key
Reply with quote

Hi,

I have a file that is split into 2 files, a 'reversal' and an 'other' file.

We compare the 2 files and eliminate any matches. We then write the clean records to a new file.

The problem is that we may have duplicate keys on both files i.e.

REVERSAL FILE

Code:
1234567890
1234567890
1111111111
2222222222


OTHER FILE

Code:
1234567890
1234567890
1111111111
2222222222
3333333333
4444444444
5555555555


I want the 'CLEAN' file to look like this:

Code:
3333333333
4444444444
5555555555


but it actually looks like this:

Code:
1234567890
3333333333
4444444444
5555555555


When we have a duplicate key it only eliminates one of the records from the CLEAN file rather than the duplicates as well.

Here is the code I have:

Code:
JOB    INPUT (FL50938O KEY(FL50938O-KEY) FL50938R KEY(FL50938R-KEY))

COMPARE-FILES                           
   IF FL50938O                         
      WS-OTHERS = WS-OTHERS + 1         
      IF NOT FL50938R                   
         PERFORM WRITE-CLEAN           
      ELSE                             
         WS-REFUNDS = WS-REFUNDS + 1   
         WS-MATCHED = WS-MATCHED + 1   
      END-IF                           
   ELSE                                 
      IF FL50938R                       
         PRINT UNMATCHED-REFUNDS       
         WS-UNMATCHED = WS-UNMATCHED + 1
         WS-REFUNDS   = WS-REFUNDS + 1 
      END-IF                           
   END-IF                               

WRITE-CLEAN. PROC                   
   FL50938C-REC       = FL50938O-REC
   FL50938C-FROM-DATE = WS-FROM-DATE
   FL50938C-TO-DATE   = WS-TO-DATE 
                                   
   PUT FL50938C                     
   WS-WRITTEN = WS-WRITTEN + 1     
END-PROC             



Is there a way I can eliminate the dups?
Back to top
View user's profile Send private message

Bill Woodger

DFSORT Moderator


Joined: 09 Mar 2011
Posts: 7225

PostPosted: Mon Sep 24, 2012 2:10 pm    Post subject: Reply to: Easytrieve - Compare files - Duplicate Key
Reply with quote

I assume that the sample data not being in sequence is just the way you made it for posting?

You're doing the "matched file processing" without any of the file-matching tests - consult your manual and try some of those. Search/browse this forum. Probably search for MATCHED will be enough.
Back to top
View user's profile Send private message
Michaelod
Warnings : 1

New User


Joined: 02 Sep 2008
Posts: 49
Location: Edinburgh

PostPosted: Mon Sep 24, 2012 2:22 pm    Post subject: Re: Reply to: Easytrieve - Compare files - Duplicate Key
Reply with quote

Bill Woodger wrote:
I assume that the sample data not being in sequence is just the way you made it for posting?

You're doing the "matched file processing" without any of the file-matching tests - consult your manual and try some of those. Search/browse this forum. Probably search for MATCHED will be enough.


Yes it should be in sequence....so it would be:

REVERSAL

Code:
1111111111
1234567890
1234567890
2222222222


OTHER

Code:
1111111111
1234567890
1234567890
2222222222
3333333333
4444444444
5555555555


Expected:

Code:
3333333333
4444444444
5555555555


Actual:

Code:
1234567890
3333333333
4444444444
5555555555
Back to top
View user's profile Send private message
Bill Woodger

DFSORT Moderator


Joined: 09 Mar 2011
Posts: 7225

PostPosted: Mon Sep 24, 2012 2:45 pm    Post subject: Reply to: Easytrieve - Compare files - Duplicate Key
Reply with quote

OK, I searched the CA forum for MATCHED for you.

Can you read through this?
Back to top
View user's profile Send private message
Michaelod
Warnings : 1

New User


Joined: 02 Sep 2008
Posts: 49
Location: Edinburgh

PostPosted: Mon Sep 24, 2012 3:23 pm    Post subject: Re: Reply to: Easytrieve - Compare files - Duplicate Key
Reply with quote

Bill Woodger wrote:
OK, I searched the CA forum for MATCHED for you.

Can you read through this?


Bill,

I've had a look but it seems to be a slightly different issue.

I can have duplicates on both files.

Here is some more example data:

REVERSAL

Code:
1111
1111
3333
5555


OTHER

Code:
1111
1111
1111
2222
3333
3333
4444
5555
5555


I'd expect my output to be:

Code:
1111
2222
3333
4444
5555


Actual is:

1111
2222
3333
4444
Back to top
View user's profile Send private message
Bill Woodger

DFSORT Moderator


Joined: 09 Mar 2011
Posts: 7225

PostPosted: Mon Sep 24, 2012 3:54 pm    Post subject: Reply to: Easytrieve - Compare files - Duplicate Key
Reply with quote

OK, let step back a... step.


Code:
111
111
222

111
111


With those as your file one and two, you will get, with your code:

Code:
111
222


Why? Because that is the way the matched file processing works.

If you want:

Code:
222


as your output, you can arrange for that.

The information and explanation is all in the topic I linked to. That topic has duplicates on both files and wants to know how to deal with them. If a different outcome is required, that does not matter for your purpose.

Do you have a manual for Easytrieve Plus? Get to the part about the Synchronized File Processing. Look at the special conditions available.

You can then use one of those special conditions with your existing code or, better, use just those conditions rather than just testing for file-presence.

Code:
111
111

111
111


If you set up a small test program to read DD * data and DISPLAY the results in that program, you should find

Code:
FILE1 Record 1 present, FILE2 Record 1 present
FILE1 Record 1 present, FILE2 Record 2 present
FILE1 Record 2 present, NO File 2 record present


Duplicates on both files is a complicated thing in Synchronized File Processing. You can code the matching yourself instead if you find the resultant code would be clearer. I'd use a "sideways match". Use File 1 as the driver. Look to File 2 from File 1. Similar type of processing to "SKIP SEQUENTIAL" except you'd do it all with GETs.
Back to top
View user's profile Send private message
Michaelod
Warnings : 1

New User


Joined: 02 Sep 2008
Posts: 49
Location: Edinburgh

PostPosted: Mon Sep 24, 2012 5:14 pm    Post subject:
Reply with quote

Tried the following and still doesnt seem to work:

Code:
COMPARE-FILES                                       
   IF NOT MATCHED                                   
      IF FL50938O                                   
        WS-OTHERS = WS-OTHERS + 1                   
        PERFORM WRITE-CLEAN                         
      END-IF                                         
      IF FL50938R                                   
        PRINT UNMATCHED-REFUNDS                     
        WS-UNMATCHED = WS-UNMATCHED + 1             
        WS-REFUNDS   = WS-REFUNDS + 1               
      END-IF                                         
   ELSE                                             
      IF LAST-DUP FL50938R OR NOT DUPLICATE FL50938R
         WS-REFUNDS = WS-REFUNDS + 1                 
        WS-MATCHED = WS-MATCHED + 1
        WS-OTHERS = WS-OTHERS + 1 
     END-IF                       
  END-IF     


Also tried it with first-dup.

Any ideas?
Back to top
View user's profile Send private message
Bill Woodger

DFSORT Moderator


Joined: 09 Mar 2011
Posts: 7225

PostPosted: Mon Sep 24, 2012 5:29 pm    Post subject: Reply to: Easytrieve - Compare files - Duplicate Key
Reply with quote

I'm sorry not to give you the exact answer, but it goes like this:

With duplicates on both files, the SFP is complex.
Because the SFP is complex, you have to understand it.
If I just give you working code and you don't understand how the match is working, you're stuck for documenting it, changing it, explaining it to colleagues.

My "rule-of-thumb" for SFP is "avoid duplicates on both files".

If this is not possible, I'd more likely code the manual matching. Reason being, next one along won't understand so well how to change the thing, or what exactly (and easily) is happening, if I use SFP.

If you are determined to stick with SFP for this, your experiments, the topic I linked to, and the manual will lead you to the correct code without me having to supply it (I can't test it, anyway). Or you'll recogmise that it is somewhat less intuitive than is suited to the program and you'll code the manual match.

If you had been using the matching conditions already (like in the other topic) the story is slightly different, as you should be able to see, for the provision of code. To use SFP with one-to-one is easy with the conditions. With one-to-many is easy with the conditions. With many-to-many is complex, even with the conditions.

If you are new to the matching conditions, you need to get a good grip of them, which you will get if you follow through on what I suggested.

If you get nowhere, check back tomorrow.
Back to top
View user's profile Send private message
Michaelod
Warnings : 1

New User


Joined: 02 Sep 2008
Posts: 49
Location: Edinburgh

PostPosted: Mon Sep 24, 2012 6:24 pm    Post subject: Re: Reply to: Easytrieve - Compare files - Duplicate Key
Reply with quote

Bill Woodger wrote:
I'm sorry not to give you the exact answer, but it goes like this:

With duplicates on both files, the SFP is complex.
Because the SFP is complex, you have to understand it.
If I just give you working code and you don't understand how the match is working, you're stuck for documenting it, changing it, explaining it to colleagues.

My "rule-of-thumb" for SFP is "avoid duplicates on both files".

If this is not possible, I'd more likely code the manual matching. Reason being, next one along won't understand so well how to change the thing, or what exactly (and easily) is happening, if I use SFP.

If you are determined to stick with SFP for this, your experiments, the topic I linked to, and the manual will lead you to the correct code without me having to supply it (I can't test it, anyway). Or you'll recogmise that it is somewhat less intuitive than is suited to the program and you'll code the manual match.

If you had been using the matching conditions already (like in the other topic) the story is slightly different, as you should be able to see, for the provision of code. To use SFP with one-to-one is easy with the conditions. With one-to-many is easy with the conditions. With many-to-many is complex, even with the conditions.

If you are new to the matching conditions, you need to get a good grip of them, which you will get if you follow through on what I suggested.

If you get nowhere, check back tomorrow.


The duplicates on both files is unavoidable. This is an existing live issue.

I'll maybe try the manual matching.
Back to top
View user's profile Send private message
dbzTHEdinosauer

Global Moderator


Joined: 20 Oct 2006
Posts: 6966
Location: porcelain throne

PostPosted: Mon Sep 24, 2012 6:35 pm    Post subject:
Reply with quote

since you don't seem to have the witherwithall to manage this
'complex' logic,
why don't you presort both files, removing dups,
then you have a simple ezytrieve program?
Back to top
View user's profile Send private message
Michaelod
Warnings : 1

New User


Joined: 02 Sep 2008
Posts: 49
Location: Edinburgh

PostPosted: Mon Sep 24, 2012 6:43 pm    Post subject:
Reply with quote

dbzTHEdinosauer wrote:
since you don't seem to have the witherwithall to manage this
'complex' logic,
why don't you presort both files, removing dups,
then you have a simple ezytrieve program?


I cant simply remove dups.

I have to match them first and then remove.

For example if i had file a:

Code:
1111
1111
1111
2222
2222
2222


and file b:

Code:
1111
2222


I would expect:

Code:
1111
1111
2222
2222


Your suggestion however would give me nothing i.e. empty file
Back to top
View user's profile Send private message
enrico-sorichetti

Global Moderator


Joined: 14 Mar 2007
Posts: 10202
Location: italy

PostPosted: Mon Sep 24, 2012 6:47 pm    Post subject: Reply to: Easytrieve - Compare files - Duplicate Key
Reply with quote

REVERSAL

Code:
1111
1111
3333
5555



OTHER

Code:
1111
1111
1111
2222
3333
3333
4444
5555
5555



I'd expect my output to be:
Code:
1111
2222
3333
4444
5555


for example the OTHER 1111 being kept will be last one or the first one ???

IMO the whole business approach is flawed ...

just curious ... the <duplicate> check is only on the key or on the whole record
Back to top
View user's profile Send private message
Michaelod
Warnings : 1

New User


Joined: 02 Sep 2008
Posts: 49
Location: Edinburgh

PostPosted: Mon Sep 24, 2012 6:50 pm    Post subject: Re: Reply to: Easytrieve - Compare files - Duplicate Key
Reply with quote

enrico-sorichetti wrote:
REVERSAL

Code:
1111
1111
3333
5555



OTHER

Code:
1111
1111
1111
2222
3333
3333
4444
5555
5555



I'd expect my output to be:
Code:
1111
2222
3333
4444
5555


for example the OTHER 1111 being kept will be last one or the first one ???

IMO the whole business approach is flawed ...

just curious ... the <duplicate> check is only on the key or on the whole record


It doesnt matter whether its first or last.....as long as the correct number of reversals are eliminated

The business approach is not flawed if you knew the context.

The use of Easytrieve under these circumstances is the flaw.
Back to top
View user's profile Send private message
dick scherrer

Site Director


Joined: 23 Nov 2006
Posts: 19270
Location: Inside the Matrix

PostPosted: Mon Sep 24, 2012 7:09 pm    Post subject:
Reply with quote

Hello,

Quote:
The use of Easytrieve under these circumstances is the flaw.
The flaw is Not Easytrieve, but rather the knowledge / creativity of those trying to use it. . .

It is rather frustrating that these software vendors write code for the masses, not necessarily as one would prefer. . .

What i understand as your objective has been solved many times using Easytrieve - it just takes a bit of work and is not necessarily automagic.

Would you know how to code this in COBOL (i surely hope so)? The same technique can be used in Easytrieve (as Bill has mentioned).
Back to top
View user's profile Send private message
enrico-sorichetti

Global Moderator


Joined: 14 Mar 2007
Posts: 10202
Location: italy

PostPosted: Mon Sep 24, 2012 7:15 pm    Post subject: Reply to: Easytrieve - Compare files - Duplicate Key
Reply with quote

Quote:
The business approach is not flawed if you knew the context.


whenever people talk about duplicates on a key which is a subset of the whole record
and tell that it does not make any difference which <duplicate> is kept...
than the logic is flawed ( wether You like it or not )
or somebody was too lazy do describe clearly the problem

since we reply on our own time and free of charge
the minimum that is expected from people asking for help is a proper description of the problem.
Back to top
View user's profile Send private message
Michaelod
Warnings : 1

New User


Joined: 02 Sep 2008
Posts: 49
Location: Edinburgh

PostPosted: Mon Sep 24, 2012 7:45 pm    Post subject: Re: Reply to: Easytrieve - Compare files - Duplicate Key
Reply with quote

enrico-sorichetti wrote:
Quote:
The business approach is not flawed if you knew the context.


whenever people talk about duplicates on a key which is a subset of the whole record
and tell that it does not make any difference which <duplicate> is kept...
than the logic is flawed ( wether You like it or not )
or somebody was too lazy do describe clearly the problem

since we reply on our own time and free of charge
the minimum that is expected from people asking for help is a proper description of the problem.


Whether we agree or not if the business approach is flawed, the problem description was clear and concise.
Back to top
View user's profile Send private message
dick scherrer

Site Director


Joined: 23 Nov 2006
Posts: 19270
Location: Inside the Matrix

PostPosted: Mon Sep 24, 2012 8:12 pm    Post subject:
Reply with quote

Hello,

Unless i have missed something, you have been told how to do what you need.

What was not clear? If you understood the code needed, implement away icon_smile.gif If there is some doubt, post what we might clarify.
Back to top
View user's profile Send private message
Michaelod
Warnings : 1

New User


Joined: 02 Sep 2008
Posts: 49
Location: Edinburgh

PostPosted: Mon Sep 24, 2012 8:24 pm    Post subject:
Reply with quote

dick scherrer wrote:
Hello,

Unless i have missed something, you have been told how to do what you need.

What was not clear? If you understood the code needed, implement away icon_smile.gif If there is some doubt, post what we might clarify.


I'm still no further forward......
Back to top
View user's profile Send private message
dick scherrer

Site Director


Joined: 23 Nov 2006
Posts: 19270
Location: Inside the Matrix

PostPosted: Mon Sep 24, 2012 8:55 pm    Post subject:
Reply with quote

Hello,

Quote:
I'm still no further forward......
Most of the work getting help from the forum is up to the requestor. Saying "no progress" is no help to someone trying to help you.

Maybe it is not yet clear that you need to toss out the "MATCHED" code and write your own code to control the processing of the 2 files. No matter what you do, the files need to be in the same key sequence.

If you do not understand how to writ ethis kind of code, there is a "Sticky" at the top of the COBOL part of the forum that is a tested, working example of matching/merging 2 files that are in sequence. This same logic has been used rather often in Easytrieve as well. It may be of value to get the cobol sample running for your files and then re-code in Easytrieve - if the deliverable has to be Easytrieve.
Back to top
View user's profile Send private message
Michaelod
Warnings : 1

New User


Joined: 02 Sep 2008
Posts: 49
Location: Edinburgh

PostPosted: Mon Sep 24, 2012 9:02 pm    Post subject:
Reply with quote

No problem. Ill have a look.

Thanks
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> CA Products All times are GMT + 6 Hours
Goto page 1, 2  Next
Page 1 of 2

 

Search our Forum:

Similar Topics
Topic Author Forum Replies Posted
No new posts High CPU consumption Job using IAM fi... aswinir JCL & VSAM 8 Thu Dec 01, 2016 8:28 pm
No new posts Limit duplicate records in the SORT pshongal SYNCSORT 6 Mon Nov 21, 2016 12:54 pm
No new posts Match or compare two files in VB Format anatol DFSORT/ICETOOL 14 Thu Nov 03, 2016 7:41 pm
This topic is locked: you cannot edit posts or make replies. How to use 2 input files in control c... Gunapala CN DFSORT/ICETOOL 23 Thu Oct 13, 2016 3:42 pm
No new posts Adding records from two files into on... shiitiizz SYNCSORT 4 Mon Sep 19, 2016 8:41 pm


Facebook
Back to Top
 
Mainframe Wiki | Forum Rules | Bookmarks | Subscriptions | FAQ | Tutorials | Contact Us