Portal | Manuals | References | Downloads | Info | Programs | JCLs | Master the Mainframes
IBM Mainframe Computers Forums Index
 
Register
 
IBM Mainframe Computers Forums Index Mainframe: Search IBM Mainframe Forum: FAQ Memberlist Usergroups Profile Log in to check your private messages Log in
 

 

ICETOOL block compare
Goto page 1, 2  Next
 
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> DFSORT/ICETOOL
View previous topic :: :: View next topic  
Author Message
CICS fan

New User


Joined: 03 Apr 2008
Posts: 82
Location: United States

PostPosted: Wed Aug 18, 2010 9:35 am    Post subject: ICETOOL block compare
Reply with quote

I have an 381 LRECL input file. Each record starts with either 'ZZ' followed by one or more 'AA' records. So data looks like:

Code:
ZZlkwehlwhflwwhflhwflwfh..........
AAjlwehwhyeohfwfehfwefefe.......
AAweouwhyfowefyeofhwefoehf........
ZZwdlhfwflfhwlfwehfwlfwelfhwf........
AAwkjwehwejhwelfhwflfw...........
ZZoweyrwoyqpwebvbvorhwfbw.....
ZZkhwepiwehlfkhlfhlhq...........
AAweljhweleqyfoehq.................
AAeku2eipuriu4iupiuipuifhlwefnlfn......
AAwejhefehfelfhflhffff........


and so on..

the requirement is to compare each block of data starting from 'ZZ' of one record till before the next 'ZZ'(so each block willbe one ZZ record and one ore more AA records or no AA records at all) and discard all duplicate blocks (exact duplicate of entire block) and keep the first block only in the output file. No sorting should be done.

Please suggest on this.
Back to top
View user's profile Send private message

dick scherrer

Site Director


Joined: 23 Nov 2006
Posts: 19270
Location: Inside the Matrix

PostPosted: Wed Aug 18, 2010 9:57 am    Post subject:
Reply with quote

Hello,

What should happen if duplicate blocks are not consecutive? Say block 3 and block 40 are duplicates?
Back to top
View user's profile Send private message
CICS fan

New User


Joined: 03 Apr 2008
Posts: 82
Location: United States

PostPosted: Wed Aug 18, 2010 10:00 am    Post subject:
Reply with quote

The first one should be retained and all subsequent blocks discarded. No sorting whatsoever should be done.
Back to top
View user's profile Send private message
dick scherrer

Site Director


Joined: 23 Nov 2006
Posts: 19270
Location: Inside the Matrix

PostPosted: Wed Aug 18, 2010 10:01 am    Post subject:
Reply with quote

Hello,

Then someone probably needs to reconsider this "requirement". . .
Back to top
View user's profile Send private message
CICS fan

New User


Joined: 03 Apr 2008
Posts: 82
Location: United States

PostPosted: Wed Aug 18, 2010 10:04 am    Post subject:
Reply with quote

This is beyond the scope of ICETOOL Sir?
Back to top
View user's profile Send private message
sqlcode1

Active Member


Joined: 08 Apr 2010
Posts: 578
Location: USA

PostPosted: Wed Aug 18, 2010 4:56 pm    Post subject:
Reply with quote

CICS fan,

Is there a limit to how many maximum AA records could follow ZZ records?

Thanks,
Back to top
View user's profile Send private message
dbzTHEdinosauer

Global Moderator


Joined: 20 Oct 2006
Posts: 6966
Location: porcelain throne

PostPosted: Wed Aug 18, 2010 5:41 pm    Post subject:
Reply with quote

CICS fan wrote:
and keep the first block only in the output file


nonsensical requirement: if the blocks are duplicate, how are you going to tell which was first.

can their be triplicate blocks?
Back to top
View user's profile Send private message
CICS fan

New User


Joined: 03 Apr 2008
Posts: 82
Location: United States

PostPosted: Wed Aug 18, 2010 5:45 pm    Post subject:
Reply with quote

See, when we compare records only, we often ensure that only first record is kept and any subsequent is discared. Therecan be triplicate blocks..or more than that.
Back to top
View user's profile Send private message
dbzTHEdinosauer

Global Moderator


Joined: 20 Oct 2006
Posts: 6966
Location: porcelain throne

PostPosted: Wed Aug 18, 2010 6:06 pm    Post subject:
Reply with quote

Quote:
See, when we compare records only, we often ensure that only first record is kept and any subsequent is discared.


I realize that with Sort utility that distinction is easily made and adhered.
but, I think the DFSORT group will tell you to write a program,
or make multiple passes (if achievable).

you have two variables
  • size of blocks - number of AA records
  • number of potential duplicate blocks


not only that, I assume you need to compare the complete 381 bytes.

nor did you answer the question, how many potential AA records can their be?

if it is less than 6 then there would be an obvious sort solution,
but if the number of AA records exceeded an amount that would require a very large record (ZZ AA AA AA AA AA) that could not be compared by sort,
the Sort team will have an interesting problem.
Back to top
View user's profile Send private message
CICS fan

New User


Joined: 03 Apr 2008
Posts: 82
Location: United States

PostPosted: Wed Aug 18, 2010 6:11 pm    Post subject:
Reply with quote

The number of AA records can be anything from equal to greater than zero.
And, yes, complete 381 bytes need to be considered.
Back to top
View user's profile Send private message
dbzTHEdinosauer

Global Moderator


Joined: 20 Oct 2006
Posts: 6966
Location: porcelain throne

PostPosted: Wed Aug 18, 2010 6:22 pm    Post subject:
Reply with quote

Quote:
The number of AA records can be anything from equal to greater than zero.


can there be 10, 15 AA records?
Back to top
View user's profile Send private message
sqlcode1

Active Member


Joined: 08 Apr 2010
Posts: 578
Location: USA

PostPosted: Wed Aug 18, 2010 7:09 pm    Post subject:
Reply with quote

CICS fan,
Quote:
The number of AA records can be anything from equal to greater than zero.

Is there a maximum number? AFAIK, currently there is no functionality in DFsort to compare block of data. So you will have trick your input file.

I was asking for maximum number of AA records under ZZ because, I was trying to develop a solution which would take all the AA record under a group and will align them horizontally. By doing this, you are playing with single record to remove duplicates. Later when you have removed duplicates, you can use OUTFIL BUILD to break them into multiple records.

I am still not sure if this method works or not but I will try if you could provide maximum number of detail records under a group.

Thanks,
Back to top
View user's profile Send private message
dick scherrer

Site Director


Joined: 23 Nov 2006
Posts: 19270
Location: Inside the Matrix

PostPosted: Wed Aug 18, 2010 8:49 pm    Post subject:
Reply with quote

Hello,

I believe that a big issue is that the duplicate blocks can be anywhere in the file - not consecutive.

If the file only has 1,000,000 records and these are not in some kind of "group sequence", it i suspect it will be problematic to identify "all of the duplicate blocks" so that only the "first" block is retained.
Back to top
View user's profile Send private message
dbzTHEdinosauer

Global Moderator


Joined: 20 Oct 2006
Posts: 6966
Location: porcelain throne

PostPosted: Wed Aug 18, 2010 8:53 pm    Post subject:
Reply with quote

point is, how can anyone prove that the retained is the first duplicate.
as I said before,
it is nonsensical to insist on retaining the 'first' of a set of dups.
why????????????????
Back to top
View user's profile Send private message
enrico-sorichetti

Global Moderator


Joined: 14 Mar 2007
Posts: 10202
Location: italy

PostPosted: Wed Aug 18, 2010 9:08 pm    Post subject: Reply to: ICETOOL block compare
Reply with quote

Hi Dick!
I am trying to understand and explain

if the records are in sequence any duplicate will do,
but if the records are <unordered> and the undupped file must retain the
original <unordered> pseudo-sequence then asking to keep the first occurrence of the duplicate <thing> seems legitimate to me icon_biggrin.gif


Code:
35
99
18
66
18
14
99

keeping the first would result  in

35
99
18
66
14
Back to top
View user's profile Send private message
dick scherrer

Site Director


Joined: 23 Nov 2006
Posts: 19270
Location: Inside the Matrix

PostPosted: Wed Aug 18, 2010 9:14 pm    Post subject:
Reply with quote

Hello,

Which may the reason for:
Quote:
. . . discard all duplicate blocks (exact duplicate of entire block) and keep the first block only in the output file. No sorting should be done.
The records/blocks are "in order" but not by something "code" might use.

It would help me (at least) to understand what this data really is (not the actual content, but what business process it is involved with). I never have done well with alphabet-soup. . . icon_rolleyes.gif
Back to top
View user's profile Send private message
dbzTHEdinosauer

Global Moderator


Joined: 20 Oct 2006
Posts: 6966
Location: porcelain throne

PostPosted: Wed Aug 18, 2010 10:15 pm    Post subject:
Reply with quote

Enrico,

thx! I understand. appreciate the 'insight'.
Back to top
View user's profile Send private message
CICS fan

New User


Joined: 03 Apr 2008
Posts: 82
Location: United States

PostPosted: Mon Aug 23, 2010 11:23 am    Post subject:
Reply with quote

Thanks enrico. couldnt have explained better. so the concensus is that ICETOOL/DFSORT cannot provide a solution here, is it?
Back to top
View user's profile Send private message
dick scherrer

Site Director


Joined: 23 Nov 2006
Posts: 19270
Location: Inside the Matrix

PostPosted: Mon Aug 23, 2010 11:07 pm    Post subject:
Reply with quote

Hello,

I suspect that the concensus is that as stated, nothing can do what is wanted. . . This has nothng to do with the capabilities of the sort or some other utility or some custom-written code. This has to do with the way the requirement has been posted.

It is doubtful that the technical capability exists to do this (again, as stated) for hgh-volume data.

If this is really wanted, suggest consideration be given to getting the groups in sequence, identifying duplicates, and then putting the output back into the original order. . .
Back to top
View user's profile Send private message
CICS fan

New User


Joined: 03 Apr 2008
Posts: 82
Location: United States

PostPosted: Wed Aug 25, 2010 10:04 am    Post subject:
Reply with quote

Thanks Dick..I understand.
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> DFSORT/ICETOOL All times are GMT + 6 Hours
Goto page 1, 2  Next
Page 1 of 2

 

Search our Forum:

Similar Topics
Topic Author Forum Replies Posted
No new posts Match or compare two files in VB Format anatol DFSORT/ICETOOL 14 Thu Nov 03, 2016 7:41 pm
No new posts How to delete second instance from Fl... Gunapala CN DFSORT/ICETOOL 6 Tue Oct 18, 2016 11:42 pm
No new posts ICETOOL/SYNCSORT - SYSPRINT output nartcr DFSORT/ICETOOL 17 Wed Oct 05, 2016 10:46 pm
No new posts COMPARE command sivatechdrive TSO/ISPF 7 Fri Sep 16, 2016 4:31 pm
No new posts SORT Block of records girishb2 DFSORT/ICETOOL 12 Thu Sep 15, 2016 8:58 am


Facebook
Back to Top
 
Mainframe Wiki | Forum Rules | Bookmarks | Subscriptions | FAQ | Tutorials | Contact Us