IBM Mainframe Forum Index
 
Log In
 
IBM Mainframe Forum Index Mainframe: Search IBM Mainframe Forum: FAQ Register
 

Change Data Capture


IBM Mainframe Forums -> DFSORT/ICETOOL
Post new topic   Reply to topic
View previous topic :: View next topic  
Author Message
Ron Klop

New User


Joined: 28 Sep 2012
Posts: 28
Location: holland

PostPosted: Fri Apr 15, 2016 2:56 pm
Reply with quote

Hai

Daily we get a file with around 10 million records. It contains a snapshot (full). We only want to process the mutations (delta). What is a quick way to sort out the mutations??
Thanx
Ron

Example

File A previous day
Code:
Recordnr   code   name
1      A   Always
2      D   Direct
7      F   Forever
8      L   Long


File B current day
Code:
Recordnr   code   name
1      A   All times
2      D   Direct
3      H   Hold
7      F   Forever


Output C
Code:
Recordnr   code   name
1      A   All times
3      H   Hold


Output C contains record 1, because the name has changed.
It does not contain record 2 and 7, because they are not changed.
It contains record 3, because it is a new one.
It does not contain record 8, because we are not interested in deletes (although if there is a way to make them recognizable in the output, it might come in handy in the future)

Code'd
Back to top
View user's profile Send private message
Bill Woodger

Moderator Emeritus


Joined: 09 Mar 2011
Posts: 7309
Location: Inside the Matrix

PostPosted: Fri Apr 15, 2016 3:17 pm
Reply with quote

Please use the Code tags to preserve spacing and use the Preview button to check before posting.

Please don't use tabs in a document and then paste them in. I attempted to demangle things to get them to line up, but the presence of tabs consumed even more time than usual, so I abandoned it.

Are the record-numbers present in the data, or just for illustration? Do you have a key? Are keys unique? If not, how would any matching go? Have you looked at SUPERCE to see if it can give you what you want?
Back to top
View user's profile Send private message
enrico-sorichetti

Superior Member


Joined: 14 Mar 2007
Posts: 10872
Location: italy

PostPosted: Fri Apr 15, 2016 3:54 pm
Reply with quote

Quote:
We only want to process the mutations (delta).


define how to identify mutations
Back to top
View user's profile Send private message
Bill Woodger

Moderator Emeritus


Joined: 09 Mar 2011
Posts: 7309
Location: Inside the Matrix

PostPosted: Fri Apr 15, 2016 4:10 pm
Reply with quote

And that makes 10,000, enrico :-)
Back to top
View user's profile Send private message
Ron Klop

New User


Joined: 28 Sep 2012
Posts: 28
Location: holland

PostPosted: Fri Apr 15, 2016 5:54 pm
Reply with quote

hai Bill

sorry, I am a newbie

recordnumbers are for illustration
code-field (A, D, F etc) is the key and they are unique
Haven't looked at SUPERCE (yet).

Enrico: what do you mean wíth 'define how to identify mutations'?? We get the data from another system. That system will not change the layout of there records, for instance by adding a mutation-indicator

greetings
Ron
Back to top
View user's profile Send private message
Ron Klop

New User


Joined: 28 Sep 2012
Posts: 28
Location: holland

PostPosted: Fri Apr 15, 2016 9:34 pm
Reply with quote

Hai Bill

can SUPERCE also be performed in batch, cause I am looking for a job / jcl that can be performed daily in production

greetings
Back to top
View user's profile Send private message
Bill Woodger

Moderator Emeritus


Joined: 09 Mar 2011
Posts: 7309
Location: Inside the Matrix

PostPosted: Fri Apr 15, 2016 10:41 pm
Reply with quote

Yes.

You'd get a listing you'd have to post-process if it is for machine rather than human consumption.

Your task is a simple JOINKEYS. If no-one beats my to it, I'll find an example, there are lots here. You will be able to identify deletes if you want.
Back to top
View user's profile Send private message
Rohit Umarjikar

Global Moderator


Joined: 21 Sep 2010
Posts: 3049
Location: NYC,USA

PostPosted: Fri Apr 15, 2016 11:19 pm
Reply with quote

As Bill said above, it is a simple joinkeys, next time please search on this forum and then post as you would find lots of examples.
If you care for the whole record and if anything is changes per record then catch that then you can use this .
Code:
JOINKEYS F1=IN1,FIELDS=(1,your total rec length,A)
JOINKEYS F2=IN2,FIELDS=(1,your total rec length,A)
REFORMAT FIELDS=(F1:1,your total rec length)   
JOIN UNPAIRED,F1,ONLY         
OPTION COPY       

F1 = File B current day
F2 = File A previous day
Back to top
View user's profile Send private message
Ron Klop

New User


Joined: 28 Sep 2012
Posts: 28
Location: holland

PostPosted: Sat Apr 16, 2016 3:06 am
Reply with quote

Thnx Rohit

what does this mean:
FIELDS(1,your total rec length,A) ?

suppose my total record length is 80
Back to top
View user's profile Send private message
Bill Woodger

Moderator Emeritus


Joined: 09 Mar 2011
Posts: 7309
Location: Inside the Matrix

PostPosted: Sat Apr 16, 2016 4:47 am
Reply with quote

On the JOINKEYS, the FIELDS parameter identifies the key for the join. You have two options, since your key is first: specify the entire record as the key - which is useful if you want to list changes as add/delete pairs; specify the key only, then use an IFTHEN=(WHEN=(logicalexpression) in the main task to check whether matched keys contain the same data.
Back to top
View user's profile Send private message
Ron Klop

New User


Joined: 28 Sep 2012
Posts: 28
Location: holland

PostPosted: Sat Apr 16, 2016 3:12 pm
Reply with quote

I really appreciate your help. Been reading about JOINKEYS in the manual, bur still find it difficult in my example.
In my example the records look like this:
"A Always". So a code with a description.

If I want to compare file B with A and want to know what has been added, updated and deleted today when comparing with yesterday, how does my JOINKEY look like then if:
    1. I want to compare on the whole record

    2. I want to compare on the key (which is code) and use IFTHEN=(WHEN= in the main task to see if the descripton has change


Is somebody willing to write it out for me??

thnx
Ron
Back to top
View user's profile Send private message
boyti ko

New User


Joined: 03 Nov 2014
Posts: 78
Location: Malaysia

PostPosted: Mon Apr 18, 2016 12:07 pm
Reply with quote

Maybe you could try doing it at least. They already spent their time explaining things to you. But in return you just want the whole code to be given to you.
Back to top
View user's profile Send private message
Bill Woodger

Moderator Emeritus


Joined: 09 Mar 2011
Posts: 7309
Location: Inside the Matrix

PostPosted: Mon Apr 18, 2016 6:58 pm
Reply with quote

Code:
//SYSIN    DD *
  JOINKEYS F1=IN1,FIELDS=(1,1,A)
  JOINKEYS F2=IN2,FIELDS=(1,1,A)
                                   
  REFORMAT FIELDS=(F1:1,80,F2:5,76)
                                   
  JOIN UNPAIRED,F1
                                   
                                   
  OPTION COPY
                                   
  OUTFIL OMIT=(5,76,CH,EQ,81,76,CH),
         BUILD=(1,80)
                                   
//IN1 DD *
A   ALL TIMES
D   DIRECT
H   HOLD
F   FOREVER
//IN2 DD *
A   ALWAYS
D   DIRECT
F   FOREVER
L   LONG


You show your data as unsorted. It would be much better if the data was sorted, otherwise it will always be sorted twice (yesterday and today).

If it were in order prior to this step, you'd specify SORTED,NOSEQCK on the JOINKEYS statements.

If you don't have an existing SORT, you could output the SORTed "today's" with an additional OUTFIL and adding the match-marker (?) to the REFORMAT and selecting all the 1's.
Back to top
View user's profile Send private message
Rohit Umarjikar

Global Moderator


Joined: 21 Sep 2010
Posts: 3049
Location: NYC,USA

PostPosted: Mon Apr 18, 2016 10:31 pm
Reply with quote

Quote:
Is somebody willing to write it out for me??

Start here
Back to top
View user's profile Send private message
Ron Klop

New User


Joined: 28 Sep 2012
Posts: 28
Location: holland

PostPosted: Tue Apr 19, 2016 10:25 pm
Reply with quote

Thnx all

Data is sorted when received. I will start working on this one of these days
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic View Bookmarks
All times are GMT + 6 Hours
Forum Index -> DFSORT/ICETOOL

 


Similar Topics
Topic Forum Replies
No new posts Data set Rec-Cnt and Byte-Cnt Testing & Performance 2
No new posts SCOPE PENDING option -check data DB2 2
No new posts Check data with Exception Table DB2 0
No new posts JCL EXEC PARM data in C Java & MQSeries 2
This topic is locked: you cannot edit posts or make replies. Automation need help in sorting the data DFSORT/ICETOOL 38
Search our Forums:

Back to Top