View previous topic :: View next topic
|
Author |
Message |
PDB
New User
Joined: 18 Sep 2007 Posts: 7 Location: India
|
|
|
|
Hi,
I need to compare two sequential files and both have account number as key field. There are 4 data fields which I need to compare. These 4 fields reside in two different types of account types. Therefore, I can have one or two entries of an account number. Also, the amount of data I am comparing is huge. I have almost 9 - 10 million records in both the files.
Here's my approach:
In JOB1 I combine the two record types of file A to make single record for every account.
In JOB2 I combine the two record types of file B to make single record for every account.
In JOB3 I compare the two files and write my output.
Now this complete process takes almost 3 hours. I tried giving more space to work files but still the process takes this much time.
Please suggest how the performance can be improved. |
|
Back to top |
|
|
dick scherrer
Moderator Emeritus
Joined: 23 Nov 2006 Posts: 19244 Location: Inside the Matrix
|
|
|
|
Hello,
How long does it take to simply read both files using easytrieve? |
|
Back to top |
|
|
chandrarkar
New User
Joined: 27 May 2008 Posts: 24 Location: india
|
|
|
|
I think if u provide the individual time taken by each individual job,
we can see which can be fine tuned. |
|
Back to top |
|
|
PDB
New User
Joined: 18 Sep 2007 Posts: 7 Location: India
|
|
|
|
It took 66 minutes to just read 1 file. I gave this parameter:
//EZTVFM DD DISP=(NEW,DELETE),
// SPACE=(CYL,(1000,1000),RLSE),
// UNIT=SYSDA |
|
Back to top |
|
|
dick scherrer
Moderator Emeritus
Joined: 23 Nov 2006 Posts: 19244 Location: Inside the Matrix
|
|
|
|
Hello,
If it took 66 minutes to read only 1 file, the 3-hour total run-time does not seem out of line. . . |
|
Back to top |
|
|
|