|
View previous topic :: View next topic
|
| Author |
Message |
Hima1985
New User
Joined: 17 Apr 2009
Posts: 70
Location: India
|
|
|
|
Hi,
Could you please suggest for below:
We are comparing two sequential files using SORT utility and need to write the matching records in one file and non matching records in another file.
Both files are LRECL of 116 / FB and the key field is from position 1 to 106.
First file contains 20 million records(no duplicates) and second file contains 35 million records(may contain duplicates).
Do we need to manually allocate Sort work files for this sort which involves comparison of such huge data ? Or SORT utility dynamically allocates all the space needed by this compare?
If we need to manually allocate Sort work files, what would be the approximate space allocation needed for each work dataset? Please suggest.
We are using Z/OS DFSORT V1R12.
| Code: |
//SYSIN DD *
JOINKEYS F1=IN1,FIELDS=(1,106,D)
JOINKEYS F2=IN2,FIELDS=(1,106,D)
JOIN UNPAIRED,F1
REFORMAT FIELDS=(F1:1,116,?)
OPTION COPY
OUTFIL FNAMES=OUT1,INCLUDE=(117,1,CH,EQ,C'B'),
BUILD=(1,116)
OUTFIL FNAMES=OUT2,INCLUDE=(117,1,CH,EQ,C'1'),
BUILD=(1,116)
/* |
Please let me know if you need more information. Thanks. |
|
| Back to top |
|
 |
dbzTHEdinosauer
Global Moderator
Joined: 20 Oct 2006
Posts: 6966
Location: porcelain throne
|
|
|
|
have you bothered to even test?
normally, sortwork should be left alone and allow sort to allocate as is needed. |
|
| Back to top |
|
|
Hima1985
New User
Joined: 17 Apr 2009
Posts: 70
Location: India
|
|
|
|
| We need promote this job to Production environment where actual data exists. So want to ensure it gets executed to avoid any space abends.. |
|
| Back to top |
|
|
dbzTHEdinosauer
Global Moderator
Joined: 20 Oct 2006
Posts: 6966
Location: porcelain throne
|
|
|
|
| Hima1985 wrote: |
| We need promote this job to Production environment where actual data exists. So want to ensure it gets executed to avoid any space abends.. |
answer my question: Have you tested? what were the results? |
|
| Back to top |
|
|
Hima1985
New User
Joined: 17 Apr 2009
Posts: 70
Location: India
|
|
|
|
| I have not tested it yet as we cannot perform volume testing here in test environment and this job is not yet implemented in production. |
|
| Back to top |
|
|
dbzTHEdinosauer
Global Moderator
Joined: 20 Oct 2006
Posts: 6966
Location: porcelain throne
|
|
|
|
Ahhh! Now I understand.
you want someone that has NO
understanding of your site,
idea of what kind of equipment,
idea of production schedule,
to say "yes this will work".
if this job is a critical business step,
then you need to test based on a production load.
anything less would not be a responsible implementation.
i am not sure what your site lacks most: competence or character. |
|
| Back to top |
|
|
dick scherrer
Moderator Emeritus
Joined: 23 Nov 2006
Posts: 19244
Location: Inside the Matrix
|
|
|
|
Hello,
Suggest you make sure the files are in sequence Before you compare them. Then, no sortwork will be needed for the compare process.
How can anyone justify turning this in to Production when full-volume testing has not been done?
Utter nonsense  |
|
| Back to top |
|
|
Bill Woodger
Moderator Emeritus
Joined: 09 Mar 2011
Posts: 7309
Location: Inside the Matrix
|
|
|
|
| What is your problem with generating the volume data? DFSORT can do that for you as well. |
|
| Back to top |
|
|
Hima1985
New User
Joined: 17 Apr 2009
Posts: 70
Location: India
|
|
|
|
Thanks for your suggestions. Ill try that.
| Quote: |
| i am not sure what your site lacks most: competence or character. |
I never expected such response from experts for which you may just ignore the post without providing any suggestions. Very sorry for wasting your valuable time. |
|
| Back to top |
|
|
dick scherrer
Moderator Emeritus
Joined: 23 Nov 2006
Posts: 19244
Location: Inside the Matrix
|
|
|
|
Hello,
You might not have expected such a response, but keep in mind that the "experts" here are as different as teachers in school - we all have different ways to make a point or get someone's attention. Rather than being offended, suggest you look at the intent of the poster. What you took exception to really does point to signifcantly serious problems in that organization.
The only reason most of us are here is to provide some assistance and help people learn. Both technical stuff and business practices.
So far, i see no waste of time. . . |
|
| Back to top |
|
|
sqlcode1
Active Member
Joined: 08 Apr 2010
Posts: 577
Location: USA
|
|
|
|
| Hima1985 wrote: |
Do we need to manually allocate Sort work files for this sort which involves comparison of such huge data ? Or SORT utility dynamically allocates all the space needed by this compare?
If we need to manually allocate Sort work files, what would be the approximate space allocation needed for each work dataset? Please suggest.
We are using Z/OS DFSORT V1R12.
...
...
Please let me know if you need more information. Thanks. |
You would have to tell DFSort about what you want to do (Dynamic Allocation or hardcoding SORTWK). It wouldn't do anything on its own. For dynamic allocation you need to define dynamic allocation.
Of course, if you manually specify SORTWK, you need to define enough SORTWK for your current (and future?) requirement.
Its really hard to give you accurate answers without you testings because some of the space requirement for sort varies from time to time even at the same shop based on space availability.
Thanks, |
|
| Back to top |
|
|
dbzTHEdinosauer
Global Moderator
Joined: 20 Oct 2006
Posts: 6966
Location: porcelain throne
|
|
|
|
i think what annoyed me most was that the TS obviously has
not bothered to even look at 10.2 Design Your Applications to Maximize Performance
and has instead decided to rely
on internet input concerning an implementation.
hey, tell your boss that I said it was ok to run in production just as you have been running in test. |
|
| Back to top |
|
|
Frank Yaeger
DFSORT Developer
Joined: 15 Feb 2005
Posts: 7129
Location: San Jose, CA
|
|
|
|
| Quote: |
| Do we need to manually allocate Sort work files for this sort which involves comparison of such huge data ? Or SORT utility dynamically allocates all the space needed by this compare? |
If you don't specify SORTWKdd DD statements, DFSORT will dynamically allocate the work space according to your site's installation defaults. We do recommend the use of dynamic allocation. If your site's default is not sufficient for your JOINKEYS job, you can override it using:
| Code: |
//JNF1CNTL DD *
OPTION DYNALLOC=(,n)
//JNF2CNTL DD *
OPTION DYNALLOC=(,n)
|
where n can be up to 255 work data sets. If you're worried that your installation default is not enough, you could start with
| Code: |
OPTION DYNALLOC=(,16)
|
as a starting point. |
|
| Back to top |
|
|
Hima1985
New User
Joined: 17 Apr 2009
Posts: 70
Location: India
|
|
|
|
| Thanks much Frank and sqlcode1. This is very informative. I will test the job with given options. |
|
| Back to top |
|
|
Marso
REXX Moderator
Joined: 13 Mar 2006
Posts: 1353
Location: Israel
|
|
|
|
You forgot to thank Dick and Bill for their suggestions:
| dick scherrer wrote: |
| Suggest you make sure the files are in sequence Before you compare them. Then, no sortwork will be needed for the compare process. |
| Bill Woodger wrote: |
| What is your problem with generating the volume data? DFSORT can do that for you as well. |
We've already read too many times sentences like "I cannot change the JCL" or "I cannot modify the program" or "I cannot do this or do that".
Sometimes it has been proven wrong, but sometimes it seems to be true.
I often wonder how you can work in such conditions... |
|
| Back to top |
|
|
|
|
