Portal | Manuals | References | Downloads | Info | Programs | JCLs | Master the Mainframes
IBM Mainframe Computers Forums Index
 
Register
 
IBM Mainframe Computers Forums Index Mainframe: Search IBM Mainframe Forum: FAQ Memberlist Usergroups Profile Log in to check your private messages Log in
 

 

Query regarding usage of Sort work files in comparision.

 
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> DFSORT/ICETOOL
View previous topic :: :: View next topic  
Author Message
Hima1985

New User


Joined: 17 Apr 2009
Posts: 70
Location: India

PostPosted: Thu May 17, 2012 3:27 pm    Post subject: Query regarding usage of Sort work files in comparision.
Reply with quote

Hi,

Could you please suggest for below:

We are comparing two sequential files using SORT utility and need to write the matching records in one file and non matching records in another file.
Both files are LRECL of 116 / FB and the key field is from position 1 to 106.

First file contains 20 million records(no duplicates) and second file contains 35 million records(may contain duplicates).

Do we need to manually allocate Sort work files for this sort which involves comparison of such huge data ? Or SORT utility dynamically allocates all the space needed by this compare?

If we need to manually allocate Sort work files, what would be the approximate space allocation needed for each work dataset? Please suggest.

We are using Z/OS DFSORT V1R12.

Code:
//SYSIN    DD  *                                     
  JOINKEYS F1=IN1,FIELDS=(1,106,D)                   
  JOINKEYS F2=IN2,FIELDS=(1,106,D)                   
                                                     
  JOIN UNPAIRED,F1                                   
                                                     
  REFORMAT FIELDS=(F1:1,116,?)                       
  OPTION COPY                                         
                                                     
  OUTFIL FNAMES=OUT1,INCLUDE=(117,1,CH,EQ,C'B'),     
  BUILD=(1,116)                                       
                                                     
  OUTFIL FNAMES=OUT2,INCLUDE=(117,1,CH,EQ,C'1'),     
  BUILD=(1,116)                                       
/*   


Please let me know if you need more information. Thanks.
Back to top
View user's profile Send private message

dbzTHEdinosauer

Global Moderator


Joined: 20 Oct 2006
Posts: 6966
Location: porcelain throne

PostPosted: Thu May 17, 2012 4:05 pm    Post subject:
Reply with quote

have you bothered to even test?

normally, sortwork should be left alone and allow sort to allocate as is needed.
Back to top
View user's profile Send private message
Hima1985

New User


Joined: 17 Apr 2009
Posts: 70
Location: India

PostPosted: Thu May 17, 2012 4:12 pm    Post subject: Reply to: Query regarding usage of Sort work files in compar
Reply with quote

We need promote this job to Production environment where actual data exists. So want to ensure it gets executed to avoid any space abends..
Back to top
View user's profile Send private message
dbzTHEdinosauer

Global Moderator


Joined: 20 Oct 2006
Posts: 6966
Location: porcelain throne

PostPosted: Thu May 17, 2012 4:32 pm    Post subject: Re: Reply to: Query regarding usage of Sort work files in co
Reply with quote

Hima1985 wrote:
We need promote this job to Production environment where actual data exists. So want to ensure it gets executed to avoid any space abends..


answer my question: Have you tested? what were the results?
Back to top
View user's profile Send private message
Hima1985

New User


Joined: 17 Apr 2009
Posts: 70
Location: India

PostPosted: Thu May 17, 2012 5:00 pm    Post subject:
Reply with quote

I have not tested it yet as we cannot perform volume testing here in test environment and this job is not yet implemented in production.
Back to top
View user's profile Send private message
dbzTHEdinosauer

Global Moderator


Joined: 20 Oct 2006
Posts: 6966
Location: porcelain throne

PostPosted: Thu May 17, 2012 5:46 pm    Post subject:
Reply with quote

Ahhh! Now I understand.

you want someone that has NO
understanding of your site,
idea of what kind of equipment,
idea of production schedule,

to say "yes this will work".

if this job is a critical business step,
then you need to test based on a production load.

anything less would not be a responsible implementation.

i am not sure what your site lacks most: competence or character.
Back to top
View user's profile Send private message
dick scherrer

Site Director


Joined: 23 Nov 2006
Posts: 19270
Location: Inside the Matrix

PostPosted: Thu May 17, 2012 6:56 pm    Post subject:
Reply with quote

Hello,

Suggest you make sure the files are in sequence Before you compare them. Then, no sortwork will be needed for the compare process.

How can anyone justify turning this in to Production when full-volume testing has not been done?

Utter nonsense icon_sad.gif
Back to top
View user's profile Send private message
Bill Woodger

DFSORT Moderator


Joined: 09 Mar 2011
Posts: 7224

PostPosted: Thu May 17, 2012 7:00 pm    Post subject: Reply to: Query regarding usage of Sort work files in compar
Reply with quote

What is your problem with generating the volume data? DFSORT can do that for you as well.
Back to top
View user's profile Send private message
Hima1985

New User


Joined: 17 Apr 2009
Posts: 70
Location: India

PostPosted: Thu May 17, 2012 7:33 pm    Post subject:
Reply with quote

Thanks for your suggestions. Ill try that.

Quote:
i am not sure what your site lacks most: competence or character.


I never expected such response from experts for which you may just ignore the post without providing any suggestions. Very sorry for wasting your valuable time.
Back to top
View user's profile Send private message
dick scherrer

Site Director


Joined: 23 Nov 2006
Posts: 19270
Location: Inside the Matrix

PostPosted: Thu May 17, 2012 7:42 pm    Post subject:
Reply with quote

Hello,

You might not have expected such a response, but keep in mind that the "experts" here are as different as teachers in school - we all have different ways to make a point or get someone's attention. Rather than being offended, suggest you look at the intent of the poster. What you took exception to really does point to signifcantly serious problems in that organization.

The only reason most of us are here is to provide some assistance and help people learn. Both technical stuff and business practices.

So far, i see no waste of time. . .
Back to top
View user's profile Send private message
sqlcode1

Active Member


Joined: 08 Apr 2010
Posts: 578
Location: USA

PostPosted: Thu May 17, 2012 9:37 pm    Post subject: Re: Query regarding usage of Sort work files in comparision.
Reply with quote

Hima1985 wrote:

Do we need to manually allocate Sort work files for this sort which involves comparison of such huge data ? Or SORT utility dynamically allocates all the space needed by this compare?

If we need to manually allocate Sort work files, what would be the approximate space allocation needed for each work dataset? Please suggest.

We are using Z/OS DFSORT V1R12.
...
...
Please let me know if you need more information. Thanks.

You would have to tell DFSort about what you want to do (Dynamic Allocation or hardcoding SORTWK). It wouldn't do anything on its own. For dynamic allocation you need to define dynamic allocation.

Of course, if you manually specify SORTWK, you need to define enough SORTWK for your current (and future?) requirement.

Its really hard to give you accurate answers without you testings because some of the space requirement for sort varies from time to time even at the same shop based on space availability.

Thanks,
Back to top
View user's profile Send private message
dbzTHEdinosauer

Global Moderator


Joined: 20 Oct 2006
Posts: 6966
Location: porcelain throne

PostPosted: Thu May 17, 2012 10:02 pm    Post subject:
Reply with quote

i think what annoyed me most was that the TS obviously has
not bothered to even look at 10.2 Design Your Applications to Maximize Performance
and has instead decided to rely
on internet input concerning an implementation.

hey, tell your boss that I said it was ok to run in production just as you have been running in test.
Back to top
View user's profile Send private message
Frank Yaeger

DFSORT Moderator


Joined: 15 Feb 2005
Posts: 7130
Location: San Jose, CA

PostPosted: Thu May 17, 2012 11:21 pm    Post subject:
Reply with quote

Quote:
Do we need to manually allocate Sort work files for this sort which involves comparison of such huge data ? Or SORT utility dynamically allocates all the space needed by this compare?


If you don't specify SORTWKdd DD statements, DFSORT will dynamically allocate the work space according to your site's installation defaults. We do recommend the use of dynamic allocation. If your site's default is not sufficient for your JOINKEYS job, you can override it using:

Code:

//JNF1CNTL DD *
   OPTION DYNALLOC=(,n)
//JNF2CNTL DD *
   OPTION DYNALLOC=(,n)


where n can be up to 255 work data sets. If you're worried that your installation default is not enough, you could start with

Code:

  OPTION DYNALLOC=(,16)


as a starting point.
Back to top
View user's profile Send private message
Hima1985

New User


Joined: 17 Apr 2009
Posts: 70
Location: India

PostPosted: Fri May 18, 2012 12:05 pm    Post subject:
Reply with quote

Thanks much Frank and sqlcode1. This is very informative. I will test the job with given options.
Back to top
View user's profile Send private message
Marso

REXX Moderator


Joined: 13 Mar 2006
Posts: 1218
Location: Israel

PostPosted: Sat May 19, 2012 4:16 am    Post subject:
Reply with quote

You forgot to thank Dick and Bill for their suggestions:

dick scherrer wrote:
Suggest you make sure the files are in sequence Before you compare them. Then, no sortwork will be needed for the compare process.
Bill Woodger wrote:
What is your problem with generating the volume data? DFSORT can do that for you as well.


We've already read too many times sentences like "I cannot change the JCL" or "I cannot modify the program" or "I cannot do this or do that".
Sometimes it has been proven wrong, but sometimes it seems to be true.

I often wonder how you can work in such conditions...
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> DFSORT/ICETOOL All times are GMT + 6 Hours
Page 1 of 1

 

Search our Forum:

Similar Topics
Topic Author Forum Replies Posted
No new posts High CPU consumption Job using IAM fi... aswinir JCL & VSAM 8 Thu Dec 01, 2016 8:28 pm
No new posts Limit duplicate records in the SORT pshongal SYNCSORT 6 Mon Nov 21, 2016 12:54 pm
No new posts How to convert the VBM file to VB or... Sulabh Agrawal JCL & VSAM 4 Fri Nov 18, 2016 1:04 pm
This topic is locked: you cannot edit posts or make replies. Db2 SQL query to convert rows into co... subratarec DB2 5 Thu Nov 17, 2016 4:51 pm
No new posts Two Selects in a single select query Rohit Umarjikar DB2 1 Fri Nov 04, 2016 8:46 pm


Facebook
Back to Top
 
Mainframe Wiki | Forum Rules | Bookmarks | Subscriptions | FAQ | Tutorials | Contact Us