IBM Mainframe Forum Index
 
Log In
 
IBM Mainframe Forum Index Mainframe: Search IBM Mainframe Forum: FAQ Register
 

sort to KSDS VSAM performance consideration


IBM Mainframe Forums -> DFSORT/ICETOOL
Post new topic   Reply to topic
View previous topic :: View next topic  
Author Message
rally

New User


Joined: 28 Jul 2005
Posts: 37

PostPosted: Sat May 12, 2007 6:09 am
Reply with quote

Hi,

We need add 1% data into the KSDS VSAM file daily,

we de/def the whole KSDS VSAM file and sort the new 1% data
and other old 99% data from SEQ to the empty KSDS VSAM.

Can we got better performance if we only MERGE the 1% data
into the old KSDS VSAM(don't del/def the file)?

Anybody can share your experience?

Thanks.
Back to top
View user's profile Send private message
dick scherrer

Moderator Emeritus


Joined: 23 Nov 2006
Posts: 19244
Location: Inside the Matrix

PostPosted: Sat May 12, 2007 7:02 am
Reply with quote

Hello,

How many records are there in the file now? How many alternate indexes are there (if any).

What is the rate of online transactions that add and/or delete records from the file during processing each day. How much batch activity is there against this file?

How often is data purged from the file or are all of the records kept "forever"? At the rate of 1% a day, the file will double in size about 3 times a year.

Does this file need to be available 7 days a week or mon-fri?

Quote:
Can we got better performance if we only MERGE the 1% data
into the old KSDS VSAM(don't del/def the file)?


There is more to performance than just how long the daily sort/load job runs. How long do the individual steps run now (del/def, sort)?
Back to top
View user's profile Send private message
rally

New User


Joined: 28 Jul 2005
Posts: 37

PostPosted: Sat May 12, 2007 9:46 pm
Reply with quote

dick scherrer wrote:
Hello,

How many records are there in the file now? How many alternate indexes are there (if any).
rally>>>total about 36,000,000 records and no AIX.

What is the rate of online transactions that add and/or delete records from the file during processing each day. How much batch activity is there against this file?
rally>>> no onliine add/delete records, only inquiry during online.
only 1 job will add about 300,000 records into this file.


How often is data purged from the file or are all of the records kept "forever"? At the rate of 1% a day, the file will double in size about 3 times a year.
rally>>> there is 1 monthly job will housekeeping this file, and about 15%
records will be purged.


Does this file need to be available 7 days a week or mon-fri?
rally>>> should be available 7 days
Quote:
Can we got better performance if we only MERGE the 1% data
into the old KSDS VSAM(don't del/def the file)?


There is more to performance than just how long the daily sort/load job runs. How long do the individual steps run now (del/def, sort)?

rally>>> about 25 minutes
Back to top
View user's profile Send private message
dick scherrer

Moderator Emeritus


Joined: 23 Nov 2006
Posts: 19244
Location: Inside the Matrix

PostPosted: Sat May 12, 2007 11:01 pm
Reply with quote

Hello,

Can you "borrow" enough disk space to a bit of testing?

My guess is that you would want to try the merge for the daily job. On a "slow" day each week (maybe tue or sat?) you could run the current process. If online performance between loads was satisfactory, you might even be able to run the del/def/load at the same time the purge is run and only need to run it monthly. Is the purge run directly from the vsam file or is a repro done, then the "old" data separated before reloading? I believe a repro/split/load would perform as well or better than deleting 15% of the data directly with individual deletes. It could also include the current day's input with very little additional time/system resources used.

Is it known if the majority of the daily inquiry processes are for the most current data or is the data requested (fairly) evenly over the entire set of data? If most of the inquiry transactions read the most recent data, this could also be a factor.
Back to top
View user's profile Send private message
rally

New User


Joined: 28 Jul 2005
Posts: 37

PostPosted: Sun May 13, 2007 7:10 am
Reply with quote

dick scherrer wrote:
Hello,

Can you "borrow" enough disk space to a bit of testing?

My guess is that you would want to try the merge for the daily job. On a "slow" day each week (maybe tue or sat?) you could run the current process. If online performance between loads was satisfactory, you might even be able to run the del/def/load at the same time the purge is run and only need to run it monthly. Is the purge run directly from the vsam file or is a repro done, then the "old" data separated before reloading? I believe a repro/split/load would perform as well or better than deleting 15% of the data directly with individual deletes. It could also include the current day's input with very little additional time/system resources used.
rally>>> Yes, I want to run the del/def/load the whole VSAM file at the same time the purge is run and only need to run it monthly. the purge run on the corresponding SEQ file, and del/def VSAM and repro the retain records back to VSAM.

Is it known if the majority of the daily inquiry processes are for the most current data or is the data requested (fairly) evenly over the entire set of data? If most of the inquiry transactions read the most recent data, this could also be a factor.
rally>>>the inquriy will read recent data also old data as per some key seq, date as part of key.


Hi Dick,

Thank you for your suggestions.
Back to top
View user's profile Send private message
dick scherrer

Moderator Emeritus


Joined: 23 Nov 2006
Posts: 19244
Location: Inside the Matrix

PostPosted: Sun May 13, 2007 9:20 am
Reply with quote

You're welcome icon_smile.gif

Please let us know how you proceed and any lessons learned. It will help some one later with a similar question.
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic View Bookmarks
All times are GMT + 6 Hours
Forum Index -> DFSORT/ICETOOL

 


Similar Topics
Topic Forum Replies
No new posts How to split large record length file... DFSORT/ICETOOL 10
No new posts Access to non cataloged VSAM file JCL & VSAM 18
No new posts JCL sort card - get first day and las... JCL & VSAM 9
No new posts Sort First/last record of a subset th... DFSORT/ICETOOL 7
No new posts Merge two VSAM KSDS files into third ... JCL & VSAM 6
Search our Forums:

Back to Top