Portal | Manuals | References | Downloads | Info | Programs | JCLs | Master the Mainframes
IBM Mainframe Computers Forums Index
 
Register
 
IBM Mainframe Computers Forums Index Mainframe: Search IBM Mainframe Forum: FAQ Memberlist Usergroups Profile Log in to check your private messages Log in
 

 

Copy VSAM to VSAM using SORT - doable?

 
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> JCL & VSAM
View previous topic :: :: View next topic  
Author Message
rmd3003

New User


Joined: 03 Jul 2006
Posts: 53

PostPosted: Sat Dec 03, 2011 5:05 am    Post subject: Copy VSAM to VSAM using SORT - doable?
Reply with quote

Hello there. I have a huge VSAM file (100 million records) which is needed to be processed (eliminating some old records). I wrote SORT (with OMIT statement). Input is one VSAM file, output is new one. But it runs kind of slow. Anything I can add to speed it up?

I know, I could've sorted it to flat file and then REPRO back to VSAM but there is time constraint and job has to finish as quickly as possible. That's why I preallocate new file, copy to it, delete old one then rename new.

So again, is there anything I can add to this SORT here to make it run faster?

Thank you in advance.

Code:
//STEP1    EXEC PGM=SORT                             
//SORTIN   DD DSN=VSAMFILE.FILE1.IN,DISP=SHR
//SORTOUT  DD DSN=VSAMFILE.FILE1.OUT,DISP=SHR     
//SORTMSG  DD SYSOUT=*                             
//SYSOUT    DD SYSOUT=*                             
//SYSUDUMP  DD SYSOUT=*                             
//*                                                 
//SYSIN     DD *                                   
    OPTION COPY                                     
    OMIT COND=(12,4,PD,LT,2999)                     



Both files (IN and OUT) have identical parameters.

Code:
DEFINE CLUSTER                       
      (NAME(VSAMFILE.FILE1.IN)     
       SHAREOPTIONS(2 3)             
       INDEXED                       
       NOIMBED                       
       NOREPLICATE                   
       NOREUSE                       
       DATACLAS(EXTVSAMC)           
       RECORDSIZE(100 500)           
       FREESPACE(50 30)             
       KEY(28 0))                   
      DATA                           
      (NAME(VSAMFILE.FILE1.IN.DATA)
       CYL(4000 2500)                 
       VOLUMES(* * * * * *)           
       CISZ(18432))                   
      INDEX                           
      (NAME(VSAMFILE.FILE1.IN.INDEX)
       CYL(750 150)                   
       VOLUMES(* * )                   
       CISZ(4096))                     
Back to top
View user's profile Send private message

Frank Yaeger

DFSORT Moderator


Joined: 15 Feb 2005
Posts: 7130
Location: San Jose, CA

PostPosted: Sat Dec 03, 2011 5:57 am    Post subject:
Reply with quote

Quote:
But it runs kind of slow.


Compared to what? Based on what criteria?

Which sort product are you using?
Back to top
View user's profile Send private message
Bill Woodger

DFSORT Moderator


Joined: 09 Mar 2011
Posts: 7315

PostPosted: Sat Dec 03, 2011 6:50 am    Post subject: Re: Copy VSAM to VSAM using SORT - doable?
Reply with quote

rmd3003 wrote:
[...]
Code:
       FREESPACE(50 30)             

[...]


Do you know what that little bunny is doing?

How much free space do you think that leaves on a Cylinder? 65%. How many cylinders in your primary? 4000. That means you have data in 1400 cylinders there, and the rest of that is freespace.

Believe me. That is not improving you job's throughput.

Somewhere, how that dataset was defined and loaded is documented. It was not loaded with that freespace. Someone has been "tuning" the dataset (I'll assume correctly) and you need to find the documentation PDQ.

Otherwise, you are in guess territory. I wouldn't like to guess about a dataset with 100,000,000 records and a deadline. Let your boss know of the problem, otherwise it'll be your bottom in a sling if it all goes wrong. If I had to just guess, I'd want as much information about the dataset contents (that'd be the data) as possible. With nothing to go on, I'd have to assume it was loaded with the defaults (by removing the freespace parameter from the define) and then ALTER it after loaded to the freespace you currently have.

In fact, looking at your posting, I'd suggest:


  1. Leave it alone until you can find out exactly how the dataset got that freespace
  2. If you have to get rid of the old records and cannot wait, write a program and delete them in situ, until you can locate the documentation
  3. There are always three things
Back to top
View user's profile Send private message
Bill Woodger

DFSORT Moderator


Joined: 09 Mar 2011
Posts: 7315

PostPosted: Sat Dec 03, 2011 7:28 am    Post subject: Reply to: Copy VSAM to VSAM using SORT - doable?
Reply with quote

Maybe the freespace is OK, if you have an extremely volatile dataset, with many inserts across the whole thing, with the inserts being generally in key order, but I suspect you actually have many inserts in one or more particular places, and that the dataset would have been loaded originallly with less freespace (possibly even zero) and then ALTERed to give the freespace for the particular characteristic inserts of the data.

A LISTCAT of the dataset would be interesting, but wouldn't (necessarily) provide a full answer.

I think DFSORT would beat REPRO in a footrace. 100 million will take time. As has been asked already, what are you comparing it to to judge it "slow".

You have a large data CI size. Is the data generally processed in sequence? Sequential inserts at the "back" of the data?

EDIT: How did you test your job? What does that VSAM file look like (LISTCAT) in development and testing environments?
Back to top
View user's profile Send private message
PeterHolland

Global Moderator


Joined: 27 Oct 2009
Posts: 2433
Location: Netherlands, Amstelveen

PostPosted: Sat Dec 03, 2011 1:35 pm    Post subject: Re: Reply to: Copy VSAM to VSAM using SORT - doable?
Reply with quote

Bill Woodger wrote:
that the dataset would have been loaded originallly with less freespace (possibly even zero) and then ALTERed to give the freespace for the particular characteristic inserts of the data.


Normally if using freespace, the vsam dataset is loaded and then altered to FSPC(0 0). If after a period there are lots of splits, a reorg will be necessary.
Back to top
View user's profile Send private message
Bill Woodger

DFSORT Moderator


Joined: 09 Mar 2011
Posts: 7315

PostPosted: Sat Dec 03, 2011 2:44 pm    Post subject: Reply to: Copy VSAM to VSAM using SORT - doable?
Reply with quote

Well, I'd do it based on the data if necessary to do anything.

I doubt that was wanted for this dataset, as the high freespace is what it has now, with 100m records.
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> JCL & VSAM All times are GMT + 6 Hours
Page 1 of 1

 

Search our Forum:

Similar Topics
Topic Author Forum Replies Posted
No new posts How to change 'K' or 'M' use Sort vice_versa DFSORT/ICETOOL 5 Thu May 18, 2017 7:11 am
No new posts Adding big TEXT lines to each record ... bshkris SYNCSORT 4 Sat May 06, 2017 1:40 am
This topic is locked: you cannot edit posts or make replies. SORT trick needed bshkris SYNCSORT 6 Tue May 02, 2017 4:35 am
No new posts copy SEQ to PDS and create stats steve-myers JCL & VSAM 5 Thu Apr 27, 2017 1:15 am
No new posts VSAM Space Allocation mrgnndhmk JCL & VSAM 7 Sat Apr 22, 2017 12:42 am


Facebook
Back to Top
 
Mainframe Wiki | Forum Rules | Bookmarks | Subscriptions | FAQ | Tutorials | Contact Us