Portal | Manuals | References | Downloads | Info | Programs | JCLs | Master the Mainframes
IBM Mainframe Computers Forums Index
 
Register
 
IBM Mainframe Computers Forums Index Mainframe: Search IBM Mainframe Forum: FAQ Memberlist Usergroups Profile Log in to check your private messages Log in
 

 

Split a dataset into number of datasets

 
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> JCL & VSAM
View previous topic :: :: View next topic  
Author Message
shobhit garg

New User


Joined: 09 Jan 2009
Posts: 8
Location: Pune

PostPosted: Thu Mar 05, 2009 5:00 pm    Post subject: Split a dataset into number of datasets
Reply with quote

Hi,

I have a file which has near about 2500000 (twenty five hundred thousand) lines.

Since it is a big dataset, it is taking hell lot of time when it runs so idea here is if we can devide this dataset into number of datasets where each dataset contains only 25000(twenty five thousand) of lines.

I am trying it but not able to succeed. Can anybody help me out ?

Let me know if yoy need more information regarding that
Back to top
View user's profile Send private message

expat

Global Moderator


Joined: 14 Mar 2007
Posts: 8593
Location: Back in jolly old England

PostPosted: Thu Mar 05, 2009 5:18 pm    Post subject:
Reply with quote

First - please search the forum as this question has been asked, and answered, quite a few times already.

Quote:
Since it is a big dataset, it is taking hell lot of time when it runs so idea here is if we can devide this dataset into number of datasets where each dataset contains only 25000(twenty five thousand) of lines.

Do you mean that it takes a hell of a long time to process ? If so, what do you expect to gain by splitting the file into multiples as you will still need to process all of the records at some stage and the only gain that I can see is if you run multiple processes in parallel. And that in itself can spawn a number of other problems to deal with.

Please give some more information about how the file is processed, and of course the bog standard questions - the LRECL and RECFM of the file.
Back to top
View user's profile Send private message
shobhit garg

New User


Joined: 09 Jan 2009
Posts: 8
Location: Pune

PostPosted: Thu Mar 05, 2009 5:29 pm    Post subject: Reply to: Split a dataset into number of datasets
Reply with quote

Thanks for the answer.

Actualy even I am not sure how these files will be processed. My job is to just create such JCL which can split the file into number of files where each file should contain only 25000 lines.
Back to top
View user's profile Send private message
ksk

Active User


Joined: 08 Jun 2006
Posts: 356
Location: New York

PostPosted: Thu Mar 05, 2009 5:47 pm    Post subject:
Reply with quote

If you have DFSORT in your shop, you can split the input file into multiples files. Just search in forum, you will get examples using DFSORT.

OR you canuse the SPLITBY function using DFSORT. Below is the card.

Code:

//SYSIN DD *
SORT FIELDS=(21,5,FS,A)
OUTFIL FNAMES=(OUT1,OUT2,OUT3),SPLITBY=10



This card splits your input file into 3 files and each file contains 10 records if your input file contains 30 records. You have to mention 3 files with DD names mantioned in the OUTFIL FNAMES.

But in your case, you have to mention 100 file names. I am not sure how effective this is in your case.
Back to top
View user's profile Send private message
Arun Raj

Moderator


Joined: 17 Oct 2006
Posts: 2275
Location: @my desk

PostPosted: Thu Mar 05, 2009 6:09 pm    Post subject:
Reply with quote

This previous topic might be of some interest to you.

SPlitting a file into number of files
Back to top
View user's profile Send private message
shobhit garg

New User


Joined: 09 Jan 2009
Posts: 8
Location: Pune

PostPosted: Fri Mar 06, 2009 1:18 pm    Post subject: Reply to: Split a dataset into number of datasets
Reply with quote

Thanks all of you,

SORT with SPLITBY is really useful and it did work for me. Once again Mnay thanks all of you icon_lol.gif
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> JCL & VSAM All times are GMT + 6 Hours
Page 1 of 1

 

Search our Forum:

Similar Topics
Topic Author Forum Replies Posted
No new posts Fileaid 2 commands instream work. In ... descann Compuware & Other Tools 2 Tue May 16, 2017 3:31 pm
No new posts SMTP the current generation number of... Jyothi Kulunde JCL & VSAM 4 Thu May 04, 2017 4:08 pm
No new posts To find out size allocated to a seque... ashek15 JCL & VSAM 15 Thu Apr 27, 2017 9:42 am
This topic is locked: you cannot edit posts or make replies. SDSF multiple spool datasets extracte... PJAlarcon CLIST & REXX 1 Fri Apr 21, 2017 10:50 pm
No new posts Dynamic split of files under groups sril.krishy DFSORT/ICETOOL 4 Mon Apr 17, 2017 1:09 pm


Facebook
Back to Top
 
Mainframe Wiki | Forum Rules | Bookmarks | Subscriptions | FAQ | Tutorials | Contact Us