Portal | Manuals | References | Downloads | Info | Programs | JCLs | Master the Mainframes
IBM Mainframe Computers Forums Index
 
Register
 
IBM Mainframe Computers Forums Index Mainframe: Search IBM Mainframe Forum: FAQ Memberlist Usergroups Profile Log in to check your private messages Log in
 

 

Split Files and run Jobs

 
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> SYNCSORT
View previous topic :: :: View next topic  
Author Message
sun_job

New User


Joined: 18 Sep 2007
Posts: 73
Location: Bangalore

PostPosted: Wed Mar 18, 2015 9:51 pm    Post subject: Split Files and run Jobs
Reply with quote

Hi

Please find my problem below:

We have a file containing 21 million records and its taking 30 minutes for doing a business logic check.

The thought is to split files 5M each dynamically and run parallel jobs, but there could be a case where input file will have more than 21 million records.

Appreciate all veterans suggestions to proceed ahead for this issue?
Back to top
View user's profile Send private message

Bill Woodger

DFSORT Moderator


Joined: 09 Mar 2011
Posts: 7234

PostPosted: Wed Mar 18, 2015 10:09 pm    Post subject: Reply to: Split Files and run Jobs
Reply with quote

If you look a SPLIT and relations on OUTFIL, you should find something that will work as you seem to want.

You get 21m-ish new records per day which need to be verified?
Back to top
View user's profile Send private message
RahulG31

Active User


Joined: 20 Dec 2014
Posts: 331
Location: USA

PostPosted: Wed Mar 18, 2015 11:40 pm    Post subject: Reply to: Split Files and run Jobs
Reply with quote

Just a small suggestion :

Rather than splitting the files in 5M each, you should decide how many split files you want (e.g 10) and then split the input file into that number.
This way you'll have a fix number of split JCLs and this will take minimum time to process the input file.
Back to top
View user's profile Send private message
sun_job

New User


Joined: 18 Sep 2007
Posts: 73
Location: Bangalore

PostPosted: Thu Mar 19, 2015 2:37 am    Post subject: Re: Reply to: Split Files and run Jobs
Reply with quote

Bill Woodger wrote:
If you look a SPLIT and relations on OUTFIL, you should find something that will work as you seem to want.

You get 21m-ish new records per day which need to be verified?


21M is just a record for a particular day. Each day the number of records will changes
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> SYNCSORT All times are GMT + 6 Hours
Page 1 of 1

 

Search our Forum:

Similar Topics
Topic Author Forum Replies Posted
No new posts High CPU consumption Job using IAM fi... aswinir JCL & VSAM 8 Thu Dec 01, 2016 8:28 pm
No new posts Match or compare two files in VB Format anatol DFSORT/ICETOOL 14 Thu Nov 03, 2016 7:41 pm
No new posts How to split the records using the am... vnktrrd DFSORT/ICETOOL 24 Fri Oct 28, 2016 7:33 pm
This topic is locked: you cannot edit posts or make replies. How to use 2 input files in control c... Gunapala CN DFSORT/ICETOOL 23 Thu Oct 13, 2016 3:42 pm
No new posts how to split records based on specifi... Venkata Ramayya DFSORT/ICETOOL 6 Wed Sep 28, 2016 3:20 am


Facebook
Back to Top
 
Mainframe Wiki | Forum Rules | Bookmarks | Subscriptions | FAQ | Tutorials | Contact Us