Portal | Manuals | References | Downloads | Info | Programs | JCLs | Mainframe wiki | Quick Ref
IBM Mainframe Computers Forums Index
 
Register
 
IBM Mainframe Computers Forums Index Mainframe: Search IBM Mainframe Forum: FAQ Memberlist Profile Log in to check your private messages Log in
 
Split Files and run Jobs

 
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> SYNCSORT
View previous topic :: :: View next topic  
Author Message
sun_job

New User


Joined: 18 Sep 2007
Posts: 73
Location: Bangalore

PostPosted: Wed Mar 18, 2015 9:51 pm    Post subject: Split Files and run Jobs
Reply with quote

Hi

Please find my problem below:

We have a file containing 21 million records and its taking 30 minutes for doing a business logic check.

The thought is to split files 5M each dynamically and run parallel jobs, but there could be a case where input file will have more than 21 million records.

Appreciate all veterans suggestions to proceed ahead for this issue?
Back to top
View user's profile Send private message

Bill Woodger

DFSORT Moderator


Joined: 09 Mar 2011
Posts: 7314

PostPosted: Wed Mar 18, 2015 10:09 pm    Post subject: Reply to: Split Files and run Jobs
Reply with quote

If you look a SPLIT and relations on OUTFIL, you should find something that will work as you seem to want.

You get 21m-ish new records per day which need to be verified?
Back to top
View user's profile Send private message
RahulG31

Active User


Joined: 20 Dec 2014
Posts: 408
Location: USA

PostPosted: Wed Mar 18, 2015 11:40 pm    Post subject: Reply to: Split Files and run Jobs
Reply with quote

Just a small suggestion :

Rather than splitting the files in 5M each, you should decide how many split files you want (e.g 10) and then split the input file into that number.
This way you'll have a fix number of split JCLs and this will take minimum time to process the input file.
Back to top
View user's profile Send private message
sun_job

New User


Joined: 18 Sep 2007
Posts: 73
Location: Bangalore

PostPosted: Thu Mar 19, 2015 2:37 am    Post subject: Re: Reply to: Split Files and run Jobs
Reply with quote

Bill Woodger wrote:
If you look a SPLIT and relations on OUTFIL, you should find something that will work as you seem to want.

You get 21m-ish new records per day which need to be verified?


21M is just a record for a particular day. Each day the number of records will changes
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> SYNCSORT All times are GMT + 6 Hours
Page 1 of 1

 

Search our Forum:

Similar Topics
Topic Author Forum Replies Posted
No new posts Compare VSAM files using DFSORT pshongal DFSORT/ICETOOL 5 Thu Nov 23, 2017 10:24 am
No new posts Split files upto certain limit based ... bubbu75 DFSORT/ICETOOL 5 Tue Nov 14, 2017 11:11 pm
No new posts Copy selected columns from 2 input fi... Poha Eater DFSORT/ICETOOL 3 Thu Nov 02, 2017 3:43 pm
No new posts Finding difference between two PS fil... Vignesh Sid SYNCSORT 5 Mon Oct 30, 2017 1:13 pm
No new posts NDM 300 files from mainframe to linux shreya19 All Other Mainframe Topics 2 Thu Oct 26, 2017 11:40 am

Facebook
Back to Top
 
Job Vacancies | Forum Rules | Bookmarks | Subscriptions | FAQ | Polls | Contact Us