Portal | Manuals | References | Downloads | Info | Programs | JCLs | Mainframe wiki | Quick Ref
IBM Mainframe Forum Index
 
Register
 
IBM Mainframe Forum Index Mainframe: Search IBM Mainframe Forum: FAQ Memberlist Profile Log in to check your private messages Log in
 
Split Files and run Jobs

 
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> SYNCSORT
View previous topic :: :: View next topic  
Author Message
sun_job

New User


Joined: 18 Sep 2007
Posts: 73
Location: Bangalore

PostPosted: Wed Mar 18, 2015 9:51 pm    Post subject: Split Files and run Jobs
Reply with quote

Hi

Please find my problem below:

We have a file containing 21 million records and its taking 30 minutes for doing a business logic check.

The thought is to split files 5M each dynamically and run parallel jobs, but there could be a case where input file will have more than 21 million records.

Appreciate all veterans suggestions to proceed ahead for this issue?
Back to top
View user's profile Send private message

Bill Woodger

DFSORT Moderator


Joined: 09 Mar 2011
Posts: 7312

PostPosted: Wed Mar 18, 2015 10:09 pm    Post subject: Reply to: Split Files and run Jobs
Reply with quote

If you look a SPLIT and relations on OUTFIL, you should find something that will work as you seem to want.

You get 21m-ish new records per day which need to be verified?
Back to top
View user's profile Send private message
RahulG31

Active User


Joined: 20 Dec 2014
Posts: 447
Location: USA

PostPosted: Wed Mar 18, 2015 11:40 pm    Post subject: Reply to: Split Files and run Jobs
Reply with quote

Just a small suggestion :

Rather than splitting the files in 5M each, you should decide how many split files you want (e.g 10) and then split the input file into that number.
This way you'll have a fix number of split JCLs and this will take minimum time to process the input file.
Back to top
View user's profile Send private message
sun_job

New User


Joined: 18 Sep 2007
Posts: 73
Location: Bangalore

PostPosted: Thu Mar 19, 2015 2:37 am    Post subject: Re: Reply to: Split Files and run Jobs
Reply with quote

Bill Woodger wrote:
If you look a SPLIT and relations on OUTFIL, you should find something that will work as you seem to want.

You get 21m-ish new records per day which need to be verified?


21M is just a record for a particular day. Each day the number of records will changes
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> SYNCSORT All times are GMT + 6 Hours
Page 1 of 1

 

Search our Forum:

Similar Topics
Topic Author Forum Replies Posted
No new posts Dynamically split large DSN list into... Ghellar CLIST & REXX 14 Tue Sep 17, 2019 8:24 pm
No new posts Pack to Unpack conversion for sending... A_programmers COBOL Programming 13 Wed Sep 11, 2019 10:46 pm
This topic is locked: you cannot edit posts or make replies. Compare two files logic in COBOL Nishant Singhal COBOL Programming 4 Sun Aug 25, 2019 2:25 am
No new posts Concatenate two files of variable rec... Thiru S DFSORT/ICETOOL 3 Wed Aug 21, 2019 11:53 pm
No new posts Copy physical Tape files to a Mainfra... Dadi Krishna Teja JCL & VSAM 11 Thu May 16, 2019 10:25 pm

Facebook
Back to Top
 
Job Vacancies | Forum Rules | Bookmarks | Subscriptions | FAQ | Polls | Contact Us