Portal | Manuals | References | Downloads | Info | Programs | JCLs | Master the Mainframes
IBM Mainframe Computers Forums Index
 
Register
 
IBM Mainframe Computers Forums Index Mainframe: Search IBM Mainframe Forum: FAQ Memberlist Usergroups Profile Log in to check your private messages Log in
 

 

Mainframe flat file splitting if maximum size limit reached

 
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> JCL & VSAM
View previous topic :: :: View next topic  
Author Message
yogeshwar_ade

Active User


Joined: 31 Aug 2006
Posts: 103
Location: INDIA

PostPosted: Wed Jan 23, 2013 8:49 pm    Post subject: Mainframe flat file splitting if maximum size limit reached
Reply with quote

I have a file say 'MF.ABC'. If it reaches size of say 500 MB then we need to split this file into two files to avoid SIZE issue during FTP.

We are trying to FTP MF.ABC file but maximum allowable size is 500 MB, if it exceeds then FTP get failed. Due to which we need to split this file if above SIZE limit reaches.

Please let me know if any other details required.

Thanks,
Yogeshwar (Yogi)
Back to top
View user's profile Send private message

superk

Moderator Team Head


Joined: 26 Apr 2004
Posts: 4648
Location: Raleigh, NC, USA

PostPosted: Wed Jan 23, 2013 8:55 pm    Post subject: Reply to: Mainframe flat file splitting if maximum size limi
Reply with quote

I don't personally know of anyway to directly check if a file exceeds a particular number of bytes. I'd suggest checking for a maximum record count.
Back to top
View user's profile Send private message
Bill Woodger

DFSORT Moderator


Joined: 09 Mar 2011
Posts: 7223

PostPosted: Wed Jan 23, 2013 9:11 pm    Post subject: Reply to: Mainframe flat file splitting if maximum size limi
Reply with quote

Which SORT product do you use? What is the RECFM and LRECL? What processing do you already have of the file so that something new doesn't cost "extra" resources?
Back to top
View user's profile Send private message
yogeshwar_ade

Active User


Joined: 31 Aug 2006
Posts: 103
Location: INDIA

PostPosted: Thu Jan 24, 2013 6:30 pm    Post subject: Reply to: Mainframe flat file splitting if maximum size limi
Reply with quote

Bill,

We are using SYNCSORT product. RECFM is FB and LRECL is 1000 for this file.
Back to top
View user's profile Send private message
Bill Woodger

DFSORT Moderator


Joined: 09 Mar 2011
Posts: 7223

PostPosted: Thu Jan 24, 2013 6:52 pm    Post subject: Reply to: Mainframe flat file splitting if maximum size limi
Reply with quote

So, consult your documentation for SPLIT, SPLIT1R and SPLITBY (if they all exist for Syncsort).

500mb of 1000-byte records is a certain number. You have various options to arrange for multiple files which could be transferred separately.

I'd arrange for headers (logical name, logical number, business/data date) and trailers (count on file) at a minimum, so that the receiver knows that files are logically the same data. Stuff like that.
Back to top
View user's profile Send private message
yogeshwar_ade

Active User


Joined: 31 Aug 2006
Posts: 103
Location: INDIA

PostPosted: Thu Jan 24, 2013 6:58 pm    Post subject: Reply to: Mainframe flat file splitting if maximum size limi
Reply with quote

You mean to say get record count for 500 MB (since its 1000 byte LRECL) and once we got record count go ahead with SORT using SPLIT option to split, right?
Back to top
View user's profile Send private message
Bill Woodger

DFSORT Moderator


Joined: 09 Mar 2011
Posts: 7223

PostPosted: Thu Jan 24, 2013 7:59 pm    Post subject: Reply to: Mainframe flat file splitting if maximum size limi
Reply with quote

Yep. You have a limit in MB for the transfer (site specific). It is "easy" to calculate a number of records which would keep you within the limit. If there are "groups" of records which need to be kept together, that is not a problem as we've already answered a couple of those recently.

So, yes, but I'm also saying do it properly, with "controls" around it, so that if there is a mess-up on the receiving system, you are covered because you've provided controls over the data.
Back to top
View user's profile Send private message
yogeshwar_ade

Active User


Joined: 31 Aug 2006
Posts: 103
Location: INDIA

PostPosted: Thu Feb 07, 2013 6:34 pm    Post subject: Reply to: Mainframe flat file splitting if maximum size limi
Reply with quote

My apology for delay in reply.

This file is not having any Header and Trailer and also we didn't need to group any specific records.

I guess now I need to first find how many records in 500 MB for 1000 LRECL file of FB type. If any pointers over how to calculate record in 500 MB will be helpful.

Secondly, I need to split file based on number of records. Say if there are 1 Million record for 500 MB file size. So for very 1 million record there will be new file to be created. If we are getting total 9 million record in file, then this file need to be splitted into 9 difdifferent files having 1 million records each.
The records in this file may vary and some time come to say 9 million and some time 20 million and we need to split this file accordingly, is there anyway we can achieve this one whether thru JCL/SORT or COBOL?

Please let me know if I missed something and requirement is not clear.
Back to top
View user's profile Send private message
Robert Sample

Global Moderator


Joined: 06 Jun 2008
Posts: 7905
Location: Bellevue, IA

PostPosted: Thu Feb 07, 2013 6:44 pm    Post subject:
Reply with quote

If you do not know how to calculate the number of records, then you belong in Beginner's and Students Forum no matter how much "experience" you have.

For a disk file on a 3390 device, half-track blocking means 27 1,000-byte records will fit in each block. This means 54 records per track and 810 records per cylinder. If you can figure out how many 1,000-byte records will occur in 500 MB, you can finish the computations yourself.

If using tape, the numbers are different but the calculation is the same (although, obviously, tracks and cylinders are not part of the calculation).
Back to top
View user's profile Send private message
yogeshwar_ade

Active User


Joined: 31 Aug 2006
Posts: 103
Location: INDIA

PostPosted: Thu Feb 07, 2013 7:18 pm    Post subject: Reply to: Mainframe flat file splitting if maximum size limi
Reply with quote

My miss Rob, I missed the basic rule and posted the query in hurry.
Back to top
View user's profile Send private message
Bill Woodger

DFSORT Moderator


Joined: 09 Mar 2011
Posts: 7223

PostPosted: Thu Feb 07, 2013 7:28 pm    Post subject: Reply to: Mainframe flat file splitting if maximum size limi
Reply with quote

500mb = 500,000,000 (approx). LRECL 1000. Equals 500,000 records.

You specify multiple OUTFIL datasets (at least 40 is you might have 20 million records, but I'd allow for "growth" as well, if there can be any).

You'd need to know how many files were output, remove the rest, and process those with data.
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> JCL & VSAM All times are GMT + 6 Hours
Page 1 of 1

 

Search our Forum:

Similar Topics
Topic Author Forum Replies Posted
No new posts Add PD field from 2nd file to PD in 1st Sushant Garje DFSORT/ICETOOL 6 Thu Dec 01, 2016 4:32 pm
This topic is locked: you cannot edit posts or make replies. Urgent Mainframe Requirments (2-10 yr... Shankar779 Mainframe Jobs 0 Sun Nov 27, 2016 1:48 pm
No new posts File Aid to File Manager conversion murali3955 IBM Tools 4 Thu Nov 24, 2016 3:41 pm
No new posts Limit duplicate records in the SORT pshongal SYNCSORT 6 Mon Nov 21, 2016 12:54 pm
No new posts How to convert the VBM file to VB or... Sulabh Agrawal JCL & VSAM 4 Fri Nov 18, 2016 1:04 pm


Facebook
Back to Top
 
Mainframe Wiki | Forum Rules | Bookmarks | Subscriptions | FAQ | Tutorials | Contact Us