IBM Mainframe Forum Index
 
Log In
 
IBM Mainframe Forum Index Mainframe: Search IBM Mainframe Forum: FAQ Register
 

Dynamic output file creation based on record count


IBM Mainframe Forums -> DFSORT/ICETOOL
Post new topic   Reply to topic
View previous topic :: View next topic  
Author Message
himanshupant

New User


Joined: 21 Mar 2007
Posts: 46
Location: India

PostPosted: Fri Jul 23, 2010 12:07 pm
Reply with quote

Hello

I have an input csv file containing a huge number of records ( say 4,00,000).
Now as MS Excel 2003 can not have more than 65,536 rows in a work sheet. I need to break this huge csv file into smaller csv files based on record count.. Say first 50,000 records in file 1, next 50,000 in file 2 , so on and so forth.

Is there any way to do this dynamically in ICETOOL / DFSORT ? A point to be noted is that I am not aware of the number of input records so the number of output files is not known to me before hand.

Furthermore if my input file contains the header as

name,age,address,sex,card-no

is there any way I can keep the same as header for each of the splitted file ?
Back to top
View user's profile Send private message
smijoss

Active User


Joined: 30 Aug 2007
Posts: 114
Location: pune

PostPosted: Fri Jul 23, 2010 5:25 pm
Reply with quote

have you checked SMART DFSORT TRICKS.pdf it has 5 methods to split a file.

regarding the header :
you can have 2 sort steps :
1) to read 1st line and create a temp sort card with header and split options
2) use the sort card created above to split file
Back to top
View user's profile Send private message
sqlcode1

Active Member


Joined: 08 Apr 2010
Posts: 577
Location: USA

PostPosted: Fri Jul 23, 2010 8:23 pm
Reply with quote

smijoss,
OP doesn't know the count of records and hence total number of output files required. He is asking for a jcl which dynamically allocates output files which has fixed number of records (50000).

Quote:
A point to be noted is that I am not aware of the number of input records so the number of output files is not known to me before hand.


himanshupant,
What is the LRECL of the input file?

Thanks,
Back to top
View user's profile Send private message
himanshupant

New User


Joined: 21 Mar 2007
Posts: 46
Location: India

PostPosted: Fri Jul 23, 2010 9:32 pm
Reply with quote

sqlcode1 wrote:
smijoss,
OP doesn't know the count of records and hence total number of output files required. He is asking for a jcl which dynamically allocates output files which has fixed number of records (50000).

Quote:
A point to be noted is that I am not aware of the number of input records so the number of output files is not known to me before hand.


himanshupant,
What is the LRECL of the input file?

Thanks,


The LRECL can be taken as 100 bytes
Back to top
View user's profile Send private message
Frank Yaeger

DFSORT Developer


Joined: 15 Feb 2005
Posts: 7129
Location: San Jose, CA

PostPosted: Fri Jul 23, 2010 10:15 pm
Reply with quote

See the "Split a file to n output files dynamically" Smart DFSORT Trick at:

www.ibm.com/support/docview.wss?rs=114&uid=isg3T7000094

Note that you must supply the maximum possible number of output DD statements in the JCL.

If you really want to set up the output DDs dynamically, you would have to create a job for the internal reader.

If you need more specific help, show an example of the records in your input file and what you expect for output. Give the RECFM and LRECL of the input file.
Back to top
View user's profile Send private message
himanshupant

New User


Joined: 21 Mar 2007
Posts: 46
Location: India

PostPosted: Sat Jul 24, 2010 6:11 am
Reply with quote

Frank Yaeger wrote:
See the "Split a file to n output files dynamically" Smart DFSORT Trick at:

www.ibm.com/support/docview.wss?rs=114&uid=isg3T7000094

Note that you must supply the maximum possible number of output DD statements in the JCL.

If you really want to set up the output DDs dynamically, you would have to create a job for the internal reader.

If you need more specific help, show an example of the records in your input file and what you expect for output. Give the RECFM and LRECL of the input file.


Thanks a ton Frank - The document really looks helpful !!!
Back to top
View user's profile Send private message
smijoss

Active User


Joined: 30 Aug 2007
Posts: 114
Location: pune

PostPosted: Sat Jul 24, 2010 3:57 pm
Reply with quote

hi sqlcode1,

never heard of JCL allocating dynamic files .. so didnt get into that ;)
Back to top
View user's profile Send private message
sqlcode1

Active Member


Joined: 08 Apr 2010
Posts: 577
Location: USA

PostPosted: Mon Jul 26, 2010 8:04 pm
Reply with quote

smijoss,
Quote:
He is asking for a jcl which dynamically allocates output files which has fixed number of records (50000).


Its not dynamic allocation through JCL but as rather setting up JCL DD statements (and other required JCL statements as well) dynamically through JOB and later submitting the job through Internal Reader.

Frank Yaeger wrote:

If you really want to set up the output DDs dynamically, you would have to create a job for the internal reader.


Thanks,
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic View Bookmarks
All times are GMT + 6 Hours
Forum Index -> DFSORT/ICETOOL

 


Similar Topics
Topic Forum Replies
No new posts How to split large record length file... DFSORT/ICETOOL 8
No new posts Extracting Variable decimal numbers f... DFSORT/ICETOOL 17
No new posts To get the count of rows for every 1 ... DB2 3
No new posts SFTP Issue - destination file record ... All Other Mainframe Topics 2
No new posts Access to non cataloged VSAM file JCL & VSAM 18
Search our Forums:

Back to Top