Portal | Manuals | References | Downloads | Info | Programs | JCLs | Master the Mainframes
IBM Mainframe Computers Forums Index
 
Register
 
IBM Mainframe Computers Forums Index Mainframe: Search IBM Mainframe Forum: FAQ Memberlist Usergroups Profile Log in to check your private messages Log in
 

 

Dynamic output file creation based on record count

 
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> DFSORT/ICETOOL
View previous topic :: :: View next topic  
Author Message
himanshupant

New User


Joined: 21 Mar 2007
Posts: 46
Location: India

PostPosted: Fri Jul 23, 2010 12:07 pm    Post subject: Dynamic output file creation based on record count
Reply with quote

Hello

I have an input csv file containing a huge number of records ( say 4,00,000).
Now as MS Excel 2003 can not have more than 65,536 rows in a work sheet. I need to break this huge csv file into smaller csv files based on record count.. Say first 50,000 records in file 1, next 50,000 in file 2 , so on and so forth.

Is there any way to do this dynamically in ICETOOL / DFSORT ? A point to be noted is that I am not aware of the number of input records so the number of output files is not known to me before hand.

Furthermore if my input file contains the header as

name,age,address,sex,card-no

is there any way I can keep the same as header for each of the splitted file ?
Back to top
View user's profile Send private message

smijoss

Active User


Joined: 30 Aug 2007
Posts: 114
Location: pune

PostPosted: Fri Jul 23, 2010 5:25 pm    Post subject:
Reply with quote

have you checked SMART DFSORT TRICKS.pdf it has 5 methods to split a file.

regarding the header :
you can have 2 sort steps :
1) to read 1st line and create a temp sort card with header and split options
2) use the sort card created above to split file
Back to top
View user's profile Send private message
sqlcode1

Active Member


Joined: 08 Apr 2010
Posts: 578
Location: USA

PostPosted: Fri Jul 23, 2010 8:23 pm    Post subject:
Reply with quote

smijoss,
OP doesn't know the count of records and hence total number of output files required. He is asking for a jcl which dynamically allocates output files which has fixed number of records (50000).

Quote:
A point to be noted is that I am not aware of the number of input records so the number of output files is not known to me before hand.


himanshupant,
What is the LRECL of the input file?

Thanks,
Back to top
View user's profile Send private message
himanshupant

New User


Joined: 21 Mar 2007
Posts: 46
Location: India

PostPosted: Fri Jul 23, 2010 9:32 pm    Post subject:
Reply with quote

sqlcode1 wrote:
smijoss,
OP doesn't know the count of records and hence total number of output files required. He is asking for a jcl which dynamically allocates output files which has fixed number of records (50000).

Quote:
A point to be noted is that I am not aware of the number of input records so the number of output files is not known to me before hand.


himanshupant,
What is the LRECL of the input file?

Thanks,


The LRECL can be taken as 100 bytes
Back to top
View user's profile Send private message
Frank Yaeger

DFSORT Moderator


Joined: 15 Feb 2005
Posts: 7130
Location: San Jose, CA

PostPosted: Fri Jul 23, 2010 10:15 pm    Post subject:
Reply with quote

See the "Split a file to n output files dynamically" Smart DFSORT Trick at:

http://www.ibm.com/support/docview.wss?rs=114&uid=isg3T7000094

Note that you must supply the maximum possible number of output DD statements in the JCL.

If you really want to set up the output DDs dynamically, you would have to create a job for the internal reader.

If you need more specific help, show an example of the records in your input file and what you expect for output. Give the RECFM and LRECL of the input file.
Back to top
View user's profile Send private message
himanshupant

New User


Joined: 21 Mar 2007
Posts: 46
Location: India

PostPosted: Sat Jul 24, 2010 6:11 am    Post subject:
Reply with quote

Frank Yaeger wrote:
See the "Split a file to n output files dynamically" Smart DFSORT Trick at:

http://www.ibm.com/support/docview.wss?rs=114&uid=isg3T7000094

Note that you must supply the maximum possible number of output DD statements in the JCL.

If you really want to set up the output DDs dynamically, you would have to create a job for the internal reader.

If you need more specific help, show an example of the records in your input file and what you expect for output. Give the RECFM and LRECL of the input file.


Thanks a ton Frank - The document really looks helpful !!!
Back to top
View user's profile Send private message
smijoss

Active User


Joined: 30 Aug 2007
Posts: 114
Location: pune

PostPosted: Sat Jul 24, 2010 3:57 pm    Post subject:
Reply with quote

hi sqlcode1,

never heard of JCL allocating dynamic files .. so didnt get into that ;)
Back to top
View user's profile Send private message
sqlcode1

Active Member


Joined: 08 Apr 2010
Posts: 578
Location: USA

PostPosted: Mon Jul 26, 2010 8:04 pm    Post subject:
Reply with quote

smijoss,
Quote:
He is asking for a jcl which dynamically allocates output files which has fixed number of records (50000).


Its not dynamic allocation through JCL but as rather setting up JCL DD statements (and other required JCL statements as well) dynamically through JOB and later submitting the job through Internal Reader.

Frank Yaeger wrote:

If you really want to set up the output DDs dynamically, you would have to create a job for the internal reader.


Thanks,
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> DFSORT/ICETOOL All times are GMT + 6 Hours
Page 1 of 1

 

Search our Forum:

Similar Topics
Topic Author Forum Replies Posted
No new posts Incorrect output when trying to add n... monica1 PL/I & Assembler 7 Fri Jan 13, 2017 5:02 pm
No new posts Copy RECFM =VB TO FB file with RECL =... sahil41352 DFSORT/ICETOOL 3 Wed Dec 28, 2016 11:29 pm
No new posts REXX DB2: Dynamic allocation of DB2.D... BHAS CLIST & REXX 3 Mon Dec 19, 2016 8:26 pm
No new posts Removing Duplicates based on certain ... chandracdac DFSORT/ICETOOL 8 Fri Dec 09, 2016 4:40 am
No new posts Add PD field from 2nd file to PD in 1st Sushant Garje DFSORT/ICETOOL 6 Thu Dec 01, 2016 4:32 pm


Facebook
Back to Top
 
Mainframe Wiki | Forum Rules | Bookmarks | Subscriptions | FAQ | Tutorials | Contact Us