IBM Mainframe Forum Index
 
Log In
 
IBM Mainframe Forum Index Mainframe: Search IBM Mainframe Forum: FAQ Register
 

SPlitting a file into number of files


IBM Mainframe Forums -> JCL & VSAM
Post new topic   This topic is locked: you cannot edit posts or make replies.
View previous topic :: View next topic  
Author Message
meen

New User


Joined: 23 Jun 2008
Posts: 15
Location: bangalore

PostPosted: Wed Oct 08, 2008 5:17 pm
Reply with quote

Hi !!

I am executing IMS program through a JCL. The output file of the program is having millions of records. I want to split that file into number of files. But the problem is that the number of records in output file are not fixed , means i dont know the number of smaller files in which the bigger file need to be divided. SO can you tell me how to split the file.
Back to top
View user's profile Send private message
Escapa

Senior Member


Joined: 16 Feb 2007
Posts: 1399
Location: IL, USA

PostPosted: Wed Oct 08, 2008 5:37 pm
Reply with quote

Quote:
But the problem is that the number of records in output file are not fixed , means i dont know the number of smaller files in which the bigger file need to be divided. SO can you tell me how to split the file.

You saying dont know number of records. This should not be issue because we can find out number of records in the file..
What will be count of o/p files if you know count lets say 10,000?

Do you mean to say you want every X number of records TO be written into one file and use new?
Back to top
View user's profile Send private message
meen

New User


Joined: 23 Jun 2008
Posts: 15
Location: bangalore

PostPosted: Wed Oct 08, 2008 5:46 pm
Reply with quote

Ya exactly...Lets say i am getting 1 million records in the file. .I want to write 10,000 records in one file and next 10,000 in other file and so on..
Each file will contain 10,000 records, but the number of records would be different each time the program runs.
Back to top
View user's profile Send private message
Arun Raj

Moderator


Joined: 17 Oct 2006
Posts: 2481
Location: @my desk

PostPosted: Wed Oct 08, 2008 6:35 pm
Reply with quote

meen,

While spltting, do you want contiguous records from the input file getting written into the same output file? Or the order does not matter?
Back to top
View user's profile Send private message
meen

New User


Joined: 23 Jun 2008
Posts: 15
Location: bangalore

PostPosted: Wed Oct 08, 2008 6:46 pm
Reply with quote

My requirement is first 10,000 in one file next 10,000 in next file and so on. That means first file contains first 10000 contiguous records.
Back to top
View user's profile Send private message
expat

Global Moderator


Joined: 14 Mar 2007
Posts: 8797
Location: Welsh Wales

PostPosted: Wed Oct 08, 2008 6:49 pm
Reply with quote

Could you enlighten me into the 'MUST DO' of splitting the file ?

I would like to understand the reasoning behind this rather odd requirement.

Are we talking about the file being created by the BMP and then split via SORT in a following jobstep, or should the creating BMP perform the split ?
Back to top
View user's profile Send private message
meen

New User


Joined: 23 Jun 2008
Posts: 15
Location: bangalore

PostPosted: Wed Oct 08, 2008 6:59 pm
Reply with quote

I am talkin about file being created by the BMP and then splitting via SORT in a following jobstep.

If creating BMP perform the split is also possible tell me that also..

Which one is feasible to code?
Back to top
View user's profile Send private message
expat

Global Moderator


Joined: 14 Mar 2007
Posts: 8797
Location: Welsh Wales

PostPosted: Wed Oct 08, 2008 7:07 pm
Reply with quote

But what is the reason for wanting the file to be split up ?

Also, it maight be a wise move to specify which sort product you use as the correct solution may differ from product to product.
Back to top
View user's profile Send private message
meen

New User


Joined: 23 Jun 2008
Posts: 15
Location: bangalore

PostPosted: Wed Oct 08, 2008 7:17 pm
Reply with quote

We have to FTP the file to the server of a system which is working on some other technology...and if we FTP the file containing such a huge number of records...it would not be possible for them to process that...

So we have to split file and pass it to other system..
Back to top
View user's profile Send private message
expat

Global Moderator


Joined: 14 Mar 2007
Posts: 8797
Location: Welsh Wales

PostPosted: Wed Oct 08, 2008 7:34 pm
Reply with quote

Which sort product are you using ? DFSORT, SYNCSORT, CA-SORT ?
Back to top
View user's profile Send private message
Frank Yaeger

DFSORT Developer


Joined: 15 Feb 2005
Posts: 7129
Location: San Jose, CA

PostPosted: Wed Oct 08, 2008 9:36 pm
Reply with quote

meen,

Perhaps one of these Smart DFSORT Tricks will help:

* Split a file to n output files dynamically

* Five ways to split a data set

You can find these tricks at:

www.ibm.com/support/docview.wss?rs=114&uid=isg3T7000094
Back to top
View user's profile Send private message
Anuj Dhawan

Superior Member


Joined: 22 Apr 2006
Posts: 6250
Location: Mumbai, India

PostPosted: Thu Oct 09, 2008 2:27 am
Reply with quote

Hi Expat,
expat wrote:
But what is the reason for wanting the file to be split up ?
I've a business reason for a similar scenario here at my shop:

There is a COBOL Batch Program which generates the print reports for the mechants, if the report is less than or equal to eight pages report should be sent in "trifold form" else it should be sent as a "flat insertion". This helps to the end worker who collects these reports - there are two different printer set-up for this - reports get printed, inserted in envelopes & are collected in different buckets; I believe, in some manner, this helps USPS personnels.
Back to top
View user's profile Send private message
Arun Raj

Moderator


Joined: 17 Oct 2006
Posts: 2481
Location: @my desk

PostPosted: Fri Oct 10, 2008 8:44 am
Reply with quote

meen,

You can use the below SYNCTOOL job to create a split job dynamically to achieve what you want.
Code:
//STEP1    EXEC PGM=SYNCTOOL                                         
//TOOLMSG  DD SYSOUT=*                                               
//DFSMSG   DD SYSOUT=*                                               
//IN       DD DSN= Your.input.file                                   
//T1       DD DSN=&&T1,DISP=(,PASS)                                   
//OUT      DD SYSOUT=*                                               
//TOOLIN   DD *                                                       
 COPY FROM(IN) TO(T1)  USING(CTL1)                                   
 SORT FROM(T1) TO(OUT) USING(CTL2)                                   
//CTL1CNTL DD *                                                       
 INREC OVERLAY=(81:SEQNUM,8,ZD,89:81,8,ZD,MOD,+5,M11,LENGTH=8)       
 OUTFIL INCLUDE=(89,8,ZD,EQ,1),                                       
    BUILD=(C' OUTFIL FILES=',SEQNUM,2,ZD,C',STARTREC=',81,8,C',',     
           C'ENDREC=',81,8,ZD,ADD,+4,M11,LENGTH=8,/,                 
           C'//SORTOF',SEQNUM,2,ZD,C' DD DSN=xxxx.SPLTFILE.F',       
           SEQNUM,2,ZD,C',UNIT=SYSDA,',/,                             
           C'//',12X,C'SPACE=(CYL,(100,100),RLSE),DISP=(,CATLG)',80:X)
//CTL2CNTL DD *                                                       
  INREC IFTHEN=(WHEN=(1,1,CH,EQ,C' '),OVERLAY=(81:C'0')),             
        IFTHEN=(WHEN=NONE,OVERLAY=(81:C'9'))                         
  SORT FIELDS=(81,1,CH,A)                                           
  OUTFIL REMOVECC,BUILD=(1,80),                                     
  HEADER1=(C'//xxxxxxxx JOB (YYYY),''SPLIT'',CLASS=A,MSGCLASS=X,',/,
           C'//        REGION=2048K,NOTIFY=&SYSUID',/,             
           C'//STEP1   EXEC PGM=SORT',/,                           
           C'//SYSOUT  DD SYSOUT=*',/,                             
           C'//SORTIN  DD DSN=Your.input.file,DISP=SHR',/,         
           C'//SYSIN   DD *',/,                                     
           C' OPTION COPY',80:X)   

The above job is for writing 5 records each into each of the split files. If you want to have N records in the each file you should modify the below control statements.
Quote:
//CTL1CNTL DD *
INREC OVERLAY=(81:SEQNUM,8,ZD,89:81,8,ZD,MOD,+N,M11,LENGTH=8)
OUTFIL INCLUDE=(89,8,ZD,EQ,1),
BUILD=(C' OUTFIL FILES=',SEQNUM,2,ZD,C',STARTREC=',81,8,C',',
C'ENDREC=',81,8,ZD,ADD,+N-1,M11,LENGTH=8,/,


After verifying the generated JCL in OUT you may change
Code:
//OUT      DD SYSOUT=*
as
Code:
//OUT      DD SYSOUT=(*,INTRDR)

I have given the final output filenames as
Code:
xxxx.SPLTFILE.F01
xxxx.SPLTFILE.F02
..............
..............
You can modify it as per your needs.
Back to top
View user's profile Send private message
Amsar

New User


Joined: 26 Sep 2008
Posts: 84
Location: Chennai

PostPosted: Sat Aug 15, 2009 12:12 am
Reply with quote

Hi All,

Solution given by arun is perfectly matching to my requirement, but we doesnot have SYNCTOOL installed. i am using DFSORT V1R5
How can i achive this?

Thanks,
Amsar
Back to top
View user's profile Send private message
dick scherrer

Moderator Emeritus


Joined: 23 Nov 2006
Posts: 19244
Location: Inside the Matrix

PostPosted: Sat Aug 15, 2009 12:14 am
Reply with quote

Hello,

Did you try the posted solution using ICETOOL rather than sort. . .
Back to top
View user's profile Send private message
Frank Yaeger

DFSORT Developer


Joined: 15 Feb 2005
Posts: 7129
Location: San Jose, CA

PostPosted: Sat Aug 15, 2009 12:44 am
Reply with quote

Amsar,

If you have DFSORT, then you have DFSORT's ICETOOL. Just change PGM=SYNCTOOL to PGM=ICETOOL to run it.

Also, in my previous post, I referred to Smart DFSORT Tricks that might help:

Quote:
Perhaps one of these Smart DFSORT Tricks will help:

* Split a file to n output files dynamically

* Five ways to split a data set

You can find these tricks at:

www.ibm.com/support/docview.wss?rs=114&uid=isg3T7000094
Back to top
View user's profile Send private message
Amsar

New User


Joined: 26 Sep 2008
Posts: 84
Location: Chennai

PostPosted: Sat Aug 15, 2009 1:14 am
Reply with quote

Hi Dick,

Sorry now i have tried with ICETOOL and job is working fine.

Thanks to Arun & Thanks for your suggetion.


Amsar.
Back to top
View user's profile Send private message
Amsar

New User


Joined: 26 Sep 2008
Posts: 84
Location: Chennai

PostPosted: Sat Aug 15, 2009 2:40 am
Reply with quote

Hi Dick / Frank,

Is it possible to get splitted records in GDG version, i have requirement to concatnate all these files in next step and i will not know the number of files.

Thanks,
Amsar
Back to top
View user's profile Send private message
dick scherrer

Moderator Emeritus


Joined: 23 Nov 2006
Posts: 19244
Location: Inside the Matrix

PostPosted: Sun Aug 16, 2009 7:08 am
Reply with quote

Hello,

I believe there is something that i misunderstand. . . Please clarify why there is a need to concatenate as input the files just split?

Also, suggest you read about Generation Data Sets in the JCL manual available vis the "IBM Manuals" link at the top of the page. I suspect that you will not use a "version", but rather a "generation". Version has a special use and i do not believe this is what you ask about. And no, you probably will not use a gdg for this requirement.

It may help if you clearly post your requirement showing some sample input, the output you want from that input and the "rules" for getting from that sample input to the output. Also mention the recfm and lrecl of the files and the relevant positions in the data to accomplish your requirement.
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   This topic is locked: you cannot edit posts or make replies. View Bookmarks
All times are GMT + 6 Hours
Forum Index -> JCL & VSAM

 


Similar Topics
Topic Forum Replies
No new posts FTP VB File from Mainframe retaining ... JCL & VSAM 1
No new posts Extract the file name from another fi... DFSORT/ICETOOL 6
No new posts How to split large record length file... DFSORT/ICETOOL 10
No new posts Extracting Variable decimal numbers f... DFSORT/ICETOOL 17
No new posts SFTP Issue - destination file record ... All Other Mainframe Topics 2
Search our Forums:

Back to Top