Portal | Manuals | References | Downloads | Info | Programs | JCLs | Master the Mainframes
IBM Mainframe Computers Forums Index
 
Register
 
IBM Mainframe Computers Forums Index Mainframe: Search IBM Mainframe Forum: FAQ Memberlist Usergroups Profile Log in to check your private messages Log in
 

 

Splitting an Output file into n number of files having 2000

 
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> DFSORT/ICETOOL
View previous topic :: :: View next topic  
Author Message
Vigneshwaran S
Warnings : 1

New User


Joined: 10 Dec 2008
Posts: 60
Location: Chennai

PostPosted: Fri Jun 19, 2009 3:39 pm    Post subject: Splitting an Output file into n number of files having 2000
Reply with quote

Hi all,

I'm having an output file which normally comes between 20,000 to 30,000 of lines everyday.

My requirement is,

I want to split this file into 'n' number files having 2000 records each.

For Example:

Suppose if have output file which is having 22000 lines i need to seperate it into 'n' different files containg 2000 record each.

I want to make this 'n' dynamic So that there is no need to change in the JCL everytime.
Back to top
View user's profile Send private message

expat

Global Moderator


Joined: 14 Mar 2007
Posts: 8593
Location: Back in jolly old England

PostPosted: Fri Jun 19, 2009 3:42 pm    Post subject:
Reply with quote

Can I ask the need for this requiremnt, why the file must be split into multiple files ?
Back to top
View user's profile Send private message
Vigneshwaran S
Warnings : 1

New User


Joined: 10 Dec 2008
Posts: 60
Location: Chennai

PostPosted: Fri Jun 19, 2009 3:47 pm    Post subject: Reply to: Splitting an Output file into n number of files ha
Reply with quote

Hi,

To FTP those reports and share it across in Notepad.

Because when we try FTPing 20000 lines we are ending up getting only 5000 lines. As we are unable to FTP such a big file.
Back to top
View user's profile Send private message
expat

Global Moderator


Joined: 14 Mar 2007
Posts: 8593
Location: Back in jolly old England

PostPosted: Fri Jun 19, 2009 3:54 pm    Post subject:
Reply with quote

Would it not perhaps be easier to investigate the FTP or server settings and see if they could be amended to be able to send the whole file, rather than faffing about splitting a file and ftp'ing umpteen little files and then maybe joining them all up again at the destination.
Back to top
View user's profile Send private message
Itanium

Active User


Joined: 22 Jan 2006
Posts: 114
Location: India

PostPosted: Fri Jun 19, 2009 4:46 pm    Post subject:
Reply with quote

This can be done be dynamically allocating the output files in COBOL Program.

Module is bpxwdyn, try googling.
Back to top
View user's profile Send private message
dick scherrer

Site Director


Joined: 23 Nov 2006
Posts: 19270
Location: Inside the Matrix

PostPosted: Fri Jun 19, 2009 8:53 pm    Post subject:
Reply with quote

Hello,

As Expat suggests, you want to correct the file transfer problem.

If you implement some cludge that creates some varying number of files every run and this process needs to keep track of how many were created and transfer them and then the target must "put it all back together again", you will introduce far more problems than solutions. . .

22000 is not such a large file. Several of ours are many gig of data and there is no data loss.
Back to top
View user's profile Send private message
Vigneshwaran S
Warnings : 1

New User


Joined: 10 Dec 2008
Posts: 60
Location: Chennai

PostPosted: Sat Jun 20, 2009 6:07 pm    Post subject: Reply to: Splitting an Output file into n number of files ha
Reply with quote

Hi all,

Sorry Dick and Expac..I need to proceed with teh same work of spliting an output file into different files. My boss wants me to this!! icon_evil.gif

This is the JCL i'm using for my requirement,
Code:


//***********************************************************
//SPLITFLS EXEC PGM=SORT
//***********************************************************
//SYSPRINT DD SYSOUT=*
//SYSOUT   DD SYSOUT=*
//SYSUDUMP DD SYSOUT=*
//SORTIN   DD DSN=FILE1......,DISP=SHR
//SORTOF01 DD DSN=OUTPUTFILE1.......,
//            DISP=(NEW,CATLG,DELETE),UNIT=SYSDA,
//            SPACE=(CYL,(1,1),RLSE),
//            RECFM=FB,LRECL=80
//SORTOF02 DD DSN=OUTPUTFILE2..............,
//            DISP=(NEW,CATLG,DELETE),UNIT=SYSDA,
//            SPACE=(CYL,(1,1),RLSE),
//            RECFM=FB,LRECL=80
//SORTOF03 DD DSN=OUTPUTFILE3..............,
//            DISP=(NEW,CATLG,DELETE),UNIT=SYSDA,
//            SPACE=(CYL,(1,1),RLSE),
//            RECFM=FB,LRECL=80
//SYSIN DD *     
  SORT FIELDS=COPY
  OUTFIL FILES=01,ENDREC=200
  OUTFIL FILES=02,STARTREC=201,ENDREC=400
  OUTFIL FILES=03,STARTREC=401,ENDREC=700
//*


I tried to count the number of records copying in each outut file. So i have changed my DFSORT step to the following,

Code:


SORT FIELDS=COPY                                                 
OUTFIL FILES=01,ENDREC=200                                       
OUTFIL REMOVECC,NODETAIL,                                       
   TRAILER1=('NO OF RECORDS IN FILE 1 : ',COUNT=(M11,LENGTH=8)) 
OUTFIL FILES=02,STARTREC=201,ENDREC=400                         
OUTFIL REMOVECC,NODETAIL,                                       
   TRAILER2=('NO OF RECORDS IN FILE 2 : ',COUNT=(M11,LENGTH=8)) 
OUTFIL FILES=03,STARTREC=401,ENDREC=700                         
OUTFIL REMOVECC,NODETAIL,                                       
   TRAILER3=('NO OF RECORDS IN FILE 3 : ',COUNT=(M11,LENGTH=8)) 


but it is giving MAXCC = 16. This i sthe error msg im getting,


[/code]

ICE213A 0 INVALID OUTFIL STATEMENT OPERAND
ICE056A 9 SORTOUT NOT DEFINED

Code:


Kindly suggest me the best solution.
Back to top
View user's profile Send private message
Vigneshwaran S
Warnings : 1

New User


Joined: 10 Dec 2008
Posts: 60
Location: Chennai

PostPosted: Sat Jun 20, 2009 7:03 pm    Post subject: Reply to: Splitting an Output file into n number of files ha
Reply with quote

Hi all,

I have creacked down all my ERRORs Finally i land up in a syntax error in the below step,

Code:


SORT FIELDS=COPY                                     
OUTFIL FILES=01,ENDREC=200,                           
TRAILER1=('TOTAL COUNT :', -
           COUNT=(COUNT=(M10,LENGTH=10))   

           


This the error msg im getting,

Code:


                                                                     
            SORT FIELDS=COPY                                         
            OUTFIL FILES=01,ENDREC=200,                             
            TRAILER1=('TOTAL ACCOUNTS COPIED IN THIS REPORT :', -   
                       COUNT=(COUNT=(M10,LENGTH=10))                 
                                     $                                     
ICE007A F SYNTAX ERROR 

                                             

I'm using trailer for the first time.Kindly help me in this issue.
Back to top
View user's profile Send private message
expat

Global Moderator


Joined: 14 Mar 2007
Posts: 8593
Location: Back in jolly old England

PostPosted: Sat Jun 20, 2009 8:52 pm    Post subject:
Reply with quote

Quote:
Sorry Dick and Expac..I need to proceed with teh same work of spliting an output file into different files. My boss wants me to this!!

Then your boss is an idiot.
Back to top
View user's profile Send private message
Frank Yaeger

DFSORT Moderator


Joined: 15 Feb 2005
Posts: 7130
Location: San Jose, CA

PostPosted: Mon Jun 22, 2009 9:32 pm    Post subject:
Reply with quote

You got the syntax error because you have COUNT twice. The correct DFSORT control statements would be:

Code:

 SORT FIELDS=COPY                                             
 OUTFIL FILES=01,ENDREC=200,                                 
    TRAILER1=('TOTAL ACCOUNTS COPIED IN THIS REPORT :', -     
         COUNT=(M10,LENGTH=10))                               


However, for a more dynamic way of doing this kind of thing, see the technique shown in the "Split a file to n output files dynamically" Smart DFSORT Trick at:

www.ibm.com/systems/support/storage/software/sort/mvs/tricks/
Back to top
View user's profile Send private message
Vigneshwaran S
Warnings : 1

New User


Joined: 10 Dec 2008
Posts: 60
Location: Chennai

PostPosted: Tue Jun 23, 2009 3:07 pm    Post subject: Reply to: Splitting an Output file into n number of files ha
Reply with quote

Hi All,

Expat i guess you are right.

A special thanks for Frank for giving that Link!!!

I have got the solution for the problem.This is the step i have used to get the output,

Code:


SORT FIELDS=COPY                                                   
 OUTFIL FILES=01,ENDREC=50,REMOVECC,                               
  TRAILER1=(2/,'RECORDS COPIED IN THIS RPT: ',COUNT=(M11,LENGTH=8))
 OUTFIL FILES=02,STARTREC=51,ENDREC=100,REMOVECC,                   
  TRAILER1=(2/,'RECORDS COPIED IN THIS RPT: ',COUNT=(M11,LENGTH=8))
 OUTFIL FILES=03,STARTREC=101,REMOVECC,                             
  TRAILER1=(2/,'RECORDS COPIED IN THIS RPT: ',COUNT=(M11,LENGTH=8))

Back to top
View user's profile Send private message
mohanrazz

New User


Joined: 30 Sep 2008
Posts: 30
Location: India

PostPosted: Mon Jan 03, 2011 4:08 pm    Post subject: Reply to: Splitting an Output file into n number of files ha
Reply with quote

Can somebody please explain what are these M10 and M11 for?
Back to top
View user's profile Send private message
dbzTHEdinosauer

Global Moderator


Joined: 20 Oct 2006
Posts: 6966
Location: porcelain throne

PostPosted: Mon Jan 03, 2011 4:16 pm    Post subject:
Reply with quote

DFSORT Edit Mask Patterns
Back to top
View user's profile Send private message
hailashwin

New User


Joined: 16 Oct 2008
Posts: 73
Location: Boston

PostPosted: Wed Jan 05, 2011 11:18 am    Post subject:
Reply with quote

Hi,
I would suggest a logic that might work for you to get the data split into n files.

Step1: Use a sort step and get the number of records.
Step2: Create a Cobol program in such a way that it should accepts the count and below template as input and create an output file in such a way that the 'Sortof' files depends on the count.

Example, if the input file has 6500 records...you would need to have 4 'Sortof' files written in the output file as below.
This output file would server as your dynamic job.

Template:
Code:

Job card
//***********************************************************
//SPLITFLS EXEC PGM=SORT
//***********************************************************
//SYSPRINT DD SYSOUT=*
//SYSOUT   DD SYSOUT=*
//SYSUDUMP DD SYSOUT=*
//SORTIN   DD DSN=FILE1......,DISP=SHR
//SORTOF01 DD DSN=OUTPUTFILE1.......,
//            DISP=(NEW,CATLG,DELETE),UNIT=SYSDA,
//            SPACE=(CYL,(1,1),RLSE),
//            RECFM=FB,LRECL=80


Output File:
Code:

Job card
//***********************************************************
//SPLITFLS EXEC PGM=SORT
//***********************************************************
//SYSPRINT DD SYSOUT=*
//SYSOUT   DD SYSOUT=*
//SYSUDUMP DD SYSOUT=*
//SORTIN   DD DSN=FILE1......,DISP=SHR
//SORTOF01 DD DSN=OUTPUTFILE1.......,
//            DISP=(NEW,CATLG,DELETE),UNIT=SYSDA,
//            SPACE=(CYL,(1,1),RLSE),
//            RECFM=FB,LRECL=80
//SORTOF02 DD DSN=OUTPUTFILE2.......,
//            DISP=(NEW,CATLG,DELETE),UNIT=SYSDA,
//            SPACE=(CYL,(1,1),RLSE),
//            RECFM=FB,LRECL=80
//SORTOF03 DD DSN=OUTPUTFILE3.......,
//            DISP=(NEW,CATLG,DELETE),UNIT=SYSDA,
//            SPACE=(CYL,(1,1),RLSE),
//            RECFM=FB,LRECL=80
//SORTOF04 DD DSN=OUTPUTFILE4.......,
//            DISP=(NEW,CATLG,DELETE),UNIT=SYSDA,
//            SPACE=(CYL,(1,1),RLSE),
//            RECFM=FB,LRECL=80
//SYSIN DD *     
  SORT FIELDS=COPY
  OUTFIL FILES=01,ENDREC=2000
  OUTFIL FILES=02,STARTREC=2001,ENDREC=4000
  OUTFIL FILES=03,STARTREC=4001,ENDREC=7000
  OUTFIL FILES=04,STARTREC=6001,ENDREC=500


Step3: Load this created job to the scheduler and it will split up your data to n datasets. If your scheduler is Control-M, you can use CTMBLT to load the job. [/code]

Again, this is only a suggestion..but I feel it would be worth trying if being built more generic.

Thanks,
Ashwin.
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> DFSORT/ICETOOL All times are GMT + 6 Hours
Page 1 of 1

 

Search our Forum:

Similar Topics
Topic Author Forum Replies Posted
No new posts High CPU consumption Job using IAM fi... aswinir JCL & VSAM 8 Thu Dec 01, 2016 8:28 pm
No new posts Add PD field from 2nd file to PD in 1st Sushant Garje DFSORT/ICETOOL 6 Thu Dec 01, 2016 4:32 pm
No new posts File Aid to File Manager conversion murali3955 IBM Tools 4 Thu Nov 24, 2016 3:41 pm
No new posts How to convert the VBM file to VB or... Sulabh Agrawal JCL & VSAM 4 Fri Nov 18, 2016 1:04 pm
No new posts CICS Roll back partially - Need to re... dwijadas CICS 4 Wed Nov 16, 2016 4:30 pm


Facebook
Back to Top
 
Mainframe Wiki | Forum Rules | Bookmarks | Subscriptions | FAQ | Tutorials | Contact Us