Portal | Manuals | References | Downloads | Info | Programs | JCLs | Master the Mainframes
IBM Mainframe Computers Forums Index
 
Register
 
IBM Mainframe Computers Forums Index Mainframe: Search IBM Mainframe Forum: FAQ Memberlist Usergroups Profile Log in to check your private messages Log in
 

 

System can only handle 5000 records per file

 
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> DFSORT/ICETOOL
View previous topic :: :: View next topic  
Author Message
Squeak6

New User


Joined: 04 Aug 2005
Posts: 36

PostPosted: Wed Aug 18, 2010 9:14 pm    Post subject: System can only handle 5000 records per file
Reply with quote

Hello...

I have to create a file(s) for a user, but their system can only handle 5000 records per file...

How do I breakup one large master file (which can have as little as 15, or as many as 35,000 records) in to smaller files (that contain 5000 records each).

The most records I’ll have in the master file is 35,000.


Thanks


Squeak…
Back to top
View user's profile Send private message

sqlcode1

Active Member


Joined: 08 Apr 2010
Posts: 578
Location: USA

PostPosted: Wed Aug 18, 2010 10:04 pm    Post subject:
Reply with quote

Squeak6,
Please refer to this post :- http://ibmmainframes.com/viewtopic.php?t=50525

Are you ok, if DFsort always create 7 output files but some of these can be empty? Is there a way to logically group your input records (for example, records with header and/or trailer)? If yes, please show sample input records.

Thanks,
Back to top
View user's profile Send private message
Frank Yaeger

DFSORT Moderator


Joined: 15 Feb 2005
Posts: 7130
Location: San Jose, CA

PostPosted: Wed Aug 18, 2010 10:49 pm    Post subject:
Reply with quote

Squeak,

If you just want to break up the input file into output files with a maximum of 5000 records, you can use this simple DFSORT job:

Code:

//S1 EXEC PGM=SORT
//SYSOUT DD SYSOUT=*
//SORTIN DD DSN=...  input file
//OUT1 DD DSN=...  outut file1
//OUT2 DD DSN=...  outut file2
...
//OUT7 DD DSN=...  outut file2
//SYSIN DD *
  OPTION COPY
  OUTFIL SPLIT1R=5000,
    FNAMES=(OUT1,OUT2,OUT3,OUT4,OUT5,OUT6,OUT7)
/*


If you want something more elaborate, you need to describe exactly what you want.
Back to top
View user's profile Send private message
Squeak6

New User


Joined: 04 Aug 2005
Posts: 36

PostPosted: Wed Aug 18, 2010 11:50 pm    Post subject:
Reply with quote

this is exactly what I need... I'll have to and a RC check step ( copy the output file to a new file to see if its empty, before I ftp the file ) I'll have to do that for each file...

thanks.

Squeak...
Back to top
View user's profile Send private message
gcicchet

Senior Member


Joined: 28 Jul 2006
Posts: 1703
Location: Australia

PostPosted: Thu Aug 19, 2010 4:27 am    Post subject:
Reply with quote

Hi Squeak,

you don't need to copy the output file to a new file to see if it's empty

Code:
 COUNT FROM(IN) EMPTY RC4   




Gerry
Back to top
View user's profile Send private message
Squeak6

New User


Joined: 04 Aug 2005
Posts: 36

PostPosted: Thu Aug 19, 2010 6:18 pm    Post subject:
Reply with quote

the code worked great, but I forgot that I have a header record that I need to add to each new output file.. ( if needed ) below is my input master file.

Code:
"First Name","Middle Name","Last Name","Employee ID","Address1"
"TIM                 ","D","SMALL               ","e9       ","



Record format . . . : FB
Record length . . . : 350
Back to top
View user's profile Send private message
Squeak6

New User


Joined: 04 Aug 2005
Posts: 36

PostPosted: Thu Aug 19, 2010 6:27 pm    Post subject:
Reply with quote

gcicchet ~ I'll need to know which one of my 7 ouptut files are empty because I'm sending the data via FTP. ( that's why I was doing an empty check step for each output file )
Back to top
View user's profile Send private message
dick scherrer

Site Director


Joined: 23 Nov 2006
Posts: 19270
Location: Inside the Matrix

PostPosted: Thu Aug 19, 2010 9:18 pm    Post subject:
Reply with quote

Hello,

What Gerry told you is that you do not need to copy the file to see if it is empty. . .
Back to top
View user's profile Send private message
Squeak6

New User


Joined: 04 Aug 2005
Posts: 36

PostPosted: Thu Aug 19, 2010 9:28 pm    Post subject:
Reply with quote

dick scherrer ~ thanks

but how do I add the header that's on my master file to each of the 7 files ( add only if the file(s) have data..
Back to top
View user's profile Send private message
Skolusu

Senior Member


Joined: 07 Dec 2007
Posts: 2205
Location: San Jose

PostPosted: Thu Aug 19, 2010 10:34 pm    Post subject:
Reply with quote

Squeak6,

Is the header part of the input file? or is it something you can add as part of reporting features?
Back to top
View user's profile Send private message
Squeak6

New User


Joined: 04 Aug 2005
Posts: 36

PostPosted: Thu Aug 19, 2010 10:53 pm    Post subject:
Reply with quote

The header is about of the orgianl master input file.. ( its just a standard header for a CSV file without totals )

Code:
"First Name","Middle Name","Last Name","Employee ID","Address1",
Back to top
View user's profile Send private message
Squeak6

New User


Joined: 04 Aug 2005
Posts: 36

PostPosted: Thu Aug 19, 2010 11:09 pm    Post subject:
Reply with quote

" The Header is apart of the orgianal input file.. "

Code:
"First Name","Middle Name","Last Name","Employee ID","Address1","Ad
"TIM                 ","D","SMALL               ","e9       ","   "
"TODD                ","O","MC CARTHY           ","e11      ","   "
"JIM                 ","W","LUCIUS              ","e16      ","   "
Back to top
View user's profile Send private message
Skolusu

Senior Member


Joined: 07 Dec 2007
Posts: 2205
Location: San Jose

PostPosted: Fri Aug 20, 2010 2:48 am    Post subject:
Reply with quote

Squeak6,

The following JCL will give you the desired results.

1. I assumed that your input is FB recfm and 80 bytes of length.
2. The FTP card is of the format PUT 'USERID.OUT1' 'OUT1.TXT'
3. The max number of records per file is 5000 and you will only process a total max of 35,000 records.
4. The header is the 1st record in the input.


Code:

//******************************************************************
//* SPLIT THE INPUT FILE INTO 7 OUT FILES EACH WITH 5000 RECORDS   *
//******************************************************************
//STEP0100 EXEC PGM=SORT               
//SYSOUT   DD SYSOUT=*                 
//SORTIN   DD DSN=Your 80 byte FB lrecl file,DISP=SHR
//OUT1     DD DSN=USERID.OUT1,DISP=(,CATLG),SPACE=(CYL,(X,Y),RLSE)
//OUT2     DD DSN=USERID.OUT2,DISP=(,CATLG),SPACE=(CYL,(X,Y),RLSE)
//OUT3     DD DSN=USERID.OUT3,DISP=(,CATLG),SPACE=(CYL,(X,Y),RLSE)
//OUT4     DD DSN=USERID.OUT4,DISP=(,CATLG),SPACE=(CYL,(X,Y),RLSE)
//OUT5     DD DSN=USERID.OUT5,DISP=(,CATLG),SPACE=(CYL,(X,Y),RLSE)
//OUT6     DD DSN=USERID.OUT6,DISP=(,CATLG),SPACE=(CYL,(X,Y),RLSE)
//OUT7     DD DSN=USERID.OUT7,DISP=(,CATLG),SPACE=(CYL,(X,Y),RLSE)
//SYSIN    DD *
  SORT FIELDS=COPY,STOPAFT=35000                                               
  OUTREC IFTHEN=(WHEN=INIT,OVERLAY=(81:SEQNUM,8,ZD,START=4998)),
  IFTHEN=(WHEN=INIT,OVERLAY=(89:81,8,ZD,DIV,+4999,M11,LENGTH=2))
                                                                 
  OUTFIL FNAMES=OUT1,BUILD=(1,80),                               
  INCLUDE=(89,2,ZD,EQ,0,OR,89,2,ZD,EQ,1)                         
                                                                 
  OUTFIL FNAMES=OUT2,BUILD=(1,80),                               
  INCLUDE=(89,2,ZD,EQ,0,OR,89,2,ZD,EQ,2)                         
                                                                 
  OUTFIL FNAMES=OUT3,BUILD=(1,80),                               
  INCLUDE=(89,2,ZD,EQ,0,OR,89,2,ZD,EQ,3)                         
                                                                 
  OUTFIL FNAMES=OUT4,BUILD=(1,80),                               
  INCLUDE=(89,2,ZD,EQ,0,OR,89,2,ZD,EQ,4)                         
                                                                 
  OUTFIL FNAMES=OUT5,BUILD=(1,80),                               
  INCLUDE=(89,2,ZD,EQ,0,OR,89,2,ZD,EQ,5)                         
                                                                 
  OUTFIL FNAMES=OUT6,BUILD=(1,80),                               
  INCLUDE=(89,2,ZD,EQ,0,OR,89,2,ZD,EQ,6)                         
                                                                 
  OUTFIL FNAMES=OUT7,BUILD=(1,80),                               
  INCLUDE=(89,2,ZD,EQ,0,OR,89,2,ZD,EQ,7)                         

//*                                                             
//******************************************************************
//* CHECK IF ANY OF THE FILES ARE EMPTY AND DELETE THEM.           *
//* WRITE THE FTP CARD IF THE FILE IS NOT EMPTY(EXLUDING HEADER).  *
//* SET MAXCC=8 IF THERE ARE NO RECORDS TO FTP.                    *
//******************************************************************
//STEP0200 EXEC PGM=IDCAMS               
//SYSPRINT DD SYSOUT=*                   
//FTPI     DD *                           
  PUT 'USERID.OUT1' 'OUT1.TXT'           
  PUT 'USERID.OUT2' 'OUT2.TXT'           
  PUT 'USERID.OUT3' 'OUT3.TXT'           
  PUT 'USERID.OUT4' 'OUT4.TXT'           
  PUT 'USERID.OUT5' 'OUT5.TXT'           
  PUT 'USERID.OUT6' 'OUT6.TXT'           
  PUT 'USERID.OUT7' 'OUT7.TXT'           
//IN1      DD DISP=SHR,DSN=USERID.OUT1                               
//IN2      DD DISP=SHR,DSN=USERID.OUT2                               
//IN3      DD DISP=SHR,DSN=USERID.OUT3                               
//IN4      DD DISP=SHR,DSN=USERID.OUT4                               
//IN5      DD DISP=SHR,DSN=USERID.OUT5                               
//IN6      DD DISP=SHR,DSN=USERID.OUT6                               
//IN7      DD DISP=SHR,DSN=USERID.OUT7                               
//*
//FTPO     DD DSN=USERID.FTPCARD,                                     
//            DISP=(MOD,PASS),SPACE=(TRK,(1,1),RLSE),                 
//            DCB=(LRECL=80,RECFM=FB,BLKSIZE=27920)                   
//*
//SYSIN    DD *                                                       
  PRINT INFILE(IN1) CHARACTER COUNT(2)                               
  IF LASTCC = 0 THEN REPRO IFILE(FTPI) OFILE(FTPO) COUNT(1)           
  ELSE IF LASTCC = 4 THEN DO                                         
       DELETE 'USERID.OUT1'                                           
       SET MAXCC=8                                                   
  END                                                                 
                                                                     
  PRINT INFILE(IN2) CHARACTER COUNT(2)                                 
  IF LASTCC = 0 THEN REPRO IFILE(FTPI) OFILE(FTPO) COUNT(1) SKIP(1)   
  ELSE IF LASTCC = 4 THEN DELETE 'USERID.OUT2'                         
                                                                       
  PRINT INFILE(IN3) CHARACTER COUNT(2)                                 
  IF LASTCC = 0 THEN REPRO IFILE(FTPI) OFILE(FTPO) COUNT(1) SKIP(2)   
  ELSE IF LASTCC = 4 THEN DELETE 'USERID.OUT3'                         
                                                                       
  PRINT INFILE(IN4) CHARACTER COUNT(2)                                 
  IF LASTCC = 0 THEN REPRO IFILE(FTPI) OFILE(FTPO) COUNT(1) SKIP(3)   
  ELSE IF LASTCC = 4 THEN DELETE 'USERID.OUT4'                         
                                                                       
  PRINT INFILE(IN5) CHARACTER COUNT(2)                                 
  IF LASTCC = 0 THEN REPRO IFILE(FTPI) OFILE(FTPO) COUNT(1) SKIP(4)   
  ELSE IF LASTCC = 4 THEN DELETE 'USERID.OUT5'                         
                                                                       
  PRINT INFILE(IN6) CHARACTER COUNT(2)                                 
  IF LASTCC = 0 THEN REPRO IFILE(FTPI) OFILE(FTPO) COUNT(1) SKIP(5)   
  ELSE IF LASTCC = 4 THEN DELETE 'USERID.OUT6'                         
                                                                       
  PRINT INFILE(IN7) CHARACTER COUNT(2)                                 
  IF LASTCC = 0 THEN REPRO IFILE(FTPI) OFILE(FTPO) COUNT(1) SKIP(6)   
  ELSE IF LASTCC = 4 THEN DELETE 'USERID.OUT7'                         

//*     
//******************************************************************
//* FTP ONLY IF THERE IS ATLEAST 1 FILE TO SEND.                   *
//******************************************************************       
//STEP0300 EXEC PGM=FTP,COND=(8,EQ,STEP0200)                         
//SYSPRINT DD SYSOUT=*                           
//INPUT    DD *           
ip addr
userid                                         
password
//         DD DSN=USERID.FTPCARD,DISP=SHR
//         DD *           
CLOSE                                                             
QUIT   
//*
Back to top
View user's profile Send private message
Squeak6

New User


Joined: 04 Aug 2005
Posts: 36

PostPosted: Fri Aug 20, 2010 7:48 pm    Post subject:
Reply with quote

Skolusu..

thank you for the code... ( I'm going to study it so I can undersand how it works )

will this include the Header in the file(s) being FTP'd?
Back to top
View user's profile Send private message
Squeak6

New User


Joined: 04 Aug 2005
Posts: 36

PostPosted: Fri Aug 20, 2010 8:10 pm    Post subject:
Reply with quote

As I need the Header to be apart of the newly created files...
Back to top
View user's profile Send private message
dick scherrer

Site Director


Joined: 23 Nov 2006
Posts: 19270
Location: Inside the Matrix

PostPosted: Sat Aug 21, 2010 3:26 am    Post subject:
Reply with quote

Hello,

Quote:
As I need the Header to be apart of the newly created files...
Possibly better understood if you posted some example(s).
Back to top
View user's profile Send private message
Skolusu

Senior Member


Joined: 07 Dec 2007
Posts: 2205
Location: San Jose

PostPosted: Mon Aug 23, 2010 10:02 pm    Post subject:
Reply with quote

Squeak6 wrote:
Skolusu..

thank you for the code... ( I'm going to study it so I can undersand how it works )

will this include the Header in the file(s) being FTP'd?


Yes. You could have run the given job and see how it works.
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> DFSORT/ICETOOL All times are GMT + 6 Hours
Page 1 of 1

 

Search our Forum:

Similar Topics
Topic Author Forum Replies Posted
No new posts Add PD field from 2nd file to PD in 1st Sushant Garje DFSORT/ICETOOL 6 Thu Dec 01, 2016 4:32 pm
No new posts File Aid to File Manager conversion murali3955 IBM Tools 4 Thu Nov 24, 2016 3:41 pm
No new posts IMS BMP program causes 878 system abend Artemk IMS DB/DC 7 Tue Nov 22, 2016 8:26 pm
No new posts Limit duplicate records in the SORT pshongal SYNCSORT 6 Mon Nov 21, 2016 12:54 pm
No new posts How to convert the VBM file to VB or... Sulabh Agrawal JCL & VSAM 4 Fri Nov 18, 2016 1:04 pm


Facebook
Back to Top
 
Mainframe Wiki | Forum Rules | Bookmarks | Subscriptions | FAQ | Tutorials | Contact Us