IBM Mainframe Forum Index
 
Log In
 
IBM Mainframe Forum Index Mainframe: Search IBM Mainframe Forum: FAQ Register
 

Dataset size increase on adding 1 byte to records
IBM Mainframe Forums -> DFSORT/ICETOOL
Post new topic   Reply to topic Quick References
View previous topic :: View next topic  
Author Message
sumannath

New User


Joined: 20 Mar 2017
Posts: 8
Location: India
PostPosted: Fri Oct 28, 2022 1:12 pm
 Reply with quote
Hi,
I have 16 files in same format which I need to merge into a single file. I am using DFSORT for the same. I've used the following step:
Code:

//STEP20   EXEC PGM=SORT                                           
//SYSOUT   DD SYSOUT=*                                             
//SORTIN   DD DISP=SHR,DSN=A.D2.W01           
//         DD DISP=SHR,DSN=A.D2.W02           
//         DD DISP=SHR,DSN=A.D2.W03           
//         DD DISP=SHR,DSN=A.D2.W04           
//         DD DISP=SHR,DSN=A.D2.W05           
//         DD DISP=SHR,DSN=A.D2.W06           
//         DD DISP=SHR,DSN=A.D2.W08           
//         DD DISP=SHR,DSN=A.D2.W10           
//         DD DISP=SHR,DSN=A.D2.W11           
//         DD DISP=SHR,DSN=A.D2.W12           
//         DD DISP=SHR,DSN=A.D2.W13           
//         DD DISP=SHR,DSN=A.D2.W14           
//         DD DISP=SHR,DSN=A.D2.W15           
//         DD DISP=SHR,DSN=A.D2.W16           
//         DD DISP=SHR,DSN=A.D2.W17           
//         DD DISP=SHR,DSN=A.D2.W18           
//SORTOUT  DD DISP=(NEW,CATLG,DELETE),                             
//            SPACE=(CYL,(150,150),RLSE),UNIT=RESDA,DSNTYPE=LARGE,
//            DSN=A.D2.ALL                                           
//SYSIN    DD *                                                   
    OPTION COPY                                                   
//*

The CPU usage is as follows:
Code:

-------- JOB  SUMMARY  STATISTICS -------  CPU TIME  ELAPSED    SERV. 
-PROCNAME STEPNAME PROGRAM     RC   EXCP     (SEC.) TIME(SEC.)  UNITS 
-         STEP20   SORT        00   190K       2.50      69.40     323K

The files have record length 128 and combined file has 40million records. The space usage is:
Code:

Current Allocation             
 Allocated cylinders : 6,226   
 Allocated extents . : 42     
                               
                               
                               
Current Utilization           
 Used cylinders  . . : 6,226   
 Used extents  . . . : 42     



Now, I used ICETOOL utility to add a byte to each record of the input files to indicate from which file (01,02,03 etc) the record is coming from. I used the following code:
Code:

//STEP20   EXEC PGM=ICETOOL                                         
//SYSOUT   DD SYSOUT=*                                               
//SRTIN01  DD DISP=SHR,DSN=A.D2.W01           
//SRTIN02  DD DISP=SHR,DSN=A.D2.W02           
//SRTIN03  DD DISP=SHR,DSN=A.D2.W03           
//SRTIN04  DD DISP=SHR,DSN=A.D2.W04           
//SRTIN05  DD DISP=SHR,DSN=A.D2.W05           
//SRTIN06  DD DISP=SHR,DSN=A.D2.W06           
//SRTIN08  DD DISP=SHR,DSN=A.D2.W08           
//SRTIN10  DD DISP=SHR,DSN=A.D2.W10           
//SRTIN11  DD DISP=SHR,DSN=A.D2.W11           
//SRTIN12  DD DISP=SHR,DSN=A.D2.W12           
//SRTIN13  DD DISP=SHR,DSN=A.D2.W13           
//SRTIN14  DD DISP=SHR,DSN=A.D2.W14           
//SRTIN15  DD DISP=SHR,DSN=A.D2.W15           
//SRTIN16  DD DISP=SHR,DSN=A.D2.W16           
//SRTIN17  DD DISP=SHR,DSN=A.D2.W17           
//SRTIN18  DD DISP=SHR,DSN=A.D2.W18           
//MRGDINPA DD DISP=(MOD,CATLG,DELETE),                               
//            SPACE=(CYL,(1500,1500),RLSE),UNIT=RESDA,DSNTYPE=LARGE,
//            DSN=A.D2.ALL                                         
//TOOLMSG  DD SYSOUT=*                                               
//DFSMSG   DD SYSOUT=*                                               
//TOOLIN   DD *                                                     
//TOOLIN   DD *                                             
  COPY FROM(SRTIN01) TO(MRGDINPA) USING(CT01)               
  COPY FROM(SRTIN02) TO(MRGDINPA) USING(CT02)               
  COPY FROM(SRTIN03) TO(MRGDINPA) USING(CT03)               
  COPY FROM(SRTIN04) TO(MRGDINPA) USING(CT04)               
  COPY FROM(SRTIN05) TO(MRGDINPA) USING(CT05)               
  COPY FROM(SRTIN06) TO(MRGDINPA) USING(CT06)               
  COPY FROM(SRTIN08) TO(MRGDINPA) USING(CT08)               
  COPY FROM(SRTIN10) TO(MRGDINPA) USING(CT10)               
  COPY FROM(SRTIN11) TO(MRGDINPA) USING(CT11)               
  COPY FROM(SRTIN12) TO(MRGDINPA) USING(CT12)               
  COPY FROM(SRTIN13) TO(MRGDINPA) USING(CT13)               
  COPY FROM(SRTIN14) TO(MRGDINPA) USING(CT14)               
  COPY FROM(SRTIN15) TO(MRGDINPA) USING(CT15)               
  COPY FROM(SRTIN16) TO(MRGDINPA) USING(CT16)               
  COPY FROM(SRTIN17) TO(MRGDINPA) USING(CT17)               
  COPY FROM(SRTIN18) TO(MRGDINPA) USING(CT18)               
/*                                                         
//CT01CNTL DD *                                             
  OUTREC OVERLAY=(129:+01,TO=BI,LENGTH=1)                   
//CT02CNTL DD *                                             
  OUTREC OVERLAY=(129:+02,TO=BI,LENGTH=1)                   
//CT03CNTL DD *                                             
  OUTREC OVERLAY=(129:+03,TO=BI,LENGTH=1)   
...


On running the job, the CPU time, elapsed time and dataset size is a follows:
Code:

-------- JOB  SUMMARY  STATISTICS -------  CPU TIME  ELAPSED    SERV. 
-PROCNAME STEPNAME PROGRAM     RC   EXCP     (SEC.) TIME(SEC.)  UNITS 
-         STEP20   ICETOOL     00   320K      26.41     880.72    1724K

Code:

Current Allocation           
 Allocated cylinders : 37,701
 Allocated extents . : 35   
                             
                             
                             
Current Utilization         
 Used cylinders  . . : 37,701
 Used extents  . . . : 35   


I want to understand why there is an increase in so much resources by only adding 1 byte to each record in the input dataset.

Thanks!
Back to top
View user's profile Send private message
Joerg.Findeisen

Senior Member


Joined: 15 Aug 2015
Posts: 1255
Location: Bamberg, Germany
PostPosted: Fri Oct 28, 2022 1:43 pm
 Reply with quote
What RECFM are you using? I would personally use BUILD instead of OVERLAY in INREC.
Back to top
View user's profile Send private message
sumannath

New User


Joined: 20 Mar 2017
Posts: 8
Location: India
PostPosted: Fri Oct 28, 2022 2:40 pm
 Reply with quote
Joerg.Findeisen wrote:
What RECFM are you using? I would personally use BUILD instead of OVERLAY in INREC.


RECFM=FB
Back to top
View user's profile Send private message
sumannath

New User


Joined: 20 Mar 2017
Posts: 8
Location: India
PostPosted: Fri Oct 28, 2022 3:24 pm
 Reply with quote
Brought down the CPU and allocated cylinders quite a lot by mentioning the DCB parameter:
Code:

//MRGDINPA DD DISP=(MOD,CATLG,DELETE),                             
//            DCB=(RECFM=FB,LRECL=129,BLKSIZE=28122),               
//            SPACE=(CYL,(1500,1500),RLSE),UNIT=RESDA,DSNTYPE=LARGE,
//            DSN=A.D2.ALL                                           


Code:

-------- JOB  SUMMARY  STATISTICS -------  CPU TIME  ELAPSED    SERV. 
-PROCNAME STEPNAME PROGRAM     RC   EXCP     (SEC.) TIME(SEC.)  UNITS 
-         STEP20   ICETOOL     00   120K       8.05      48.54     547K


Code:

Current Allocation             
 Allocated cylinders : 12,451   
 Allocated extents . : 18       
                               
                               
                               
Current Utilization             
 Used cylinders  . . : 12,451   
 Used extents  . . . : 18       


So, the question is how does mentioning DCB decrease the resources substantially? I thought DFSORT determines the record length automatically.
Back to top
View user's profile Send private message
Joerg.Findeisen

Senior Member


Joined: 15 Aug 2015
Posts: 1255
Location: Bamberg, Germany
PostPosted: Fri Oct 28, 2022 3:47 pm
 Reply with quote
sumannath wrote:
So, the question is how does mentioning DCB decrease the resources substantially? I thought DFSORT determines the record length automatically.

It does. Have you tried BUILD instead of OVERLAY?
Back to top
View user's profile Send private message
sergeyken

Senior Member


Joined: 29 Apr 2008
Posts: 2023
Location: USA
PostPosted: Fri Oct 28, 2022 4:51 pm
 Reply with quote
sumannath wrote:

So, the question is how does mentioning DCB decrease the resources substantially? I thought DFSORT determines the record length automatically.

Please, take a look at the SORT log (under your //SYSOUT DD)

It says everything about the finally used datasets attributes, record size, number of records, etc. You can get much more info from there rather than from the forum, regarding your own datasets.

BTW, using parameter BLKSIZE=0 allows zOS to automatically calculate the most suitable block size, depending also on the particular device used for storage.
Back to top
View user's profile Send private message
Joerg.Findeisen

Senior Member


Joined: 15 Aug 2015
Posts: 1255
Location: Bamberg, Germany
PostPosted: Fri Oct 28, 2022 5:16 pm
 Reply with quote
I thought these details were known before posting. icon_confused.gif
Back to top
View user's profile Send private message
sergeyken

Senior Member


Joined: 29 Apr 2008
Posts: 2023
Location: USA
PostPosted: Fri Oct 28, 2022 8:17 pm
 Reply with quote
Joerg.Findeisen wrote:
I thought these details were known before posting. icon_confused.gif

I doubt they were...
Back to top
View user's profile Send private message
Rohit Umarjikar

Global Moderator


Joined: 21 Sep 2010
Posts: 3053
Location: NYC,USA
PostPosted: Mon Oct 31, 2022 8:40 am
 Reply with quote
Remove DCB and try SDB=YES.
www.ibm.com/docs/en/zos/2.1.0?topic=statements-option-control-statement
www-40.ibm.com/servers/resourcelink/svc00100.nsf/pages/zOSV2R3sc236882/$file/icet100_v2r3.pdf
Sample ex
ibmmainframes.com/about47793.html
ibmmainframes.com/about57592.html
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic View Bookmarks
All times are GMT + 6 Hours
Forum Index -> DFSORT/ICETOOL

 

Similar Topics
Topic Forum Replies
No new posts Compare 2 files and retrive records f... DFSORT/ICETOOL 3
No new posts Compare 2 files(F1 & F2) and writ... JCL & VSAM 8
No new posts FINDREP - Only first record from give... DFSORT/ICETOOL 3
No new posts Compare only first records of the fil... SYNCSORT 7
No new posts Pulling a fixed number of records fro... DB2 2
Search our Forums:

Back to Top
IBMMainframes.com is not an official and/or affiliated with IBM® in anyway
Board Rules | FAQ | Downloads | Wiki | SiteMap | Contact Us