Portal | Manuals | References | Downloads | Info | Programs | JCLs | Master the Mainframes
IBM Mainframe Computers Forums Index
 
Register
 
IBM Mainframe Computers Forums Index Mainframe: Search IBM Mainframe Forum: FAQ Memberlist Usergroups Profile Log in to check your private messages Log in
 

 

QSAM Buffering Options

 
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> COBOL Programming
View previous topic :: :: View next topic  
Author Message
rockish

Active User


Joined: 05 Jun 2009
Posts: 185
Location: Planet Earth

PostPosted: Tue Sep 08, 2009 4:21 pm    Post subject: QSAM Buffering Options
Reply with quote

I have a COBOL program which writes about 80,192,091 records into a QSAM file. This step consumes a total of about 2.5 CPU minutes and over 15 clock minutes.
The total EXCP count consumed on this file is about 431141.
This shows that there should be one write operation for every 186 records.
The record length of the file is 150. The Blocksize of the output file is currently 27900. which is nothing but 150 * 186.. Hence one write operation takes place whenever a block is getting filled..

My doubt now is, will a I-O operation take place whenever a block gets filled ? Or when 'n' number of blocks that can fill a track gets filled ?

Because here in my case, a WRITE takes place whenever one block is getting filled and not when 2 blocks that constitutes a track gets filled.

I have a COBOL program which writes about 80,192,091 records into a QSAM file. This step consumes a total of about 2.5 CPU minutes and over 15 clock minutes.
The total EXCP count consumed on this file is about 431141.
This shows that there should be one write operation for every 186 records.
The record length of the file is 150. The Blocksize of the output file is currently 27900. which is nothing but 150 * 186.. Hence one write operation takes place whenever a block is getting filled..

My doubt now is, will a I-O operation take place whenever a block gets filled ? Or when 'n' number of blocks that can fill a track gets filled ?

Are there any other ways of improving the performance of a COBOL write with the help of QSAM buffering options ???
Back to top
View user's profile Send private message

Robert Sample

Global Moderator


Joined: 06 Jun 2008
Posts: 7932
Location: Bellevue, IA

PostPosted: Tue Sep 08, 2009 4:49 pm    Post subject:
Reply with quote

Do you have DCB=BUFNO= on the file? If so, how many buffers are you giving it? If not, add DCB=BUFNO=30 (or 50 or 100 or ...) on your DD statement and you could see some improvement. With half-track blocking, I tell my programmers to use at least 30 buffers although there are times when 50 or 100 improves performance. I've seen some old studies that indicate giving more than 200 buffers decreased performance but I haven't had a chance to study that myself.
Back to top
View user's profile Send private message
rockish

Active User


Joined: 05 Jun 2009
Posts: 185
Location: Planet Earth

PostPosted: Tue Sep 08, 2009 4:57 pm    Post subject:
Reply with quote

Thanks Robert for your reply.

I was actually just about to add that to my post.
The JCL has DCB=BUFNO=5 specified in it.

But could you pls clarify on how this works ? As I already typed, i find a WRITE operation taking place whenever a block is getting filled, then where does this BUFNO come into picture ??

Is it like, the records will get written into buffers and whenever the number of the writes reaches the BUFNO specified, the actual write takes place ? In that case, is it like increasing BUFNO may not reduce EXCP, but just decrease the elapsed time alone ???
Back to top
View user's profile Send private message
Robert Sample

Global Moderator


Joined: 06 Jun 2008
Posts: 7932
Location: Bellevue, IA

PostPosted: Tue Sep 08, 2009 5:09 pm    Post subject:
Reply with quote

DCB=BUFNO=5 is the default for QSAM. It is a default that has been set for many,many years and is extremely out of date. If you're running on a z10, the machine can run 4 billion instructions per second (what that means, exactly, is another question we won't discuss today) whereas an ESCON channel is pumping 17 million bytes per second through it (which could be many disk reads or writes, of course). Do we see a 235-to-1 difference? So with 5 buffers the system fills each, writes it, goes to the next. After the 5th one is filled, the system then has to wait for the first I/O operation to complete before starting to fill the first one again.

DO NOT become concerned with the actual physical writes -- that is well beyond the control of applications programmers; accept that the system will do so as efficiently as possible given the constraints you, the programmer, put on it (such as limiting it to only 5 buffers).

While actual results may vary, I have personally cut run time for a job by 80% (75 minutes to 15 minutes) merely by changing the buffers for each file from the default to something in the 30 to 50 range. Note that doing this may increase region requirements for the program.
Back to top
View user's profile Send private message
rockish

Active User


Joined: 05 Jun 2009
Posts: 185
Location: Planet Earth

PostPosted: Tue Sep 08, 2009 5:14 pm    Post subject:
Reply with quote

Thanks for your explanation Robert.

I will test my program with different BUFNO values and will publish the most optimal one I was able to use.
Back to top
View user's profile Send private message
dbzTHEdinosauer

Global Moderator


Joined: 20 Oct 2006
Posts: 6966
Location: porcelain throne

PostPosted: Tue Sep 08, 2009 7:50 pm    Post subject:
Reply with quote

Not sure if it applies to your version of cobol, but insure that you are not using the WRITE Immediate option.
Back to top
View user's profile Send private message
Singaram

New User


Joined: 19 Aug 2009
Posts: 19
Location: India

PostPosted: Tue Sep 08, 2009 11:20 pm    Post subject:
Reply with quote

Can you please clarify the relation of BUFNO value and the BLKSIZE for QSAM??
Back to top
View user's profile Send private message
Robert Sample

Global Moderator


Joined: 06 Jun 2008
Posts: 7932
Location: Bellevue, IA

PostPosted: Wed Sep 09, 2009 12:03 am    Post subject:
Reply with quote

BUFNO is the number of buffers used by QSAM for reading (or writing) the file.

BLKSIZE is the physical size read or written -- the size of the data "chunks" (I know, not a very technical term) written to the disk at one time.

The amount of memory required by a program for the file buffers is the number of buffers times the block size; since buffers can now go above the line the performance impact of more buffers is lessened.
Back to top
View user's profile Send private message
Terry Heinze

JCL Moderator


Joined: 14 Jul 2008
Posts: 1239
Location: Richfield, MN, USA

PostPosted: Wed Sep 09, 2009 3:08 am    Post subject:
Reply with quote

Mr. J, I'm guessing that your guess will be correct: Increasing BUFNO will not significantly affect the EXCP count but you will probably see a slight decrease in elapsed time. Keep in mind that unless you have a dedicated machine, elapsed time is not a good measurement because of a variable job mix. Please let us know the results of your benchmark.
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> COBOL Programming All times are GMT + 6 Hours
Page 1 of 1

 

Search our Forum:

Similar Topics
Topic Author Forum Replies Posted
No new posts DB2 Unload format options Susanta DB2 2 Fri Aug 12, 2016 5:42 pm
No new posts Endevor - Compiler Options. Chetan Kumar Mainframe Interview Questions 4 Tue Oct 20, 2015 6:32 pm
No new posts DFSORT with Reformat Options jegan_selvan DFSORT/ICETOOL 4 Tue Oct 14, 2014 4:52 pm
No new posts Compiler Options tmaheswara CICS 2 Mon Mar 11, 2013 11:59 am
No new posts How to use OPTIMIZE options in PL/I f... CyberKing PL/I & Assembler 3 Thu Jan 31, 2013 9:56 pm


Facebook
Back to Top
 
Mainframe Wiki | Forum Rules | Bookmarks | Subscriptions | FAQ | Tutorials | Contact Us