QSAM Buffering Options

rockish · Posted: Tue Sep 08, 2009 4:21 pm

I have a COBOL program which writes about 80,192,091 records into a QSAM file. This step consumes a total of about 2.5 CPU minutes and over 15 clock minutes.
The total EXCP count consumed on this file is about 431141.
This shows that there should be one write operation for every 186 records.
The record length of the file is 150. The Blocksize of the output file is currently 27900. which is nothing but 150 * 186.. Hence one write operation takes place whenever a block is getting filled..

My doubt now is, will a I-O operation take place whenever a block gets filled ? Or when 'n' number of blocks that can fill a track gets filled ?

Because here in my case, a WRITE takes place whenever one block is getting filled and not when 2 blocks that constitutes a track gets filled.

I have a COBOL program which writes about 80,192,091 records into a QSAM file. This step consumes a total of about 2.5 CPU minutes and over 15 clock minutes.
The total EXCP count consumed on this file is about 431141.
This shows that there should be one write operation for every 186 records.
The record length of the file is 150. The Blocksize of the output file is currently 27900. which is nothing but 150 * 186.. Hence one write operation takes place whenever a block is getting filled..

My doubt now is, will a I-O operation take place whenever a block gets filled ? Or when 'n' number of blocks that can fill a track gets filled ?

Are there any other ways of improving the performance of a COBOL write with the help of QSAM buffering options ???

Robert Sample · Posted: Tue Sep 08, 2009 4:49 pm

Do you have DCB=BUFNO= on the file? If so, how many buffers are you giving it? If not, add DCB=BUFNO=30 (or 50 or 100 or ...) on your DD statement and you could see some improvement. With half-track blocking, I tell my programmers to use at least 30 buffers although there are times when 50 or 100 improves performance. I've seen some old studies that indicate giving more than 200 buffers decreased performance but I haven't had a chance to study that myself.

rockish · Posted: Tue Sep 08, 2009 4:57 pm

Thanks Robert for your reply.

I was actually just about to add that to my post.
The JCL has DCB=BUFNO=5 specified in it.

But could you pls clarify on how this works ? As I already typed, i find a WRITE operation taking place whenever a block is getting filled, then where does this BUFNO come into picture ??

Is it like, the records will get written into buffers and whenever the number of the writes reaches the BUFNO specified, the actual write takes place ? In that case, is it like increasing BUFNO may not reduce EXCP, but just decrease the elapsed time alone ???

Robert Sample · Posted: Tue Sep 08, 2009 5:09 pm

DCB=BUFNO=5 is the default for QSAM. It is a default that has been set for many,many years and is extremely out of date. If you're running on a z10, the machine can run 4 billion instructions per second (what that means, exactly, is another question we won't discuss today) whereas an ESCON channel is pumping 17 million bytes per second through it (which could be many disk reads or writes, of course). Do we see a 235-to-1 difference? So with 5 buffers the system fills each, writes it, goes to the next. After the 5th one is filled, the system then has to wait for the first I/O operation to complete before starting to fill the first one again.

DO NOT become concerned with the actual physical writes -- that is well beyond the control of applications programmers; accept that the system will do so as efficiently as possible given the constraints you, the programmer, put on it (such as limiting it to only 5 buffers).

While actual results may vary, I have personally cut run time for a job by 80% (75 minutes to 15 minutes) merely by changing the buffers for each file from the default to something in the 30 to 50 range. Note that doing this may increase region requirements for the program.

rockish · Posted: Tue Sep 08, 2009 5:14 pm

Thanks for your explanation Robert.

I will test my program with different BUFNO values and will publish the most optimal one I was able to use.

dbzTHEdinosauer · Posted: Tue Sep 08, 2009 7:50 pm

Not sure if it applies to your version of cobol, but insure that you are not using the WRITE Immediate option.

Singaram · New User Joined: 19 Aug 2009 Posts: 19 Location: India

Can you please clarify the relation of BUFNO value and the BLKSIZE for QSAM??

Robert Sample · Posted: Wed Sep 09, 2009 12:03 am

BUFNO is the number of buffers used by QSAM for reading (or writing) the file.

BLKSIZE is the physical size read or written -- the size of the data "chunks" (I know, not a very technical term) written to the disk at one time.

The amount of memory required by a program for the file buffers is the number of buffers times the block size; since buffers can now go above the line the performance impact of more buffers is lessened.

Terry Heinze · Posted: Wed Sep 09, 2009 3:08 am

Mr. J, I'm guessing that your guess will be correct: Increasing BUFNO will not significantly affect the EXCP count but you will probably see a slight decrease in elapsed time. Keep in mind that unless you have a dedicated machine, elapsed time is not a good measurement because of a variable job mix. Please let us know the results of your benchmark.