Portal | Manuals | References | Downloads | Info | Programs | JCLs | Mainframe wiki | Quick Ref
IBM Mainframe Computers Forums Index
 
Register
 
IBM Mainframe Computers Forums Index Mainframe: Search IBM Mainframe Forum: FAQ Memberlist Profile Log in to check your private messages Log in
 
COBOL Performance Tuning Tips

 
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> COBOL Programming
View previous topic :: :: View next topic  
Author Message
zulfukharali

New User


Joined: 11 May 2006
Posts: 12

PostPosted: Fri Oct 13, 2006 2:48 am    Post subject: COBOL Performance Tuning Tips
Reply with quote

My standalone COBOL program is processing an input file (sequential file) of 50 million records. The program is taking quiet long time an approx of 10 hours to execute it.
The compiler options used are
PARM=('APOST,NUMPROC(MIG),OPT,BUFSIZE(10K)',
'RENT,RES,NOSEQ,TRUNC(OPT),TERM,SSRANGE',
'')

Please suggest me the best way to tune out my program.

Thanks
Zulfi
Back to top
View user's profile Send private message

MFRASHEED

Active User


Joined: 14 Jun 2005
Posts: 186
Location: USA

PostPosted: Fri Oct 13, 2006 3:31 am    Post subject: Re: Performance Tuning COBOL
Reply with quote

Run STROBE on program and check where time is spent during processing.

Search forum for 'STROBE' and you will find information. One just link with documentation is:


http://ibmmainframes.com/viewtopic.php?t=13272&highlight=strobe

Hope this helps.
Back to top
View user's profile Send private message
mmwife

Super Moderator


Joined: 30 May 2003
Posts: 1592

PostPosted: Fri Oct 13, 2006 3:58 am    Post subject:
Reply with quote

Hi Zulfi,

There are many things you might be able to do, but IMMEDIATELY, if not sooner, git rid of the SSRANGE. It's designed for testing, not production and it plays hell w/performance.
Back to top
View user's profile Send private message
DavidatK

Active Member


Joined: 22 Nov 2005
Posts: 700
Location: Troy, Michigan USA

PostPosted: Fri Oct 13, 2006 3:59 am    Post subject: Re: Performance Tuning COBOL
Reply with quote

Zulfi,

A little more information.

-for 10 hours of processing, how much CPU does the program use?
-what other major processing is the program doing for each record?
-how is the input dataset defined? LRECL, BLKSZ?

50 million records doesn't seem like that extreme of an amount. We process 175 million records in less than 30 min, with considerable processing per record.

You can try this on your Input dataset JCl to see if it will help

DCB=BUFNO=50

This will allow the system to read 50 buffers of data each time it accesses the dataset, instead of the site default of probably 1 or 2.

Dave
Back to top
View user's profile Send private message
zulfukharali

New User


Joined: 11 May 2006
Posts: 12

PostPosted: Fri Oct 13, 2006 4:20 am    Post subject: Re: Performance Tuning COBOL
Reply with quote

David,

The input file is defined as LRECL=142 and it is a report pulled out from SAR. Let me try with BUFNO option as u suggested. When compiling i have given the BUFSIZE of 10K and i couldn't see any difference.
Let me try with this option.

Thanks David, mmwife and Rasheed for your quick response.

Zulfi
Back to top
View user's profile Send private message
mmwife

Super Moderator


Joined: 30 May 2003
Posts: 1592

PostPosted: Fri Oct 13, 2006 5:23 am    Post subject:
Reply with quote

Hi Zulfi,

The buffers alloced by the BUFSIZE option are used by the compiler not your pgm, so that won't help or hurt.

And don't foget SSRANGE. It's a killer.
Back to top
View user's profile Send private message
zulfukharali

New User


Joined: 11 May 2006
Posts: 12

PostPosted: Fri Oct 13, 2006 11:26 pm    Post subject: Re: Performance Tuning COBOL
Reply with quote

Hi Jack,

Still it is taking lon time to execute. As you suggested, I had removed SSRANGE options and as per DAVE suggested i had given BUFNO in my input file. But it is not turning out as expected.

The process on each record is it will look for each and every transaction and if it rejected/error/valid the it will pull out the required information and writes into the respective reject/error/valid output files.
It is doing lot of input and output operations on files.

The input file is splitted into various different input files according to requirement and each splitted input file is undergoing the above process of a record.
This is the process happening in the code. Please suggest to help me out in tuning the performance.

Thanks a million.
Regards,
Zulfi
Back to top
View user's profile Send private message
DavidatK

Active Member


Joined: 22 Nov 2005
Posts: 700
Location: Troy, Michigan USA

PostPosted: Sat Oct 14, 2006 12:55 am    Post subject: Re: Performance Tuning COBOL
Reply with quote

Zulfi,

There are really only four broad reasons why a program runs a long time.
1-there is that much work to do, the program is running efficiently, and it is just going to run that long, but from what you describe, 10 hours for 50 million records seems exceptionally long.
2-Your program is waiting on I/O. You are doing excessive read-write operations, possibly on small unblocked records.
3-The program is using excessive CPU, it?s executing unnecessary code.
4-The resources you need are being used by another program. In this case, moving the execution time may solve the problem, or at least reduce it.

Here are a few things you can look into.

If you are reading Keyed VSAM files heavily, 1-read the VSAM file only if the previous read was a different key, otherwise, the data you need is already in the buffer. 2-Look into using the BLSR subsystem to load the VSAM file into memory. (Search the forum for BLSR).

Building on this, you should perform any major function only if the result can be different then the previous time you performed the function. This can be a CPU burner also. i.e. If you moving constants to your output record, move them into the record only once, and follow the procedure described below for ?INITIALIZE?

You can add DCB=BUFNO to all of your sequential input and output files. This will reduce the number of physical I/Os the system performs.

If you are using the ?INITIALIZE? statement for each input, output record, STOP. This is a real CPU burner. The more complex the copybook you are initializing, the worse it is. Look into using ?INITIALIZE? only once per record format, move the initialized record to a WS-HOLD-XXXXX area the same size as the copybook, then whenever you need to do an initialization of the record copy the ?01? level of WS-HOLD-XXXXX to the ?01? level of the copybook.

If this is a DB2 program, look into the SELECT, UPDATE, INSERT, DELETE statements to make sure they are efficient. An inefficient DB2 request can also be a CPU burner and an I/O hog. i.e. If you have timestamps as part of your key

WHERE START-TS <= :WS-PROCESS-TIMESTAMP
AND END-TS >= :WS-PROCESS-TIMESTAMP

Is usually pretty efficient, but

WHERE :WS-PROCESS-TIMESTAMP BETWEEN START-TS AND END-TS

Is a killer. The first example will use the indexes to find your records, the second forces a read of every record in the table, for each request.

MFRASHEED's suggestion is good also, a 'STOBE' of the running program can tell you where the time is being spent, then you can put your efforts in that area

Dave
Back to top
View user's profile Send private message
mmwife

Super Moderator


Joined: 30 May 2003
Posts: 1592

PostPosted: Sat Oct 14, 2006 10:56 pm    Post subject:
Reply with quote

Hi Zulfi,

It's difficult to get a feel for what you're facing w/o seeing the pertinent code.

If you can cut & paste the JCL, FDs and some (not all) of the process code that would explain the overall logic, it would be helpful. If you can't, I understand.

Some other thoughts:

Are you doing table searches, inspects, multiple(many) open/closes of files, using intrinsic functions, etc.?
Back to top
View user's profile Send private message
zulfukharali

New User


Joined: 11 May 2006
Posts: 12

PostPosted: Sun Oct 15, 2006 6:07 am    Post subject: Re: Performance Tuning COBOL
Reply with quote

Hi Jack,

Actually i thought of do g cut and copy the piece of code. I will do it on Monday. I would like to thanks all of you people for the time and effort you spent on this.

The process is for each X, Y and Z, process each transaction. If the transaction has error code not zeroes write into error file. If the transaction is rejected represented by 'R' at some offset, then write into rejected file. If it is a valid (not rejected or error) then write into output file. Once the process is completed for a X, Y and Z, then rewrite the processed records of input file as spaces. So that read records wont process the same X, Y and Z. This should go till end of report or end of file.

All the input-output files are declared as sequential file. I have tried all the suggestions given. But nothing turning out. Hope when i give a sample piece of code and JCL that will help a lot more to understand and to get the solution.
I am not doing any table searches, inspect. But there are many IO operation on files are involved.

Thanks
Zulfi
Back to top
View user's profile Send private message
zulfukharali

New User


Joined: 11 May 2006
Posts: 12

PostPosted: Mon Oct 16, 2006 11:11 pm    Post subject: Re: Performance Tuning COBOL
Reply with quote

Jack,
Please find the piece of code altered for security reasons. This is where the main logic revolves.

Thanks
Zulfi
Back to top
View user's profile Send private message
DavidatK

Active Member


Joined: 22 Nov 2005
Posts: 700
Location: Troy, Michigan USA

PostPosted: Tue Oct 17, 2006 2:05 am    Post subject: Re: Performance Tuning COBOL
Reply with quote

Zulfi,

It's a little difficult to follow exactly what your trying to accomplish, but there is one overwhelming observation, you?re opening/closing files way to many times. This is undoubtedly the root of your performance problem.

In your notes you say that sysin will have only one or two records. Either build a small COBOL table and read the records into that, or set up a couple of WS variables after interrogating sysin. You should never have to re-read sysin.

And the X1 file, I?m not clear why that file is being opened and closed so many times, there must be a better approach that can be taken.

Except for academic exercises, I can?t remember ever writing a program where a file was opened, closed, and re-opened in 38 years of programming (yes, I know, I?m old)

I?d be glad to help with some suggestions, but I need to know exactly what your program requirements are.

Dave
Back to top
View user's profile Send private message
mmwife

Super Moderator


Joined: 30 May 2003
Posts: 1592

PostPosted: Tue Oct 17, 2006 5:38 am    Post subject:
Reply with quote

David is right on about the open/closes. That's why I mentioned it in my post too, but I didn't seriously think your app did that.

I worked on a pgm (actually, I wrote it) that o/ced a temp file created for every account that was billed. The file was again o/ced to read it.

The process generated 80K o/cs. The job ran 3+ hours. I told the project mgr that we should eliminate them but she said no.

I got tired waiting for my tests to end, so I went ahead and used a table to house each set of bill pages instead of a temp file.

The job ran in 10 mins.!!! I knew o/cs were CPU burners, but I had no idea of how much!

So take David's advice!
Back to top
View user's profile Send private message
zulfukharali

New User


Joined: 11 May 2006
Posts: 12

PostPosted: Wed Oct 18, 2006 12:03 am    Post subject: Re: Performance Tuning COBOL
Reply with quote

Hi Jack, Dave and Rasheed.

Thanks a lot for your suggestions and advice. I altered the program avoiding lot of i-o operation of files and it took me only 3 mins to execute.

Once again thanks a million for your advice and thanks for forum.

Thanks and Regards,
Zulfi
Back to top
View user's profile Send private message
DavidatK

Active Member


Joined: 22 Nov 2005
Posts: 700
Location: Troy, Michigan USA

PostPosted: Wed Oct 18, 2006 12:09 am    Post subject: Re: Performance Tuning COBOL
Reply with quote

Below is the explanation for each of the compiler options mentioned above and how they impact the performance:-

Code:
ARITH - EXTEND or COMPAT :
The ARITH compiler option allows you to control the maximum number of digits allowed for numeric variables in your program.ARITH(EXTEND), the maximum number of digits is 31 - Slower. ARITH(COMPAT), the maximum number of digits is 18 - Faster

AWO or NOAWO :
APPLY WRITE-ONLY processing for physical sequential files with VB format.
APPLY WRITE-ONLY , the file buffer is written to the output device when there is notenough space in the buffer for the next record.Without APPLY WRITE-ONLY, the file buffer is written to the output device when there is not enough space in the buffer for the maximum size record.
If the application has a large variation in the size of the records to be written, using APPLY WRITE-ONLY can result in a performance savings since this will generally result in fewer I/O calls.
NOAWO is the default.

DATA(24) or DATA(31) :
Specifies whether reentrant program data areas reside above or below the 16-MB line. With DATA(24) reentrant programs must reside below the 16-MB line. With DATA(31) reentrant programs can reside above the 16-MB line.
DATA(31) is the default.

DYNAM or NODYNAM :
DYNAM ,Changes the behavior of CALL literal statements to load subprograms dynamically at run time. Call path length is longer - slower NODYNAM ,CALL literal statements cause subprograms to be statically link-edited in the load module. Call path length is - faster
NODYNAM is the default.

FASTSRT or NOFASTSRT :
FASTSRT ,Specifies fast sorting by the IBM DFSORT licensed program. - FasterNOFASTSRT ,Specifies that Enterprise COBOL will do SORT or MERGE input/output. - Slower
NOFASTSRT is the default

NUMPROC - NOPFD, MIG, or PFD :
Handles packed/zoned decimal signs as follows:
NUMPROC(NOPFD), sign fix-up processing is done for all references to these numeric data items. NUMPROC(MIG), sign fix-up processing is done only for receiving fields (and not for sendingfields) of arithmetic and MOVE statements. NUMPROC(PFD), the compiler assumes that the data has the correct sign and bypasses this sign fix-up processing.
For performance sensitive applications, NUMPROC(PFD) is recommended when possible.NUMPROC(NOPFD) is the default.

OPTIMIZE(STD), OPTIMIZE(FULL), or NOOPTIMIZE :
Optimizes the object program. OPTIMIZE has the suboptions of (STD/FULL). OPTIMIZE(FULL) provides improved runtime performance, over both the OS/VS COBOL and VS COBOL II OPTIMIZE option, because the compiler discards unused data items and does not generate code for any VALUE clauses for these data items.
NOOPTIMIZE is generally used while a program is being developed when frequent compiles arenecessary. NOOPTIMIZE also makes it easier to debug a program since code is not moved;
NOOPTIMIZE is the default.

RENT or NORENT :
Using the RENT compiler option causes the compiler to generate some additional code to ensure that the program is reentrant.
On the average, RENT was equivalent to NORENT.
RENT is the default.

RMODE - AUTO, 24, or ANY :
Allows NORENT programs to have RMODE(ANY).
When using NORENT, the RMODE option controls where the WORKING-STORAGE will reside.With RMODE(24), the WORKING-STORAGE will be below the 16 MB line. With RMODE(ANY), theWORKING-STORAGE can be above the 16 MB line.
RMODE(AUTO) is the default.

SSRANGE or NOSSRANGE :
SSRANGE - At run time, checks validity of subscript, index, and reference modification references. Its slower than NOSSRANGE.
NOSSRANGE is the default.

TEST or NOTEST :
TEST - Produces object code usable by Debug Tool for the product. It is slower than NOTEST.
NOTEST is the default.

THREAD or NOTHREAD :
THREAD -Enables a COBOL program for execution in a run unit with multiple POSIX threads or PL/I tasks. It is slower than NOTHREAD.
NOTHREAD is the default.

TRUNC - BIN, STD, or OPT :
Truncates final intermediate results.
TRUNC(STD) Truncates numeric fields according to PICTURE specification of the binary receiving field TRUNC(OPT) Truncates numeric fields in the most optimal way TRUNC(BIN) Truncates binary fields based on the storage they occupy
On an average performance analysis -
TRUNC(OPT) > TRUNC(STD) > TRUNC(BIN)
TRUNC(STD) is the default.
Back to top
View user's profile Send private message
mmwife

Super Moderator


Joined: 30 May 2003
Posts: 1592

PostPosted: Wed Oct 18, 2006 7:13 am    Post subject:
Reply with quote

Come on Zulfi, admit it, you're making fun of me. icon_lol.gif
Back to top
View user's profile Send private message
zulfukharali

New User


Joined: 11 May 2006
Posts: 12

PostPosted: Thu Oct 19, 2006 1:28 am    Post subject: Re: Performance Tuning COBOL
Reply with quote

Really Jack,
I couldn't believe the same program which took 10 hrs to execute, it took only 3 mins or less. More number of I-O operation is a real killer.
The operations on the table are too fast than on the files.

Once again thanks a bunch to Jack, Dave and Rasheed.

Thanks
Zulfi
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> COBOL Programming All times are GMT + 6 Hours
Page 1 of 1

 

Search our Forum:

Similar Topics
Topic Author Forum Replies Posted
This topic is locked: you cannot edit posts or make replies. Extract all "IF" Statements... Adarsh Damodaran CLIST & REXX 1 Wed Sep 06, 2017 9:28 am
No new posts ALPHABETIC check in COBOL vidyaa COBOL Programming 8 Thu Aug 17, 2017 7:13 pm
No new posts XML Parse for COBOL 5.2 Bhanu Praveen COBOL Programming 1 Sat Jul 08, 2017 8:58 pm
No new posts Dynamic output file creation in cobol... smileheal COBOL Programming 7 Thu Jun 15, 2017 10:53 pm
No new posts Search utility(3.14) using JCl and Cobol RKS3 COBOL Programming 12 Sat Jun 10, 2017 10:24 pm

Facebook
Back to Top
 
Job Vacancies | Forum Rules | Bookmarks | Subscriptions | FAQ | Polls | Contact Us