IBM Mainframe Forum Index
 
Log In
 
IBM Mainframe Forum Index Mainframe: Search IBM Mainframe Forum: FAQ Register
 

Performance Issue - COBOL - DB2 & Memory


IBM Mainframe Forums -> COBOL Programming
Post new topic   Reply to topic
View previous topic :: View next topic  
Author Message
santosh.iim

New User


Joined: 20 Apr 2006
Posts: 9

PostPosted: Mon Oct 26, 2009 4:29 pm
Reply with quote

We were trying to optimize Database reads and in the process found that there was a unnecessary read to a table. This is actually a duplicate read of the same table for the same key.

The first read happens in a module A. The Duplicate read is happening in a module B.

A calls C, C calls D, D calls E, E calls B.

we need data of A into Module B, So we tried fetching data in A and Passed it over through to B via Pointers/Assembler Tokens.

A and B are above the line modules.

While earlier with Duplicate reads to the same table, it took 12 minutes. Currently with the above mentioned procedure, it's taking 19 mins with CPU time also being doubled.

We were of the impression that reducing I/O operations is always good for performance of the Job. Not able to make out of this anamoly[/b]
Back to top
View user's profile Send private message
Robert Sample

Global Moderator


Joined: 06 Jun 2008
Posts: 8696
Location: Dubuque, Iowa, USA

PostPosted: Mon Oct 26, 2009 4:56 pm
Reply with quote

Quote:
We were of the impression that reducing I/O operations is always good for performance of the Job
This is completely the wrong impression. There are two main performance issues that can occur -- a job can be I/O-bound (that is, the job is waiting for I/O to complete), or it can be CPU-bound (that is, the job is waiting for CPU processing to complete). Knowing which your job is experiencing is critical to any performance changes. For example, if the job is CPU-bound, you could cut out ALL I/O and the job's CPU time and run time would go down very little if any at all. And vice-versa, if the job is I/O-bound, changing the program to reduce the time to process data will have little (if any) impact on run time.

I suspect you may have a CPU-bound process (although it would take a lot more data to be convincing). If so, reducing one I/O would not reduce the CPU or run time and could well increase it. Doing thorough analysis requires tools such as STROBE or APA to be able to delve into the executing code.
Back to top
View user's profile Send private message
GuyC

Senior Member


Joined: 11 Aug 2009
Posts: 1281
Location: Belgium

PostPosted: Mon Oct 26, 2009 5:30 pm
Reply with quote

Reading two times the same ROW won't (most of the time) make your process have more I/O's.
It probably will give you more getpages and getpages cost CPU, Unnecessary calls to DB2 also cost CPU.
Thus avoiding a SELECT or FETCH will reduce CPU.
If the process needed to avoid the Select costs more than the CPU gained you could end up with more CPU than before.

Where, in this case, the higher CPU comes from, I have no idea.
Back to top
View user's profile Send private message
Steve Davies

New User


Joined: 15 Oct 2009
Posts: 32
Location: UK

PostPosted: Mon Oct 26, 2009 7:38 pm
Reply with quote

More really needs to be know about how you are doing this. If you get your data in module A, you then have your data, you set a pointer to the address of the data retrieved by module A. This pointer must then be passed down the chain of modules (through the Linkage) until it is to be processed by module B, so module B can set a working storage record to the address of the pointer, and thereby access the data passed from module A without the need to go to the database and retrieve it again.

In theory this sounds OK, as you are just passing a pointer value down until it is needed. So why so much extra CPU then? How are you doing this? How many times does module B use the pointer passed down from module A?

I totally agree that STROBE or APA would be a good tool to use to help you find out what's going on. You could do a run before and after your change so you can compare and see what's causing the extra CPU usage.

As you may have guessed by now I have no answers for you, but it sounds like there could be something wrong with the way you are doing this. I'd be interested to know.
Back to top
View user's profile Send private message
dick scherrer

Moderator Emeritus


Joined: 23 Nov 2006
Posts: 19244
Location: Inside the Matrix

PostPosted: Mon Oct 26, 2009 8:45 pm
Reply with quote

Hello and welcome to the forum,

Quote:
While earlier with Duplicate reads to the same table, it took 12 minutes. Currently with the above mentioned procedure, it's taking 19 mins with CPU time also being doubled.
This is known as "snatching defeat from the jaws of victory" or "reverse tuning". . . icon_smile.gif

Quote:
We were of the impression that reducing I/O operations is always good for performance of the Job.
Often, reducing i/o improves performance because not only are the i/o's not done, neither is the unneeded duplicate processing.

In order to determine where to "tune" you need to make sure it is understood where the resources are currently being used. From the very little posted, we most likely cannot contribute anything specific yet. Does the new process repeatedly take 19 minutes or was there only one test run?
Back to top
View user's profile Send private message
santosh.iim

New User


Joined: 20 Apr 2006
Posts: 9

PostPosted: Mon Oct 26, 2009 11:52 pm
Reply with quote

This is my first post and i can't believe the number of views and replies i got.
Thank you all.
I wasn't sure of the replies and hence put the question a little vague. Sorry for this. Please find some more details to it.

Before
CPU TIME = 2.49 TOTAL TIME = 12.9 minutes

After
CPU TIME = 4.45 TOTAL TIME = 19.8 minutes

Our intention was to cut down CPU time and not Total time. and as a part of it we may taken up optimization of one such Job.

The Job runs on almost entire database in production. We replicated the same in test regions with fewer data ofcourse. The above statistics were obtained were almost same in about 10 times that we ran the Job.
So, the Statistics are not of one run.

Since this would run on entire database, one such removal of duplicate read in test region almost saved us from fetching 1,00,000 records from one such table. This we thought would help us beat CPU time which turned out otherwise.

Also, now we had to save all this 1,00,000 record data into a Working storage area and pass through to required module. Each record of this has around 15 bytes of data. Could this have been an issue? if so what best can we do to cut this out..?

Please let me know if more details are required..


P.S: we will be running strobe tomorrow. I will post them once we are done.

Thank you
Back to top
View user's profile Send private message
santosh.iim

New User


Joined: 20 Apr 2006
Posts: 9

PostPosted: Mon Oct 26, 2009 11:57 pm
Reply with quote

As from the test data, I can safely till that Module A would lead down to Module B almost 6400 times in one run. This is based on a column in some other table.

Would passing pointer depend on number of times it's called and number of modules it's passed through.
Back to top
View user's profile Send private message
dbzTHEdinosauer

Global Moderator


Joined: 20 Oct 2006
Posts: 6966
Location: porcelain throne

PostPosted: Tue Oct 27, 2009 12:55 am
Reply with quote

are you processing this 'passed 1,000,000 items' as a COBOL internal table?

why are you collecting all the db2 rows/columns in a table at one time?
understanding your process at this point will help us.
Back to top
View user's profile Send private message
santosh.iim

New User


Joined: 20 Apr 2006
Posts: 9

PostPosted: Tue Oct 27, 2009 1:53 pm
Reply with quote

Yes, These are being passed as COBOL Internal table...The idea of collecting DB2 rows into the COBOL table is to prevent duplicate read of DB2 Table.
Back to top
View user's profile Send private message
dick scherrer

Moderator Emeritus


Joined: 23 Nov 2006
Posts: 19244
Location: Inside the Matrix

PostPosted: Tue Oct 27, 2009 7:22 pm
Reply with quote

Hello,

So instead of reading some data again, the process now reads the data, stores the data in an array (using more cpu) and then in the called process re-process the data from the array (again more cpu). . .

Quote:
The idea of collecting DB2 rows into the COBOL table is to prevent duplicate read of DB2 Table.
If the query was such that it ran a long, long time to return only a few rows, this might help, but if the query directly reads only those rows needed to process, building and using the array may well cost more. "Snatching defeat. . .".

Why is "all" of the data needed 6,400 times (something to think about)?

Suggest someone review the entire process (not just some individual pieces) to determine if the approach will work well for high volume. Testing with only a few rows may show no problems (they exist, but may be nearly invisible), but when the "real" load is processed, it takes too many resources.
Back to top
View user's profile Send private message
Terry Heinze

JCL Moderator


Joined: 14 Jul 2008
Posts: 1249
Location: Richfield, MN, USA

PostPosted: Tue Oct 27, 2009 8:12 pm
Reply with quote

Quote:
I wasn't sure of the replies and hence put the question a little vague.
Please don't do this. It results in a lot of people guessing at solutions and wasting a lot of time. Post as much information as you can about your requirement/problem at first.
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic View Bookmarks
All times are GMT + 6 Hours
Forum Index -> COBOL Programming

 


Similar Topics
Topic Forum Replies
No new posts Replace each space in cobol string wi... COBOL Programming 2
No new posts COBOL -Linkage Section-Case Sensitive COBOL Programming 1
No new posts SFTP Issue - destination file record ... All Other Mainframe Topics 2
No new posts COBOL ZOS Web Enablement Toolkit HTTP... COBOL Programming 0
No new posts Calling DFSORT from Cobol, using OUTF... DFSORT/ICETOOL 5
Search our Forums:

Back to Top