Long running CICS transaction - Debug help

Natarajan R A · New User Joined: 11 Jun 2014 Posts: 12 Location: INDIA

Hi,

I got a long running CICS transaction in production region. From looking into Mainview found out that most of its transaction abends takes a high CPU time and a particular file got high Elaspsed time.

Tried to replicate in test region. But in test region, for the particular day input tested with the corresponding files data from production, all went successfully without long running issue.

Note: In testing, for the programs have added Displays.

As not able to get long running in testing for the given inputs similar to production, not sure what is causing for long running in production.

Could anyone suggest the possibities for long running. So that I would check in that direction too. .

Thanks in advance. .

Rohit Umarjikar · Posted: Wed Jun 11, 2014 4:18 pm

Welcome, As you say, did you check what was holding the file while CICS was trying to access?

Natarajan R A · New User Joined: 11 Jun 2014 Posts: 12 Location: INDIA

I am not having much knowledge on how to check what resource / something holding that file . .

Would I want to approach CICS administrator for checking the usage of that file by other resource at that time . ?

May this, holding of the file by other resource, be one of the possibilities for long running . .

Regards,
Nataraj

Rohit Umarjikar · Posted: Wed Jun 11, 2014 4:48 pm

Yes, please talk to CICS admin and he/she should be able to provide you the information.

Natarajan R A · New User Joined: 11 Jun 2014 Posts: 12 Location: INDIA

From other forums, I came across of adding traces for a particular long running transaction . .
Will it lead to any close of determining the issue . . ?

Is there any option that I could see the resource names that are using the files . ?

Bill O'Boyle · Posted: Wed Jun 11, 2014 6:05 pm

Traces are normally managed and invoked by your CICS Tech Support group.

Is the transaction abending with an "AICA" abend (Runaway Task)?

If so, the Trace may be beneficial as this may pinpoint a looping-logic error between CICS API's or the successive issuing (looping) of the same CICS-API.

Get together with Tech Support and they should be able to assist.

HTH....

Natarajan R A · New User Joined: 11 Jun 2014 Posts: 12 Location: INDIA

We are facing AKC3 abend . .

For this abend would trace be beneficial . . ?

Any suggestions on how to proceed fo this abend . .

Natarajan R A · New User Joined: 11 Jun 2014 Posts: 12 Location: INDIA

As I tested with displays for this long running transaction . . I am not seeing any looping . . so I'm not sure how to proceed further . .

enrico-sorichetti · Posted: Wed Jun 11, 2014 6:27 pm

IIRC ... AKC3 is a TASK purge abend ...
You will have to discover the reason for the purge
operator, CICS, ZNEP ??
the cics log message sequence should tell

look also at the transact dump to determine the abed reason.

Bill O'Boyle · Posted: Wed Jun 11, 2014 6:32 pm

AKC3 indicates that CICS views this transaction as a potential system hog and needs to purge it before the region goes Short-On-Storage. Usually (but not exclusively), the DTIMOUT parameter (FMT=MMSS) defined to this Transaction (PCT definition) governs the time it takes until CICS purges the task.

HTH....

Natarajan R A · New User Joined: 11 Jun 2014 Posts: 12 Location: INDIA

On analysing this long running transaction I started using Dumpmaster . .
But I am not able to figure out the reason for the cause . .

From the dumps . I could see the program and the corresponding CICS statement executed at the time of abend. .
From the transaction info I could see a UserID . Dtimeout = 0000600, SPURGE = YES, runaway limit = 0060000.
Not sure what is dtimeout, Spurge and userid to whom it denotes to .

For checking from the cics log . have I want to look into MSGUSR step . ? let me know on that .

Natarajan R A · New User Joined: 11 Jun 2014 Posts: 12 Location: INDIA

From the dumps . . as the task abended with AKC3, due to purge . it would show CICS statement at the time of the task abended right . ?

From the transaction info I could see a specific users UserID . . So if it was purged by User . at that abeded time only the failed CICS statement would come right . ?

I am not sure on this . Pls guide me if I'm wrong . .

Rohit Umarjikar · Posted: Fri Jun 13, 2014 9:07 pm

Did you talk to CICS Admin as told by us?

Natarajan R A · New User Joined: 11 Jun 2014 Posts: 12 Location: INDIA

Ya Rohit. . As mentioned in earlier post I had a talk with CICS Admin regarding any other resource holding that file while CICS trying to access . . They told 'll check and let me know . .

Also requested them to add trace for that particular transaction. Will it lead to any closer to the problem. . ?

Could anyone guide me where to look for added traces . . In CICS Dumpmaster is where should I check it out . ?

Bill O'Boyle · Posted: Mon Jun 16, 2014 5:05 pm

Have you printed the CICS Transaction Dump? There should be a TRACE in there.

Get your CICS System Programmer to assist you with the JCL to print the Dump as the Dump Utility program is dependent upon your CICS Version/Release. For example, for CICS/TS 4.1, the Utility program is named DFHDU660.

He/She should also be assisting you in solving this error. There really isn't much more that we can suggest.

HTH....

Natarajan R A · New User Joined: 11 Jun 2014 Posts: 12 Location: INDIA

In the long running CICS transaction, I could see the files in process and a particular file got lot of I/O requests and large amount of time spent can be seen.

File name Requests Time
XYZ 1223622 4:20.037

Browses - 376498; Reads - 282377; Updates - 188249; Miscellaneous Req - 376498

Browse time - 0.127296; Read time - 1.190672; Update time - 4:18.247; Misc request time - 0.471968

These details have obtained using omegamon.

While checking for update operations for that long running transaction, I could see Read-update; Rewrite CICS functions.

I am not sure is this update function what may cause for the long running. Like holding any particular record in the file.

Could anyone pls explain or guide on this. Also it would he helpful, in guiding me to check for other alternatives if this may not be the reason for long running.

Traces has been setup and have got a trace file during the long running. But I am not able to read/understand the trace contents in it.

Need some manuals on reading the trace / guidance on the same, so that I could try with that too.

Thanks.

Bill O'Boyle · Posted: Thu Jul 17, 2014 6:08 pm

Here's a helpful Trace Entries PDF link, from our friends at GSE-Nordic -

www.gse-nordic.org/2013/S105%20-%20CICS%20Debug%20and%20Trace.pdf

Note that the default Trace Table Size (defined to the SIT as "TRTABSZ" and is normally 4096K) may not be large enough to capture all the Trace Entries.

You may have to request an increase by speaking with your CICS Technical staff and/or CICS System Programmer(s).

Do you have IBM's Fault Analyzer at your shop?

HTH....

Natarajan R A · New User Joined: 11 Jun 2014 Posts: 12 Location: INDIA

I am still trying to read trace with referrals from the documents with help of Google.

publibz.boulder.ibm.com/cgi-bin/bookmgr_OS390/BOOKS/DFHS6B04/1.1.16?SHELF=ez2dgz50&DT=20080110144427

But still finding it hard to figure it out.

This is the first time I am hearing about Fault analyzer. Could you please explain briefly about the tool . and whom I could ask.

Is it the IBM specialist for my application that I should approach about the usage of the tool.

Regards

Bill O'Boyle · Posted: Fri Jul 18, 2014 4:24 pm

IBM Fault Analyzer for z/OS Tutorial (watch the wraparound) -