Portal | Manuals | References | Downloads | Info | Programs | JCLs | Master the Mainframes
IBM Mainframe Computers Forums Index
 
Register
 
IBM Mainframe Computers Forums Index Mainframe: Search IBM Mainframe Forum: FAQ Memberlist Usergroups Profile Log in to check your private messages Log in
 

 

Job running too long, nearly for 3 hrs.

 
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> JCL & VSAM
View previous topic :: :: View next topic  
Author Message
KumaranJeeva

New User


Joined: 10 Apr 2008
Posts: 7
Location: London

PostPosted: Thu Apr 10, 2008 9:09 pm    Post subject: Job running too long, nearly for 3 hrs.
Reply with quote

Hi,

While running a Mainframe job that executes a program to process records from a table and write into a file, it's taking nearly 3 frs to process around 8,00,000 records for LRECL - 150.
Did any of you face the same situation?
Please help as it's an emergency.
Back to top
View user's profile Send private message

Craq Giegerich

Senior Member


Joined: 19 May 2007
Posts: 1512
Location: Virginia, USA

PostPosted: Thu Apr 10, 2008 9:18 pm    Post subject:
Reply with quote

Some details would make it a lot easier to offer suggestions. JCL program details etc.
Back to top
View user's profile Send private message
KumaranJeeva

New User


Joined: 10 Apr 2008
Posts: 7
Location: London

PostPosted: Thu Apr 10, 2008 9:26 pm    Post subject: Reply to: Job running too long, nearly for 3 hrs.
Reply with quote

A batch program is invoked through this Job. The program reads nearly 8 million records from Database and writes two output files one with volume of 16 million and another with 8 million records.
Please let me know if you need more information.
Back to top
View user's profile Send private message
Phrzby Phil

Active Member


Joined: 31 Oct 2006
Posts: 961
Location: Richmond, Virginia

PostPosted: Thu Apr 10, 2008 10:50 pm    Post subject:
Reply with quote

Could you possibly help someone with this little info?

How, e.g., is the program getting the records from the table? What does the WHERE look like? Got indexes? How's your CPU time?
Back to top
View user's profile Send private message
KumaranJeeva

New User


Joined: 10 Apr 2008
Posts: 7
Location: London

PostPosted: Thu Apr 10, 2008 11:36 pm    Post subject: Reply to: Job running too long, nearly for 3 hrs.
Reply with quote

YEah it's reading from couple of tables with different cursors, processing the data and writing them into two file. The were condition is simple primary key chking with no joins at all for the cursors. The CPU time taken for this jobs is 21 mins only.
I'd be glad to provide any more info needed.
Back to top
View user's profile Send private message
Craq Giegerich

Senior Member


Joined: 19 May 2007
Posts: 1512
Location: Virginia, USA

PostPosted: Thu Apr 10, 2008 11:43 pm    Post subject:
Reply with quote

What are the output files like -- RECFM, LRECL, BLKSIZE, tape or disk?
Back to top
View user's profile Send private message
Craq Giegerich

Senior Member


Joined: 19 May 2007
Posts: 1512
Location: Virginia, USA

PostPosted: Thu Apr 10, 2008 11:48 pm    Post subject:
Reply with quote

Well I hope the job has finished, it has been 3 hours and you still have not given us enough information to answer your emergency request!
Back to top
View user's profile Send private message
dick scherrer

Site Director


Joined: 23 Nov 2006
Posts: 19270
Location: Inside the Matrix

PostPosted: Fri Apr 11, 2008 1:29 am    Post subject:
Reply with quote

Hello,

As has been mentioned, you have provided little or nothing that we can use to help you. . . .

Quote:
it's reading from couple of tables with different cursors, processing the data and writing them into two file.
Might help if you posted the SQL code that defines these.

How many rows are in the 2 tables? It may be far faster to unload the tables and use the sequential files as input to your process.

Has this job run faster previously? Is this the first attempt to run this (with the full volume)?

Might your wall-time be caused by contention within the database due to other tasks using these tables while your job is running?
Back to top
View user's profile Send private message
KumaranJeeva

New User


Joined: 10 Apr 2008
Posts: 7
Location: London

PostPosted: Fri Apr 11, 2008 7:35 pm    Post subject: Reply to: Job running too long, nearly for 3 hrs.
Reply with quote

This is the first attempt to run this job with full volume. The data in one table is around 8 million records and in another is 80,000 records. The unload process and using the sequential file process is not acceptable.
There are two cursors used. The second cursor is inside the first cursor.
As i've mentioned the cursors are based on simple conditions and but fetches more those 8 million records.
I mainly want to know if any of you have faced similar situation.
Back to top
View user's profile Send private message
dick scherrer

Site Director


Joined: 23 Nov 2006
Posts: 19270
Location: Inside the Matrix

PostPosted: Fri Apr 11, 2008 9:38 pm    Post subject:
Reply with quote

Hello,

Quote:
The unload process and using the sequential file process is not acceptable
Who dictated this? What business reason is ther to not do this if the runtime would drop to part of an hour versus multiple hours.

Quote:
I mainly want to know if any of you have faced similar situation.
Yes, regularly, on different systems with different performance issues.

It sounds like you want to "match" data in the 2 tables. I suggest you try the unload and process (and if you need to match the 2 files, use a 2-file match/merge, not some kludge with arrays. . .
Back to top
View user's profile Send private message
KumaranJeeva

New User


Joined: 10 Apr 2008
Posts: 7
Location: London

PostPosted: Fri Apr 11, 2008 9:47 pm    Post subject: Reply to: Job running too long, nearly for 3 hrs.
Reply with quote

The problem is we're not supposed to change the existing job format.
Moreover it's based on keys. I'm not familiar about any flat files which can be handled with keys.
We're not trying to match two tables, we're trying to do a chk before taking the data from the tables and loading into a file.
Back to top
View user's profile Send private message
dick scherrer

Site Director


Joined: 23 Nov 2006
Posts: 19270
Location: Inside the Matrix

PostPosted: Fri Apr 11, 2008 10:09 pm    Post subject:
Reply with quote

Hello,

Quote:
I'm not familiar about any flat files which can be handled with keys.
In a flat file, keys refer to the field(s) the file os sorted by.

Quote:
We're not trying to match two tables, we're trying to do a chk
What kind of check? To make sure something from one table exists or does not exist in the other? That is a match. . .

If you post the nested cursor code, it may help someone make a suggestion.

Quote:
The problem is we're not supposed to change the existing job format.
Typically, i agree with this "rule". However, if something is implemented in such a way as to be unsupportable, the format may need to be changed. What if it ran 40 hours rather than only a few? I suspect the no format change rule would be waived to get the process to run.
Back to top
View user's profile Send private message
Anuj Dhawan

Senior Member


Joined: 22 Apr 2006
Posts: 6258
Location: Mumbai, India

PostPosted: Fri Apr 11, 2008 10:21 pm    Post subject:
Reply with quote

Hi,

If you post a little about how does the program
Quote:
process records from a table and write into a file
, in literal sense the code you are using, it might fetch better suggestions.
Back to top
View user's profile Send private message
Itanium

Active User


Joined: 22 Jan 2006
Posts: 114
Location: India

PostPosted: Sat Apr 12, 2008 9:05 am    Post subject:
Reply with quote

dick,
Hats-off to your patience.

KumaranJeeva,
Please help us to help you !!!

Thanks.
Back to top
View user's profile Send private message
KumaranJeeva

New User


Joined: 10 Apr 2008
Posts: 7
Location: London

PostPosted: Mon Apr 14, 2008 3:53 pm    Post subject: Reply to: Job running too long, nearly for 3 hrs.
Reply with quote

Hi all,

Thanks for all your suggestions. After making a change in the query, the problem is solved now.

Once again thank you very much for your suggestions.
Back to top
View user's profile Send private message
Phrzby Phil

Active Member


Joined: 31 Oct 2006
Posts: 961
Location: Richmond, Virginia

PostPosted: Mon Apr 14, 2008 6:13 pm    Post subject:
Reply with quote

After all of this assistance, wouldn't you like to share what you've learned with the community?
Back to top
View user's profile Send private message
dick scherrer

Site Director


Joined: 23 Nov 2006
Posts: 19270
Location: Inside the Matrix

PostPosted: Mon Apr 14, 2008 7:33 pm    Post subject:
Reply with quote

Hello Kumaran J,

Thank you for posting that your process is now working.

If you can post what you did to improve performance, it will probably help someone else who encounters a similar problem.
Back to top
View user's profile Send private message
KumaranJeeva

New User


Joined: 10 Apr 2008
Posts: 7
Location: London

PostPosted: Mon Apr 14, 2008 7:36 pm    Post subject:
Reply with quote

The change was a code error. There was nothing to do with optimization. That's why i didn't post it here. But whenever a situation arise like this, mostly the culprit will be either in the query and the positioning of the query in the code.
Back to top
View user's profile Send private message
Phrzby Phil

Active Member


Joined: 31 Oct 2006
Posts: 961
Location: Richmond, Virginia

PostPosted: Mon Apr 14, 2008 11:18 pm    Post subject:
Reply with quote

Still - so secretive. An experienced user may not learn from your "code error," but a new user certainly will. How about giving it a try?

If you are embarassed by the error - and who hasn't been? - remember, we really don't know who you are, just who you are pretending to be.
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> JCL & VSAM All times are GMT + 6 Hours
Page 1 of 1

 

Search our Forum:

Similar Topics
Topic Author Forum Replies Posted
No new posts How to move a long alphanumeric data ... lind sh COBOL Programming 6 Mon Dec 05, 2016 7:51 pm
No new posts Problem in Running Query via JCL vickey_dw DB2 3 Tue Oct 18, 2016 11:11 pm
No new posts Syntax for running batch history repo... polymathtarun CA Products 1 Tue Jun 21, 2016 1:51 pm
No new posts DFSORT split long VB lines efficiently BridgetBrackenbury DFSORT/ICETOOL 2 Fri Feb 12, 2016 5:10 am
No new posts Jobs from iseries not running in sequ... lyd All Other Mainframe Topics 4 Mon Jan 18, 2016 1:17 am


Facebook
Back to Top
 
Mainframe Wiki | Forum Rules | Bookmarks | Subscriptions | FAQ | Tutorials | Contact Us