Portal | Manuals | References | Downloads | Info | Programs | JCLs | Master the Mainframes
IBM Mainframe Computers Forums Index
 
Register
 
IBM Mainframe Computers Forums Index Mainframe: Search IBM Mainframe Forum: FAQ Memberlist Usergroups Profile Log in to check your private messages Log in
 

 

Maximum no of checkpoints that can be issued by IMS
Goto page 1, 2  Next
 
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> IMS DB/DC
View previous topic :: :: View next topic  
Author Message
abin

Active User


Joined: 14 Aug 2006
Posts: 198

PostPosted: Thu Sep 27, 2007 5:58 pm    Post subject: Maximum no of checkpoints that can be issued by IMS
Reply with quote

Hi,

Could anybody tell me the maximum no: of checkpoints that can be issued ny IMS.

I am expecting a 8,000,000 no: of checkpoints. Will IMS handle this.

Thanks,

Abin.
Back to top
View user's profile Send private message

Sandy Zimmer

Active Member


Joined: 13 Jun 2007
Posts: 826
Location: Wilmington, DE

PostPosted: Mon Oct 01, 2007 10:13 pm    Post subject:
Reply with quote

Why so many?
Back to top
View user's profile Send private message
Sandy Zimmer

Active Member


Joined: 13 Jun 2007
Posts: 826
Location: Wilmington, DE

PostPosted: Tue Oct 02, 2007 9:46 pm    Post subject:
Reply with quote

You realize that at a sync point (GOBACK, CHKP, or GU to the IO-PCB), IMS flushes the dynamic log file for your program and flushes positioning on all the data bases in your program. You then need to reposition yourself (GU). So, find a logical place to issue checkpoint. There is a lot more that happens, so be very careful.

One of the most wonderful things about IMS is the recovery feature.
Back to top
View user's profile Send private message
Bitneuker

CICS Moderator


Joined: 07 Nov 2005
Posts: 1105
Location: The Netherlands at Hole 19

PostPosted: Wed Oct 03, 2007 9:00 pm    Post subject:
Reply with quote

Quote:
GU to the IO-PCB


When we talk about an MPP checkpoints will be made automatically by IMS.

As or the 8 mio checkpoints; how much segment modifications are you processing against the number of modification records, in what environment (my guess is batch or BMP) and some other info. Is the question academical or a real business process?
Back to top
View user's profile Send private message
Sandy Zimmer

Active Member


Joined: 13 Jun 2007
Posts: 826
Location: Wilmington, DE

PostPosted: Wed Oct 03, 2007 9:15 pm    Post subject:
Reply with quote

Exactly, George! Abin - this must be for a BMP. If you take a checkpoint too often, you will be defeating the purpose. Think about the number of updates you are doing and the backout capibilities of IMS.
Back to top
View user's profile Send private message
Bitneuker

CICS Moderator


Joined: 07 Nov 2005
Posts: 1105
Location: The Netherlands at Hole 19

PostPosted: Wed Oct 03, 2007 9:24 pm    Post subject:
Reply with quote

Well Sandy, let's wait for his requirements. Just posting whether or not IMS will coop with 8 mio checkpoints doesn't bring much. Of course IMS will coop with it but as you also stated; what are the effects on performance and.......are you willing to accept a recovery of 10 minutes or 1 hour? As all laws in ICT count: describe your business case, understand it and before seeking for advice publish it. Without this information a topic is useless. In the early 70-ties I followed a course in which I had to study a book: "The psygoligy (is this english???????? no it can't be) for computer programming". Several chapters described "egoless programming" exposing yourself and your errors to th world and get answers so all would learn. That's what I usally miss in the topics. Just a question without background.
Back to top
View user's profile Send private message
abin

Active User


Joined: 14 Aug 2006
Posts: 198

PostPosted: Mon Oct 08, 2007 11:48 am    Post subject:
Reply with quote

Hi All,

I am very sorry for being so late to reply. I was out of station for a week. Here is the situatuion of my process.

The process is awfully complex which reads near 16 DB's and updates 1 DB per record. It is a BMP. I am issuing 1 checkpoint for 5 records. This is to avoid DB deadlocks which frequented during system testing. 5 record limit is suggested by DBA and it is working fine and so is acceptable.

Why I asked about the maximum no: of checkpoints is because, input for the process is going to be too huge. Is there is any upper limit of checkpoints for IMS.

Most probably design will be changed by reducing no: of input for each processing.

Thanks for all the reply. But my original question about maximum check point is unanswered. Please let me know if any body knows about such an upper limit. If there is an upper limit, knowledge may help me in my design of process.

Please let me know if any more clarrification is needed.

Thanks and Regards,
Abin.
Back to top
View user's profile Send private message
Bitneuker

CICS Moderator


Joined: 07 Nov 2005
Posts: 1105
Location: The Netherlands at Hole 19

PostPosted: Mon Oct 08, 2007 4:01 pm    Post subject:
Reply with quote

Did some info research.......given 8 mio inputrecords and a checkpoint every 5th record.

Quote:
You can issue a checkpoint every 1/10th of a second (because of the granularity of the timestamp in the log record - that may have changed in current releases). As long as the chkpt id is unique (you have 8bytes to use so that's a 64bit number) you can issue as many as you like.

With 8 million records, one chkpt every five records is 1.6million tenths of a second, which is 160,000 seconds, 2667minutes, 44.4hours to run it.

You really don't need to checkpoint that frequently. It will produce a massive shed-load of archived SLDS if you run that way.

I'll lend you my baseball bat so that you can educate the DBA in the error of his/her ways icon_wink.gif
Back to top
View user's profile Send private message
abin

Active User


Joined: 14 Aug 2006
Posts: 198

PostPosted: Mon Oct 08, 2007 5:38 pm    Post subject:
Reply with quote

Hi George ,

Thanks for the info. Quote:
Quote:
You can issue a checkpoint every 1/10th of a second (because of the granularity of the timestamp in the log record - that may have changed in current releases).
.

My code issues an average of 10 check pints per second. As for DBA, I think they calculated to the utmost. I was there when the system was tested for checkpoint decision. It was a hell seeing the process breaks down for checkpoint upto 10. Main problem there are many other jobs which are not expendable running at same time using same DB's.

Thanks,
Abin.
Back to top
View user's profile Send private message
Sandy Zimmer

Active Member


Joined: 13 Jun 2007
Posts: 826
Location: Wilmington, DE

PostPosted: Mon Oct 08, 2007 5:59 pm    Post subject:
Reply with quote

Abin, Do you databases ever come offline? There are ways to do updates that do not require checkpoint restart.
Back to top
View user's profile Send private message
Bitneuker

CICS Moderator


Joined: 07 Nov 2005
Posts: 1105
Location: The Netherlands at Hole 19

PostPosted: Mon Oct 08, 2007 6:04 pm    Post subject:
Reply with quote

Anothe warning i.e regarding online:


Quote:
IMS can handle high checkpoint rates; What is the determining factor here is the size of the checkpoint record, the number and size of the OLDS and the message queue space allocations.

For each symbollic checkpoint you will get at number of log records depending on what the BMP is doing and whether or not it is going against an external subsystem. With respect to the checkpoint records, each symbollic checkpoint creates 2 log records: a type x'41' record and a type x'18' record.
In reality you will get 1 to n type x'18' records. The type x'18' record breaks down each checkpoint into 1K increments; depending on the size of the checkpoint record. Thus an 8K checkpoint record will produce 8 type x'18'
records and 1 type x'41' log record.

The larger the size of the checkpoint record, the larger the number of type x'18' log records per checkpoint and the longer it will take to write the checkpoint record out to the OLDS. So if your checkpoint record size is high, your system becomes I/O bound.

If this occurs this also means that IMS will not be able to keep up with online activity so your online transactions will begin to queue up. Depending on how high your online activity is and your allocations, you will either fill up a message queue or you will run out of available OLDS. Unless there is some automated system to detect this and force the BMP to abend, the DBA has ensured themself some additional work under very adverse conditions.


and maybe helpfull:

Quote:
BMC Software is providing a product which is streamlining the number of issued checkpoint calls to a more appropriate frequency (let's say max every 2 secs or every 5 secs or every 100th issued call or ....), ... controlled from outside the application.
Back to top
View user's profile Send private message
abin

Active User


Joined: 14 Aug 2006
Posts: 198

PostPosted: Mon Oct 08, 2007 6:05 pm    Post subject:
Reply with quote

Hi Sandy,

Are you suggesting DL/I Batch mode. Any way answer is no. Because DBA wont allow code without checkpoint. DB's comes offline a few hours in a fortnight for some repairs.

Thanks,
Abin.
Back to top
View user's profile Send private message
Sandy Zimmer

Active Member


Joined: 13 Jun 2007
Posts: 826
Location: Wilmington, DE

PostPosted: Mon Oct 08, 2007 6:06 pm    Post subject:
Reply with quote

Are these databases being updated online at the same time that this BMP process is running?
Back to top
View user's profile Send private message
abin

Active User


Joined: 14 Aug 2006
Posts: 198

PostPosted: Mon Oct 08, 2007 6:10 pm    Post subject:
Reply with quote

Yes Sandy thats correct.

regards,
Abin.
Back to top
View user's profile Send private message
Sandy Zimmer

Active Member


Joined: 13 Jun 2007
Posts: 826
Location: Wilmington, DE

PostPosted: Mon Oct 08, 2007 6:10 pm    Post subject:
Reply with quote

I wonder if your DBA does image copies? Tell him to make an extra and run in DLI mode against the image copy - then do a restore. Maybe he should have thought of something like this?
Back to top
View user's profile Send private message
Bitneuker

CICS Moderator


Joined: 07 Nov 2005
Posts: 1105
Location: The Netherlands at Hole 19

PostPosted: Mon Oct 08, 2007 6:12 pm    Post subject:
Reply with quote

Talk again to your DBA or maybe Sandy can help you to redesign.......

Quote:
You can issue any number of checkpoints from a BMP. You would probably want to make the CHKPID field a hex number so you could keep it straight, but I know of no limit to the total number of checkpoint that a single BMP could take.

Now, to add to the overwhelming sentiment of everyone else on this list, this is NOT a good idea. The overhead alone of taking all of these checkpoints will greatly exceed the total processing requirement for the BMP. Each checkpoint will cause the database buffers to be flushed. In addition, if there are any GSAM files, the files will be temporarily closed and reopened as a result of each checkpoint. It grows even worse if there is some DB2 in the mix because of all of the related DB2 commit activity which can negate a lot of the things that DB2 does for performance.

My experience with BMP performance indicates that the run time and resource consumption will go up exponentially when you introduce a very short checkpoint interval as suggested here.

Back to top
View user's profile Send private message
abin

Active User


Joined: 14 Aug 2006
Posts: 198

PostPosted: Mon Oct 08, 2007 6:21 pm    Post subject:
Reply with quote

I'm afraid they wont allow us to run in DLI mode even if on an image as I said before. Backing up of the DB's daily invloves huge overload. This system is really huge.

Thanks for your suggestion.

Regards,
Abin.
Back to top
View user's profile Send private message
Bitneuker

CICS Moderator


Joined: 07 Nov 2005
Posts: 1105
Location: The Netherlands at Hole 19

PostPosted: Mon Oct 08, 2007 6:29 pm    Post subject:
Reply with quote

Quote:
One BMP in particular was issuing 60 Checkpoints per second instead of 1. The elapsed time of the BMPs was
11+ minutes. The Checkpoint frequency was changed to 1 Checkpoint per
second
and the elapsed time of the same job was reduced to 17 seconds.


So it seems all there's left is to talk to your DBA again and point out what will happen if they insist on that frequency of checkpointing.
Back to top
View user's profile Send private message
Sandy Zimmer

Active Member


Joined: 13 Jun 2007
Posts: 826
Location: Wilmington, DE

PostPosted: Mon Oct 08, 2007 6:39 pm    Post subject:
Reply with quote

What kind of system? What were the requirements? Why so much input and where is it coming from?
Back to top
View user's profile Send private message
Bitneuker

CICS Moderator


Joined: 07 Nov 2005
Posts: 1105
Location: The Netherlands at Hole 19

PostPosted: Mon Oct 08, 2007 6:46 pm    Post subject:
Reply with quote

Quote:
Thanks for all the reply. But my original question about maximum check point is unanswered. Please let me know if any body knows about such an upper limit. If there is an upper limit, knowledge may help me in my design of process.


Well, as you may have read in the quotes there's no limit (watch the length of the checkpoint-id). And if you definitely want your checkpoint per 5 records: case close icon_question.gif
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> IMS DB/DC All times are GMT + 6 Hours
Goto page 1, 2  Next
Page 1 of 2

 

Search our Forum:

Similar Topics
Topic Author Forum Replies Posted
No new posts What is the maximum number of sort wo... Pravina M SYNCSORT 2 Mon Mar 14, 2016 4:51 pm
No new posts CEE3250C The system or user abend AEY... kishpra CICS 3 Fri Aug 07, 2015 1:01 pm
No new posts S0C7 issued on a CALL statement RedDevil711 ABENDS & Debugging 8 Sun Feb 01, 2015 3:57 pm
No new posts How to find the maximum value of a pa... Susheel singh IMS DB/DC 8 Mon Mar 24, 2014 11:39 am
No new posts List of console commands issued? vasanthz All Other Mainframe Topics 9 Fri Feb 21, 2014 11:45 pm


Facebook
Back to Top
 
Mainframe Wiki | Forum Rules | Bookmarks | Subscriptions | FAQ | Tutorials | Contact Us