IBM Mainframe Forum Index
 
Log In
 
IBM Mainframe Forum Index Mainframe: Search IBM Mainframe Forum: FAQ Register
 

Maximum no of checkpoints that can be issued by IMS


IBM Mainframe Forums -> IMS DB/DC
Post new topic   Reply to topic
View previous topic :: View next topic  
Author Message
abin

Active User


Joined: 14 Aug 2006
Posts: 198

PostPosted: Thu Sep 27, 2007 5:58 pm
Reply with quote

Hi,

Could anybody tell me the maximum no: of checkpoints that can be issued ny IMS.

I am expecting a 8,000,000 no: of checkpoints. Will IMS handle this.

Thanks,

Abin.
Back to top
View user's profile Send private message
Sandy Zimmer

Active Member


Joined: 13 Jun 2007
Posts: 826
Location: Wilmington, DE

PostPosted: Mon Oct 01, 2007 10:13 pm
Reply with quote

Why so many?
Back to top
View user's profile Send private message
Sandy Zimmer

Active Member


Joined: 13 Jun 2007
Posts: 826
Location: Wilmington, DE

PostPosted: Tue Oct 02, 2007 9:46 pm
Reply with quote

You realize that at a sync point (GOBACK, CHKP, or GU to the IO-PCB), IMS flushes the dynamic log file for your program and flushes positioning on all the data bases in your program. You then need to reposition yourself (GU). So, find a logical place to issue checkpoint. There is a lot more that happens, so be very careful.

One of the most wonderful things about IMS is the recovery feature.
Back to top
View user's profile Send private message
Bitneuker

CICS Moderator


Joined: 07 Nov 2005
Posts: 1104
Location: The Netherlands at Hole 19

PostPosted: Wed Oct 03, 2007 9:00 pm
Reply with quote

Quote:
GU to the IO-PCB


When we talk about an MPP checkpoints will be made automatically by IMS.

As or the 8 mio checkpoints; how much segment modifications are you processing against the number of modification records, in what environment (my guess is batch or BMP) and some other info. Is the question academical or a real business process?
Back to top
View user's profile Send private message
Sandy Zimmer

Active Member


Joined: 13 Jun 2007
Posts: 826
Location: Wilmington, DE

PostPosted: Wed Oct 03, 2007 9:15 pm
Reply with quote

Exactly, George! Abin - this must be for a BMP. If you take a checkpoint too often, you will be defeating the purpose. Think about the number of updates you are doing and the backout capibilities of IMS.
Back to top
View user's profile Send private message
Bitneuker

CICS Moderator


Joined: 07 Nov 2005
Posts: 1104
Location: The Netherlands at Hole 19

PostPosted: Wed Oct 03, 2007 9:24 pm
Reply with quote

Well Sandy, let's wait for his requirements. Just posting whether or not IMS will coop with 8 mio checkpoints doesn't bring much. Of course IMS will coop with it but as you also stated; what are the effects on performance and.......are you willing to accept a recovery of 10 minutes or 1 hour? As all laws in ICT count: describe your business case, understand it and before seeking for advice publish it. Without this information a topic is useless. In the early 70-ties I followed a course in which I had to study a book: "The psygoligy (is this english???????? no it can't be) for computer programming". Several chapters described "egoless programming" exposing yourself and your errors to th world and get answers so all would learn. That's what I usally miss in the topics. Just a question without background.
Back to top
View user's profile Send private message
abin

Active User


Joined: 14 Aug 2006
Posts: 198

PostPosted: Mon Oct 08, 2007 11:48 am
Reply with quote

Hi All,

I am very sorry for being so late to reply. I was out of station for a week. Here is the situatuion of my process.

The process is awfully complex which reads near 16 DB's and updates 1 DB per record. It is a BMP. I am issuing 1 checkpoint for 5 records. This is to avoid DB deadlocks which frequented during system testing. 5 record limit is suggested by DBA and it is working fine and so is acceptable.

Why I asked about the maximum no: of checkpoints is because, input for the process is going to be too huge. Is there is any upper limit of checkpoints for IMS.

Most probably design will be changed by reducing no: of input for each processing.

Thanks for all the reply. But my original question about maximum check point is unanswered. Please let me know if any body knows about such an upper limit. If there is an upper limit, knowledge may help me in my design of process.

Please let me know if any more clarrification is needed.

Thanks and Regards,
Abin.
Back to top
View user's profile Send private message
Bitneuker

CICS Moderator


Joined: 07 Nov 2005
Posts: 1104
Location: The Netherlands at Hole 19

PostPosted: Mon Oct 08, 2007 4:01 pm
Reply with quote

Did some info research.......given 8 mio inputrecords and a checkpoint every 5th record.

Quote:
You can issue a checkpoint every 1/10th of a second (because of the granularity of the timestamp in the log record - that may have changed in current releases). As long as the chkpt id is unique (you have 8bytes to use so that's a 64bit number) you can issue as many as you like.

With 8 million records, one chkpt every five records is 1.6million tenths of a second, which is 160,000 seconds, 2667minutes, 44.4hours to run it.

You really don't need to checkpoint that frequently. It will produce a massive shed-load of archived SLDS if you run that way.

I'll lend you my baseball bat so that you can educate the DBA in the error of his/her ways icon_wink.gif
Back to top
View user's profile Send private message
abin

Active User


Joined: 14 Aug 2006
Posts: 198

PostPosted: Mon Oct 08, 2007 5:38 pm
Reply with quote

Hi George ,

Thanks for the info. Quote:
Quote:
You can issue a checkpoint every 1/10th of a second (because of the granularity of the timestamp in the log record - that may have changed in current releases).
.

My code issues an average of 10 check pints per second. As for DBA, I think they calculated to the utmost. I was there when the system was tested for checkpoint decision. It was a hell seeing the process breaks down for checkpoint upto 10. Main problem there are many other jobs which are not expendable running at same time using same DB's.

Thanks,
Abin.
Back to top
View user's profile Send private message
Sandy Zimmer

Active Member


Joined: 13 Jun 2007
Posts: 826
Location: Wilmington, DE

PostPosted: Mon Oct 08, 2007 5:59 pm
Reply with quote

Abin, Do you databases ever come offline? There are ways to do updates that do not require checkpoint restart.
Back to top
View user's profile Send private message
Bitneuker

CICS Moderator


Joined: 07 Nov 2005
Posts: 1104
Location: The Netherlands at Hole 19

PostPosted: Mon Oct 08, 2007 6:04 pm
Reply with quote

Anothe warning i.e regarding online:


Quote:
IMS can handle high checkpoint rates; What is the determining factor here is the size of the checkpoint record, the number and size of the OLDS and the message queue space allocations.

For each symbollic checkpoint you will get at number of log records depending on what the BMP is doing and whether or not it is going against an external subsystem. With respect to the checkpoint records, each symbollic checkpoint creates 2 log records: a type x'41' record and a type x'18' record.
In reality you will get 1 to n type x'18' records. The type x'18' record breaks down each checkpoint into 1K increments; depending on the size of the checkpoint record. Thus an 8K checkpoint record will produce 8 type x'18'
records and 1 type x'41' log record.

The larger the size of the checkpoint record, the larger the number of type x'18' log records per checkpoint and the longer it will take to write the checkpoint record out to the OLDS. So if your checkpoint record size is high, your system becomes I/O bound.

If this occurs this also means that IMS will not be able to keep up with online activity so your online transactions will begin to queue up. Depending on how high your online activity is and your allocations, you will either fill up a message queue or you will run out of available OLDS. Unless there is some automated system to detect this and force the BMP to abend, the DBA has ensured themself some additional work under very adverse conditions.


and maybe helpfull:

Quote:
BMC Software is providing a product which is streamlining the number of issued checkpoint calls to a more appropriate frequency (let's say max every 2 secs or every 5 secs or every 100th issued call or ....), ... controlled from outside the application.
Back to top
View user's profile Send private message
abin

Active User


Joined: 14 Aug 2006
Posts: 198

PostPosted: Mon Oct 08, 2007 6:05 pm
Reply with quote

Hi Sandy,

Are you suggesting DL/I Batch mode. Any way answer is no. Because DBA wont allow code without checkpoint. DB's comes offline a few hours in a fortnight for some repairs.

Thanks,
Abin.
Back to top
View user's profile Send private message
Sandy Zimmer

Active Member


Joined: 13 Jun 2007
Posts: 826
Location: Wilmington, DE

PostPosted: Mon Oct 08, 2007 6:06 pm
Reply with quote

Are these databases being updated online at the same time that this BMP process is running?
Back to top
View user's profile Send private message
abin

Active User


Joined: 14 Aug 2006
Posts: 198

PostPosted: Mon Oct 08, 2007 6:10 pm
Reply with quote

Yes Sandy thats correct.

regards,
Abin.
Back to top
View user's profile Send private message
Sandy Zimmer

Active Member


Joined: 13 Jun 2007
Posts: 826
Location: Wilmington, DE

PostPosted: Mon Oct 08, 2007 6:10 pm
Reply with quote

I wonder if your DBA does image copies? Tell him to make an extra and run in DLI mode against the image copy - then do a restore. Maybe he should have thought of something like this?
Back to top
View user's profile Send private message
Bitneuker

CICS Moderator


Joined: 07 Nov 2005
Posts: 1104
Location: The Netherlands at Hole 19

PostPosted: Mon Oct 08, 2007 6:12 pm
Reply with quote

Talk again to your DBA or maybe Sandy can help you to redesign.......

Quote:
You can issue any number of checkpoints from a BMP. You would probably want to make the CHKPID field a hex number so you could keep it straight, but I know of no limit to the total number of checkpoint that a single BMP could take.

Now, to add to the overwhelming sentiment of everyone else on this list, this is NOT a good idea. The overhead alone of taking all of these checkpoints will greatly exceed the total processing requirement for the BMP. Each checkpoint will cause the database buffers to be flushed. In addition, if there are any GSAM files, the files will be temporarily closed and reopened as a result of each checkpoint. It grows even worse if there is some DB2 in the mix because of all of the related DB2 commit activity which can negate a lot of the things that DB2 does for performance.

My experience with BMP performance indicates that the run time and resource consumption will go up exponentially when you introduce a very short checkpoint interval as suggested here.

Back to top
View user's profile Send private message
abin

Active User


Joined: 14 Aug 2006
Posts: 198

PostPosted: Mon Oct 08, 2007 6:21 pm
Reply with quote

I'm afraid they wont allow us to run in DLI mode even if on an image as I said before. Backing up of the DB's daily invloves huge overload. This system is really huge.

Thanks for your suggestion.

Regards,
Abin.
Back to top
View user's profile Send private message
Bitneuker

CICS Moderator


Joined: 07 Nov 2005
Posts: 1104
Location: The Netherlands at Hole 19

PostPosted: Mon Oct 08, 2007 6:29 pm
Reply with quote

Quote:
One BMP in particular was issuing 60 Checkpoints per second instead of 1. The elapsed time of the BMPs was
11+ minutes. The Checkpoint frequency was changed to 1 Checkpoint per
second
and the elapsed time of the same job was reduced to 17 seconds.


So it seems all there's left is to talk to your DBA again and point out what will happen if they insist on that frequency of checkpointing.
Back to top
View user's profile Send private message
Sandy Zimmer

Active Member


Joined: 13 Jun 2007
Posts: 826
Location: Wilmington, DE

PostPosted: Mon Oct 08, 2007 6:39 pm
Reply with quote

What kind of system? What were the requirements? Why so much input and where is it coming from?
Back to top
View user's profile Send private message
Bitneuker

CICS Moderator


Joined: 07 Nov 2005
Posts: 1104
Location: The Netherlands at Hole 19

PostPosted: Mon Oct 08, 2007 6:46 pm
Reply with quote

Quote:
Thanks for all the reply. But my original question about maximum check point is unanswered. Please let me know if any body knows about such an upper limit. If there is an upper limit, knowledge may help me in my design of process.


Well, as you may have read in the quotes there's no limit (watch the length of the checkpoint-id). And if you definitely want your checkpoint per 5 records: case close icon_question.gif
Back to top
View user's profile Send private message
Sandy Zimmer

Active Member


Joined: 13 Jun 2007
Posts: 826
Location: Wilmington, DE

PostPosted: Tue Oct 09, 2007 2:40 am
Reply with quote

abin, I will not give up on your problem. I have thought about it all day - intermittendly - icon_wink.gif You need to point your DBA to this forum or at least copy the QUOTES that Bitneuker posted. These DBA people come from all over the world and are top-notch at what they do. I know some of them thru reputation. They really do know what they are doing. Please learn what IMS can and cannot do - just because you can do it, does not make it good. It is very-very powerful and these people have been doing this for years. You really need to know IMS to design a system. I will help with whatever you need.
Back to top
View user's profile Send private message
abin

Active User


Joined: 14 Aug 2006
Posts: 198

PostPosted: Tue Oct 09, 2007 10:07 am
Reply with quote

Hello Sandy,

Thanks for your kindfull regard to my problem icon_biggrin.gif . As I pointed out previously current checkpoint level is working without any adversary effect on the system. To note out current process works with an input of 250K records at maximum. So, checkpoint limit was not a big issue here.

I was seeking out for a solution which we were proposing to implement for future. The program with checkpoint I was talking about was actually just one code, though a bigger one in the entire process. For some time being we will stick to the old proccess. I sure will take up knowledge we discussed to DBA if the alternate with 80 mio comes up. I have already posted this as an issue to the people concerned.

Quote:
What kind of system?
IMS DB/DC. Handles both online and batch systems. There are an awfull lot of code running at same time.
Quote:
What were the requirements?
icon_neutral.gif
Quote:
Why so much input and where is it coming from
? Batch processing of already accumulated records via online.

Quote:
Well, as you may have read in the quotes there's no limit (watch the length of the checkpoint-id). And if you definitely want your checkpoint per 5 records: case close

Yes George and Thanks. If the new system comes up as a real one then I will certainly reopen the case icon_smile.gif .

Thanks,
Abin.
Back to top
View user's profile Send private message
abin

Active User


Joined: 14 Aug 2006
Posts: 198

PostPosted: Tue Oct 09, 2007 10:24 am
Reply with quote

And yes If new system comes up. I sure will press them to increase the checkpoint count. This infact involves a huge amount of controlled system testing in production.

Thanks,
Abin.
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic View Bookmarks
All times are GMT + 6 Hours
Forum Index -> IMS DB/DC

 


Similar Topics
Topic Forum Replies
No new posts SORT ERROR PARAMETER VALUE EXCEEDS M... DFSORT/ICETOOL 12
No new posts Issued a CMD call for /DIS & it r... IMS DB/DC 3
No new posts Increase the Maximum Length of LRECL ... JCL & VSAM 5
No new posts Logic to skip Maximum time-stamp and ... DB2 2
No new posts What is the maximum number of sort wo... SYNCSORT 2
Search our Forums:

Back to Top