Will massive file definition impact CICS performance?

dejunzhu · Active User Joined: 08 May 2008 Posts: 390 Location: China

let's say we have over 30,000 tellers in my shop, and I want to define a ESDS file which is used to store log records for each teller, so, over 30,000 ESDS file will be defined into CICS.

I'm concerning about whether this will give impact to CICS performance?

Robert Sample · Posted: Fri Jan 11, 2013 8:47 am

From the CICS V3.1 System Definition Guide manual:

Bill O'Boyle · Posted: Fri Jan 11, 2013 8:51 am

30,000 Files

A good example of a bad idea

enrico-sorichetti · Posted: Fri Jan 11, 2013 12:58 pm

topic will be locked shortly to prevent further stupidity...

But really Your organization should consider hiring a properly skilled consultant !

Bill Woodger · Posted: Fri Jan 11, 2013 2:39 pm

There'd be TIOT size to consider as well, even if it were possible to get anywhere close to it.

And what would you do to process all of these in batch? 30,000 copies of each references in each piece JCL, one step running 30,000 times, or "dynamic allocation" with 30,000 datasets to really ensure that no-one can ever follow what is going on?

What problems have you come to in your design just using one file and a reference?

Pete Wilson · Posted: Fri Mar 01, 2013 7:57 pm

I would assume a separate entry within the same ESDS for each individual teller, otherwise.....time fa coffee

dejunzhu · Active User Joined: 08 May 2008 Posts: 390 Location: China

Robert Sample · Posted: Sun Mar 17, 2013 5:13 am

Bill Woodger · Posted: Sun Mar 17, 2013 5:42 am

From your system design there will be a natural split that can be made for these log files. It will not be an individual level, it is also unlikely to be "everything jammed in one". Somewhere in between you will find the point where the log files will give you the most benefit. If those logs have no particular relevant content beyond the technical detail, then put the split at a level where the number of datasets involved can still be managed by humans, and nowhere near approaching any limits, and then write some stuff to give you the detailed information you need.

You will not have "developers" nosing around in Production log files (I hope). If you do, prepare to have your Audit department jump down your throats. If they don't, expect all types of scandal and corruption and incompetence with your data generally. If your system is not Auditable, then it will fall apart. If it is Auditable, then you don't consider doing things this way.

How you can continue on your original route having been shown it was not even possible, I don't know.

dick scherrer · Posted: Mon Mar 18, 2013 7:37 am

Hello,

Many of us have been following this effort for multiple years.

If/when this application ever goes live with the full compliment of users and data, we should prepare for the deluge of problem questions . . .

d

Gary McDowell · Active User Joined: 15 Oct 2012 Posts: 139 Location: USA

Bill O'Boyle · Posted: Tue Mar 19, 2013 6:32 am

Your TIOT draws 16-Bit storage from z/OS. For a TIOT with 30,000 files (which has been explained to you that this is not possible), you'd been playing with fire as you've also reduced the amount of DSA (16-Bit CICS storage) who also draws real storage from z/OS.

Conclusion: I don't believe you'd be able to start a region with such a huge amount of 16-Bit storage being taken from z/OS as well as a only a fraction of necessary shortage allocated to CICS/DSA. Remember, there are still many CICS Internal Modules, which require below-the-line storage. You'd wind up with an S80A abend and a U1800 (and no, that's not a submarine).

You need to get a group of savvy technicians together, draw pictures, order a couple of pizzas, submit opinions and keep doing this until you have a better plan.

Otherwise, you may want to consider getting your CV updated and circulated just prior to the attempted (bad idea) implementation and your all but inevitable demise.

Anuj Dhawan · Posted: Tue Mar 19, 2013 9:14 am

dejunzhu · Active User Joined: 08 May 2008 Posts: 390 Location: China

dejunzhu · Active User Joined: 08 May 2008 Posts: 390 Location: China

Nic Clouston · Posted: Tue Mar 19, 2013 1:15 pm

TIOT - loads of references if you google TIOT IBM including what TIOT is an acronym for on the results page.
Regarding:

Pete Wilson · Posted: Tue Mar 19, 2013 1:31 pm

Task I/O Table (aka (TIOT)

For every dataset you add to a job or started task, and for every volume that the dataset(s) reside on there's ~4kb of storage taken from the TIOT assigned to your job or task. The TIOT is limited in size depending on Z/OS level, so trying to add 30k files to your CICS region would blow up your TIOT well before it gets near 30k, let alone 10k files. I think max TIOT is around 64kb up until I think Z/OS 1.12 or 1.13 when XTIOT became available and extended the capacity of TIOT.

For the sake of your reputation look for a way of storing this data in as few datasets as technically possible. It doubt it will be a problem for you even if these files get quite large.

Anuj Dhawan · Posted: Tue Mar 19, 2013 2:46 pm

Thanks Nic.

dejunzhu - please ignore my comment, it was for Bill, not for you.

Garry Carroll · Posted: Tue Mar 19, 2013 4:47 pm

Why not include your teller-ids in the log records, have one huge ESDS to contain all these log records and use your SORT product in batch to extract the records for the teller-id(s) you're interested in?

Garry.

dick scherrer · Posted: Wed Mar 20, 2013 12:30 am

Hello,