Joined: 26 Apr 2004 Posts: 4652 Location: Raleigh, NC, USA
Guys, I know this sounds like a rookie question, but bear with me. Ever since last weekend, we've been dealing with a bunch of data contention issues for our batch job streams that have caused problems and delays. These are jobs that process incoming data transmissions, and all use the same set of datasets:
- two sequential log datasets updated with DISP=MOD
- four GDG's, all updated with (+1),DISP=(NEW,CATLG).
- one of the GDG's is allocated in the last step as input with (+1),DISP=OLD
All GDG's are defined with LIMIT=255,NOERASE,SCRATCH
Two issues have been discussed with the storage management team, and their response has me puzzled. So, my two questions:
- What is supposed to happen when the limit is reached and a new +1 dataset is cataloged?
- What is supposed to happen when generation 9999 has been reached and a new +1 dataset is cataloged?
Joined: 06 Jun 2008 Posts: 8697 Location: Dubuque, Iowa, USA
For your first question, the manual is Access Method Services for Catalogs in the DFSMS bookshelf. The V1R13 manual says
Quote:
EMPTY|NOEMPTY
Specifies what action is to be taken for the catalog entries for the GDG base when the number of generation data sets in the GDG base is equal to the LIMIT value and another GDS is to be cataloged. The disposition of the actual data sets uncatalogued from the GDG base is determined by the setting of the SCRATCH/NOSCRATCH parameter for the GDG base.
EMPTY
remove all GDS entries from GDG base when a new GDS is created that causes the GDG LIMIT to be exceeded.
Abbreviation: EMP
NOEMPTY
remove only the oldest GDS entry when a new GDS is created that causes GDG LIMIT to be exceeded.
Abbreviation: NEMP
as well as
Quote:
SCRATCH|NOSCRATCH
Specifies what action is to be taken for a generation data set located on disk volumes when the data set is uncataloged from the GDG base as a result of EMPTY/NOEMPTY processing. For generation data sets located on tape, this parameter has no effect.
You can override the SCRATCH|NOSCRATCH attribute when issuing the DELETE command.
SCRATCH
The GDS is deleted from all disks it occupies when uncatalogued from the GDG base, regardless of whether it is SMS-managed or not.
Abbreviation: SCR
NOSCRATCH
If the data set is a non-SMS managed data set it is not removed from any of the volumes it occupies. If the data set is an SMS-managed data set it is recataloged as a non-VSAM data set in rolled-off status, and is no longer associated with the GDG base. It is not deleted from any of the SMS-managed volumes it occupies.
Abbreviation: NSCR
For your second question, the manual is JCL User's Guide in the MVS bookshelf (again quoted from V1R13) in the Appendix on Generation Data Sets:
Quote:
Creating an SMS-Managed Generation Data Set
z/OS MVS JCL User's Guide
SA22-7598-07
When creating a new SMS-managed generation data set, always code the DSNAME and DISP parameters and optionally, code the DATACLAS and STORCLAS parameters.
DSNAME Parameter
In the DSNAME parameter, code the name of the GDG followed by a number, +1 to +255, in parentheses. If this is the first data set being added to a GDG in the job, code +1 in parentheses. Each time in the job you add a data set to the same GDG, increase the number by one.
When referring to this data set in a subsequent job step, code the relative generation number used to create it on the DSNAME parameter. You cannot refer to this data set in the step in which it was created. At the end of the job, the system updates the relative generation numbers of all generations in the group to reflect the additions.
Note:
If the relative generation number makes the absolute generation number exceed G9999Vyy, wraparound occurs. In an integrated catalog facility catalog, if you create a new generation data set with a relative generation number, such as (+1), and an absolute generation number of G9999Vyy exists in the GDG base, the wraparound generates number G0001Vyy. (For information about absolute generation numbers and version numbers, in the form GxxxxVyy, see z/OS DFSMS Using Data Sets.)
DATACLAS and STORCLAS Parameters
If the ACS routines do not select the needed data class or storage class, code the DATACLAS or STORCLAS parameters (and any DD parameters needed to override attributes in the data class or storage class).
Quote:
Creating a Non-SMS-Managed Generation Data Set
z/OS MVS JCL User's Guide
SA22-7598-07
When creating a new non-SMS-managed generation data set, always code the DSNAME, DISP, and UNIT parameters and optionally, code the VOLUME, SPACE, LABEL, and DCB parameters.
DSNAME Parameter
In the DSNAME parameter, code the name of the GDG followed by a number, +1 to +255, in parentheses. If this is the first data set being added to a GDG in the job, code +1 in parentheses. Each time in the job you add a data set to the same GDG, increase the number by one.
When referring to this data set in a subsequent job step, code the relative generation number used to create it on the DSNAME parameter. You cannot refer to this data set in the step in which it was created. At the end of the job, the system updates the relative generation numbers of all generations in the group to reflect the additions.
Note:
If the relative generation number makes the absolute generation number exceed G9999Vyy, wraparound occurs. In an integrated catalog facility catalog, if you create a new generation data set with a relative generation number, such as (+1), and an absolute generation number of G9999Vyy exists in the GDG base, the wraparound generates number G0001Vyy.
(For information about absolute generation numbers and version numbers, in the form GxxxxVyy, see z/OS DFSMS Using Data Sets.)
One thing I would be cautious about, since you mentioned data transfers, is the relative generation number. For FTP, for example, there is a daemon running as a started task (FTPD1 or something similar, usually) and I do not know if the relative generations would be updated after each data transmission, or after the FTPD1 started task completes. MAINVIEW on my system indicates FTPD1 has been running since the last IPL; I assume therefore that relative generation numbers are updated at some point but that may vary by site and would be something to check.
Joined: 26 Apr 2004 Posts: 4652 Location: Raleigh, NC, USA
The explanation provided by the Storage Management Team is that, since the number of GDG's had exceeded the LIMIT, the older generations had rolled-off, and had been archived. When the attempt is made to catlalog the new generation, each of the rolled-off generations has to be recovered first, and then rolled-in, and only after all of them had been recovered, would the new catalog attempt occur, after which they would be rolled-out again. And this series of events would keep happening over and over each time a new generation needs to be cataloged.
They also stated that once the 9999 generations was reached that any attempts to catalog a new +1 dataset would fail since the new generation dataset would not replace the rolled-out dataset with the same name.
Joined: 09 Mar 2011 Posts: 7309 Location: Inside the Matrix
Don't leave us hanging like that, please...
Were your Storage people's responses relevant to the problem? They seem too far out from what is expected for them to be making it up as they go along. With SCRATCH, for instance, there would (normally) be nothing to be archived or retrieved. If the SCRATCH was not obeyed, the now-individual-datasets have no connection to the GDG other than the similarity of name. Etc. You don't want us to just guess, do you?
Joined: 26 Apr 2004 Posts: 4652 Location: Raleigh, NC, USA
The contention issues seem to have gone away. From what I understand they deleted all of the rolled-off generations and any others that had been archived.
Joined: 09 Mar 2011 Posts: 7309 Location: Inside the Matrix
I can understand that if -254 is archived and +1 is created, -254 has to come back before it can be scratched. Perhaps a specific MOD,DELETE in IEFBR14 for -254 could overcome that?
Rolled-off generatiions need to be recalled? That doesn't make sense. If it is actually happening, someone could find out why. Plus, if they are SCRATCH, why are they still "there" when they've rolled-off?
It may be working now, but I'd suspect that more has been deleted than was strictly necessary, although if no-one is moaning about it, perhaps some of what was archived should have been deleted instead already.
If the rolled-off DSNs existed (weren't scratched) they would look like they were part of the GDG. Listing the catalog with x.x.x.G* would find them. Listing the catalog of the GDG would not. Some "exit" doing something "clever" with archive/retrieval? Blind guess.
What you were told is tosh. Yet acting upon it seems to have resolved the problem. So somewhere part of what you have been told is actually relevant to what was causing the problem, in a co-incidental sort of way. To guess further, the problem itself has not been solved, but the symptoms have been removed and the symptoms from a previous run was what was causing the new problems.
I guess you don't have access to the necessary sysouts. Your Storage people aren't going to look into it further, as they have the kudos already for fixing it.
As suggested the GnnnnV00 number wraps around from 9999 to 0001 which works OK, and the 0002 would be deleted as the 0001 is created (assuming SCRATCH is specified on the GDG base and GDG ROLLOFF ACTION is set to EXPIRE in the Mgmtclas assigned to the datasets).
Even if the BASE has NOSCRATCH and Mgmtclas GDG ROLLOFF ACTION is NOT set to EXPIRE in the Mgmtclas, the dataset simply get ROLLED-OFF and will be migrated according to the Mgmtclas values Primary Days and/or #GDG ON PRIMARY.
The issue with contention is usually caused by one of two things:
1. DFHSM trying to migrate older generation, and this ENQ's on the GDG base and prevents new generations being cataloged.
2. More than one job trying to create a new generation in the same base at the same time...again causing an ENQ on the base.
