We have a peculiar situation where we thought ICETOOL to abend but it did not. We take in two different files in one ICETOOL step and assemble both to a common layout on two temp files. Then we back reference the temp files in the same step to concatenate the derived files to one output file.
Since the SORTOF01 and SORTOF02 is allocated less primary space, it uses both primary & secondary space and therefore data gets written to mutliple volumes. Then when we back reference the temp files in MRGINPUT and it chops off several thousand records without even issuing a warning.
Please find my questions below:
1. Can you please suggest a solution on how to back reference if we have files that are on multiple volumes being backreferenced?
2. Could you please confirm on how we can make the SORT card in CTL3CNTL return a non zero return code (or make it abend) if it senses that it is trying to chop off records?
3. Could you please confirm if it is okay to proceed with back reference in any of our furture implementation as it seem to result in unpredictable results like in this one (unless we know answer for Question 1). This taking into consideration that if we had to use splice operator in our project we have to rely on this technique and when we really don't have control over data being written on multiple volumes in DISK?
The problem with your suggested solution is that we cannot use disposition of MOD in our installation.
Do you have any other way in which this could be accomplished without using MOD as a disposition?
Also, could you please guide us if question 1 in my first post could resolve this issue - "Can you please suggest a solution on how to back reference if we have files that are on multiple volumes being backreferenced?"
I'm planning to use SPLICE operator in one of my future project and will that be an issue now since I'm planning back reference there as well.
Are you suggesting that back reference should be avoided?
Joined: 23 Nov 2006 Posts: 19244 Location: Inside the Matrix
Hello,
Quote:
The problem with your suggested solution is that we cannot use disposition of MOD in our installation.
There might be a reason to prohibit MOD for production (real) datasets, but what business/technical reason might there be to prohibit MOD for a temporary dataset to be used in only one step?
The jcl shows CATLG for the T1 dataset, but i believe this is a really "throwaway" dataset and is not needed after this step completes.
Joined: 15 Feb 2005 Posts: 7129 Location: San Jose, CA
Quote:
The problem with your suggested solution is that we cannot use disposition of MOD in our installation.
Do you have any other way in which this could be accomplished without using MOD as a disposition?
Ravi,
Not sure why you can't use MOD. Has you site just banned MOD data sets arbitrarily?
I'm not a JCL expert, but I would think you could just break up your one ICETOOL step into two ICETOOL steps to avoid backreferencing. For example, have the first ICETOOL step with the first two operators (COPY, COPY) and SORTOF1 and SORTOF2, and have the second ICETOOL step with the third operator (SORT or SPLICE) and MRGINPUT using the SORTOF1 and SORTOF2 data set names rather than referback.
Note that there is no difference in efficiency between using one ICETOOL step or two ICETOOL steps in this way - you're still doing the same number of passes over the data.
Quote:
Are you suggesting that back reference should be avoided?
I'm stating that there is a system restriction associated with back reference that may affect you. It's a system restriction for any product, not just DFSORT.
Joined: 15 Feb 2005 Posts: 7129 Location: San Jose, CA
Quote:
the "second or succeeding DD statement" are output that will be concatenated as input together?
Not sure what you're asking. The situation is that a second DD statement uses referback for a previous output data set.
In the job shown in the first post, MRGINPUT uses referback for two data sets that are concatenated together. But it would be true if MRGINPUT only referred back to a single data set as well.
Thanks. I see that the only solution from your post could be that by specifying the same volume for SORTOF01 & SORTOF02 files in this particular example. But I think it may not be correct to specify volumes for DASD files. Pls. correct me if I'm wrong??
Dick/Frank,
On your question on why not to use MOD?
------------------------------------------------
The usage of MOD requires special permission from our assurance group. This is the reason I tried not to use MOD. I think there was a technical reason to not to use MOD when such a DD was being referenced in a program invoked through say a DB2 Checkpoint/Restart utility like BMC.
But I can double check to see if we still have this restriction.
Frank,
Thanks again. I see that back reference shall be avoided by splitting to multiple steps. The workaround I'm thinking now is to increase the space to a logic level such that the files are on one volume. But going forward I shall go with splitting the format & copy operation into two steps.
Do you see any better workaround for this problem?
Joined: 23 Nov 2006 Posts: 19244 Location: Inside the Matrix
Hello Ravi,
Quote:
But I can double check to see if we still have this restriction.
Even if the general restriction still exists, you might talk about an exception for this step as it is a "sort" step and the concern about database and/or restart would not apply. Also, the dataset in queston is for use only within the step, i believe. . . FWIW.
Joined: 14 Mar 2007 Posts: 8797 Location: Welsh Wales
Thanks to this problem being highlighted sometime last year, I took the precaution of ending the ICETOOL step when the datasets were written out to the one dataset and then coding a second ICETOOL step to do the SPLICE / SELECT processing, which worked with no problems.
Joined: 18 Jul 2007 Posts: 2146 Location: At my coffee table
raviraman sr wrote:
Thanks. I see that the only solution from your post could be that by specifying the same volume for SORTOF01 & SORTOF02 files in this particular example. But I think it may not be correct to specify volumes for DASD files. Pls. correct me if I'm wrong??
Isn't there a way of specifying only a single SYSDA volume?
And why the referback on input for the VOL=, it is a cataloged dataset, isn't it?
Or how about?
You will have the same thing you already have now (when it does work) and won't have the referback potential of missing data.
BTW, && temp datasets and MOD is the more elegant form for this kind of JCL, but, to each his own......grin......
Joined: 15 Feb 2005 Posts: 7129 Location: San Jose, CA
Quote:
won't have the referback potential of missing data.
Hmmm ... I read through the doc again and I don't see where this is restricted to referback situations. The restriction seems to be for any DD statement that requests the same data set as a previous DD statement for output.
I'm guessing that the same problem can occur with catalogued data sets when referback is NOT used such as in the example you show. Anybody know for sure?
Joined: 18 Jul 2007 Posts: 2146 Location: At my coffee table
Frank Yaeger wrote:
Hmmm ... I read through the doc again and I don't see where this is restricted to referback situations. The restriction seems to be for any DD statement that requests the same data set as a previous DD statement for output.
Excellent point, after rereading, I have to agree, unless there is something that the (fill in the name of the correct process here) does differently during the referback as opposed to having full and direct access to the cataloged information....
Quote:
I'm guessing that the same problem can occur with catalogued data sets when referback is NOT used such as in the example you show. Anybody know for sure?
Do you think the below would work if the files (TEMPOUT*) being created is in the same step? I was of the opinion that the below would not work and this is the reason I thought Splice Operator Examples had it as a back reference.
Joined: 18 Jul 2007 Posts: 2146 Location: At my coffee table
The question came up as to whether or not it was the referback, i.e.,
//MRGINPUT DD DISP=(SHR),
// DSN=*.SORTOF01
or just any reference to a dataset created in the same step. i.e.,
//MRGINPUT DD DISP=(SHR),
// DSN=&USER..TEMPOUT1
that is causing the loss of data....
Based upon your original JCL, see if you have a setup that (regularly) fails with the referback (*.SORTOF01), then try to see if it fails (regularly) with the actual DSN (&USER..TEMPOUT1).
And, unless it is needed, drop the 'VOL=REF=*.SORTOF01' in both cases, (unless somebody knows why it should be needed?)......
Or try it in both, who knows, that may be the culprit....grin.....
I have attempted the below earlier and it will give JCL interpreter error for dataset not found. I originally had only temp datasets (&&TEMP**) for the "throwaway" files but due to my site restrictions I catalogued with 2 days retention.
When referred with DISP=SHR as below
Code:
//MRGINPUT DD DISP=(SHR),
// DSN=&USER..TEMPOUT1
The JCL Interpreter tries to check whether the file is in catalogue. Since it is not found/created in the prev. step, it throws up interpreter error.
Only way I think this can be overcome is by specifying DISP=MOD which we will have to check with our assurance team.
