We are using batch pipes in one of our jobs and now we are facing an issue. Please advice us on this issue. The document with details of the JCL's and the pipe error is added as an attachment.
Joined: 23 Nov 2006 Posts: 19243 Location: Inside the Matrix
Hello,
As others have mentioned, you need to post your info using copy/paste and the "Code" tag at the top of the Reply panel. Attachemnts are not usable by many of our contributors and "screenshots" just make a mess and i delete them. Copy/Paste/"Code" is the way to go.
Once you hae entered your post, you can Preview your post to see how your post will appear to the forum. When you are satisfied with the way your post looks, then press Submit.
Note that INPUT1 is overridden in JCL2 with the same PIPE dataset.
********************************************************************
Here JCL1 acts as writer.
JCL2 acts as reader.
Pipe dataset: TEST.PIPE.SORT.OUTPUT
JCL1 and JCL2 ran concurrently.
Here input files are sorted by JCL1 and as each record is written into pipe dataset ‘TEST.PIPE.SORT.OUTPUT’, JCL2 reads each record from the pipe dataset and uses it in executing the Cobol code.
When the Number of records in TEST.INPUT.FILE1 is 4377415 and Number of records in TEST.INPUT.FILE2 is 10660319, we are facing an issue with the above code.
The description of the issue is below.
INPUT1 - READER IS WAITING FOR OPEN SYNC
PIPE IN ERROR. REASON=PARTICIPANT JOB2 IN ERROR
INPUT1 - ERROR DISTRIBUTED FROM ANOTHER PARTICIPANT.
ACTION = ABEND
JOB FAILED BY JOB OPTIMIZER PIPES
Additional Information:
PIPE PARAMETERS FOR DD INPUT1:
PIPE-FLG - BATCH-PI
RD-EROPT - IMMED
ADD-WTR - 0
NOOP-WT - 5
I/O-ACT - ASKOPER
SYNC-ACT - ASKOPER
BLKSIZE - 32000
FIT-DD
RDR-FLAG - NO-EOFWT
SYNC-TYP - OPEN
REQ-PRTS - 0
REQ-RDR - 1
NOOP-ACT - ASKOPER
SYNC-WT - 30
BUFF# - 99
LRECL - 4000
ERROR-CC - 0
REQ-WTR - 1
ADD-RDR - 0
I/O-WT - 5
RECFM – 90
Please advise us on the above issue.
Joined: 23 Nov 2006 Posts: 19243 Location: Inside the Matrix
Hello,
Is this the first attempt to use batch pipes on this system?
Are both jobs submitted at the same time?
Your sort appears to be large rather than small. I mention this because the sort will need to process for a while before there is any output ready to be written as the actual sorted output. Also (once upon a time) different releases of various sort products might use the output dataset as work space while performing the sort. How would your process react to that? How would the pipes process know which writes to sortout were "real" (i.e. the final output) and which were the sort processing internally? This may not be a concern - i do not know how the various sort products run these days.
It would be interesting to see the jes messages from both jobs as well as the diagnostic info presented by them.
Job1 specifies (new,catlg) for the sortout and i'm not sure what the system will do when the second job tries to allocate the same dataset. This should also be in the jes outputs.
Joined: 23 Nov 2006 Posts: 19243 Location: Inside the Matrix
Hi Expat,
Once upon a time, some sorts did. The "last thing" the sort did was to write the completed sorted output to the "front" of the sortout dataset. I don't know if this is still done or not.
When an internal sort was used that specified "output procedure" there as no need to copy the sorted data to the front of some output file and that overhead was saved. All that was needed was to return the records in the correct sequence to the code and where on the dasd the sorted data came from was no concern to the program.
Joined: 28 Jul 2006 Posts: 1702 Location: Australia
Hi,
Quote:
Does sort write data to sortout before the data is completely sorted ?
The answer to that as far as I'm concerned is no, many jobs use the same dataset as input and output.
Also when input is on cart in order to save tape drives, you can AFF the SORTOUT to the SORTIN DD, which tells me the SORTOUT DD is not opened until all of the data has been read in and the SORTIN DD has been closed.
If the data is not completely sorted how can you can guarantee the data will be written in the correct sequence ?
Joined: 23 Nov 2006 Posts: 19243 Location: Inside the Matrix
Hi Gerry,
Quote:
The answer to that as far as I'm concerned is no, many jobs use the same dataset as input and output.
In the cases you mention, no the sort would not write to the output "in the middle" of working.
In other cases the sort might use the sortout dataset (assuming dasd and not the input dataset) as additional work space to make the sort run faster.
As i mentioned earlier, i do not know if this is still done, but there was a time when it was done.
FWIW - if i find production jcl that writes over the input in a sort step (and it is not a transient dataset - i.e. used only in "this" job) i get the jcl changed. Not much worse than sorting the transmitted input file over itself and losing the data due to some problem. Problems shouldn't happen, but datasets shouldn't be put at risk either.
Maybe we'll learn why batch pipes is being attempted for this "non-pipe" process. . .
