I'm trying to work on a automation here, and I need some guidance. Here's the problem:
I have around 80-100 JCL members in a dataset, every week based on a criteria I need to select and submit only selected JCL's. I get the list in a text file. Moreover, before submitting the JCL members, I need to make a couple of changes and then submit them one by one. So, we end up spending a lot of time on these.
Now, we have come up with a solution, to tackle this. We broke down the problem into individual parts, and came up with a idea for each. But we don't know if this can be implemented. Here's the idea:
First, since we get the list of JCL's that has to be run in a text file, we thought of FTP'ing the file to the mainframes, and store that as a PS. Now on mainframes we have a Base PDS which contains the 100 odd JCL members. We were thinking of comparing this PDS with the PS and copying the matched members to a new PDS(Only the member names need to be compared). Once this is done, we will have a PDS with only the JCL members we need to submit.
Second, once we have the PDS with the selected members, we can use the file manager option which has a option to make changes to the entire PDS at once (Ex: We can give the C ALL XXX YYY command to change all the members inside the PDS at once). If possible we can either write a JCL which accesses this option or we can just directly go and use the option a couple of times to make the changes required. Either way, since going to the option directly from the ISPF menu doesn't actually make much of a time difference.
Third, I googled for a way to submit all the jobs thru a single JCL, instead of submitting them 1 by 1(This was the bottom line problem why we had to do this whole process) . But again I need to ask you guys, I have a PDS with JCL members that need to be submitted 1 by 1, so instead that, is there a way where we can write a JCL which when submitted can run these jobs 1 by 1. We did search for this, and we saw a couple of people having the same problem, and it seems there is a way of doing this. IEBGENER can acheive this with the help of INTRDR.
I apologize for such a big post, but any help would be greatly appreciated. Thank you.
Joined: 17 Oct 2006 Posts: 2481 Location: @my desk
Quote:
I have a PDS with JCL members that need to be submitted 1 by 1
Hello and welcome to the forums. Does each job have to wait until the previous job's completion? We had a similar requirement wherein every job had a last step which will submit the subsequent job through internal reader. Submit the first job alone and you are done!
Despite being your first post here, you have done a good job of explaining your requirement above. Appreciate that
Joined: 26 Apr 2004 Posts: 4652 Location: Raleigh, NC, USA
Ani Shetty wrote:
First, since we get the list of JCL's that has to be run in a text file, we thought of FTP'ing the file to the mainframes, and store that as a PS. Now on mainframes we have a Base PDS which contains the 100 odd JCL members. We were thinking of comparing this PDS with the PS and copying the matched members to a new PDS(Only the member names need to be compared). Once this is done, we will have a PDS with only the JCL members we need to submit.
Sounds good.
Ani Shetty wrote:
Second, once we have the PDS with the selected members, we can use the file manager option which has a option to make changes to the entire PDS at once (Ex: We can give the C ALL XXX YYY command to change all the members inside the PDS at once). If possible we can either write a JCL which accesses this option or we can just directly go and use the option a couple of times to make the changes required. Either way, since going to the option directly from the ISPF menu doesn't actually make much of a time difference.
Again, sounds good. I don't have File Manager available so I can't comment much. I would, personally, copy the individual PDS members to a single sequential file (IEBPTPCH), and make the global changes (SORT).
Ani Shetty wrote:
Third, I googled for a way to submit all the jobs thru a single JCL, instead of submitting them 1 by 1(This was the bottom line problem why we had to do this whole process) . But again I need to ask you guys, I have a PDS with JCL members that need to be submitted 1 by 1, so instead that, is there a way where we can write a JCL which when submitted can run these jobs 1 by 1. We did search for this, and we saw a couple of people having the same problem, and it seems there is a way of doing this. IEBGENER can acheive this with the help of INTRDR.
What do you mean by running jobs 1 by 1? I presume that, if you're not just somehow single-threading them, that you want to have each job submit the next using INTRDR.
Joined: 28 Jan 2012 Posts: 316 Location: Room: TREE(3). Hilbert's Hotel
Hi...
Quote:
Third, I googled for a way to submit all the jobs thru a single JCL
Once you have all your required JCL in a separate PDS, you can write a rexx program to edit them in the order in which you have to run.
For ex..if you want to run JCL1, JCL2 & JCL3 in the same order..and JCL2 will run only after JCL1 and JCL3 will run after JCL2...then your REXX program will edit JCL1 to add the IEBGENER (INTRDR) step for JCL2 and you JCL2 will have IEBGENER step for JCL3.
You Rexx program will take the input as the list of JCLs in order to run and the PDS member which contains this JCL...and then will call a macro to edit each PDS member, in that order,i.e, your first JCL will be edited to add an IEBGENER step for JCL2, JCL2 for JCL3 and so on..
Does each job have to wait until the previous job's completion? We had a similar requirement wherein every job had a last step which will submit the subsequent job through internal reader. Submit the first job alone and you are done!
Thank you! Not quite, We don't have to wait for the previous job to finish, we can submit them.
Quote:
Sounds good.
Thanks! interesting, I searched and nowhere I could get a satisfactory answer for comparing a PDS and a PS. Now, if it's possible, how do I go about it. When I searched the forum I got this post from a guy who seems to have the same exact question, here's the link:
http://ibmmainframes.com/about47399.html
You guys had answered this, Can you give a little more explanation on how to do this.
Quote:
Again, sounds good. I don't have File Manager available so I can't comment much. I would, personally, copy the individual PDS members to a single sequential file (IEBPTPCH), and make the global changes (SORT).
Sure, will consider this option too.
Quote:
What do you mean by running jobs 1 by 1? I presume that, if you're not just somehow single-threading them, that you want to have each job submit the next using INTRDR.
Qkay, let me put it this way, instead of visiting each member, making the changes, and submitting them. Is there a way to automate this? Do have to go give the submit command to over 80 jobs everytime. Can't we automate this? We were just considering using Internal reader, Is there a better way of doing this?
Quote:
Once you have all your required JCL in a separate PDS, you can write a rexx program to edit them in the order in which you have to run.
For ex..if you want to run JCL1, JCL2 & JCL3 in the same order..and JCL2 will run only after JCL1 and JCL3 will run after JCL2...then your REXX program will edit JCL1 to add the IEBGENER (INTRDR) step for JCL2 and you JCL2 will have IEBGENER step for JCL3.
Thank you! But as I told they don't have to run in any particular order, you just go wham wham and submit all of them asap.
Quote:
If you change the members to NOT have a JOB statement, you could generate a JOB statement for submission and have only one job to run.
Do you mean I can write a JCL with a JOB statement, that can trigger all those JCL members?
Joined: 28 Jan 2012 Posts: 316 Location: Room: TREE(3). Hilbert's Hotel
Okay..as I understand it..first you need to copy the JCL into a new PDS using the PS file as input...so you need a way to compare the JCL member names in PDS and your PS file and the copy them into a new PDS.
What you can do is..get a list of the members of JCL PDS into sequential dataset and use your sort product to JOIN the two files and copy them using IEBCOPY.
Check this post..it is quite similar to what you are looking for.
Joined: 23 Nov 2006 Posts: 19243 Location: Inside the Matrix
Hello,
Quote:
But as I told they don't have to run in any particular order, you just go wham wham and submit all of them asap.
I surely hope NOT!
You will have considerable complaints from the Operatons and/or Scheduling people if they all have differet jobnames and run in different classes or even 1 class if there are several initiators with that class.
Quote:
Do you mean I can write a JCL with a JOB statement, that can trigger all those JCL members?
No. Not at all. It should not be difficult to create a JOB statement and copy all of the needed members into one sequential file, globally edit that single file, and submit this via the internal reader. It is a rather straightforward process and care should be taken Not to make it difficult.
2 ways to do it. the first is with IEBPTPCH to produce a PS file. that's difficult and too many manual items to deal with though it can be done.
the second way is the best if and only if you have SAS. You need merely write a simple macro that reads in a list then creates a SELECT list for a super simple procedure called PROC SOURCE.
PROC SOURCE then does the heavy lifting for you that IEBPTPCH only does part way so that you can then feed the sequential file back into your PDS once you've made whatever changes you want to that sequential file - all the time keeping the manufactured control cards in place for IEBUPDTE to automatically feed back into your PDS.
Once you've done that then you really need to manually go to TSO 3.5 and reset the statistics to suit.
Also if you've decided to use either IEBPTPCH or SAS's PROC SOURCE and you've decided to use either DFSORT or SyncSort to maninpulate the sequential file, remember to utilize the EQUAL option.
The EQUAL option will ensure that your records are in the proper order because they are in a special sequence in order to be streamed into IEBUPDTE.
EQUALS ensures that the input sequence is the same before the SORT or MERGE instruction is invoked and probably right about the time of the INREC instruction.
from the DFSORT programming guide:
EQUALS The sequence of records that collate identically [even if COPY] is preserved from input to output - with either the MERGE or SORT instruction.
In this particular case obviously there will be no sort only copy because the IEBUPDTE instructions need to be in the particular order it was created using someting like IEBPTPCH. Sort was merely an example I used because it is a fine file-to-file utility. However, unless a programmer explicity uses EQUALS then the sort program ignores the previous collating sequence and the result though usually 99.9% correct [and often 100%] - there's no guarantee without being explicit.
Joined: 07 Dec 2007 Posts: 2205 Location: San Jose
Dale Robertson wrote:
from the DFSORT programming guide:
EQUALS The sequence of records that collate identically [even if COPY] is preserved from input to output - with either the MERGE or SORT instruction.
Dale Robertson,
Can you please point me the link where you have seen that in DFSORT programming guide?
It would be wise if you actually did some tests as well as to read in all of the places where that option exists as reading will only get you so far because often times the documentation is incomplete or misleading sadly.
I gave plenty of test jcl in the past week but chances are that it was ignored and more probably that I will get harrassed further if I repeat it. It's a little discouraging.
Realize also that there is more documentation with SyncSort and even SAS Programming Tips: Guide to Efficient SAS Programming [1st Edition 1991] - which surprisingly gave the best advice on EQUALS!!!
Joined: 03 Oct 2009 Posts: 1787 Location: Bloomington, IL
Dale Robertson wrote:
It would be wise if you actually did some tests as well as to read in all of the places where that option exists as reading will only get you so far because often times the documentation is incomplete or misleading sadly.
Presumably you were not aware that Sri Kolusu is a DFSORT developer.
It would be wise if you actually did some tests as well as to read in all of the places where that option exists as reading will only get you so far because often times the documentation is incomplete or misleading sadly.
Thank you very much for the DFSORT links. But my question is still unanswered, I want to see the specific link where EQUALS is used even for COPY operations as you seem to highlight here
Dale Robertson wrote:
from the DFSORT programming guide:
EQUALS The sequence of records that collate identically [even if COPY] is preserved from input to output - with either the MERGE or SORT instruction.
So can you please show me where the EQUALS is being used for COPY operation.
Dale Robertson wrote:
It would be wise if you actually did some tests as well as to read in all of the places where that option exists as reading will only get you so far because often times the documentation is incomplete or misleading sadly.
In your prior posts you were implying that documentation had it described and now you are saying that it is incomplete or misleading ?
As for testing how about you do just 1 test for me.
1. Create a file with LRECL=5042
2. Write 50 or as many as you feel like records into that file with the following data
Code:
Position 1 - 5000 = spaces
Position 5001 = 'DALE ROBERTSON DUPLICATE RECORD # '
Position 5034 = 8 byte sequence number to number the records
Now use a COPY operation to copy with NOEQUALS and show me that DFSORT did NOT COPY the records as is.
The maximum length of the collected control fields (sort fields) is 4088 bytes with Equals ON.
The above created dataset has spaces in first 5000 bytes and all the records are duplicates. So without EQUALS option I am not guaranteed to get the records as it is. ? Is that what you are inferring? Never Knew SORT products supported keys greater than 4k. The largest LRECL you can have on z/OS is 32767. Which utility would you use to COPY such files?
Dale Robertson wrote:
Realize also that there is more documentation with SyncSort and even SAS Programming Tips: Guide to Efficient SAS Programming [1st Edition 1991] - which surprisingly gave the best advice on EQUALS!!!
Good luck.
r
Please do NOT QUOTE Syncsort documentation as a source for how DFSORT works.
