IBM Mainframe Forum Index
 
Log In
 
IBM Mainframe Forum Index Mainframe: Search IBM Mainframe Forum: FAQ Register
 

Comparing a PS & PDS, submitting many jobs thru a single


IBM Mainframe Forums -> JCL & VSAM
Post new topic   Reply to topic
View previous topic :: View next topic  
Author Message
Ani Shetty

New User


Joined: 28 Jun 2013
Posts: 2
Location: India

PostPosted: Sat Jun 29, 2013 5:16 pm
Reply with quote

Hello,

I'm trying to work on a automation here, and I need some guidance. Here's the problem:

I have around 80-100 JCL members in a dataset, every week based on a criteria I need to select and submit only selected JCL's. I get the list in a text file. Moreover, before submitting the JCL members, I need to make a couple of changes and then submit them one by one. So, we end up spending a lot of time on these.

Now, we have come up with a solution, to tackle this. We broke down the problem into individual parts, and came up with a idea for each. But we don't know if this can be implemented. Here's the idea:

First, since we get the list of JCL's that has to be run in a text file, we thought of FTP'ing the file to the mainframes, and store that as a PS. Now on mainframes we have a Base PDS which contains the 100 odd JCL members. We were thinking of comparing this PDS with the PS and copying the matched members to a new PDS(Only the member names need to be compared). Once this is done, we will have a PDS with only the JCL members we need to submit.

Second, once we have the PDS with the selected members, we can use the file manager option which has a option to make changes to the entire PDS at once (Ex: We can give the C ALL XXX YYY command to change all the members inside the PDS at once). If possible we can either write a JCL which accesses this option or we can just directly go and use the option a couple of times to make the changes required. Either way, since going to the option directly from the ISPF menu doesn't actually make much of a time difference.

Third, I googled for a way to submit all the jobs thru a single JCL, instead of submitting them 1 by 1(This was the bottom line problem why we had to do this whole process) . But again I need to ask you guys, I have a PDS with JCL members that need to be submitted 1 by 1, so instead that, is there a way where we can write a JCL which when submitted can run these jobs 1 by 1. We did search for this, and we saw a couple of people having the same problem, and it seems there is a way of doing this. IEBGENER can acheive this with the help of INTRDR.

I apologize for such a big post, but any help would be greatly appreciated. Thank you.
Back to top
View user's profile Send private message
Arun Raj

Moderator


Joined: 17 Oct 2006
Posts: 2481
Location: @my desk

PostPosted: Sat Jun 29, 2013 6:09 pm
Reply with quote

Quote:
I have a PDS with JCL members that need to be submitted 1 by 1
Hello and welcome to the forums. Does each job have to wait until the previous job's completion? We had a similar requirement wherein every job had a last step which will submit the subsequent job through internal reader. Submit the first job alone and you are done!

Despite being your first post here, you have done a good job of explaining your requirement above. Appreciate that icon_smile.gif
Back to top
View user's profile Send private message
superk

Global Moderator


Joined: 26 Apr 2004
Posts: 4652
Location: Raleigh, NC, USA

PostPosted: Sat Jun 29, 2013 6:17 pm
Reply with quote

Ani Shetty wrote:

First, since we get the list of JCL's that has to be run in a text file, we thought of FTP'ing the file to the mainframes, and store that as a PS. Now on mainframes we have a Base PDS which contains the 100 odd JCL members. We were thinking of comparing this PDS with the PS and copying the matched members to a new PDS(Only the member names need to be compared). Once this is done, we will have a PDS with only the JCL members we need to submit.


Sounds good.

Ani Shetty wrote:

Second, once we have the PDS with the selected members, we can use the file manager option which has a option to make changes to the entire PDS at once (Ex: We can give the C ALL XXX YYY command to change all the members inside the PDS at once). If possible we can either write a JCL which accesses this option or we can just directly go and use the option a couple of times to make the changes required. Either way, since going to the option directly from the ISPF menu doesn't actually make much of a time difference.


Again, sounds good. I don't have File Manager available so I can't comment much. I would, personally, copy the individual PDS members to a single sequential file (IEBPTPCH), and make the global changes (SORT).

Ani Shetty wrote:

Third, I googled for a way to submit all the jobs thru a single JCL, instead of submitting them 1 by 1(This was the bottom line problem why we had to do this whole process) . But again I need to ask you guys, I have a PDS with JCL members that need to be submitted 1 by 1, so instead that, is there a way where we can write a JCL which when submitted can run these jobs 1 by 1. We did search for this, and we saw a couple of people having the same problem, and it seems there is a way of doing this. IEBGENER can acheive this with the help of INTRDR.


What do you mean by running jobs 1 by 1? I presume that, if you're not just somehow single-threading them, that you want to have each job submit the next using INTRDR.
Back to top
View user's profile Send private message
mistah kurtz

Active User


Joined: 28 Jan 2012
Posts: 316
Location: Room: TREE(3). Hilbert's Hotel

PostPosted: Sat Jun 29, 2013 9:53 pm
Reply with quote

Hi...

Quote:
Third, I googled for a way to submit all the jobs thru a single JCL


Once you have all your required JCL in a separate PDS, you can write a rexx program to edit them in the order in which you have to run.

For ex..if you want to run JCL1, JCL2 & JCL3 in the same order..and JCL2 will run only after JCL1 and JCL3 will run after JCL2...then your REXX program will edit JCL1 to add the IEBGENER (INTRDR) step for JCL2 and you JCL2 will have IEBGENER step for JCL3.

You Rexx program will take the input as the list of JCLs in order to run and the PDS member which contains this JCL...and then will call a macro to edit each PDS member, in that order,i.e, your first JCL will be edited to add an IEBGENER step for JCL2, JCL2 for JCL3 and so on..

Your edit macro will look something like below:
Code:
/*REXX*/
ADDRESS ISREDIT "MACRO PROCESS"
"ISPEXEC VGET (ENDEXEC) SHARED"
"ISPEXEC VGET (JCLNAME) SHARED"
"ISPEXEC VGET (MDSN)    SHARED"

ISREDIT 'LINE_AFTER .ZL = "//STEP99  EXEC PGM=IEBGENER"'
ISREDIT 'LINE_AFTER .ZL = "//SYSUT1   DD DSN='MDSN'('JCLNAME'),DISP=SHR"'
ISREDIT 'LINE_AFTER .ZL = "//SYSUT2   DD SYSOUT=(*,INTRDR)"'
ISREDIT 'LINE_AFTER .ZL = "//SYSPRINT DD  SYSOUT=*"'

"SAVE"
"END"
EXIT

Here MDSN is the PDS name which has all the JCL and JCLNAME is the variable containg JCL member name passed from your main REXX program.

This is what I can think of...hope it helps..
Back to top
View user's profile Send private message
dick scherrer

Moderator Emeritus


Joined: 23 Nov 2006
Posts: 19244
Location: Inside the Matrix

PostPosted: Sun Jun 30, 2013 3:34 am
Reply with quote

Hello,

If you change the members to NOT have a JOB statement, you could generate a JOB statement for submission and have only one job to run.
Back to top
View user's profile Send private message
Ani Shetty

New User


Joined: 28 Jun 2013
Posts: 2
Location: India

PostPosted: Sun Jun 30, 2013 12:11 pm
Reply with quote

Quote:
Does each job have to wait until the previous job's completion? We had a similar requirement wherein every job had a last step which will submit the subsequent job through internal reader. Submit the first job alone and you are done!

Thank you! Not quite, We don't have to wait for the previous job to finish, we can submit them.

Quote:
Sounds good.

Thanks! interesting, I searched and nowhere I could get a satisfactory answer for comparing a PDS and a PS. Now, if it's possible, how do I go about it. When I searched the forum I got this post from a guy who seems to have the same exact question, here's the link:
http://ibmmainframes.com/about47399.html
You guys had answered this, Can you give a little more explanation on how to do this.

Quote:
Again, sounds good. I don't have File Manager available so I can't comment much. I would, personally, copy the individual PDS members to a single sequential file (IEBPTPCH), and make the global changes (SORT).

Sure, will consider this option too.

Quote:
What do you mean by running jobs 1 by 1? I presume that, if you're not just somehow single-threading them, that you want to have each job submit the next using INTRDR.

Qkay, let me put it this way, instead of visiting each member, making the changes, and submitting them. Is there a way to automate this? Do have to go give the submit command to over 80 jobs everytime. Can't we automate this? We were just considering using Internal reader, Is there a better way of doing this?

Quote:
Once you have all your required JCL in a separate PDS, you can write a rexx program to edit them in the order in which you have to run.
For ex..if you want to run JCL1, JCL2 & JCL3 in the same order..and JCL2 will run only after JCL1 and JCL3 will run after JCL2...then your REXX program will edit JCL1 to add the IEBGENER (INTRDR) step for JCL2 and you JCL2 will have IEBGENER step for JCL3.

Thank you! But as I told they don't have to run in any particular order, you just go wham wham and submit all of them asap.

Quote:
If you change the members to NOT have a JOB statement, you could generate a JOB statement for submission and have only one job to run.

Do you mean I can write a JCL with a JOB statement, that can trigger all those JCL members?

URL fixed
Back to top
View user's profile Send private message
mistah kurtz

Active User


Joined: 28 Jan 2012
Posts: 316
Location: Room: TREE(3). Hilbert's Hotel

PostPosted: Sun Jun 30, 2013 1:09 pm
Reply with quote

Okay..as I understand it..first you need to copy the JCL into a new PDS using the PS file as input...so you need a way to compare the JCL member names in PDS and your PS file and the copy them into a new PDS.

What you can do is..get a list of the members of JCL PDS into sequential dataset and use your sort product to JOIN the two files and copy them using IEBCOPY.

Check this post..it is quite similar to what you are looking for.
Back to top
View user's profile Send private message
gcicchet

Senior Member


Joined: 28 Jul 2006
Posts: 1702
Location: Australia

PostPosted: Sun Jun 30, 2013 2:15 pm
Reply with quote

Hi,

you can use something like this to submit the jobs from PDS

Code:
//STEP0001 EXEC PGM=SORT                                             
//SYSOUT   DD SYSOUT=*                                               
//TOOLMSG  DD SYSOUT=*                                               
//DFSMSG   DD SYSOUT=*                                               
//SORTIN   DD * list of jobs to be subbed                           
J                                                                   
J01                                                                 
J0000003                                                             
//SORTOUT  DD SYSOUT=(A,INTRDR)                                     
//SYSIN    DD *                                                     
  OPTION COPY                                                       
  INREC IFTHEN=(WHEN=INIT,                                           
                BUILD=(C'(',1,8,C')',81:SEQNUM,3,ZD)),               
        IFTHEN=(WHEN=INIT,                                           
               BUILD=(1,10,SQZ=(SHIFT=LEFT),81:SEQNUM,3,ZD))         
*                                                                   
  OUTFIL REMOVECC,IFOUTLEN=80,                                       
         IFTHEN=(WHEN=(81,3,ZD,EQ,1),                               
    BUILD=(C'//SORTIN   DD DISP=SHR,DSN=PDS-WITH-JCL',1,10)),       
*                                                                   
         IFTHEN=(WHEN=(81,3,ZD,GT,1),                               
    BUILD=(C'//         DD DISP=SHR,DSN=PDS-WITH-JCL',1,10)),       
*                                                                   
  HEADER1=(C'//SUB#JCL  JOB GERRY,''COPY  '',',/,                   
           C'//             CLASS=P,         ',/,                   
           C'//             MSGCLASS=X,      ',/,                   
           C'//             NOTIFY=&SYSUID,  ',/,                   
           C'//             TYPRUN=HOLD      ',/,                   
           C'//*                             ',/,                   
           C'//STEP0001 EXEC PGM=SORT',/,                           
           C'//SYSOUT   DD SYSOUT=*',/, ,                           
           C'//SYSIN    DD *',/,                                     
           C'  SORT FIELDS=COPY',/,                                 
           C'//SORTOUT  DD SYSOUT=(A,INTRDR)')                       
//*                                                                 



Gerry
Back to top
View user's profile Send private message
dick scherrer

Moderator Emeritus


Joined: 23 Nov 2006
Posts: 19244
Location: Inside the Matrix

PostPosted: Mon Jul 01, 2013 4:49 am
Reply with quote

Hello,

Quote:
But as I told they don't have to run in any particular order, you just go wham wham and submit all of them asap.
I surely hope NOT!

You will have considerable complaints from the Operatons and/or Scheduling people if they all have differet jobnames and run in different classes or even 1 class if there are several initiators with that class.

Quote:
Do you mean I can write a JCL with a JOB statement, that can trigger all those JCL members?
No. Not at all. It should not be difficult to create a JOB statement and copy all of the needed members into one sequential file, globally edit that single file, and submit this via the internal reader. It is a rather straightforward process and care should be taken Not to make it difficult.
Back to top
View user's profile Send private message
Dale Robertson

New User


Joined: 21 Jun 2013
Posts: 44
Location: U.S.A.

PostPosted: Mon Jul 15, 2013 11:29 pm
Reply with quote

Ani Shetty,

2 ways to do it. the first is with IEBPTPCH to produce a PS file. that's difficult and too many manual items to deal with though it can be done.

the second way is the best if and only if you have SAS. You need merely write a simple macro that reads in a list then creates a SELECT list for a super simple procedure called PROC SOURCE.

PROC SOURCE then does the heavy lifting for you that IEBPTPCH only does part way so that you can then feed the sequential file back into your PDS once you've made whatever changes you want to that sequential file - all the time keeping the manufactured control cards in place for IEBUPDTE to automatically feed back into your PDS.

Once you've done that then you really need to manually go to TSO 3.5 and reset the statistics to suit.

r
Back to top
View user's profile Send private message
Dale Robertson

New User


Joined: 21 Jun 2013
Posts: 44
Location: U.S.A.

PostPosted: Tue Jul 16, 2013 12:10 am
Reply with quote

Ani Shetty,

Also if you've decided to use either IEBPTPCH or SAS's PROC SOURCE and you've decided to use either DFSORT or SyncSort to maninpulate the sequential file, remember to utilize the EQUAL option.

The EQUAL option will ensure that your records are in the proper order because they are in a special sequence in order to be streamed into IEBUPDTE.

Make sense?

r
Back to top
View user's profile Send private message
Bill Woodger

Moderator Emeritus


Joined: 09 Mar 2011
Posts: 7309
Location: Inside the Matrix

PostPosted: Tue Jul 16, 2013 12:37 am
Reply with quote

It is EQUALS, not EQUAL.

Even if DFSORT/SyncSort are used, there is no need to actually SORT the data, so EQUALS will not come into it.
Back to top
View user's profile Send private message
Dale Robertson

New User


Joined: 21 Jun 2013
Posts: 44
Location: U.S.A.

PostPosted: Tue Jul 16, 2013 1:00 am
Reply with quote

Let me be a little more clear then please.

EQUALS ensures that the input sequence is the same before the SORT or MERGE instruction is invoked and probably right about the time of the INREC instruction.

from the DFSORT programming guide:
EQUALS The sequence of records that collate identically [even if COPY] is preserved from input to output - with either the MERGE or SORT instruction.

In this particular case obviously there will be no sort only copy because the IEBUPDTE instructions need to be in the particular order it was created using someting like IEBPTPCH. Sort was merely an example I used because it is a fine file-to-file utility. However, unless a programmer explicity uses EQUALS then the sort program ignores the previous collating sequence and the result though usually 99.9% correct [and often 100%] - there's no guarantee without being explicit.

That is a unique item to sort.
Back to top
View user's profile Send private message
dick scherrer

Moderator Emeritus


Joined: 23 Nov 2006
Posts: 19244
Location: Inside the Matrix

PostPosted: Tue Jul 16, 2013 1:03 am
Reply with quote

Hello,

Pleasde do not "explain" what most of us (helpers) already know and have provided advice concerning. . .

I see no reason that EQUAL/S has even been mentioned. It is surely not a benefit to the original question?
Back to top
View user's profile Send private message
Skolusu

Senior Member


Joined: 07 Dec 2007
Posts: 2205
Location: San Jose

PostPosted: Tue Jul 16, 2013 3:26 am
Reply with quote

Dale Robertson wrote:
from the DFSORT programming guide:
EQUALS The sequence of records that collate identically [even if COPY] is preserved from input to output - with either the MERGE or SORT instruction.


Dale Robertson,

Can you please point me the link where you have seen that in DFSORT programming guide?

Thanks
Back to top
View user's profile Send private message
Dale Robertson

New User


Joined: 21 Jun 2013
Posts: 44
Location: U.S.A.

PostPosted: Tue Jul 16, 2013 6:07 pm
Reply with quote

Skolusu,

publibz.boulder.ibm.com/cgi-bin/bookmgr_OS390/DOWNLOAD/ice1ca61.pdf?DN=SC26-7523-07&DT=20111118120619

publibz.boulder.ibm.com/cgi-bin/bookmgr_OS390/DOWNLOAD/ice1ci20.pdf?DN=SC26-7524-03&DT=20100625092952

publibz.boulder.ibm.com/cgi-bin/bookmgr_OS390/DOWNLOAD/ice1ct20.pdf?DN=SC26-7526-02&DT=20100625093551

publibz.boulder.ibm.com/cgi-bin/bookmgr_OS390/Shelves/all13be9?filter=dfsort

It would be wise if you actually did some tests as well as to read in all of the places where that option exists as reading will only get you so far because often times the documentation is incomplete or misleading sadly.

I gave plenty of test jcl in the past week but chances are that it was ignored and more probably that I will get harrassed further if I repeat it. It's a little discouraging.

Realize also that there is more documentation with SyncSort and even SAS Programming Tips: Guide to Efficient SAS Programming [1st Edition 1991] - which surprisingly gave the best advice on EQUALS!!!

Good luck.

r
Back to top
View user's profile Send private message
Akatsukami

Global Moderator


Joined: 03 Oct 2009
Posts: 1788
Location: Bloomington, IL

PostPosted: Tue Jul 16, 2013 8:17 pm
Reply with quote

Dale Robertson wrote:
It would be wise if you actually did some tests as well as to read in all of the places where that option exists as reading will only get you so far because often times the documentation is incomplete or misleading sadly.

Presumably you were not aware that Sri Kolusu is a DFSORT developer.
Back to top
View user's profile Send private message
dick scherrer

Moderator Emeritus


Joined: 23 Nov 2006
Posts: 19244
Location: Inside the Matrix

PostPosted: Tue Jul 16, 2013 8:40 pm
Reply with quote

Hello,

Quote:
Presumably you were not aware that Sri Kolusu is a DFSORT developer.

Probably not - along with not realizing that several here are quite knowledgable and have much deeper exposure/understanding . . .
Back to top
View user's profile Send private message
Dale Robertson

New User


Joined: 21 Jun 2013
Posts: 44
Location: U.S.A.

PostPosted: Tue Jul 16, 2013 8:42 pm
Reply with quote

then why don't you know anything about something as simplistic as EQUALS/NOEQUALS?

The reason is simple. you don't try anything on your own. you only read about it or take Bill Woodgr's word for it.
Back to top
View user's profile Send private message
dick scherrer

Moderator Emeritus


Joined: 23 Nov 2006
Posts: 19244
Location: Inside the Matrix

PostPosted: Tue Jul 16, 2013 8:55 pm
Reply with quote

Hello,

We are more aware of "sort stuff" than you are willing to believe.

Quote:
The reason is simple. you don't try anything on your own.
Possibly the silliest statement posted yet this year. We'll overlook it as frustration.

It must be tough believing you know more than just about anyone and being disagreed with by several of us . . .

Be aware that future posts like this will probably get edited/tossed.
Back to top
View user's profile Send private message
Skolusu

Senior Member


Joined: 07 Dec 2007
Posts: 2205
Location: San Jose

PostPosted: Tue Jul 16, 2013 9:38 pm
Reply with quote

Dale Robertson wrote:
Skolusu,

publibz.boulder.ibm.com/cgi-bin/bookmgr_OS390/DOWNLOAD/ice1ca61.pdf?DN=SC26-7523-07&DT=20111118120619

publibz.boulder.ibm.com/cgi-bin/bookmgr_OS390/DOWNLOAD/ice1ci20.pdf?DN=SC26-7524-03&DT=20100625092952

publibz.boulder.ibm.com/cgi-bin/bookmgr_OS390/DOWNLOAD/ice1ct20.pdf?DN=SC26-7526-02&DT=20100625093551

publibz.boulder.ibm.com/cgi-bin/bookmgr_OS390/Shelves/all13be9?filter=dfsort

It would be wise if you actually did some tests as well as to read in all of the places where that option exists as reading will only get you so far because often times the documentation is incomplete or misleading sadly.


Thank you very much for the DFSORT links. But my question is still unanswered, I want to see the specific link where EQUALS is used even for COPY operations as you seem to highlight here

Dale Robertson wrote:

from the DFSORT programming guide:
EQUALS The sequence of records that collate identically [even if COPY] is preserved from input to output - with either the MERGE or SORT instruction.


So can you please show me where the EQUALS is being used for COPY operation.


Dale Robertson wrote:

It would be wise if you actually did some tests as well as to read in all of the places where that option exists as reading will only get you so far because often times the documentation is incomplete or misleading sadly.


In your prior posts you were implying that documentation had it described and now you are saying that it is incomplete or misleading ?

As for testing how about you do just 1 test for me.

1. Create a file with LRECL=5042
2. Write 50 or as many as you feel like records into that file with the following data
Code:

Position 1 - 5000   = spaces
Position 5001       = 'DALE ROBERTSON DUPLICATE RECORD # '
Position 5034       = 8 byte sequence number to number the records



Now use a COPY operation to copy with NOEQUALS and show me that DFSORT did NOT COPY the records as is.

The maximum length of the collected control fields (sort fields) is 4088 bytes with Equals ON.

The above created dataset has spaces in first 5000 bytes and all the records are duplicates. So without EQUALS option I am not guaranteed to get the records as it is. ? Is that what you are inferring? Never Knew SORT products supported keys greater than 4k. The largest LRECL you can have on z/OS is 32767. Which utility would you use to COPY such files?

Dale Robertson wrote:

Realize also that there is more documentation with SyncSort and even SAS Programming Tips: Guide to Efficient SAS Programming [1st Edition 1991] - which surprisingly gave the best advice on EQUALS!!!

Good luck.

r


Please do NOT QUOTE Syncsort documentation as a source for how DFSORT works.
Back to top
View user's profile Send private message
Bill Woodger

Moderator Emeritus


Joined: 09 Mar 2011
Posts: 7309
Location: Inside the Matrix

PostPosted: Wed Jul 17, 2013 1:05 pm
Reply with quote

Dale, perhaps you can post the piece from the SAS documentation? Exactly as it is, with no editorial from yourself.
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic View Bookmarks
All times are GMT + 6 Hours
Forum Index -> JCL & VSAM

 


Similar Topics
Topic Forum Replies
No new posts submitting feedback for manuals All Other Mainframe Topics 3
No new posts How to create a list of SAR jobs with... CA Products 3
No new posts Help in Automating Batch JCL jobs mon... JCL & VSAM 3
No new posts How to append a PS file into multiple... JCL & VSAM 3
No new posts Submit multiple jobs from a library t... JCL & VSAM 14
Search our Forums:

Back to Top