|
View previous topic :: View next topic
|
| Author |
Message |
Jeff Almond
New User
Joined: 30 Mar 2021
Posts: 4
Location: United Kingdom
|
|
|
|
First post here and looking for some suggestions.
I have approx 500 files that have been transferred to an MVS system. I need to copy them to PDS and considered IEBGENER. Wrapped the copy in a PROC and executed it like so:
| Code: |
//TOPDS PROC DSIN=MISSING.INPUT.FILE,DSOUT=NULL
//GENER EXEC PGM=IEBGENER
//SYSPRINT DD SYSOUT=*
//SYSIN DD DUMMY
//SYSUT1 DD DISP=SHR,DSN=&DSIN
//SYSUT2 DD DISP=SHR,
// DSN=AAAA.BBB.CCCCCC.DDDDDD.EEEEEEE.TEST(&DSOUT)
//SYSPRINT DD SYSOUT=*
//SYSIN DD DUMMY
// PEND
//PDS1 EXEC TOPDS,DSIN=XXXXXX.YYYYY.ZZZZZZZZ.WWWWW,DSOUT=ZZZZZZZ |
This works fine for a small number of datasets but errors with too many EXEC's if I have too many.
Q. What is the best way to copy these datasets? This is a regular job so don't want to be commenting out EXEC's to run only 100 or so at a time...
Code tags added |
|
| Back to top |
|
 |
Joerg.Findeisen
Senior Member
Joined: 15 Aug 2015
Posts: 1337
Location: Bamberg, Germany
|
|
|
|
| Consider using PDSE instead of PDS, first step. |
|
| Back to top |
|
|
Pandora-Box
Global Moderator
.jpg)
Joined: 07 Sep 2006
Posts: 1592
Location: Andromeda Galaxy
|
|
|
|
Do you want 500 physical sequential to be copied to one single PDS member?
Or a One to one mapping |
|
| Back to top |
|
|
Jeff Almond
New User
Joined: 30 Mar 2021
Posts: 4
Location: United Kingdom
|
|
|
|
| Pandora-Box wrote: |
Do you want 500 physical sequential to be copied to one single PDS member?
Or a One to one mapping |
500 sequential to 500 members |
|
| Back to top |
|
|
Pandora-Box
Global Moderator
Joined: 07 Sep 2006
Posts: 1592
Location: Andromeda Galaxy
|
|
|
|
| I would recommend for LMCOPY would come in handy I believe there should be samples in forum |
|
| Back to top |
|
|
Jeff Almond
New User
Joined: 30 Mar 2021
Posts: 4
Location: United Kingdom
|
|
|
|
| Pandora-Box wrote: |
| I would recommend for LMCOPY would come in handy I believe there should be samples in forum |
Thanks for the help, I'll read up on this. |
|
| Back to top |
|
|
sergeyken
Senior Member
Joined: 29 Apr 2008
Posts: 2142
Location: USA
|
|
|
|
| If you do not find LMCOPY to be useful for some reason, it can be also implemented in REXX, with complexity seriously below average. |
|
| Back to top |
|
|
sergeyken
Senior Member
Joined: 29 Apr 2008
Posts: 2142
Location: USA
|
|
|
|
| If you used a PDSE as your output library, and also split your 500-steps job into 2-50 smaller jobs (less than 255 steps each), then you might run several of your jobs in parallel. It may speed up the whole process, with minimum efforts required. |
|
| Back to top |
|
|
Willy Jensen
Active Member
Joined: 01 Sep 2015
Posts: 734
Location: Denmark
|
|
|
|
Something like this perhaps?
//ADD EXEC PGM=IEBUPDTE,PARM=NEW
//SYSPRINT DD SYSOUT=*
//SYSUT2 DD DISP=SHR,DSN=target.pds
//SYSIN DD *
./ ADD NAME=member1
// DISP=SHR,DSN=input.dsn1
./ ADD NAME=member2
// DISP=SHR,DSN=input.dsn2
and so on.. |
|
| Back to top |
|
|
sergeyken
Senior Member
Joined: 29 Apr 2008
Posts: 2142
Location: USA
|
|
|
|
Something is missing:
| Code: |
//ADD EXEC PGM=IEBUPDTE,PARM=NEW
//SYSPRINT DD SYSOUT=*
//SYSUT2 DD DISP=SHR,DSN=target.pds
//SYSIN DD *
./ ADD NAME=member1
// DD DISP=SHR,DSN=input.dsn1
// DD *
./ ADD NAME=member2
// DD DISP=SHR,DSN=input.dsn2
// DD *
./ ADD NAME=member3
. . . . and so on.. |
P.S.
Total number of DD statements per one single job step is also limited to some value... |
|
| Back to top |
|
|
Joerg.Findeisen
Senior Member
Joined: 15 Aug 2015
Posts: 1337
Location: Bamberg, Germany
|
|
|
|
| sergeyken wrote: |
P.S.
Total number of DD statements per one single job step is also limited to some value... |
Depending on TIOT Size set in ALLOC member. Sample:
| Code: |
| TIOT SIZE: 64 (MAX DDS: 3273) |
|
|
| Back to top |
|
|
steve-myers
Active Member
Joined: 30 Nov 2013
Posts: 917
Location: The Universe
|
|
|
|
A few thoughts.
Several people have suggested running several streams in parallel. It's dangerous and it won't really save much time even when the PDS is really a PDSE, which, in theory, can handle multiple streams adding members in parallel. Allocating the input data set is what uses the time.
The JCL only solution, similar to your initial idea, is viable, though you have to split it into several jobs because of limits on number of steps in a job. Don't worry about DD statement limits in a single step that some people have mentioned. No way you'll run into that limit.
I presume your input data sets all have a name like XXX.member.YYY. If you have even basic programming skills it should not be too terribly difficult to extract the member name portion of the data set name and prepare your copy job, whether it's JCL only or something like an IDCAMS run where you create
REPRO INDATASET( ... ) OUTDATASET( PDS(member))
The advantage of the IDCAMS solution is you can run all the REPRO commands as one ugly job step. |
|
| Back to top |
|
|
Joerg.Findeisen
Senior Member
Joined: 15 Aug 2015
Posts: 1337
Location: Bamberg, Germany
|
|
|
|
| steve-myers wrote: |
| The JCL only solution, similar to your initial idea, is viable, though you have to split it into several jobs because of limits on number of steps in a job. Don't worry about DD statement limits in a single step that some people have mentioned. No way you'll run into that limit. |
From my experience, at some point you will run into that limit. |
|
| Back to top |
|
|
steve-myers
Active Member
Joined: 30 Nov 2013
Posts: 917
Location: The Universe
|
|
|
|
| Joerg.Findeisen wrote: |
| steve-myers wrote: |
| The JCL only solution, similar to your initial idea, is viable, though you have to split it into several jobs because of limits on number of steps in a job. Don't worry about DD statement limits in a single step that some people have mentioned. No way you'll run into that limit. |
From my experience, at some point you will run into that limit. |
IEBGENER requires 4 DD statements. My IMPORT idea allocates 2 "temporary" allocations that can be freed by allocation if IMPORT is lazy about freeing its allocations. In 50+ years I don't think I've ever run into too many DD statements, not to be confused with running out of DD DYNAM DD statements in MVT. Get real. |
|
| Back to top |
|
|
Joerg.Findeisen
Senior Member
Joined: 15 Aug 2015
Posts: 1337
Location: Bamberg, Germany
|
|
|
|
| @steve-myers: I have no doubts that you have never hit the limit, but others have. Never rely on hard limits even if they seem far away. Just saying. |
|
| Back to top |
|
|
enrico-sorichetti
Superior Member
Joined: 14 Mar 2007
Posts: 10888
Location: italy
|
|
|
|
LMCOPY is not performancewise the best choice IMO
the ts would have to write anyway a rexx script to do it
I would write a rexx script to ...
read a file containing ...
source dataset name, target dataset name, target member name
for each record allocate input(sortin), output(sortout) , invoke dfsort to do the copy
the proper allocate with reuse and a free afterwards would take care of the number of allocated datasets limitations
( tested for 1000 datasets )
see here for the overall logic and the discussion of the reuse parameter
ibmmainframes.com/viewtopic.php?t=57579&highlight=alloc+reuse
and here
ibmmainframes.com/viewtopic.php?t=52353&postdays=0&postorder=asc&start=25 |
|
| Back to top |
|
|
Jeff Almond
New User
Joined: 30 Mar 2021
Posts: 4
Location: United Kingdom
|
|
|
|
| Thanks to all who took the time and replied. I'm going to go with the IDCAMS suggestion because as stated "you can run all the REPRO commands as one ugly job step" and this is really the driver. I've read up on all the other suggestions and learned things in the process so whilst I may not be using your suggestion it has been of value to me. Once again, thanks all. |
|
| Back to top |
|
|
Pandora-Box
Global Moderator
Joined: 07 Sep 2006
Posts: 1592
Location: Andromeda Galaxy
|
|
|
|
Hi Enrico
Thanks much better approach indeed  |
|
| Back to top |
|
|
|
|
