|
View previous topic :: View next topic
|
| Author |
Message |
ManSab
New User
Joined: 23 Sep 2006
Posts: 13
Location: Pune
|
|
|
|
Need some expert advice on BMC unload execution time reduction:
Have a scenario where I need to change IMS key fields(data not definition). I am using BMC unload utility with an UserExit; where key data is changed. After successful unload with correction, I reload data back. I am able to accomplish on smaller databases.
However for larger databases (approx. 10 million record size), while unloading (with UserExit) the job ran more than 16 hours thrice before I cancelled it. DBA and I tried various options to reduce the execution time but no luck. Options we tried:
- Changed DFSVSAMP parameters (increased IOBF )
- Job class and Region parameter.
- Changed UserExit default BUFSPACE(20) to BUFSPACE(400)
- Few more setting based on BMC recommendation - DFSURGU1(SHORT), EXPAND(NO) ,LPCK(NO)
Unload JCL:
| Code: |
//UNLOAD EXEC PGM=DFSRRC00,PARM='ULU,DFSURGU0,XRXDB',
// REGION=3072K
//IMS DD DSN=XXXXXX.DBDLIB,DISP=SHR
//SYSPRINT DD SYSOUT=*
//DFSVSAMP DD DSN=XXXXXX.XXXXX.MACLIB(XRVSAMP),DISP=SHR
//DFSURGU1 DD DSN= XXXXXX.XXXXX.XRXRIDUN.SS301,
// UNIT=(TAPE,,DEFER),DCB=BUFNO=5,
// LABEL=(4,,,RETPD=0060),
// DISP=(NEW,CATLG,KEEP),
// VOLUME=(,RETAIN)
//DFSURGU2 DD DSN= XXXXXX.XXXXX.XRXRIDUN.SS302,
// UNIT=(TAPE,,DEFER),DCB=BUFNO=5,
// LABEL=4,
// DISP=(NEW,CATLG,KEEP),
// VOLUME=(,RETAIN)
//XRXDB DD DSN= XXXXXX.XXXXX.SS300.B4,
// DISP=(OLD,KEEP,KEEP)
//XRXDB DD DSN= XXXXXX.XXXXX.SS00I.B4,
// DISP=(OLD,KEEP,KEEP)
//PLUSIN DD *
UNLOAD USEREXIT(MDCTAK1) BUFSPACE(400) DFSURGU1(SHORT) EXPAND(NO) -
LPCK(NO)
//* |
Appreciate your advice ? - Thanks |
|
| Back to top |
|
 |
dick scherrer
Moderator Emeritus
Joined: 23 Nov 2006
Posts: 19243
Location: Inside the Matrix
|
|
|
|
Hello,
How many output tape volumes were created?
What was the wait time between mounts?
No unload of only 10mil records should take anywhere near that long. |
|
| Back to top |
|
|
Ed Goodman
Active Member
Joined: 08 Jun 2011
Posts: 556
Location: USA
|
|
|
|
How long does it take without the exit routine?
I'm asking because of a problem I had MAAAANNNNYYYY years ago doing something similar. (right after 9/11)
In my case, we neglected to make the exit routine program resident. So each segment passed to it would re-load the damn thing. It was taking forever too.
Here are the highlights from the compile JCL I made to make this work. It was COBOL II, and I'm certain there is a better way.
I had to rebuild the IGZEOPT module with reentrant/reuse (or something like that)
| Code: |
//CMP EXEC PGM=IGYCRCTL, REGION=1024K
// PARM=('BUF(32760),ADV,RENT,DATA(31),FDUMP,F(I)',
// ',NONUM,OFF',
// ',NOCMPR2,SEQ,NUMPROC(PFD),NOSSR,RES,NODYNAM')
//MYLIB DD DSN=WLF6.WNEG.LOAD,DISP=SHR
//SYSLIN DD DSN=&&LOADSET,DISP=(OLD,DELETE)
// DD DSN=WLF6.WNEG.JCL(ASMINCL),DISP=SHR
INCLUDE MYLIB(IGZEOPT)
|
I know that we ended up creating a new SCLM type for these exits, and that's what I always use now.
I'm hoping that someone here can tell you a better way to keep a program loaded. |
|
| Back to top |
|
|
Ed Goodman
Active Member
Joined: 08 Jun 2011
Posts: 556
Location: USA
|
|
|
|
This angered me that I couldn't remember...so I went hunting!
I found the SCLM hooks that are in use with Enterprise COBOL
| Code: |
Member WTSO.XXXX.SOURCE(CEEUOPT) has:
CEEUOPT CSECT
CEEUOPT AMODE ANY
CEEUOPT RMODE ANY
CEEXOPT RTEREUS=(ON),
TRAP=(OFF)
END
Assembled into
WTSO.XXXX.LOAD(CEEUOPT)
The SCLM language macro for the Link Edit has:
BMCLE370 FLMSYSLB WTSO.XXXX.LOAD
*
FLMLANGL LANG=BMCLE370,CANEDIT=N,VERSION=L370V1.0
FLMTRNSL CALLNAM='BMCLKED/370',
FUNCTN=BUILD,
COMPILE=HEWL,
VERSION=F64,
GOODRC=4,
PORDER=3,
OPTIONS=(MAP,XREF,LIST,SIZE=(256K,48K),LET,REUS)
*
* 1 (* SYSLIN *)
FLMALLOC IOTYPE=S,KEYREF=SINC,RECFM=FB,LRECL=80,
RECNUM=190000,DDNAME=SYSLIN
FLMCPYLB WTSO.XXXX.LOAD
*
* 2 (* LOAD MODULE NAME *)
FLMALLOC IOTYPE=L,KEYREF=LOAD
FLMCPYLB WTSO.XXXX.LOAD
*
* 3 (* SYSLMOD *)
FLMALLOC IOTYPE=P,KEYREF=LOAD,RECFM=U,LRECL=0,
RECNUM=190000,DIRBLKS=40,BLKSIZE=6144
FLMCPYLB WTSO.XXXX.LOAD
*
* 4 (* SYSLIB *)
FLMALLOC IOTYPE=A,DDNAME=SYSLIB
FLMCPYLB WTSO.XXXX.LOAD
FLMCPYLB SYS1.SCEELKED
FLMCPYLB SYS1.SIGYCOMP
FLMCPYLB SYS1.LINKLIB
The SCLM language parser (Compile) has:
FLMTRNSL CALLNAM='BMCEXIT',
FUNCTN=BUILD,
DSNAME=SYS1.SIGYCOMP,
COMPILE=IGYCRCTL,
VERSION=1.0,
GOODRC=4,
PORDER=3,
OPTIONS=(APOST,C,DATA(31),NODYN,LIB,NORENT,XREF,
OFF,MAP,NUMPROC(PFD),NOCMPR2,FLAG(I,W),
NOADV,TEST(NONE,SYM),LANGUAGE(EN),SEQ,BUF(32760))
|
I also realized that we have a vestigial line of code in all of our exits. At the start of the Procedure Division:
IF (FIRST-TIME-SWITCH EQUAL SPACES)
MOVE HIGH-VALUES TO FIRST-TIME-SWITCH
DISPLAY 'START DB LOAD'
This was added by me to show that the program only loaded once. You could add something similar to prove you have it working.
If it's not, you see that display statement for every input segment. |
|
| Back to top |
|
|
ManSab
New User
Joined: 23 Sep 2006
Posts: 13
Location: Pune
|
|
|
|
Ed Goodman :
- UserExit gets called, I did check with display statement including DISPLAY 'START DB LOAD' . Had to remove displays because of additional overhead and to avoid operations team questions.
- Unload without userexit completes in 60 sec in prod and in 80 seconds in test (approx).
- You may be spot on with program not being resident. I will check on this.
dick scherrer:
I don't see multiple volumes but one in job log. I will check with storage support to confirm.
Will keep you posted.... - Thanks |
|
| Back to top |
|
|
dick scherrer
Moderator Emeritus
Joined: 23 Nov 2006
Posts: 19243
Location: Inside the Matrix
|
|
|
|
Hello,
I believe you've shown that the userexit is the culprit.
Work with someone on your system who provides performance measurement support and see how much of the time is in the exit code (cpu or i/o or both).
As Ed mentioned, constantly loading a module over and over will cause a Lot of unneeded overhead.
Originally, i thought the time it took to mount tapes might be a problem. We can forget that as only one volume is needed  |
|
| Back to top |
|
|
ManSab
New User
Joined: 23 Sep 2006
Posts: 13
Location: Pune
|
|
|
|
Ed Goodman and dick scherrer:
I debugged the unload step (userexit), and it does get loaded every time segment is passed on to it. Will compile with these parameters now:
| Code: |
//COMPX EXEC PGM=IGYCRCTL,REGION=7M,
// PARM=(APOST,DYNAM,FLAG(W,E),LIB,MAP,NOSEQ,OFFSET,
// RENT,SSRANGE,SOURCE,XREF,VBREF,DATA(31),NOSSR,RES) |
Thanks |
|
| Back to top |
|
|
ManSab
New User
Joined: 23 Sep 2006
Posts: 13
Location: Pune
|
|
|
|
with compile parameter changes, job still running from 2 hours..
Ed Goodman :
Can you please elaborate this :
| Quote: |
I had to rebuild the IGZEOPT module with reentrant/reuse (or something like that)
|
|
|
| Back to top |
|
|
dick scherrer
Moderator Emeritus
Joined: 23 Nov 2006
Posts: 19243
Location: Inside the Matrix
|
|
|
|
Hello,
Might there be a parameter specific to BMC to tell the run Not to reload every time?
BMC support should be able to tell you how to proceed to get around all of these loads. |
|
| Back to top |
|
|
Ed Goodman
Active Member
Joined: 08 Jun 2011
Posts: 556
Location: USA
|
|
|
|
Like I said, this was 11 years ago. BMC may have changed something by now. I just remember that they couldn't help us then, and that the instructions/hints to change the compile were in the BMC manual.
Mansab, just changing the compile will NOT work. You also have to work with the link editor. The item you asked me to elaborate on is about changing the options that get used during link-edit.
If you are on Enterprise COBOL, you'll want to use what's in my second post dealing with the CEE version. |
|
| Back to top |
|
|
Ed Goodman
Active Member
Joined: 08 Jun 2011
Posts: 556
Location: USA
|
|
|
|
From my current BMC manual:
You may receive positive results by using NORENT and NODYNAM as compile options.
When using a COBOL/370 exit with the Unload function or the Reload function, you
may need to assemble an application-specific copy of module CEEUOPT with
RTEREUS(ON) and TRAP=OFF. If your current LE/370 run-time keywords module
specifies RTEREUS(OFF) and TRAP=ON, you can assemble a CEEUOPT module with
RTEREUS(ON) and TRAP=OFF to another library and INCLUDE it in the link-edit of your
user exit. The CEEUOPT module must be assembled with AMODE(31). For details
about the LE environment, see the IBM Language Environment manuals. |
|
| Back to top |
|
|
ManSab
New User
Joined: 23 Sep 2006
Posts: 13
Location: Pune
|
|
|
|
So this is how it crumbled down...........
per Ed advice and BMC Manual
1) Assembled and link/edit CEEUOPT parameter changes into a different LOADLIB.
| Code: |
//SYSIN DD *
CEEUOPT CSECT
CEEUOPT AMODE 31
CEEUOPT RMODE ANY
CEEXOPT RTEREUS=(ON),
TRAP=(OFF,NOSPIE)
END
/*
|
2) Included above CEEUOPT load during link-edit of the UserExit.
| Code: |
//LINK EXEC PGM=IEWL,
// PARM=(LIST,XREF,REUS,NORENT,AMODE(31),RMODE(ANY))
//SYSPRINT DD SYSOUT=*
//SYSTERM DD SYSOUT=*
//SYSLMOD DD DSN=XXXX.XXXX.LINKLIB,DISP=SHR
//SYSLIN DD DSN=&&COMP1,DISP=(OLD,PASS)
// DD *
INCLUDE SYSLIB(CEEUOPT)
NAME USEXCV80(R)
//SYSLIB DD DSN= XXXX. CEEUOPT.LOADLIB,DISP=SHR
|
3) Submitted unload job for 10 million records, and guess what it completed in 20 SECONDS.
Thanks much Ed and Dick, you guys are awesome..!!! |
|
| Back to top |
|
|
Ed Goodman
Active Member
Joined: 08 Jun 2011
Posts: 556
Location: USA
|
|
|
|
Can I get WOOT WOOT!?!?!
Seriously, how good do you feel right now? It's awesome when it works! |
|
| Back to top |
|
|
dick scherrer
Moderator Emeritus
Joined: 23 Nov 2006
Posts: 19243
Location: Inside the Matrix
|
|
|
|
What was the Hannibal Smith saying:
I love it when a plan comes together"
Good to hear it is working - thank you for letting us know 
Oh, yeah, aaand:
WOOT WOOT!?!?!
d |
|
| Back to top |
|
|
Ed Goodman
Active Member
Joined: 08 Jun 2011
Posts: 556
Location: USA
|
|
|
|
| Now, the question is, Is there a way to use 'RTEREUS=(ON)' for a single link-edit? Or does it have to be part of the CEEUOPT member? |
|
| Back to top |
|
|
|
|
|
