IBM Mainframe Forum Index
 
Log In
 
IBM Mainframe Forum Index Mainframe: Search IBM Mainframe Forum: FAQ Register
 

Add a step counter in a stack ?


IBM Mainframe Forums -> CLIST & REXX
Post new topic   Reply to topic
View previous topic :: View next topic  
Author Message
rodferrn

New User


Joined: 14 Jul 2020
Posts: 14
Location: Brazil

PostPosted: Fri Jul 17, 2020 7:00 am
Reply with quote

Hi everyone, so this is my first post ever however I've been following this forum for so many years now and I always enjoy reading all the content.
So, HEY!

Basically, as I mentioned in the subject, I'm looking for a way to add a step counter inside my rexx stack, however, I reached a point where I'm not sure where to go from here.. basically, my rexx will read data from 2 datasets (source / target) and for each 1 row that is read on these datasets, 4 job steps are created, so first I do write a job card to the member and then once each dataset is read those from source/target the new 4 steps are created, so every time the stack resets and goes over again until it reaches the end of the file..

Is there a way to put a step counter for each step that it is generated, however, do not reset this counter ?? I was investigating MAKEBUFF command.. perhaps this is what I am looking for ??

Follow the code;

As you may see, each step has a STEP'S' on it, I tried putting a counter with S = 0 ; S = S + 1 but it didnt work due the stack I guess..

Appreciate the support if possible.

Thanks !

Code:

/*-------------------------------------------------------------------*/     
 "newstack"                        /*                                */     
 Outn=0                            /* Clear the variable  OutNumber  */     
 JobN=0                            /* Clear the variable: JobNumber  */     
/*--------------------------------*/                                         
 do dsn=1 to dsls.0                 /* init DataSetList for Source    */     
 do dsn=1 to dslt.0                 /* init DataSetList for Target    */     
                                                                             
    SRCName = WORD(DSLs.dsn,1)      /* Variable to identify source dataset */
    TGTName = WORD(DSLt.dsn,1)      /* Variable to identify target dataset */
                                                                             
/* QUEUE Below is used to generate steps whenever a dataset is read from     
   Source and Target members contained in JobLib                     */     
                                                                             
    QUEUE '//STEP'S'  EXEC PGM=IKJEFT01'                                     
    QUEUE '//SYSTSPRT  DD  SYSOUT=*'                                         
    QUEUE '//SYSTSIN   DD  *'                                               
    QUEUE ' PROF NOPREF '                                                   
    QUEUE ' HSEND FRRECOV FROMCOPYPOOL('CPNAME') - '                         
    QUEUE ' DSNAME('SRCName') - '                                           
    QUEUE ' NEWNAME('TGTName') '                                             
    QUEUE '//STEP'S'  EXEC PGM=IKJEFT01,DYNAMNBR=20            '             
    QUEUE '//SYSPROC  DD  DSN=DB2.CLONE.REXX,DISP=SHR  '               
    QUEUE '//SYSTSPRT DD  SYSOUT=*  '                                   
    QUEUE '//SYSTSIN  DD  * '                                           
    QUEUE '  %SLEEP 5 '                                                 
    QUEUE '//STEP'S'  EXEC PGM=IDCAMS '                                 
    QUEUE '//SYSPRINT DD   SYSOUT=* '                                   
    QUEUE '//SYSIN    DD   * '                                         
    QUEUE ' ALTER 'TGTNAME' STORCLAS(SCSTD) '                           
    QUEUE '//STEP'S'   EXEC PGM=IKJEFT01 '                             
    QUEUE '//SYSTSPRT  DD  SYSOUT=* '                                   
    QUEUE '//SYSTSIN   DD  * '                                         
    QUEUE '  PROF NOPREF '                                             
    QUEUE '  HMIGRATE ('TGTNAME') MOVE '                               
                                                                       
    if queued()//N_Split=0 then call Write                             
 end                               /*                                */
    if queued()>0 then call write  /*                                */
      cc=bpxwdyn('free dd('dd1')') /*                                */
 "delstack"                        /*                                */
 exit 0                            /*                                */
Back to top
View user's profile Send private message
prino

Senior Member


Joined: 07 Feb 2009
Posts: 1306
Location: Vilnius, Lithuania

PostPosted: Fri Jul 17, 2020 10:56 am
Reply with quote

Two do's, one end. Your code will not work.
Back to top
View user's profile Send private message
rodferrn

New User


Joined: 14 Jul 2020
Posts: 14
Location: Brazil

PostPosted: Fri Jul 17, 2020 11:25 am
Reply with quote

Now that you mentioned... its weird but its working as I need.... will look into it thank you.. but still haven't figured out that counter thou.. :-(
Back to top
View user's profile Send private message
Willy Jensen

Active Member


Joined: 01 Sep 2015
Posts: 712
Location: Denmark

PostPosted: Fri Jul 17, 2020 7:22 pm
Reply with quote

Variables are not affected by the stack. Both DOs are using the same control variable, surely that is not intended? And I can't see you setting the S variable anywhere. You need to initialise S before the first DO and then update it inside the second DO.
Back to top
View user's profile Send private message
rodferrn

New User


Joined: 14 Jul 2020
Posts: 14
Location: Brazil

PostPosted: Fri Jul 17, 2020 9:46 pm
Reply with quote

Thanks Willy for the tip, so I managed to add the step counter.
I know the code doesn't look perfect, I still need to go thru and review it and will spend some time always to improve and remove unnecessary stuff and adding more stuff, I always wanted to do coding so I am enjoying it each successful step (and learning with the bad choices lol)

So, following your tip, I managed to add the step counter.. but I got hit by another issue.. for each listed dataset in my input file, 4 steps are generated.. so the step counter is adding a value on a 4 step scale .. would there be a way to add one step counter for each one of the 4 steps ? like .. something continuous STEP1 , STEP2 , STEP3 , STEP4 , STEP 5 , STEP6.. perhaps create one variable for each of the 4 QUEUED steps ?

Here's the code now;

Code:

/*-------------------------------------------------------------------*/       
 "newstack"                        /*                                */       
 Outn=0                            /* Clear the variable  OutNumber  */       
 JobN=0                            /* Clear the variable: JobNumber  */       
 S=0                               /* Variable for Step Counter      */       
/*--------------------------------*/                                         
 do dsn=1 to dsls.0                 /* init DataSetList for Source    */     
 do dsn=1 to dslt.0                 /* init DataSetList for Target    */     
/**********************************/                                         
 If S>=0 then                       /* Init Step counter              */     
    do                                                                       
      S = S + 1                                                               
    end                                                                       
                                                                             
    SRCName = WORD(DSLs.dsn,1)      /* Variable to identify source dataset */
    TGTName = WORD(DSLt.dsn,1)      /* Variable to identify target dataset */
                                                                             
/* QUEUE Below is used to generate steps whenever a dataset is read from     
   Source and Target members contained in JobLib                     */       
                                                                             
    QUEUE '//STEP'S'  EXEC PGM=IKJEFT01'                                     
    QUEUE '//SYSTSPRT  DD  SYSOUT=*'                                         
    QUEUE '//SYSTSIN   DD  *'                                     
    QUEUE ' PROF NOPREF '                                         
    QUEUE ' HSEND FRRECOV FROMCOPYPOOL('CPNAME') - '             
    QUEUE ' DSNAME('SRCName') - '                                 
    QUEUE ' NEWNAME('TGTName') '                                 
    QUEUE '//STEP'S'  EXEC PGM=IKJEFT01,DYNAMNBR=20            ' 
    QUEUE '//SYSPROC  DD  DSN=DB2.CLONE.REXX,DISP=SHR  '         
    QUEUE '//SYSTSPRT DD  SYSOUT=*  '                             
    QUEUE '//SYSTSIN  DD  * '                                     
    QUEUE '  %SLEEP 5 '                                           
    QUEUE '//STEP'S'  EXEC PGM=IDCAMS '                           
    QUEUE '//SYSPRINT DD   SYSOUT=* '                             
    QUEUE '//SYSIN    DD   * '                                   
    QUEUE ' ALTER 'TGTNAME' STORCLAS(SCSTD) '                     
    QUEUE '//STEP'S'   EXEC PGM=IKJEFT01 '                       
    QUEUE '//SYSTSPRT  DD  SYSOUT=* '                             
    QUEUE '//SYSTSIN   DD  * '                                   
    QUEUE '  PROF NOPREF '                                       
    QUEUE '  HMIGRATE ('TGTNAME') MOVE '                         
                                                                 
    if queued()//N_Split=0 then call Write                       
 end                               /* end do for dsls (source)       */
    if queued()>0 then call write  /*                                */
      cc=bpxwdyn('free dd('dd1')') /*                                */
 end                               /* end do for dslt (target)       */
 "delstack"                        /*                                */
 exit 0                            /*                                */


Thanks again for all the valuable feedback.
Back to top
View user's profile Send private message
rodferrn

New User


Joined: 14 Jul 2020
Posts: 14
Location: Brazil

PostPosted: Fri Jul 17, 2020 9:59 pm
Reply with quote

Sorry for double post, just to provide an example on how the job gets created after running code updated provided in my last update;
So it will generate a STEP1 4 times , then moves to STEP2 4 times.. and so on .. what I'm trying to accomplish here is to setup some variables that for each step it will add a step counter and keep updating that counter once every dataset is referenced.

Code:

//*--------------------------------------------------------- 
//* Job generated by FRRECOV rexx program                     
//*--------------------------------------------------------- 
//STEP1  EXEC PGM=IKJEFT01                                   
//SYSTSPRT  DD  SYSOUT=*                                     
//SYSTSIN   DD  *                                             
 PROF NOPREF                                                 
 HSEND FRRECOV FROMCOPYPOOL(DSN$BHSAPPWS$LG) -               
 DSNAME(PWS1.ADSNLOAD) -                                     
 NEWNAME(SID1.ADSNLOAD)                                       
//STEP1  EXEC PGM=IKJEFT01,DYNAMNBR=20                       
//SYSPROC  DD  DSN=DB2.CLONE.REXX,DISP=SHR                   
//SYSTSPRT DD  SYSOUT=*                                       
//SYSTSIN  DD  *                                             
  %SLEEP 5                                                   
//STEP1  EXEC PGM=IDCAMS                                     
//SYSPRINT DD   SYSOUT=*                                     
//SYSIN    DD   *                                             
 ALTER SID1.ADSNLOAD STORCLAS(SCSTD)                         
//STEP1   EXEC PGM=IKJEFT01                                   
//SYSTSPRT  DD  SYSOUT=*                                     
//SYSTSIN   DD  *                                             
   PROF NOPREF                                   
   HMIGRATE (SID1.ADSNLOAD) MOVE                 
 //STEP2  EXEC PGM=IKJEFT01                       
 //SYSTSPRT  DD  SYSOUT=*                         
 //SYSTSIN   DD  *                               
  PROF NOPREF                                     
  HSEND FRRECOV FROMCOPYPOOL(DSN$BHSAPPWS$LG) -   
  DSNAME(PWS1.CONFIG.DWS1PSMP) -                 
  NEWNAME(SID1.CONFIG.DWS1PSMP)                   
 //STEP2  EXEC PGM=IKJEFT01,DYNAMNBR=20           
 //SYSPROC  DD  DSN=DB2.CLONE.REXX,DISP=SHR       
 //SYSTSPRT DD  SYSOUT=*                         
 //SYSTSIN  DD  *                                 
   %SLEEP 5                                       
 //STEP2  EXEC PGM=IDCAMS                         
 //SYSPRINT DD   SYSOUT=*                         
 //SYSIN    DD   *                               
  ALTER SID1.CONFIG.DWS1PSMP STORCLAS(SCSTD)     
 //STEP2   EXEC PGM=IKJEFT01                     
 //SYSTSPRT  DD  SYSOUT=*                         
 //SYSTSIN   DD  *                               
  PROF NOPREF                           
  HMIGRATE (SID1.CONFIG.DWS1PSMP) MOVE 
Back to top
View user's profile Send private message
Nic Clouston

Global Moderator


Joined: 10 May 2007
Posts: 2455
Location: Hampshire, UK

PostPosted: Sat Jul 18, 2020 1:46 am
Reply with quote

Code:
    QUEUE ' DSNAME('SRCName') - '                                 
    QUEUE ' NEWNAME('TGTName') '                                 
    s = s + 1
    QUEUE '//STEP'S'  EXEC PGM=IKJEFT01,DYNAMNBR=20            ' 
    QUEUE '//SYSPROC  DD  DSN=DB2.CLONE.REXX,DISP=SHR  '         
Back to top
View user's profile Send private message
Willy Jensen

Active Member


Joined: 01 Sep 2015
Posts: 712
Location: Denmark

PostPosted: Sat Jul 18, 2020 2:50 am
Reply with quote

Updated sample. I have made the assumption that the stems DSNL and DSLT are paired, so that source dataset 1 is in DSNLS.1 and target dataset 1 is in DSNLT.1, and so forth. Hence you only need one loop. Then I understand that you want each and every step number increased. The step number will be right-adjusted 4 bytes wide left-padded with 0. Finally I have used EXECIO directly instead of calling the WRITE routine. I hope that this is what you had in mind.
Code:
 "newstack"                        /*                                */     
 Outn=0                            /* Clear the variable  OutNumber  */     
 JobN=0                            /* Clear the variable: JobNumber  */     
 S=0                               /* Variable for Step Counter      */     
 do dsn=1 to dsls.0                /* init DataSetList for Source    */     
   SRCName = WORD(DSLs.dsn,1)      /* Variable to identify source dataset */
   TGTName = WORD(DSLt.dsn,1)      /* Variable to identify target dataset */
/* QUEUE Below is used to generate steps whenever a dataset is read from     
   Source and Target members contained in JobLib                     */     
   S = S + 1                       /* increase step nr               */     
   QUEUE '//STEP'right(s,4,0)'  EXEC PGM=IKJEFT01'                                     
   QUEUE '//SYSTSPRT  DD  SYSOUT=*'                                         
   QUEUE '//SYSTSIN   DD  *'                                                 
   QUEUE ' PROF NOPREF '                                                     
   QUEUE ' HSEND FRRECOV FROMCOPYPOOL('CPNAME') - '                         
   QUEUE ' DSNAME('SRCName') - '                                             
   QUEUE ' NEWNAME('TGTName') '                                             
   S = S + 1                       /* increase step nr               */     
   QUEUE '//STEP'right(s,4,0)'  EXEC PGM=IKJEFT01,DYNAMNBR=20'             
   QUEUE '//SYSPROC  DD  DSN=DB2.CLONE.REXX,DISP=SHR  '                     
   QUEUE '//SYSTSPRT DD  SYSOUT=*  '                                         
   QUEUE '//SYSTSIN  DD  * '                                                 
   QUEUE '  %SLEEP 5 '                                                       
   S = S + 1                       /* increase step nr               */     
   QUEUE '//STEP'right(s,4,0)'  EXEC PGM=IDCAMS '                                       
   QUEUE '//SYSPRINT DD   SYSOUT=* '                                         
   QUEUE '//SYSIN    DD   * '                                               
   QUEUE ' ALTER 'TGTNAME' STORCLAS(SCSTD) '                                 
   S = S + 1                       /* increase step nr               */     
   QUEUE '//STEP'right(s,4,0)'   EXEC PGM=IKJEFT01 '                                   
   QUEUE '//SYSTSPRT  DD  SYSOUT=* '                                         
   QUEUE '//SYSTSIN   DD  * '                                               
   QUEUE '  PROF NOPREF '                                                   
   QUEUE '  HMIGRATE ('TGTNAME') MOVE '                                     
   if queued()//N_Split=0 then "execio" queued() "diskr" dd1 /* write*/     
 end                               /* end do for dsls (source)       */     
 "execio" queued() "diskr" dd1 "(finis)"  /* write and close         */     
 cc=bpxwdyn('free dd('dd1')')      /*                                */     
 "delstack"                        /*                                */     
 exit 0                            /*                                */     

Question, why sub-setting the writes, do you really expect thousands of lines generated?
Back to top
View user's profile Send private message
Joerg.Findeisen

Senior Member


Joined: 15 Aug 2015
Posts: 1231
Location: Bamberg, Germany

PostPosted: Sat Jul 18, 2020 3:16 am
Reply with quote

A job can have a maximum of 255 job steps. This maximum includes all steps in any procedures the EXEC statements call. icon_exclaim.gif

Also remove the unnecessary SLEEP step and modify the HSEND to make use of the WAIT parameter instead.
Code:
   QUEUE ' HSEND WAIT FRRECOV FROMCOPYPOOL('CPNAME') - '                         
   QUEUE ' DSNAME('SRCName') - '                                             
   QUEUE ' NEWNAME('TGTName') '
Back to top
View user's profile Send private message
rodferrn

New User


Joined: 14 Jul 2020
Posts: 14
Location: Brazil

PostPosted: Sat Jul 18, 2020 3:26 am
Reply with quote

Hi Willy ! The sample you wrote worked exactly the way I needed !
Thank you very very much for your help, and indeed, these source/target dataset records works in pair, so whenever a record is added on those steps from source, the same record (however with different HLQ) will be added on TGTNAME.

yes, in fact I'm expecting thousand lines to be retrieved but wanted to add the step counter so I can split the jobs across different members whenever it reaches the maximum number of 255 steps (240 is the number I've had in mind).

Also, thanks for explaining each update you've made to the sample, the good thing is that I was able to understand so I believe I'm getting there.

Thanks again!
Back to top
View user's profile Send private message
rodferrn

New User


Joined: 14 Jul 2020
Posts: 14
Location: Brazil

PostPosted: Sat Jul 18, 2020 3:28 am
Reply with quote

Joerg.Findeisen wrote:
A job can have a maximum of 255 job steps. This maximum includes all steps in any procedures the EXEC statements call. icon_exclaim.gif

Also remove the unnecessary SLEEP step and modify the HSEND to make use of the WAIT parameter instead.
Code:
   QUEUE ' HSEND WAIT FRRECOV FROMCOPYPOOL('CPNAME') - '                         
   QUEUE ' DSNAME('SRCName') - '                                             
   QUEUE ' NEWNAME('TGTName') '


Hi Joerg, thanks for the HSEND WAIT suggestion, I'll look into it and definitely remove the SLEEP step. Nice one !! Just added this sleep step because each step was processed to fast causing HSM to fail the next FRRECOV command.

Thank you so much!
Back to top
View user's profile Send private message
Joerg.Findeisen

Senior Member


Joined: 15 Aug 2015
Posts: 1231
Location: Bamberg, Germany

PostPosted: Sat Jul 18, 2020 10:57 am
Reply with quote

@rodferrn: When you are able to issue DFHSM administrator commands, do not mix with user commands.

Change:
Code:
QUEUE '  HMIGRATE ('TGTNAME') MOVE '

to:
Code:
QUEUE '  HSEND WAIT MIGRATE DSN('TGTNAME') MOVE '
Back to top
View user's profile Send private message
rodferrn

New User


Joined: 14 Jul 2020
Posts: 14
Location: Brazil

PostPosted: Sat Jul 18, 2020 12:08 pm
Reply with quote

Joerg, I really have no words to thank you enough for all the good suggestions.

By removing the SLEEP step (it also reduced A LOT the number of steps in the jobs) and adding the HSEND WAIT command on both FRRECOV and MIGRATE it reduced the jobs runtime considerable, below, same number of steps and same datasets ;

Old run using sleep;
2.38 MINUTES EXECUTION TIME

New run without Sleep and adding WAIT parm
0.20 MINUTES EXECUTION TIME

Thank you folks!! Will keep trying my best ! icon_biggrin.gif
Back to top
View user's profile Send private message
Willy Jensen

Active Member


Joined: 01 Sep 2015
Posts: 712
Location: Denmark

PostPosted: Sat Jul 18, 2020 5:04 pm
Reply with quote

Well, if you really want to reduce the number of steps, then remember that the ALTER command can also be issued from TSO. So you could run everything in one step. Or even from your REXX. Or write a small driver REXX doing one block of HSEND, ALTER and HMIGRATE and then your original REXX would just generate the call to that REXX. Many ways to skin that particular cat.
Back to top
View user's profile Send private message
rodferrn

New User


Joined: 14 Jul 2020
Posts: 14
Location: Brazil

PostPosted: Wed Jul 22, 2020 3:55 am
Reply with quote

Thanks for your feedback Willy, the problem I have in running everything in a single step is that, when it comes to multiple files (such as application vsam tablespace files) then I would need basically to double the amount of storage in my storage group, because first I would do the FRRECOV with newname (for all datasets) and then proceed with alter storage class and hsm move. So in order to have this workaround working I would need to treat each dataset at once after proceeding to the next one.

If you allow me to do a one more quick question, I'm trying to use the LMDINIT / LMDLIST for VSAM Cluster/Data however, even by coding the DSNDBC (to list only cluster datasets) still my output shows both cluster and data files, any ideas why ?

Follow the code;

Code:

"ISPEXEC LMDINIT LISTID(IDSID) LEVEL(SAP"SID".DSNDBC.*)"           
cnt=1                                                             
DO FOREVER                                                         
   "ISPEXEC LMDLIST LISTID("IDSID") OPTION(LIST) DATASET(DSSID) " 
   IF RC = 0 THEN                                                 
    DO                                                             
     DATA.CNT = DSSID                                             
     CNT = CNT + 1                                                 
    END                                                           
   ELSE LEAVE                                                     
END                                                               


Thanks again for all the amazing support and suggestions.
Att.
Back to top
View user's profile Send private message
Willy Jensen

Active Member


Joined: 01 Sep 2015
Posts: 712
Location: Denmark

PostPosted: Wed Jul 22, 2020 2:02 pm
Reply with quote

Interesting, LMDLIST returns the data- and index components, even though they do not match the LEVEL parameter.
You can add STATS(YES) to LMDLIST and then test the ZDLSIZE variable for being null, meaning cluster.
Back to top
View user's profile Send private message
rodferrn

New User


Joined: 14 Jul 2020
Posts: 14
Location: Brazil

PostPosted: Fri Jul 24, 2020 12:46 am
Reply with quote

Hi Willy, thanks for you reply again, I've been reading about the STATS(YES) and ZDLSIZE but I'm still trying to make sense of those 2 lol.

Ugh, it is funny how we move from a plan to another when we are trying to design a solution .. I've just removed the N_Split variable because for that one I was spamming multiple jobs based on the number of rows, but once JCL has a 255 steps/execs constraint I have to limit jobs to 255, given the number of vsam cluster datasets I have for my application this might even generate hundreds of jobs.. so I'm looking into some way to run a single step where it invoke the rexx and the rexx script will run the 3 set of commands for all datasets ... right now I was trying to limit the jobs generation by step counter .. I tried IF S = 255 then call Write however this will limit one single job with 255 steps ..

Will look into something like you said (running a single JCL where the rexx is invoked and all commands are processed by the rexx itself).

Anyway, just with the good tips and suggestions this is how the procedure look so far ... (its small work but its decent work ha!) icon_biggrin.gif

Code:

/*********************************************************************/     
/* During program init insert copy pool name to generate frrecov jobs*/     
/*********************************************************************/     
SAY 'Insert Copy Pool name eg. DSN$BHSAPPWS$DB / DSN$BHSAPPWS$LG'           
PULL CPNAME                                                                 
CPNAME = CPNAME                                                             
                                                                           
/* If user does not provide CPNAME fail exec */                             
 IF CPNAME = '' THEN DO                                                     
                     SAY ' MISSING COPY POOL NAME '                         
                     SAY ' PLEASE REPEAT EXEC AND INPUT COPY POOL NAME '   
                     EXIT 16                                               
                END                                                         
/*-------------------------------------------------------------------*/     
 CC=BPXWDYN('ALLOC DA('JobLib'('SRCDSN')) SHR RTDDN(DD1)')                 
 "EXECIO * DISKR" DD1 "(STEM DSLs. FINIS)"                                 
 CC=BPXWDYN('FREE DD('DD1')')  /* Open SRC Member in PDS for data reading */
                                                                           
 CC=BPXWDYN('ALLOC DA('JobLib'('TGTDSN')) SHR RTDDN(DD2)')                 
 "EXECIO * DISKR" DD2 "(STEM DSLt. FINIS)"                                 
 CC=BPXWDYN('FREE DD('DD2')')  /* Open TGT Member in PDS for data reading */
/*-------------------------------------------------------------------*/     
 "newstack"                        /*                                */       
 Outn=0                            /* Clear the variable  OutNumber  */       
 JobN=0                            /* Clear the variable: JobNumber  */       
 S=0                               /* Variable for Step Counter      */       
/*--------------------------------*/                                         
 do dsn=1 to dsls.0                 /* init DataSetList for Source    */     
 do dsn=1 to dslt.0                 /* init DataSetList for Target    */     
/**********************************/                                         
                                                                             
    SRCName = WORD(DSLs.dsn,1)      /* Variable to identify source dataset */
    TGTName = WORD(DSLt.dsn,1)      /* Variable to identify target dataset */
                                                                             
/* QUEUE Below is used to generate steps whenever a dataset is read from     
   Source and Target members contained in JobLib                     */       
                                                                             
    S = S + 1                                                                 
    QUEUE '//STEP'right(s,4,0)'  EXEC PGM=IKJEFT01'                           
    QUEUE '//SYSTSPRT  DD  SYSOUT=*'                                         
    QUEUE '//SYSTSIN   DD  *'                                                 
    QUEUE ' PROF NOPREF '                                                     
    QUEUE ' HSEND WAIT FRRECOV FROMCOPYPOOL('CPNAME') - '                     
    QUEUE ' DSNAME('SRCName') - '                                         
    QUEUE ' NEWNAME('TGTName') '                                           
    S = S + 1                                                             
    QUEUE '//STEP'right(s,4,0)'  EXEC PGM=IDCAMS '                         
    QUEUE '//SYSPRINT DD   SYSOUT=* '                                     
    QUEUE '//SYSIN    DD   * '                                             
    QUEUE ' ALTER 'TGTNAME' STORCLAS(SCSTD) '                             
    S = S + 1                                                             
    QUEUE '//STEP'right(s,4,0)'  EXEC PGM=IKJEFT01 '                       
    QUEUE '//SYSTSPRT  DD  SYSOUT=* '                                     
    QUEUE '//SYSTSIN   DD  * '                                             
    QUEUE '  PROF NOPREF '                                                 
    QUEUE '  HSEND WAIT MIGRATE - '                                       
    QUEUE '  DSN('TGTNAME') MOVE '                                         
                                                                           
    CALL Write                                                             
                                                                           
 end                               /* end do for dsls (source)       */   
      cc=bpxwdyn('free dd('dd1')') /*                                */   
 "delstack"                        /*                                */   
 exit 0                            /*                                */   
/*-------------------------------------------------------------------*/     
/* ---  Start call Write      -------------------------------------- */     
/*-------------------------------------------------------------------*/     
Write:                                                                       
 Outn=Outn+1                       /*Add +1 to the OutNumber variable*/     
 Jobn=Jobn+1                       /*Add +1 to the JobNumber variable*/     
                                                                             
 say 'New Job 'NameRun''Outn' created at record' dsn 'will contain',         
     queued() 'records'  /* Print on user screen how many Jobs are created */
                                                                             
 CC=BPXWDYN('ALLOC DA('JobLib'('NameRun''Outn')) SHR DD('DD1') REUSE')       
                                                                             
 call JobCard                      /* CALL JobCard insert jcl card   */     
 "EXECIO" QUEUED() "DISKW" DD1 "(FINIS)"                                     
 Return 0                                                                   
/*-------------------------------------------------------------------*/ 
/*-------------------------------------------------------------------*/ 
/* ---  Start call JobCard    -------------------------------------- */ 
/*-------------------------------------------------------------------*/ 
JobCard:                                                                 
If Jobn>=0 then                                                         
   do                              /*                                */ 
     Jobn = Jobn + 0                                                     
   end                             /*                                */ 
jcl.1='//FRRC'Jobn'  JOB  (@HSG,BA01),SP3,CLASS=A,MSGCLASS=H,      '     
jcl.2='//         MSGLEVEL=(1,1),NOTIFY=&SYSUID,REGION=0M          '     
jcl.3='//*---------------------------------------------------------'     
jcl.4='//* Job generated by FRRECOV rexx program                   '     
jcl.5='//*---------------------------------------------------------'     
jcl.0=5                                                                 
                                   /*                                */ 
do n=jcl.0 to 1 by -1              /*                                */ 
   push jcl.n                      /*                                */ 
end                                /*                                */ 
Return                                                                   


PS. Btw this one doesnt have counter to limit each job by 255 steps.. I couldn't find that yet.. perhaps something simple.. but I need to go deep in my research and study.

Cheers folks!!
Back to top
View user's profile Send private message
Pedro

Global Moderator


Joined: 01 Sep 2006
Posts: 2546
Location: Silicon Valley

PostPosted: Fri Jul 24, 2020 4:21 am
Reply with quote

I do not like that variable DSN is used for both of these DO clauses. I think it should be two different variable names (though I did not examine the earlier logic thoroughly.

Code:
 do dsn=1 to dsls.0                 /* init DataSetList for Source    */     
 do dsn=1 to dslt.0                 /* init DataSetList for Target    */     


More likely, there is a one to one correspondence in your dsls. and dslt. stem variables. It seems more plausible that you only need one DO loop here.


Quote:
some way to run a single step where it invoke the rexx

I think you need to rely heavily on ADDRESS TSO to issue your commands.
Code:
Address TSO
'PROF NOPREF '             

/*--------------------------------*/                                         
Do dsn=1 to dsls.0                  /* init DataSetList for Source    */     
                                              /* init DataSetList for Target    */   
                                                                             
    SRCName = WORD(DSLs.dsn,1)      /* Variable to identify source dataset */
    TGTName = WORD(DSLt.dsn,1)      /* Variable to identify target dataset */   
     
    'HSEND WAIT FRRECOV FROMCOPYPOOL('CPNAME') ',
       'DSNAME('SRCName') ',
       'NEWNAME('TGTName') '

    'ALTER 'TGTNAME' STORCLAS(SCSTD) '

    'HSEND WAIT MIGRATE ',
       'DSN('TGTNAME') MOVE '

  End
End


The use of WAIT on FRRECOV makes this possible. I do not think WAIT on MIGRATE is needed unless you are very tight on storage.
Back to top
View user's profile Send private message
Willy Jensen

Active Member


Joined: 01 Sep 2015
Posts: 712
Location: Denmark

PostPosted: Fri Jul 24, 2020 7:07 pm
Reply with quote

The double 'DO's have been noted before. Something else, if you just want to look at catalog info then you can use the IGGCSI00 program, which does not incur the cost of reading the VTOC like LMDLIST STATS(YES) does. IBM supplies a sample in SYS1.SAMPLIB(IGGCSIRX) , but there are numerous other implementations. Search this forum, or have a look at my CSIREXX here
Back to top
View user's profile Send private message
rodferrn

New User


Joined: 14 Jul 2020
Posts: 14
Location: Brazil

PostPosted: Wed Jul 29, 2020 6:18 am
Reply with quote

Hello Pedro and Willy, wow!! Thank you so much for all the input.
Using Adress TSO just resolved all my problems with step limit in JCL. icon_biggrin.gif

The more I work on this project the more I feel like learning and eager to improve the code. So, I've generated an input member (SRCSAP) containing 45K datasets of SAPsid.DSNDBC.** , running a single job with this exec (using address TSO) is working perfectly, but the first runtime took 4 hours to complete (not bad, definitely not bad) but looking to improve the code or the efficiency of the solution, I removed the WAIT parameter (as suggested by Pedro) and was able to reduce the runtime to 3 hours and a half.
Now, on a second test I compiled my rexx and I managed reduce that runtime even more.

Now, I am thinking about serializing my execution (still using address tso) but instead of processing a single source/target member I was thinking about doing a dataset count by 10k or 12k and after each 10-12k generate a new member in DB2.CLONE.NEW.sid;

The current solution (below) will generate one source and one target member and add all source datasets (45k) on the same member, so I am looking for a way to do (perhaps a NESTED LOOP ? ) to generate a new member lets say SRCSPx and TGTSPx once 10 or 12k files are read..

Would a nested loop resolve this ??

This is the current code;

Code:

"ISPEXEC LMDINIT LISTID(ID3) LEVEL(SAP"SID".DSNDBC.*)"               
                                                                     
cntid3=1                                                             
DO FOREVER                                                           
"ISPEXEC LMDLIST LISTID("ID3") OPTION(LIST) DATASET(DSNSAP)"         
   IF RC = 0 THEN                                                   
    DO                                                               
     DATAsap.cntid3 = DSNSAP                                         
     cntid3 = cntid3 + 1                                             
    END                                                             
   ELSE LEAVE                                                       
END                                                                 
                                                                     
  "ISPEXEC LMDLIST LISTID("ID3") OPTION(FREE)"                       
  "ALLOC DD("SRCSAP") DA(DB2.CLONE.NEW."SID"("SRCSAP")) SHR REU"     
  "EXECIO * DISKW  "SRCSAP" (STEM DATAsap. FINIS"                   
  "FREE DD("SRCSAP")"                                               
  "ALLOC DD("TGTSAP") DA(DB2.CLONE.NEW."SID"("TGTSAP")) SHR REU"     
  Push "TESTE"                                                       
  "EXECIO 1 DISKW "TGTSAP" (FINIS"                                   
  "FREE DD("TGTSAP")"                                               
Back to top
View user's profile Send private message
Willy Jensen

Active Member


Joined: 01 Sep 2015
Posts: 712
Location: Denmark

PostPosted: Wed Jul 29, 2020 1:02 pm
Reply with quote

Something like (not tested, from the top of my head).
Notes:
1 I use BPXWDYN for allocation so that I can retrieve a generated DDname.
2. I use count= 1 to 999999 to save a coount-up operation. not a big saving but still.
3. When count is divisable with 10000 then a reallocation is done
4. I leave the loop for LMDLIST rc<>0, which I find simpler.
I hope this is somewhat what yo are loooking for.
Code:
mbrn=0
ddname=''
Call Realloc
DO count=1 to 999999
   "ISPEXEC LMDLIST LISTID("ID3") OPTION(LIST) DATASET(DSNSAP)"         
   IF RC <> 0 THEN leave                                                   
   if count//10000=0 then call realloc                                                     
   DATAsap.count = DSNSAP                                         
END
cc=Bpxwdyn('free dd('ddname')')
... some code ..
ReAalloc:
 if ddname<>'' then cc=Bpxwdyn('free dd('ddname')')
 mbrn=mbrn+1
 cc=Bpxwdyn('alloc da(dataset.name(mbr'mbrn')) shr rtddn(ddname)')
 return 0
Back to top
View user's profile Send private message
Willy Jensen

Active Member


Joined: 01 Sep 2015
Posts: 712
Location: Denmark

PostPosted: Wed Jul 29, 2020 4:42 pm
Reply with quote

Just realized that, despite my general preference for BPXWDYN over ALLOCATE, in this case "ALLOCATE DA('dataset.name(mbr"mbrn")) DD(ddname) SHR REUSE" is the simpler option.
Back to top
View user's profile Send private message
rodferrn

New User


Joined: 14 Jul 2020
Posts: 14
Location: Brazil

PostPosted: Fri Jul 31, 2020 1:55 am
Reply with quote

Hey Willy, thanks again for your reply, but I'm still having issues to write down the logic to split my DSNDBD files into multiple members.

Whenever you guys give me a sample, what I try to do is, instead of simply copy and paste I try to write down my own logic based on the provided code, in that way, plus reading manuals is the way I'm using to learn rexx (and programming actually).

Now, allow me to paste the entire logic below and explain a little bit (with the current logic I have 2 issues that I'm trying to adress).

So the logic basically asks for a SID (subsystem id) based on that SID, a dataset will be created and then LMDINIT / LMDLIST comes into play to list all datasets based on SAPsid.DSNDBD.** , now, using one database as example, I have 20165 DSNDBD files, so the logic has mainly 2 issues

1) Code is generating 2 members in DB2.CLONE.NEW.sid where SRCSAP1 goes from dataset number 1 to number 10.000 (which is ok) however the second member generated SRCSAP2 will copy the same datasets from row 1 to row 10.000 and continue until it reaches 20.002 (and then it stops). Now the second member is still adding the first 10k files (which were supposed to be on member 1 only).

2) Since its doing "IF count // 10001=0 THEN CALL Realloc" count goes only until 20.002 files, so its not writing all 20.165 files into the members.

Now, I have a guess about issue number 1, since I'm doing LMDINIT / LMDLIST, I think between the loop (from SRCSAP1 to SRCSAP2) the list is deleted and recreated so it will add all the 20002 datasets in member 2 ? I thought about creating a 3rd member containing the full list and then split this list into 2 new members splitting by 10.000 each ? Not sure if that would be the correct approach to follow ?

Here's the current code;

Code:

/* REXX */                                                                   
 SAY 'PLEASE INFORM A SUBSYSTEM ID NAME E.G. PWS , YPE , YPC '               
 SAY 'Member number is not required (Do not use E.G. YPE1 , YPC2) '         
                                                                             
 PULL SID                                                                   
IF SID = '' THEN DO                                                         
             SAY 'MISSING SUBSYSTEM NAME, PROGRAM WILL END'                 
             EXIT 16                                                         
END                                                                         
  ELSE SAY "DATASET DB2.CLONE.NEW."SID" WILL BE ALLOCATED"                   
                                                                             
CALL ALLOC_DS                                                               
                                                                             
ALLOC_DS:     /* This procedure will create the new dataset that will store 
                 Source / Target files and also FRRECOV jobs     */         
                                                                             
DSN = SYSDSN(DB2.CLONE.NEW.""SID"") /* When SID specified, create dataset */
IF DSN = 'OK' THEN DO   /* If DSN already exist, program will exit */       
         SAY "DATASET ALREADY EXISTS. EXITING PROGRAM, GOODBYE!"             
EXIT 4                                                                       
END                                                                         
ELSE                   /* If DSN does not exist, create and continue exec */
 "ALLOC DA(DB2.CLONE.NEW."SID") NEW REU RECFM(F B) LRECL(80)",               
 "DSORG(PO) SPACE(10,10) CYLINDERS UNIT(SYSDA) DSNTYPE(LIBRARY)"             
 SAY "DATASET DB2.CLONE.NEW."SID" ALLOCATED SUCCESSFULY "                     
 SAY                                                                         
 SAY                                                                         
 SAY                                                                         
                                                                             
UPDATE_DS:                                                                   
/* UPDATE_DS procedure will list all DB2 source files and create a member     
   for each set of datasets                                 */               
                                                                             
/* FETCH STEP 3: Will fetch data based on SAPSID parsed argument */           
                                                                             
 SAY "CREATING MEMBER (SRCSAP) TO LIST SOURCE DB2 DATASETS"                   
 SAY "CREATING MEMBER (TGTSAP) WITH RENAME FROM SRCSAP FILE"                 
 SAY                                                                         
 SAY "LOADING DATASET LIST IN SRCSAP MEMBER. PLEASE BE PATIENT"               
 SAY                                                                         
                                                                             
"ISPEXEC LMDINIT LISTID(ID3) LEVEL(SAP"SID".DSNDBD.*)"                       
                                                                             
mbn = 0                                                                 
CALL Realloc                                                             
  DO count = 1 to 999999                                                 
"ISPEXEC LMDLIST LISTID("ID3") OPTION(LIST) DATASET(DSNSAP)"             
   IF RC <> 0 THEN CALL LIST_DS                                         
   IF count // 10001=0 THEN CALL Realloc                                 
     DATAsap.count = DSNSAP                                             
END                                                                     
                                                                         
Realloc:                                                                 
  "ISPEXEC LMDLIST LISTID("ID3") OPTION(FREE)"                           
  "ALLOC DD("SRCSAP""mbn") DA(DB2.CLONE.NEW."SID"("SRCSAP""mbn")) SHR"   
  "EXECIO * DISKW  "SRCSAP""mbn" (STEM DATAsap. FINIS"                   
  "FREE DD("SRCSAP""mbn")"                                               
  mbn = mbn + 1                                                         
RETURN 0                                                                 
                                                                         
/* LIST_DS will show on screen the new members generated                 
           For Source and Target files                   */             
                                                                         
LIST_DS:                                                                 
 SAY "LISTING NEWLY CREATED MEMBERS"   
 "LISTDS DB2.CLONE.NEW."SID" MEMBERS"   
EXIT 0                                 
Back to top
View user's profile Send private message
Nic Clouston

Global Moderator


Joined: 10 May 2007
Posts: 2455
Location: Hampshire, UK

PostPosted: Fri Jul 31, 2020 3:16 pm
Reply with quote

After writing your 10,000 records you then have to remove them i.e. DROP DATASAP - as you were previously told.
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic View Bookmarks
All times are GMT + 6 Hours
Forum Index -> CLIST & REXX

 


Similar Topics
Topic Forum Replies
No new posts How to get a stack trace on a looping... ABENDS & Debugging 5
No new posts Return codes-Normal & Abnormal te... JCL & VSAM 7
No new posts How to append a PS file into multiple... JCL & VSAM 3
No new posts Two input files & writing counter... DFSORT/ICETOOL 12
No new posts convert file from VB to FB and use tr... DFSORT/ICETOOL 8
Search our Forums:

Back to Top