IBM Mainframe Forum Index
 
Log In
 
IBM Mainframe Forum Index Mainframe: Search IBM Mainframe Forum: FAQ Register
 

STEM usage in REXX


IBM Mainframe Forums -> CLIST & REXX
Post new topic   Reply to topic
View previous topic :: View next topic  
Author Message
Nileshkul

New User


Joined: 09 May 2016
Posts: 43
Location: India

PostPosted: Thu Jan 05, 2023 5:10 pm
Reply with quote

Hi,

I am trying to use STEM in REXX to get record count. I am able to get record count for file up to 50 K records (data approx 40 Mb). Just want to make sure if there is any limit of number of records/storage in bytes that can be successfully processed by STEM command to get record count.

Any help would be great.

Many Thanks - Nilesh
Back to top
View user's profile Send private message
sergeyken

Senior Member


Joined: 29 Apr 2008
Posts: 1650

PostPosted: Thu Jan 05, 2023 6:27 pm
Reply with quote

What did you get so far?

What did you try?

What are your results? (If any?)

Please, clarify this sentence: "Trying to use STEM to get record count..." It sounds like
Code:
Record_Count = STEM
Back to top
View user's profile Send private message
Nileshkul

New User


Joined: 09 May 2016
Posts: 43
Location: India

PostPosted: Thu Jan 05, 2023 7:57 pm
Reply with quote

Thanks for reply. Apologies, I did not provided enough details earlier.

I have tried code something like below:
"ALLOC DA('DSN1.DSN2.DSN3)') F(INPUT) SHR REUSE"
"EXECIO * DISKR INPUT (STEM LINES. FINIS"

SAY "REC COUNT:" LINES.0

My query is - the above code working fine up to 50 K records (data approx 40 Mb) - so will it work for any number of records of data like 1 million or so.. OR is there any limit on storage that above code will probably fail with huge files.

Many Thanks - Nilesh
Back to top
View user's profile Send private message
sergeyken

Senior Member


Joined: 29 Apr 2008
Posts: 1650

PostPosted: Thu Jan 05, 2023 8:11 pm
Reply with quote

Nileshkul wrote:
Thanks for reply. Apologies, I did not provided enough details earlier.

I have tried code something like below:
Code:
"ALLOC DA('DSN1.DSN2.DSN3)') F(INPUT) SHR REUSE"
"EXECIO * DISKR INPUT (STEM LINES. FINIS"

SAY "REC COUNT:" LINES.0


My query is - the above code working fine up to 50 K records (data approx 40 Mb) - so will it work for any number of records of data like 1 million or so.. OR is there any limit on storage that above code will probably fail with huge files.

Many Thanks - Nilesh


FYI:
1) Learn how to use the Code button when posting your samples

2) Your code actually is trying to read the whole dataset content into memory. For a huge dataset you'll get the memory-related ABEND much faster than LINES.0 is assigned its value...

3) There is no way to get the dataset number of records without reading it record-by-record.
Back to top
View user's profile Send private message
Nileshkul

New User


Joined: 09 May 2016
Posts: 43
Location: India

PostPosted: Thu Jan 05, 2023 8:58 pm
Reply with quote

Thanks for quick response. Also thanks about Code button use details.

Is there any known limit or threshold I can use to guess number of records it would work fine without memory issue. I meant the limit in memory size in Mb that I can use for my calculations.

Many Thanks - Nilesh
Back to top
View user's profile Send private message
sergeyken

Senior Member


Joined: 29 Apr 2008
Posts: 1650

PostPosted: Thu Jan 05, 2023 9:06 pm
Reply with quote

Nileshkul wrote:
Thanks for quick response. Also thanks about Code button use details.

Is there any known limit or threshold I can use to guess number of records it would work fine without memory issue. I meant the limit in memory size in Mb that I can use for my calculations.

Many Thanks - Nilesh


It is a very-very bad idea: to load the whole dataset into memory, only to find the number of records.
The available limit of memory depends on multiple settings provided by your zOS administrator; request your own Helpdesk, or whatever is available.

There is no other way (at non system administration level) to get the number of records in a dataset besides reading the records one-by-one (or by relatively small groups), WITHOUT KEEPING THEIR CONTENT IN MEMORY. You need to organize a sort of loop to do this.
Back to top
View user's profile Send private message
sergeyken

Senior Member


Joined: 29 Apr 2008
Posts: 1650

PostPosted: Thu Jan 05, 2023 9:19 pm
Reply with quote

Something like this:

Code:
NumRecords = 0
EndOfFile = 0
Do While EndOfFile = 0
   "MAKEBUF"      /* make buffer to read */
   "EXECIO 1000 DISKR INPUT"  /* read up to 1000 records into stack */
   If RC <> 0 Then EndOfFile = 1
   NumRecords = NumRecords + Queued()  /* accumulated number of records */
   "DROPBUF"     /* eliminate records in buffer */
End
Say "Number of records is" NumRecords
Back to top
View user's profile Send private message
sergeyken

Senior Member


Joined: 29 Apr 2008
Posts: 1650

PostPosted: Thu Jan 05, 2023 9:27 pm
Reply with quote

Cannot guess: what is the goal, to calculate the number of records in advance, before reading them?

If it is only required to add new records at the end, then DISP=MOD is used. Otherwise, it seems to be some useless activity...
Back to top
View user's profile Send private message
Willy Jensen

Active Member


Joined: 01 Sep 2015
Posts: 639
Location: Denmark

PostPosted: Thu Jan 05, 2023 9:48 pm
Reply with quote

Not very elegant, but you can use ICETOOL like so:
Code:
 cc=bpxwdyn('alloc dd(dd1) da(your.dataset) shr reuse')
 cc=bpxwdyn('alloc dd(toolin) new delete reuse',           
            'lrecl(80) recfm(f,b) blksize(4080)',           
            'tracks space(1,1) unit(sysda)')               
 cc=bpxwdyn('alloc dd(toolmsg) new delete reuse',           
            'tracks space(1,1) unit(sysda)')               
 cc=bpxwdyn('alloc dd(dfsmsg) dummy reuse')                 
 parse value 1 ' COUNT FROM(DD1)' with r.0 r.1             
 "execio 1 diskw toolin (stem r. finis)"                   
 Address Attchmvs "ICETOOL"                                 
 "execio * diskr toolmsg (stem lst. finis)"                 
 do n=1 to lst.0                                           
   if word(lst.n,1)='ICE628I' then say lst.n               
 end                                                       
 "free  dd(toolmsg dfsmsg dd1 toolin)"                     
Back to top
View user's profile Send private message
sergeyken

Senior Member


Joined: 29 Apr 2008
Posts: 1650

PostPosted: Thu Jan 05, 2023 9:56 pm
Reply with quote

Quote:
You don't need a musket to shoot a butterfly


I'm pretty sure, the real problem is in the wrong general design of the application.
Back to top
View user's profile Send private message
Pedro

Global Moderator


Joined: 01 Sep 2006
Posts: 2385
Location: Silicon Valley

PostPosted: Thu Jan 05, 2023 10:13 pm
Reply with quote

I also am unsure of the need to get the number of records. Please clarify.

But I have this suggestion. Use the LISTDSI function to retrieve overall data set information. Use SYSALLOC and SYSUSED to determine how much of the data set is used. And maybe also use SYSLRECL and SYSBLKSIZE and SYSBLKSTRK to guess at how many records are in the dataset. (sorry, I do not have an example of the math required)
Back to top
View user's profile Send private message
Nileshkul

New User


Joined: 09 May 2016
Posts: 43
Location: India

PostPosted: Thu Jan 05, 2023 10:50 pm
Reply with quote

sergeyken wrote:
Something like this:

Code:
NumRecords = 0
EndOfFile = 0
Do While EndOfFile = 0
   "MAKEBUF"      /* make buffer to read */
   "EXECIO 1000 DISKR INPUT"  /* read up to 1000 records into stack */
   If RC <> 0 Then EndOfFile = 1
   NumRecords = NumRecords + Queued()  /* accumulated number of records */
   "DROPBUF"     /* eliminate records in buffer */
End
Say "Number of records is" NumRecords


Thanks a lot for this simple way of getting record count, This is very helpful, really Thanks again.

Many Thanks - Nilesh
Back to top
View user's profile Send private message
Nileshkul

New User


Joined: 09 May 2016
Posts: 43
Location: India

PostPosted: Thu Jan 05, 2023 10:52 pm
Reply with quote

Willy Jensen wrote:
Not very elegant, but you can use ICETOOL like so:
Code:
 cc=bpxwdyn('alloc dd(dd1) da(your.dataset) shr reuse')
 cc=bpxwdyn('alloc dd(toolin) new delete reuse',           
            'lrecl(80) recfm(f,b) blksize(4080)',           
            'tracks space(1,1) unit(sysda)')               
 cc=bpxwdyn('alloc dd(toolmsg) new delete reuse',           
            'tracks space(1,1) unit(sysda)')               
 cc=bpxwdyn('alloc dd(dfsmsg) dummy reuse')                 
 parse value 1 ' COUNT FROM(DD1)' with r.0 r.1             
 "execio 1 diskw toolin (stem r. finis)"                   
 Address Attchmvs "ICETOOL"                                 
 "execio * diskr toolmsg (stem lst. finis)"                 
 do n=1 to lst.0                                           
   if word(lst.n,1)='ICE628I' then say lst.n               
 end                                                       
 "free  dd(toolmsg dfsmsg dd1 toolin)"                     


Thanks for sharing this, this is helpful.
Back to top
View user's profile Send private message
don.leahy

Active Member


Joined: 06 Jul 2010
Posts: 745
Location: Whitby, ON, Canada

PostPosted: Sat Jan 07, 2023 8:11 am
Reply with quote

I would bet that the ICETOOL approach would be the fastest. Why not try both approaches and share the results?
Back to top
View user's profile Send private message
sergeyken

Senior Member


Joined: 29 Apr 2008
Posts: 1650

PostPosted: Tue Jan 10, 2023 8:07 pm
Reply with quote

don.leahy wrote:
I would bet that the ICETOOL approach would be the fastest. Why not try both approaches and share the results?


Either yes, or no.

For relatively small datasets the allocation of DD-names, and dynamic LINK to the utility might take longer than a straightforward reading the records.

For a huge enough datasets the utility-call approach may be faster.
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic View Bookmarks
All times are GMT + 6 Hours
Forum Index -> CLIST & REXX

 


Similar Topics
Topic Forum Replies
No new posts LISTC in REXX CLIST & REXX 4
No new posts AI writing DFSORT, REXX codes.. All Other Mainframe Topics 3
No new posts I want to append a record in an exist... CLIST & REXX 17
No new posts Access library name in REXX CLIST & REXX 6
No new posts access library name in REXX CLIST & REXX 2
Search our Forums:

Back to Top