Portal | Manuals | References | Downloads | Info | Programs | JCLs | Master the Mainframes
IBM Mainframe Computers Forums Index
 
Register
 
IBM Mainframe Computers Forums Index Mainframe: Search IBM Mainframe Forum: FAQ Memberlist Usergroups Profile Log in to check your private messages Log in
 

 

Storage Abend

 
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> CLIST & REXX
View previous topic :: :: View next topic  
Author Message
Amb

New User


Joined: 12 Mar 2010
Posts: 64
Location: India

PostPosted: Mon May 16, 2011 10:15 am    Post subject: Storage Abend
Reply with quote

I am trying to execute a REXX program which processes a huge file.

I am getting an error saying
"EXEC IO error. Unable to obtain storage. Machine storage exhausted."

This is a critical REXX program. Is there any way I can increase the storage capacity for this?
Back to top
View user's profile Send private message

prino

Active Member


Joined: 07 Feb 2009
Posts: 982
Location: Oostende, Belgium

PostPosted: Mon May 16, 2011 10:26 am    Post subject:
Reply with quote

Answer to last question: NO

General remark: You do not process huge files with REXX, ever!
Back to top
View user's profile Send private message
Stefan

Active User


Joined: 12 Jan 2006
Posts: 110
Location: Germany

PostPosted: Mon May 16, 2011 12:51 pm    Post subject: Re: Storage Abend
Reply with quote

If it's really necessary to read a huge dataset with REXX you could try to read it "in sections". Put your EXECIO in a loop and specify a fixed number of records to be read per cycle, for example "EXECIO 100 DISKR ... ". Then process these 100 records and continue with reading the next 100 records into the same variables.

But in general I agree with prino that huge files should be processed by compiled programs.

Maybe you could pre-process your dataset with some kind of utility such as SORT or IDCAMS to extract only the data which is needed. So it might be possible to reduce the amount of data to be processed by your REXX program.
Give us an example of what your business needs are.
Back to top
View user's profile Send private message
DB2 Guy

New User


Joined: 28 Oct 2008
Posts: 98
Location: Cubicle

PostPosted: Mon May 16, 2011 2:57 pm    Post subject: Reply to: Storage Abend
Reply with quote

Quote:
But in general I agree with prino that huge files should be processed by compiled programs.
I do not speak REXX much but does the above sentence mean if the REXX code is complied (using Change Man or any other version control tool), it is good to use REXX with huge file opposed to when it is used as interpreted-language?
Back to top
View user's profile Send private message
enrico-sorichetti

Global Moderator


Joined: 14 Mar 2007
Posts: 10201
Location: italy

PostPosted: Mon May 16, 2011 3:04 pm    Post subject: Reply to: Storage Abend
Reply with quote

the issue is about the improper use of EXECIO
the format
EXECIO * READ <yak yak yak>
will read into the stack or into a stem which means in storage
ALL the records of the file
irrelevant of the interpreted compiled mode

so if the file is huge an insufficient storage abend is most likely to occur
Back to top
View user's profile Send private message
PeterHolland

Global Moderator


Joined: 27 Oct 2009
Posts: 2422
Location: Netherlands, Amstelveen

PostPosted: Mon May 16, 2011 3:04 pm    Post subject:
Reply with quote

Compiled programs means in this case anything BUT rexx. Like Cobol, PL/I, Assembler etc. Compiled Rexx means (a part of the) interpreting is compiled, storage handling will not be changed.
Back to top
View user's profile Send private message
prino

Active Member


Joined: 07 Feb 2009
Posts: 982
Location: Oostende, Belgium

PostPosted: Mon May 16, 2011 3:54 pm    Post subject: Re: Reply to: Storage Abend
Reply with quote

DB2 Guy wrote:
I do not speak REXX much but does the above sentence mean if the REXX code is complied (using Change Man or any other version control tool), it is good to use REXX with huge file opposed to when it is used as interpreted-language?

Neither REXX (nor any other language) is ever compiled by ChangeMan (or any other version control tool)!
Back to top
View user's profile Send private message
PeterHolland

Global Moderator


Joined: 27 Oct 2009
Posts: 2422
Location: Netherlands, Amstelveen

PostPosted: Mon May 16, 2011 4:07 pm    Post subject: Re: Reply to: Storage Abend
Reply with quote

prino wrote:
DB2 Guy wrote:
I do not speak REXX much but does the above sentence mean if the REXX code is complied (using Change Man or any other version control tool), it is good to use REXX with huge file opposed to when it is used as interpreted-language?

Neither REXX (nor any other language) is ever compiled by ChangeMan (or any other version control tool)!


Give the guy a break, he means probably a changeman invoked compile process. Like invoking compiles/assemblings in ispf.
Back to top
View user's profile Send private message
prino

Active Member


Joined: 07 Feb 2009
Posts: 982
Location: Oostende, Belgium

PostPosted: Mon May 16, 2011 4:20 pm    Post subject: Re: Reply to: Storage Abend
Reply with quote

PeterHolland wrote:
prino wrote:
DB2 Guy wrote:
I do not speak REXX much but does the above sentence mean if the REXX code is complied (using Change Man or any other version control tool), it is good to use REXX with huge file opposed to when it is used as interpreted-language?

Neither REXX (nor any other language) is ever compiled by ChangeMan (or any other version control tool)!

Give the guy a break, he means probably a changeman invoked compile process. Like invoking compiles/assemblings in ispf.

Then he should say so. It's the same with people talking about REXX Panels.
Back to top
View user's profile Send private message
Pedro

Senior Member


Joined: 01 Sep 2006
Posts: 2019
Location: Silicon Valley

PostPosted: Mon May 16, 2011 7:22 pm    Post subject: Reply to: Storage Abend
Reply with quote

note: I agree that rexx is not a good choice for huge files.

Quote:
the issue is about the improper use of EXECIO

You can use EXECIO to read the file in chunks so that you possibly avoid exhausting the storage.

Quote:
Is there any way I can increase the storage capacity for this?

You can increase the region size when you logon or in the batch job. 'Huge' is a relative term. You might be able to get by just by increasing the region size to a huge value.
Back to top
View user's profile Send private message
DB2 Guy

New User


Joined: 28 Oct 2008
Posts: 98
Location: Cubicle

PostPosted: Mon May 16, 2011 7:24 pm    Post subject: Reply to: Storage Abend
Reply with quote

Thank you Enrico and Peter.

Robert - Terminology is critical in IT, I know and understand that. Most of us (at least me ) are born when market was taken over by the tools such as Change-Man and/or endeavor. In my career, I've never executed, IGYCRCTL, to COMPILE components and then IEWL, to LINK-EDIT Component and possibly will never write my own JCL to do that, it's 2011. It's pretty common to say, "compiled using ChangeMan", around many shops I had been to.

If there is some compilation error in a COBOL-program compile, I won't talk about the under-the-cover-asssembler-code in error. I'll talk about one-level-up -- I'll talk about COBOL error. I'm compiling my programs using Change-man, I won't say it's copiled by IGYCRCTL+IEWL, I'll talk one level up - it's compiled using Change-Man.

We young musketeers are anyways, dealing with bad managers/leads, don't sound like them, please...icon_smile.gif
Back to top
View user's profile Send private message
Kjeld

Active User


Joined: 15 Dec 2009
Posts: 365
Location: Denmark

PostPosted: Tue May 17, 2011 7:32 pm    Post subject: Re: Reply to: Storage Abend
Reply with quote

DB2 Guy wrote:
Quote:
But in general I agree with prino that huge files should be processed by compiled programs.
I do not speak REXX much but does the above sentence mean if the REXX code is complied (using Change Man or any other version control tool), it is good to use REXX with huge file opposed to when it is used as interpreted-language?

If you want to process large files by interpreted code I will suggest SAS, if it is available at the site. It will usually handle records one by one, and intermediate data will be written to external DASD storage.
Back to top
View user's profile Send private message
valyk

Active User


Joined: 16 Apr 2008
Posts: 104
Location: South Carolina

PostPosted: Tue May 17, 2011 8:15 pm    Post subject: Re: Reply to: Storage Abend
Reply with quote

DB2 Guy wrote:
Thank you Enrico and Peter.

Robert - Terminology is critical in IT, I know and understand that. Most of us (at least me ) are born when market was taken over by the tools such as Change-Man and/or endeavor. In my career, I've never executed, IGYCRCTL, to COMPILE components and then IEWL, to LINK-EDIT Component and possibly will never write my own JCL to do that, it's 2011. It's pretty common to say, "compiled using ChangeMan", around many shops I had been to.

If there is some compilation error in a COBOL-program compile, I won't talk about the under-the-cover-asssembler-code in error. I'll talk about one-level-up -- I'll talk about COBOL error. I'm compiling my programs using Change-man, I won't say it's copiled by IGYCRCTL+IEWL, I'll talk one level up - it's compiled using Change-Man.

We young musketeers are anyways, dealing with bad managers/leads, don't sound like them, please...icon_smile.gif


I can totally sympathize with you. As a 'young muskateer' myself, I started programming on the mainframe several years ago. I started in the application area of my shop and quickly realized that I did not want to spend my career as an application programmer. Then I hopped at an opportunity to be a systems programmer.

Companies now-a-days do not care about training application programmers. They expect you to have a super high level understanding of what goes on, and are happy at leaving it at that.

If you want a good understanding of how things work, switch to a systems programmer job. You will not be disappointed. Granted, you will probably work your butt off, but you will feel like you are actually accomplishing something.
Back to top
View user's profile Send private message
Victor Niu

New User


Joined: 11 Mar 2010
Posts: 59
Location: China,Shanghai

PostPosted: Wed May 18, 2011 11:29 am    Post subject:
Reply with quote

rexx works below 2G address space.

You need not deal with too huge data by rexx,I believe you can divide the data of the huge file into parts and handle one part each time&#65292;then one by one.

I have experienced the similar problem,if your pgm used over 1g memory and still increasing,it will be very dangerous.you check the pgm's overhead by RMF.
Back to top
View user's profile Send private message
gcicchet

Senior Member


Joined: 28 Jul 2006
Posts: 1703
Location: Australia

PostPosted: Wed May 18, 2011 1:11 pm    Post subject:
Reply with quote

Hi,

unless there is a requirement, don't use STEM

Read a record
Process the record
Write the record
and you won't have any storage issues.
I used the following code to process 11415828 records, FB and LRECL of 133.

Code:
DO WHILE (RETURN_CODE ¬= EOFFLAG)   
   "EXECIO 1 DISKR OLD"             
   RETURN_CODE = RC                 
   IF RETURN_CODE = 0 THEN         
      DO                           
       IN_CNTR = IN_CNTR + 1       
        PARSE PULL RECORD           
                                   
            OUT_CNTR = OUT_CNTR + 1;
            QUEUE RECORD;           
            "EXECIO * DISKW NEW";   
   END;                             
END;                               


Now apart from this test I ran, I would never use REXX for large files, it took 26 mins CPU to process the above file.

I copied the same file using DFSORT in less than a minute.

Gerry
Back to top
View user's profile Send private message
dbzTHEdinosauer

Global Moderator


Joined: 20 Oct 2006
Posts: 6966
Location: porcelain throne

PostPosted: Wed May 18, 2011 2:22 pm    Post subject:
Reply with quote

Gerry,

do you run your test in foreground or background? icon_lol.gif
Back to top
View user's profile Send private message
gcicchet

Senior Member


Joined: 28 Jul 2006
Posts: 1703
Location: Australia

PostPosted: Wed May 18, 2011 2:57 pm    Post subject:
Reply with quote

Hi Dick,

my test is in batch, I wouldn't lock up my session for that amount of time.


Gerry
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> CLIST & REXX All times are GMT + 6 Hours
Page 1 of 1

 

Search our Forum:

Similar Topics
Topic Author Forum Replies Posted
No new posts IMS BMP program causes 878 system abend Artemk IMS DB/DC 7 Tue Nov 22, 2016 8:26 pm
No new posts abend sort based on count records in ... anatol DFSORT/ICETOOL 5 Mon Oct 17, 2016 10:10 pm
No new posts hot to get details when "EXEC CI... Andi1982 CICS 11 Tue Sep 20, 2016 5:01 pm
No new posts S213-14 ABEND explantion Apurva Mehta JCL & VSAM 5 Thu Jul 07, 2016 10:56 pm
No new posts How to determine where is error in ab... jackzhang75 PL/I & Assembler 13 Fri Apr 15, 2016 2:01 am


Facebook
Back to Top
 
Mainframe Wiki | Forum Rules | Bookmarks | Subscriptions | FAQ | Tutorials | Contact Us