View previous topic :: View next topic
|
Author |
Message |
mosinjamadar
New User
Joined: 26 Sep 2007 Posts: 42 Location: pune
|
|
|
|
Hi all
I am getting IRX0662E error when reading a flat file which contains 10,000,000 records. Below is peace of code which i am using to read the file
"EXECIO * DISKR INDD (STEM NEWVAR."
error is
NO OF RECORDS
IRX0662E EXECIO error. Unable to obtain storage.
please help me out with any other way I can over come this
thanks |
|
Back to top |
|
|
enrico-sorichetti
Superior Member
Joined: 14 Mar 2007 Posts: 10873 Location: italy
|
|
|
|
already debated to death,
the EXCIO as coded will read into storage all the dataset records
for large number of records that' s the least You can expect
process them in chunks , search the forum for samples
another suggestion ...
review Your process and evaluate alternatives
for large datasets processing REXX is not the best tool around |
|
Back to top |
|
|
expat
Global Moderator
Joined: 14 Mar 2007 Posts: 8797 Location: Welsh Wales
|
|
|
|
Quote: |
for large datasets processing REXX is not the best tool around |
Very true. I usually cut out as much rubbish as possible from the input bt preprocessing with a SORT step. That way I only get the records and fields that I need. Makes it so much quicker. |
|
Back to top |
|
|
dick scherrer
Moderator Emeritus
Joined: 23 Nov 2006 Posts: 19244 Location: Inside the Matrix
|
|
|
|
Hello,
Quote: |
for large datasets processing REXX is not the best tool around |
My experience has this more strongly negative. . .
REXX may be the worst possible tool for processing large datasets"
There are just too many alternatives to the brute force reading of entire huge files in rexx. . .
As i've mentioned before, i suppose this is ok if your family has the mainframe hardware account. . . |
|
Back to top |
|
|
bala kalluri Warnings : 1 New User
Joined: 08 Jan 2009 Posts: 21 Location: INDIA
|
|
|
|
Even I'm facing the same problem, rexx isn't able to handle huge amounts of data.
Could you tell me, what alternate way have you chosen. |
|
Back to top |
|
|
dick scherrer
Moderator Emeritus
Joined: 23 Nov 2006 Posts: 19244 Location: Inside the Matrix
|
|
|
|
Hello,
Depending on what you want to do, you could use real code or possibly the sort product that is in use on your system. |
|
Back to top |
|
|
bala kalluri Warnings : 1 New User
Joined: 08 Jan 2009 Posts: 21 Location: INDIA
|
|
|
|
HI All,
I have huge number of data in a flat file.
My requirement is to find a particular string in the file.
IF i implement the logic of reading record one by one and searching for a particulart string, then its throwing "storage exhaust" problem.
If not rexx, is there any program in cobol which can help me out to find the record number which contains the searched string details.
Thanks,
Bala |
|
Back to top |
|
|
superk
Global Moderator
Joined: 26 Apr 2004 Posts: 4652 Location: Raleigh, NC, USA
|
|
|
|
bala kalluri wrote: |
My requirement is to find a particular string in the file. |
Why aren't you using a utility like sort (either DFSORT or Syncsort)? |
|
Back to top |
|
|
dbzTHEdinosauer
Global Moderator
Joined: 20 Oct 2006 Posts: 6966 Location: porcelain throne
|
|
|
|
or the 3 dot option searchfor? |
|
Back to top |
|
|
expat
Global Moderator
Joined: 14 Mar 2007 Posts: 8797 Location: Welsh Wales
|
|
|
|
Do you really mean reading "one by one" or do you mean trying to load a stem with the datset.
This has been discussed numerous times, and you can process the file into a stem in manageable segements, or you could (and most probably should) use your sort product to do this. |
|
Back to top |
|
|
dick scherrer
Moderator Emeritus
Joined: 23 Nov 2006 Posts: 19244 Location: Inside the Matrix
|
|
|
|
Probably, i oversimplfy, but ten million records should not be processed in rexx. . .
d |
|
Back to top |
|
|
bala kalluri Warnings : 1 New User
Joined: 08 Jan 2009 Posts: 21 Location: INDIA
|
|
|
|
I want read record line by line and to capture the one which satisfies my search criteria. I have huge number of records in range of 10-20 lakhs.
Reading records in stem variable will not help, it will throw storage exhausted error.
I cant use sort utility here, all records contains different data. |
|
Back to top |
|
|
expat
Global Moderator
Joined: 14 Mar 2007 Posts: 8797 Location: Welsh Wales
|
|
|
|
Please elaborate why you can not use sort to select only those records which contain the required string. Using the SS capabilities of DFSORT should not cause you any problems.
RECFM & LRECL of the input file, and possible positions of the required string. |
|
Back to top |
|
|
dbzTHEdinosauer
Global Moderator
Joined: 20 Oct 2006 Posts: 6966 Location: porcelain throne
|
|
|
|
Quote: |
I cant use sort utility here, all records contains different data |
well, I would hope. why would you have 10-20 lakhs of the same record? |
|
Back to top |
|
|
|