|
View previous topic :: View next topic
|
| Author |
Message |
ribinmathew
New User
Joined: 13 Jan 2009
Posts: 5
Location: Pune
|
|
|
|
Hi All,
I'm in the process of developing a tool in REXX which has to search for a string in flat PS file and when found, i need to copy some particular number of records above and below the line containing that particular string to another PS file.
The real problem I'm facing is, the file which has to be read contains very huge number of records so that operation of reading and searching using stem variables is not possible. So I need to go for a STACK operation of reading file into STACK and thereafter writing records from STACK into file is required. As I'm not that familier to STACK operations, can anybody help me out.
Thanks in advance |
|
| Back to top |
|
 |
Pedro
Global Moderator
Joined: 01 Sep 2006
Posts: 2547
Location: Silicon Valley
|
|
|
|
| Quote: |
| operation of reading and searching using stem variables is not possible. |
The typical problem with large files is that your region gets full (you run out of private memory).
You still use memory when you use the STACK! Surprise! For a large number of records, your private memory will still get full.
When working with large files, the recommendation is to use a process that reads a manageable number of records, say 10000, into the stem with EXECIO. Then process those 10000 before continuing to the next. |
|
| Back to top |
|
|
ribinmathew
New User
Joined: 13 Jan 2009
Posts: 5
Location: Pune
|
|
|
|
Hi,
Thanks for your reply. Actually I'm facing some probs with processing the file after splitting it, as you suggested. I used the method of populating a consistent number of records into stem variables and process it and then continue the same process with calling the same logic repeatedly. But the problem is that you need to resume reading the file from the record where I stopped processing during the last iteration and I did so by coding the same logic repeatedly depending on the number of records present record. But this is not a consistent logic or solution for the problem I'm facing because number of records in each file varies from thousands to lakhs. Can any one suggest a solution for this problem in file processing.
Thnaks in advance. |
|
| Back to top |
|
|
expat
Global Moderator
Joined: 14 Mar 2007
Posts: 8797
Location: Welsh Wales
|
|
|
|
| Read THIS topic so see how it works. |
|
| Back to top |
|
|
dick scherrer
Moderator Emeritus
Joined: 23 Nov 2006
Posts: 19244
Location: Inside the Matrix
|
|
|
|
Hello,
You might consider doing this in batch using something other than rexx. . .
I believe it is neither a good candidate for an "online" process, nor a good candidate for rexx  |
|
| Back to top |
|
|
Peter Poole
New User
Joined: 07 Jan 2009
Posts: 50
Location: Scotland
|
|
|
|
| ribinmathew wrote: |
Hi All,
I'm in the process of developing a tool in REXX which has to search for a string in flat PS file and when found, i need to copy some particular number of records above and below the line containing that particular string to another PS file.
Thanks in advance |
Are the number(s) of required records above and below a successfully selected line constant(s)?
If so, all you need to do is set up an array of the required number of previous records and then process the file one record at a time. When you get a hit, pull in the previous lines from that array and call a routine to read the next set of lines. When you don't get a hit, shuffle the records in the array up by one and add the current line to the bottom of the array.
It's probably not the most elegant solution, but it should do what you asked for.
Cheers. |
|
| Back to top |
|
|
Pedro
Global Moderator
Joined: 01 Sep 2006
Posts: 2547
Location: Silicon Valley
|
|
|
|
| Quote: |
| But this is not a consistent logic or solution for the problem I'm facing because number of records in each file varies from thousands to lakhs. |
It is not clear what problem you are having. Use EXECIO to ask for a certain number of records. But I think the zero element of the stem will have the actual number for records retrieved. It should not matter how many records are actually in the file. |
|
| Back to top |
|
|
Peter Poole
New User
Joined: 07 Jan 2009
Posts: 50
Location: Scotland
|
|
|
|
As I understand the spec (Mathew, please correct me if I'm speaking through my bottom!)
On finding a qualifying record, he needs to take x records previous to that one and y records subsequent to that. If he reads the file in stemmed 'blocks' of 500 - or any other number - and gets a match on the seond record of the second 'block' then his previous record requirement may well take him to the last few records in 'block' 1, which he's just overwritten by reading 'block' 2....
Cheers! |
|
| Back to top |
|
|
enrico-sorichetti
Superior Member
Joined: 14 Mar 2007
Posts: 10873
Location: italy
|
|
|
|
I do not understand why people want to complicate their lives
there is a wondeful utility called ISRSUPC that produces this output
| Code: |
==MSG> -CAUTION- Profile changed to CAPS OFF (from CAPS ON) because data
==MSG> contains lower case characters.
000001 1 ISRSUPC - MVS/PDF FILE/LINE/WORD/BYTE/SFOR COMPARE UTILITY- ISPF
000002 LINE-# SOURCE SECTION SRCH DSN: ENRICO.DAF.DISTR(D
000003
000004 4 //* DSN's of SYS1.xxxxxxxx map into SMPE DDDEFs of
000005 6 //STEPLIB DD DISP=SHR,DSN=SYS1.SASMMOD1,
000006 9 // DD DISP=SHR,DSN=SYS1.SGLOSAMP, Allocation Optimi
000007 11 // DD DISP=SHR,DSN=SYS1.SASFPMAC, Ba
000008 13 // DD DISP=SHR,DSN=SYS1.MACLIB,
000009 15 // DD DISP=SHR,DSN=SYS1.MODGEN,
000010 17 // DD DISP=SHR,DSN=SYS1.SBDTMAC, Bulk Data Transf
000011 19 // DD DISP=SHR,DSN=SYS1.SEZACMAC,
000012 21 // DD DISP=SHR,DSN=SYS1.SICEUSER,
000013 23 // DD DISP=SHR,DSN=SYS1.NFSMAC,
000014 25 // DD DISP=SHR,DSN=SYS1.SDVGMAC0, NetView FT
000015 27 // DD DISP=SHR,DSN=SYS1.SDBNIMAC,
000016 113 * DFPACS Set SMFRECNO to nnn in SYS1.SACBCNTL(IGDACSSC)
000017 1608 MNOTE 255,'* macro was not found - it is usually in SYS1.SGL
|
| Code: |
Command ===> Scroll ===> CSR
000018 1625 MNOTE 255,'* macro was not found - it is usually in SYS1.SASF
000019 1642 MNOTE 255,'* macro was not found - it is usually in SYS1.SBDT
000020 1659 MNOTE 255,'* macro was not found - it is usually in SYS1.SICE
000021 1676 MNOTE 255,'* macro was not found - it is usually in SYS1.SDVG
000022 1693 MNOTE 255,'* macro was not found - it is usually in SYS1.SDBN
000023 1710 MNOTE 255,'* macro was not found - it is usually in SYS1.SEZA
000024 1727 MNOTE 255,'* macro was not found - it is usually in SYS1.SEZA
000025 3077 CLC 0(17,R3),=C'SYS1.DFDSS.DEFRAG' Special DEFRAG ?
000026
000027 1 ISRSUPC - MVS/PDF FILE/LINE/WORD/BYTE/SFOR COMPARE UTILITY- ISPF
000028 SEARCH-FOR SUMMARY SECTION SRCH DSN: ENRICO.DAF.DISTR(D
000029
000030 LINES-FOUND LINES-PROC DATASET-W/LNS DATASET-WO/LNS COMPARE-COLS L
000031 22 16269 1 0 1:80
000032
000033 PROCESS OPTIONS USED: ANYC
000034
000035 THE FOLLOWING PROCESS STATEMENTS (USING COLUMNS 1:72) WERE PROCESSED:
000036 SRCHFOR 'SYS1.'
000037
|
as You can see the isrsupc output shows the line where the string was found
it' s trivial to build from the srchfor output a sort step to copy the wanted records |
|
| Back to top |
|
|
Peter Poole
New User
Joined: 07 Jan 2009
Posts: 50
Location: Scotland
|
|
|
|
If people didn't want to complicate their lives, 90% of today's IT projects might never have been initiated 
I pretty certain you could probably do the whole process in one swell foop using FileAid, but that would not answer the question asked.
Cheers. |
|
| Back to top |
|
|
enrico-sorichetti
Superior Member
Joined: 14 Mar 2007
Posts: 10873
Location: italy
|
|
|
|
the point is that the TS is trying to use the wrong tool  |
|
| Back to top |
|
|
Peter Poole
New User
Joined: 07 Jan 2009
Posts: 50
Location: Scotland
|
|
|
|
TS?
Cheers. |
|
| Back to top |
|
|
dick scherrer
Moderator Emeritus
Joined: 23 Nov 2006
Posts: 19244
Location: Inside the Matrix
|
|
|
|
Hello,
TS = Topic Starter. . .
One thought is that while some things are possible, maybe they shouldn't be done. . . Unless somone in the family has the hardware account |
|
| Back to top |
|
|
enrico-sorichetti
Superior Member
Joined: 14 Mar 2007
Posts: 10873
Location: italy
|
|
|
|
| TS ==> topic/thread starter |
|
| Back to top |
|
|
enrico-sorichetti
Superior Member
Joined: 14 Mar 2007
Posts: 10873
Location: italy
|
|
|
|
another point....
thats not really a STACK that is needed
( but, as usual, the terminology depends on the poster's background)
from my point of view a STACK ( without a qualifier ) is a LIFO thing
the TS really needs a FIFO thing
and I would rather call it a QUEUE or SLIDING window
and the logic gets really nice when the records hit by the search string fall within the keep distance
( if You work with the sliding window logic it is pretty easy to do it ) |
|
| Back to top |
|
|
expat
Global Moderator
Joined: 14 Mar 2007
Posts: 8797
Location: Welsh Wales
|
|
|
|
I'd go with Enricos suggestion of knowing where the lines are, and then simply building the reads around them.
The OP should pursue this track. |
|
| Back to top |
|
|
Peter Poole
New User
Joined: 07 Jan 2009
Posts: 50
Location: Scotland
|
|
|
|
TS - grazi. We live, we learn.
For the rest, well:
"There's more than one way to skin a cat" he said, ignoring the piteous mewling as he pinned the tiny paws to the easel....
Mathew, has this helped at all, or have you lost the will to live by now?
Cheers. |
|
| Back to top |
|
|
|
|
