|
View previous topic :: View next topic
|
| Author |
Message |
dbzTHEdinosauer
Global Moderator
Joined: 20 Oct 2006
Posts: 6966
Location: porcelain throne
|
|
|
|
I am wondering how many
pic9(n) display
fields there are? |
|
| Back to top |
|
 |
Bill Woodger
Moderator Emeritus
Joined: 09 Mar 2011
Posts: 7309
Location: Inside the Matrix
|
|
|
|
| dbzTHEdinosauer wrote: |
I am wondering how many
pic9(n) display
fields there are? |
I predict none. I predict each of the 165 bytes is a piece of text which, in some cases, may have been splitted into a second item, and to compare them to the original you need to stick them back together. All the originals in one gigantic table, all the potentially splitted ones in another.
They call me The Mindreader. Well, it sounded something like that.
For "splitted" it is archaic. Shakespeare used it. Also still in use at the end of the 19th Century.
Kunal Jain, do you want to tell us a bit more about your task? If that is the whole thing, you definitely don't need two 80meg tables. |
|
| Back to top |
|
|
Bill Woodger
Moderator Emeritus
Joined: 09 Mar 2011
Posts: 7309
Location: Inside the Matrix
|
|
|
|
| kunal jain wrote: |
Basically I need to compare the records of two VSAM files.
This comparision is a bit complex in a sense that
1. there are some acceptable differences which i need to ignore
2. Record has a. item key b. item description. one item key may contain lacs of description distinguished by sequence number..
3. the description coming in two files for an item key is not is sequence. So to compare all the description for an item-key, I am planning to store them in two internal tables. then i am planning to check for an item-key if every description of 1st Table is present in 2nd table(anywhere) or not.If not present in 2nd Table then it will go to output report as Deleted description and for vice versa case as Inserted record.
[...] |
So, you've covered the "acceptable" differences elsewhere.
In addition to the 165 bytes of description, each file record has a item key and a sequence number. There may be 100,000s of descriptions for one item key, if I take you literally (so you allow for 500,000).
The descriptions are not in sequence, which presumably must mean, somehow, that the sequence numbers are not in sequence? Or do you mean that the descriptions are, for instance, in time sequence, so that sequence 0000001 to sequence 0000002 for a particular key might be seperated by one or more description records for another key?
So far you have left Key (might not be necessary) and Sequence Number out of your table. In goes the Sequence, and that is another 2meg gone. Not much more than 50% business growth and your solution comes to a dead halt. OK, you could bodge it about to add more working-storages to keep it going, but it isn't a particularly bright idea.
I don't know whether I am one of enrico's timewasters (and can't work out if it would be good to be one, or not) but my reason to keep poking at this thing is this. When we get our first idea how to solve a problem it can be difficult, at first, to accept that we might not have the best solution. It's our solution, we like it. We're going to stick to it. How to break that down? Just bashing may entrench the idea, rather than weaken faith in it. Some other options might get things going in a different way.
Kunal Jain, unless you can show more information as to why, you don't need your gigantic tables for this task. How many records do you have on file? What percentage of keys have 100,000s of descriptions? Are you just reporting? How often are you running the job? |
|
| Back to top |
|
|
kunal jain
New User
Joined: 19 May 2011
Posts: 59
Location: India
|
|
|
|
The file has around 15 crore records. For a very few Item-key, there are around 8 lacs of item description. Around 1 % of item-keys has 1lacs to 9 lacs of records.
I need to report the mismatch of the item descriptions(& also the extra ones if exist in any of the file). This Comparision is done on a monthly basis. |
|
| Back to top |
|
|
kunal jain
New User
Joined: 19 May 2011
Posts: 59
Location: India
|
|
|
|
| To add more, this comparision is to verify the o/p of new process with o/p of old process. |
|
| Back to top |
|
|
Bill Woodger
Moderator Emeritus
Joined: 09 Mar 2011
Posts: 7309
Location: Inside the Matrix
|
|
|
|
OK, sorry Kunal Jain, what is a crore? I knew, by accident, a lac/lakh, so I'm guessing a million or 10 million, but makes a difference.
If you have 8 lacs already, how are they going in a 5 lac table? And you have 9 lacs as well? |
|
| Back to top |
|
|
kunal jain
New User
Joined: 19 May 2011
Posts: 59
Location: India
|
|
|
|
Sorry for the confusion...
2 VSAM Files has around 150 million records.
Only 1 % of item key has upto 5 lacs records.. |
|
| Back to top |
|
|
Bill Woodger
Moderator Emeritus
Joined: 09 Mar 2011
Posts: 7309
Location: Inside the Matrix
|
|
|
|
OK, so even your gigantic table is already full. Unless you can absolutely solidly say there will be no growth of descriptions beyond the 500,000 (or so) you already have.
Unfortunately, "only 1%" of 150 million is still 1.5 million. So that's a lot of use of your tables, so you'd have to do a very good job at accessing the tables or it will "run like a dog with no legs" as we say.
Can you explain about the file structure?
Can you look back at questions already asked?[/code] |
|
| Back to top |
|
|
kunal jain
New User
Joined: 19 May 2011
Posts: 59
Location: India
|
|
|
|
150 million is the total item description in all for all the item-keys.
File has around 10k item keys and only 1%(of 10k) keys has upto 5 lacs item description. Both the file has same structure
File structure:
FD INPUT-FILE1
RECORD 165
LABEL RECORD STANDARD.
01 INPUT1-REC.
05 ITEM-KEY PIC X(17).
05 ITEM-DESC PIC X(148). |
|
| Back to top |
|
|
Bill Woodger
Moderator Emeritus
Joined: 09 Mar 2011
Posts: 7309
Location: Inside the Matrix
|
|
|
|
| OK, where's the Sequence Number you mentioned? How is the key constructed? Are they unique? What type of VSAM file is it? |
|
| Back to top |
|
|
kunal jain
New User
Joined: 19 May 2011
Posts: 59
Location: India
|
|
|
|
ITEM KEY-->
05 ITEM-kEY.
10 KEY PIC x(7).
10 SEQ pic X(10).
VSAM File is KSDS.
For the item desc of the same key, just seq keeps on incrementing for every desc so as to distinguished between the desc of the same key.
Please let me know if any more info is needed |
|
| Back to top |
|
|
Bill Woodger
Moderator Emeritus
Joined: 09 Mar 2011
Posts: 7309
Location: Inside the Matrix
|
|
|
|
A bit late getting round to this one, and "off topic", but..
| dbzTHEdinosauer wrote: |
some shops do not allow NOSSRANGE.
though what you have provide, BILL, will work (and I have used it myself, sometimes),
another possible solution
(if you do not have to reference both tables in the same statement)
is use one 01, use two 05's, one redefining the other, and
flip-flop between table a and table b with a set (the 01 linkage item) to pointer phrase.
actually, I agree with Bill, and a non-large table process be used.
problem with two 500,000 165-char item tables,
you are trying to load everything.
as I said before,
what are you going to do whtn 500,000 165-char items is not enough?
since you need to compare both at the same time,
move an item from table a to working storage
flip the address
move an item from table b to working storage,
do the compare.
but again,
you think that you can load the world to memory and play with it.
can't always do that, might as well learn how to design a complete process
instead of attempting to solve the problem in one program. |
I suppose if the rules are SSRANGE, then you have to compile with it (options can be "locked" at installation, so they can make it so you really can't change it). And I suppose it is the "spirit" of the law not to turn off the checking at run time :-)
I still don't like SSRANGE. I'm open to being convinced.
For "the flip the address" I'd suggest
| Code: |
01 L-ADDRESS-SWAP-BYTE-1 REDEFINES gigantic-table-1
05 FILLER PIC X.
|
And the same for the other table.
Otherwise, every time you flip the address the program will do size-of-gigantic/4096 operations to make the redundant BLLs addressable. For 80 meg, that is close to 20,000 unnecessary BLL settings per flip. The only BLL you need (for subscripting/indexing) is the first one and setting the address of the above will get that for you.
I'm hoping this is not relevant to the current requirment, but for anyone searching later who can't avoid gigantic tables. |
|
| Back to top |
|
|
dbzTHEdinosauer
Global Moderator
Joined: 20 Oct 2006
Posts: 6966
Location: porcelain throne
|
|
|
|
SSRANGE keeps the animals in line.
hate wasting time debugging things that SSRANGE catches,
when rookies (or experienced programmers who are actually rookies)
do dumb things with their code.
But, I do not advocate SSRANGE in production, does use a few resources.
side topic:
I always use 1 byte linkage area. then I float a linkage definition on that address, thus the BLL re-alignment does not enter in.
as far as trying to explain a 1 byte linkage area, floating another linkage area on that aderess, was just too much to hope for. so, i did not.
my only thoughts of this thread,
were well stated by enrico, earlier:
why waste good technical solutions on bad design? |
|
| Back to top |
|
|
Bill Woodger
Moderator Emeritus
Joined: 09 Mar 2011
Posts: 7309
Location: Inside the Matrix
|
|
|
|
OK, get the SSRANGE point. Maybe you should have an "SSRANGE box", like a "swear box". Any time SSRANGE kicks in, culprit has to kick in 10 bucks. To be divided equally at the end of the month between those who have not contributed that month.
My BLL point wasn't for you Dick, I just panicked a little that one day someone will actually do a search on the forum, and it'll be this topic they need and then they'll go off an do it "as is" and blame, well, you actually, so what the heck. I don't suppose I have many Cobol tricks that you don't know :-)
You're right and enrico's right. |
|
| Back to top |
|
|
Bill O'Boyle
CICS Moderator
Joined: 14 Jan 2008
Posts: 2501
Location: Atlanta, Georgia, USA
|
|
|
|
To avoid going beyond an array HIBOUNDS, calculate this array-max up front in HOUSEKEEPING and use this max as your HIBOUNDS throughout the code. It will always be correct, regardless of the OCCURS.
Example -
| Code: |
03 WS-FWORD PIC 9(08) COMP.
03 WS-ARRAY-REC.
05 WS-ARRAY-ENTRY OCCURS 100 TIMES
INDEXED BY X-WS-AE, X-WS-AE-MAX
PIC X(256).
DIVIDE LENGTH OF WS-ARRAY-REC BY LENGTH OF WS-ARRAY-ENTRY (1)
GIVING WS-FWORD.
SET X-WS-AE-MAX TO WS-FWORD.
|
Hard coding array-max's are for the birds....
Bill |
|
| Back to top |
|
|
enrico-sorichetti
Superior Member
Joined: 14 Mar 2007
Posts: 10873
Location: italy
|
|
|
|
please avoid using local jargon or Your own language
since we take time to understand Your problem it would be nice on Your side to spare us the trouble of understanding and translating Your language
( crores, laks, and friends )
almost time to lock the topic, looks like it is going nowhere and is just wasting everybody' s time  |
|
| Back to top |
|
|
Bill O'Boyle
CICS Moderator
Joined: 14 Jan 2008
Posts: 2501
Location: Atlanta, Georgia, USA
|
|
|
|
Enrico,
Does this mean I need to stop using using the "FWORD"? 
Bill |
|
| Back to top |
|
|
enrico-sorichetti
Superior Member
Joined: 14 Mar 2007
Posts: 10873
Location: italy
|
|
|
|
Naah !
IIRC the FWORD has been a standard term in IT jargon for a loong time
( HWORD, FWORD, DWORD, ... ) |
|
| Back to top |
|
|
Akatsukami
Global Moderator
Joined: 03 Oct 2009
Posts: 1788
Location: Bloomington, IL
|
|
|
|
| enrico-sorichetti wrote: |
Naah !
IIRC the FWORD has been a standard term in IT jargon for a loong time
( HWORD, FWORD, DWORD, ... ) |
I've never heard a colleague use "fnord". Of course, I wouldn't  |
|
| Back to top |
|
|
Nic Clouston
Global Moderator
Joined: 10 May 2007
Posts: 2455
Location: Hampshire, UK
|
|
|
|
| Why not just run SUPERC or COMPAREX if you have that installed? |
|
| Back to top |
|
|
Bill Woodger
Moderator Emeritus
Joined: 09 Mar 2011
Posts: 7309
Location: Inside the Matrix
|
|
|
|
| Bill O'Boyle wrote: |
To avoid going beyond an array HIBOUNDS, calculate this array-max up front in HOUSEKEEPING and use this max as your HIBOUNDS throughout the code. It will always be correct, regardless of the OCCURS.
Example -
| Code: |
03 WS-FWORD PIC 9(08) COMP.
03 WS-ARRAY-REC.
05 WS-ARRAY-ENTRY OCCURS 100 TIMES
INDEXED BY X-WS-AE, X-WS-AE-MAX
PIC X(256).
DIVIDE LENGTH OF WS-ARRAY-REC BY LENGTH OF WS-ARRAY-ENTRY (1)
GIVING WS-FWORD.
SET X-WS-AE-MAX TO WS-FWORD.
|
Hard coding array-max's are for the birds....
Bill |
This is nice.
However, I'd like to have-my-cake-and-eat-it by also having the hard-coded max (maybe a "seed cake", for the birds).
Reason is, when someone comes along with the intention of changing the table, extending the occurs, and what they do is... just change the array-max... It would actually be nice if Cobol let you do that, or to get the max from the OCCURS clause, but it doesn't. Now, with your idea Bill, maybe we can cover that as well.
| Code: |
03 WS-ARRAY-MAX PIC 9(08) COMP VALUE 100.
03 WS-FWORD PIC 9(08) COMP.
03 WS-ARRAY-REC.
05 WS-ARRAY-ENTRY OCCURS 100 TIMES
INDEXED BY X-WS-AE, X-WS-AE-MAX
PIC X(256).
DIVIDE LENGTH OF WS-ARRAY-REC BY LENGTH OF WS-ARRAY-ENTRY (1)
GIVING WS-FWORD.
IF WS-FWORD NOT EQUAL TO WS-ARRAY-MAX
report and gracefully collapse
END-IF
SET X-WS-AE-MAX TO WS-FWORD.
|
I know it might seem redundant, but it catches those times that the OCCURS should have been changed, but was not, and the max was. |
|
| Back to top |
|
|
kunal jain
New User
Joined: 19 May 2011
Posts: 59
Location: India
|
|
|
|
Thank you all for your advice and suggestion.
I have got the clue of what can be the path to my solution.
I am very pleased to receive in-depth analysis and suggestion to my problem.
Thank you so much..  |
|
| Back to top |
|
|
|
|
