IBM Mainframe Forum Index
 
Log In
 
IBM Mainframe Forum Index Mainframe: Search IBM Mainframe Forum: FAQ Register
 

Which one is Better Array Processing or File Processing


IBM Mainframe Forums -> JCL & VSAM
Post new topic   Reply to topic
View previous topic :: View next topic  
Author Message
satmaar
Currently Banned

New User


Joined: 11 Feb 2009
Posts: 16
Location: Bangalore

PostPosted: Thu Aug 19, 2010 3:48 pm
Reply with quote

Hi,

I would like to know which one is better for processing interms of time, internal array(READ/WRITE) and processing of VSAM KSDS File(READ/WRITE) for about 300K Records?

Thanks,
satmaar
Back to top
View user's profile Send private message
Bill O'Boyle

CICS Moderator


Joined: 14 Jan 2008
Posts: 2501
Location: Atlanta, Georgia, USA

PostPosted: Thu Aug 19, 2010 3:58 pm
Reply with quote

I'm unsure as to how you'd load the array internally into your program. This needs to be explained a little better.

How many of the 300K records will be WRITES?

Too many random WRITES can be costly and inefficient, but what that "too many" is requires background information and research on your part.

If you estimate that this file may grow in the future and the number of WRITES becomes significant, then you may reach a point where it might be a better choice to write all the records to a temporary flat-file and then SORT by key.

Also, ensure that you're using adequate buffers for this file. The AMORG JCL parameter is the place to check.

Bill
Back to top
View user's profile Send private message
dbzTHEdinosauer

Global Moderator


Joined: 20 Oct 2006
Posts: 6966
Location: porcelain throne

PostPosted: Thu Aug 19, 2010 3:59 pm
Reply with quote

satmaar wrote:
internal array(READ/WRITE)


what do you mean with todays' made-up word?
Back to top
View user's profile Send private message
satmaar
Currently Banned

New User


Joined: 11 Feb 2009
Posts: 16
Location: Bangalore

PostPosted: Thu Aug 19, 2010 5:08 pm
Reply with quote

Bill,Thanks for your Reply.
Let me explain, We are Declaring array in a program and it is being used for Writing from other input file and later this array is used for input to create other file in the same program.

So what i wanted to know is instead of creating this array in program,if we could already have this info copied into any files will it give lesser processing time or it ll give same time as array process.

Hope you understood.please let me know if i could elaborate little more.

Thanks,
Satmaar
Back to top
View user's profile Send private message
satmaar
Currently Banned

New User


Joined: 11 Feb 2009
Posts: 16
Location: Bangalore

PostPosted: Thu Aug 19, 2010 5:10 pm
Reply with quote

dbzTHEdinosauer wrote:
satmaar wrote:
internal array(READ/WRITE)

Actually i wanted to say this array was declared inside the program.Sorry for the confussion
what do you mean with todays' made-up word?
Back to top
View user's profile Send private message
dbzTHEdinosauer

Global Moderator


Joined: 20 Oct 2006
Posts: 6966
Location: porcelain throne

PostPosted: Thu Aug 19, 2010 5:33 pm
Reply with quote

COBOL does not have internal arrays.
there are host variable arrays for db2.
the term in cobol is COBOL INTERNAL Table.

array is a term used for JAVA, C, etc....

and your requirement definition lacks specifics.

what actually are you doing. saying I want to write and read is not professional.
to receive any useful help from us you need to describe what your requirement is.
as Bill said, size is very important.
also, what processing of the records is required? reformat?

all we know is that you want to do something with a vsam file
and you have mentioned a figure of 300k records.
are these part of the vsam file, are they another file?
Back to top
View user's profile Send private message
Bill O'Boyle

CICS Moderator


Joined: 14 Jan 2008
Posts: 2501
Location: Atlanta, Georgia, USA

PostPosted: Thu Aug 19, 2010 7:28 pm
Reply with quote

The COBOL internal table option should be eliminated as an option.

If the internal table were defined to hold 300K+ records and each record is (conservatively speaking) 100-bytes long, that's 30M+ bytes of memory alone, just for a single table.

This would rival the size of nearly all load modules....

Bill
Back to top
View user's profile Send private message
PeterHolland

Global Moderator


Joined: 27 Oct 2009
Posts: 2481
Location: Netherlands, Amstelveen

PostPosted: Thu Aug 19, 2010 9:08 pm
Reply with quote

Bill O'Boyle wrote:
The COBOL internal table option should be eliminated as an option.

If the internal table were defined to hold 300K+ records and each record is (conservatively speaking) 100-bytes long, that's 30M+ bytes of memory alone, just for a single table.

This would rival the size of nearly all load modules....

Bill


And its not possible that storage in COBOL is allocated dynamically?
Back to top
View user's profile Send private message
dick scherrer

Moderator Emeritus


Joined: 23 Nov 2006
Posts: 19244
Location: Inside the Matrix

PostPosted: Thu Aug 19, 2010 9:15 pm
Reply with quote

Hello,

Quote:
I would like to know which one is better for processing interms of time, internal array(READ/WRITE) and processing of VSAM KSDS File(READ/WRITE) for about 300K Records?
Rather than giving us 2 "solutions" to rate or choose between, you would get more usable info if you described the requirement you need to accomplish.

Once people know what the actual business process needs to support, better suggestions can be made.
Back to top
View user's profile Send private message
Bill O'Boyle

CICS Moderator


Joined: 14 Jan 2008
Posts: 2501
Location: Atlanta, Georgia, USA

PostPosted: Thu Aug 19, 2010 10:18 pm
Reply with quote

Peter Holland said -
Quote:

And its not possible that storage in COBOL is allocated dynamically?

It is certainly possible, by obtaining external storage via CEEGTST, an MVS GETMAIN or a STORAGE OBTAIN, but 30M+ is a sizable amount and this was just a guess, because the LRECL was not furnished.

Plus, I don't know what version/release of COBOL the OP is running?

In COBOL2, the largest size for an 01 level is 16MB-1. But of course, later version/releases allow a greater size.

However (as we're frequent lurkers of this site), many times these shops are still running COBOL2 or (even worse), OS/VS COBOL, which has a 128K 01 level maximum size.

If/when the table grows exponentially, to where 100MB+ is required, wouldn't that be the time to sit back and question whether this was a good idea to begin with as well as seek out and pummel the designers and reward the non-participants? (Managerial ethics, ya know).

Just my 2ยข.... icon_wink.gif

Bill
Back to top
View user's profile Send private message
PeterHolland

Global Moderator


Joined: 27 Oct 2009
Posts: 2481
Location: Netherlands, Amstelveen

PostPosted: Fri Aug 20, 2010 1:47 am
Reply with quote

Bill,

i was wondering about this :

Quote:

This would rival the size of nearly all load modules....


Beside that, for array processing like the TS suggested, means reading the data in memory, building array/table processing (Dick Brenholtz will be puking now because of my terms).
I think its easier to process VSAM KSDS data if we are talking about huge
quantities of records.
Back to top
View user's profile Send private message
Bill O'Boyle

CICS Moderator


Joined: 14 Jan 2008
Posts: 2501
Location: Atlanta, Georgia, USA

PostPosted: Fri Aug 20, 2010 2:46 am
Reply with quote

Peter,

Actually, what I was trying to say was, if the table were part of COBOL WS, the load module would be huge and its size would probably be larger than any other load module in the shop.

I can't think of a reason to have such a large load module and suggested acquiring external storage.

But yes, I can hear Dick puking as I type this. icon_eek.gif

I guess this is why he changed his location to the Porcelain Throne? icon_wink.gif

Bill
Back to top
View user's profile Send private message
dbzTHEdinosauer

Global Moderator


Joined: 20 Oct 2006
Posts: 6966
Location: porcelain throne

PostPosted: Fri Aug 20, 2010 7:55 am
Reply with quote

My, my, my - is the heretofore even-steven Bill O'Boyle developing a sarcastic bent to his posts? icon_lol.gif

and my new location has more to do with the origin of my posts,
rather than my reaction to others'.

and just for the record:
dynamic allocations are perfect when you are not sure of your potential requirement,
but if you know that you are going to use a 100,000 item table,
allocate for it at compile time. That way the scheduling of the Job containing the step requiring large resources can be planned for and accomodated to insure that there are always adequate resources.

I love dynamic expansion of working-storage:
  • provides a venue for outside the box coding
  • separates the kids from the adults
Back to top
View user's profile Send private message
Bill Dennis

Active Member


Joined: 17 Aug 2007
Posts: 562
Location: Iowa, USA

PostPosted: Fri Aug 20, 2010 8:13 am
Reply with quote

If access time is a concern, you may be able to allocate a UNIT=VIO device in your JCL. This simulates a disk file but resides in memory so no special GETMAINs. Some shops limit the size of VIO files so as not to over-commit memory.

Heard an analogy that reading from memory is like reaching into the mailbox outside the front door. Reading from disk is like getting in the car and driving to the main post office.
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic View Bookmarks
All times are GMT + 6 Hours
Forum Index -> JCL & VSAM

 


Similar Topics
Topic Forum Replies
No new posts How to split large record length file... DFSORT/ICETOOL 10
No new posts Extracting Variable decimal numbers f... DFSORT/ICETOOL 17
No new posts SFTP Issue - destination file record ... All Other Mainframe Topics 2
No new posts Access to non cataloged VSAM file JCL & VSAM 18
No new posts Need help for File Aid JCL to extract... Compuware & Other Tools 23
Search our Forums:

Back to Top