View previous topic :: View next topic
|
Author |
Message |
Rahul_kumar Warnings : 2 New User
Joined: 24 Jun 2006 Posts: 50
|
|
|
|
Hi,
I have a requirement to load replace a table from multiple input files received from external system.I cannot merge the files into a single file as the size of each file is quite huge(around 2 GB of data in each file).
The approach which I am following is to empty the table first and then Load resume with each file.Can anyone suggest any better approach of doing it.
Can we concatenate the files and do a load replace?
An urgent response would be appreciated.
Thanks,
Rahul |
|
Back to top |
|
|
daveporcelan
Active Member
Joined: 01 Dec 2006 Posts: 792 Location: Pennsylvania
|
|
|
|
You could do a Load Replace on the first file, which will do an 'empty' as you put it.
Then do a Load Resume on the other files.
The concatenation of files should work as well.
I would guess the later would be a little more efficient.
A test with two smaller files would not be a bad idea.
I am calm in my response, only you feel the urgency. |
|
Back to top |
|
|
Bill O'Boyle
CICS Moderator
Joined: 14 Jan 2008 Posts: 2501 Location: Atlanta, Georgia, USA
|
|
|
|
We are not sitting on some cloud, waiting for queries to be sent our way.
To phrase your request as "Urgent" shows disrespect for any member who voluntarily gives their free time to this site and they're obliged to provide an answer sooner than later.
"Thanks in Advance" (or the TIA acronym) is another phrase which indicates you expect a reply and this also rubs people the wrong way.
Mr. Bill |
|
Back to top |
|
|
Marso
REXX Moderator
Joined: 13 Mar 2006 Posts: 1353 Location: Israel
|
|
|
|
Use of
Code: |
//SYSREC DD DISP=SHR,DSN=big.file1
// DD DISP=SHR,DSN=big.file2
// DD DISP=SHR,DSN=big.file3
//SYSIN DD *
LOAD DATA REPLACE |
works perfectly for me each time I need. |
|
Back to top |
|
|
gylbharat
Active Member
Joined: 31 Jul 2009 Posts: 565 Location: Bangalore
|
|
|
|
Rahul_kumar wrote: |
Hi,
I have a requirement to load replace a table from multiple input files received from external system.I cannot merge the files into a single file as the size of each file is quite huge(around 2 GB of data in each file).
Thanks,
Rahul |
As you mentioned... Size of each file is more than 2G. It would be better to have a partitioned tablespace and load data parallelly .. |
|
Back to top |
|
|
dick scherrer
Moderator Emeritus
Joined: 23 Nov 2006 Posts: 19244 Location: Inside the Matrix
|
|
|
|
Hello,
In most places, 2G is considered a rather modest size. |
|
Back to top |
|
|
gylbharat
Active Member
Joined: 31 Jul 2009 Posts: 565 Location: Bangalore
|
|
|
|
dick scherrer wrote: |
Hello,
In most places, 2G is considered a rather modest size. |
TS said... 2G for each file... Considering 4-5 files... data becomes around 10GB.
Quote: |
Segmented : This type of table space holds more than one table. The available space is divided into groups of pages called segments, each the same size. Each segment contains rows from only one table. A segmented table space can have between 1 and 32 VSAM linear data sets, all of which are either user defined or in the same storage group. The maximum size of a data set in the segmented table space is 2 gigabytes. And so, the maximum size of a segmented table space is 64 gigabytes (2 gigabytes multiplied by 32 data sets).
|
So this segmented tablespace will have around 5-6 liner datasets... Considering increase in data gradually... it would be better to have a partitioned tablespace. Partition will help in parallelism of query and data loading. |
|
Back to top |
|
|
dick scherrer
Moderator Emeritus
Joined: 23 Nov 2006 Posts: 19244 Location: Inside the Matrix
|
|
|
|
Hello,
Quote: |
TS said... 2G for each file... Considering 4-5 files... data becomes around 10GB. |
Which is still not very much. . .
Quote: |
Considering increase in data gradually... it would be better to have a partitioned tablespace. |
I saw no "gradually" . . . This is a complete reload and this data may be used for reference or decision support rather than any transaction processing.
Quote: |
Partition will help in parallelism of query and data loading. |
As the data is to be loaded en masse and we know nothing about the use of it after loading, this might be premature. Possibly TS will clarify. |
|
Back to top |
|
|
Rahul_kumar Warnings : 2 New User
Joined: 24 Jun 2006 Posts: 50
|
|
|
|
Hi All,
Thanks for the reply everyone!
@Bill O'Boyle - Will keep your suggestion in mind next time! |
|
Back to top |
|
|
gylbharat
Active Member
Joined: 31 Jul 2009 Posts: 565 Location: Bangalore
|
|
|
|
Hi Rahul,
Can you please share... what solution is now getting implemented in your shop? |
|
Back to top |
|
|
Anuj Dhawan
Superior Member
Joined: 22 Apr 2006 Posts: 6250 Location: Mumbai, India
|
|
|
|
Bill O'Boyle wrote: |
We are not sitting on some cloud, waiting for queries to be sent our way.
To phrase your request as "Urgent" shows disrespect for any member who voluntarily gives their free time to this site and they're obliged to provide an answer sooner than later.
"Thanks in Advance" (or the TIA acronym) is another phrase which indicates you expect a reply and this also rubs people the wrong way.
|
Agree. Specially, TIA is preety irritating. |
|
Back to top |
|
|
|