Doubts regarding Merging

Rahul_kumar · New User Joined: 24 Jun 2006 Posts: 50

Hi,

Will there be any perfomance issue if I merge 10 files having length 10000 bytes and 10 files having length as 1000 bytes.

The files with different length have the same number of records and the actual data length is less than 1000 bytes in both type of files.

superk · Posted: Wed Apr 27, 2011 4:45 pm

Personally, I seriously doubt that any difference will be all that noticible.

Phrzby Phil · Posted: Wed Apr 27, 2011 5:04 pm

I would suggest testing, but I'm tired of suggesting that, so don't test it.

Rahul_kumar · New User Joined: 24 Jun 2006 Posts: 50

I have tested and the performance is good in case of the less record length files.Want to know the reason?

Robert Sample · Posted: Wed Apr 27, 2011 5:23 pm

Because block size matters -- LRECL does not. LRECL is merely a way of dividing up the physical block stored on the device. If the block isze is 8000, you can use LRECL 80 and get 100 records from the block, or use LRECL 100 and get 80 records from the block. Which one to use? Unless you want to see some funny-looking data, use the LRECL that the file was built with. However, do not expect LRECL to have a performance impact -- it is a logical construct, not a physical construct.

enrico-sorichetti · Posted: Wed Apr 27, 2011 5:36 pm

why not do a bit of arithmetic Yourself .....

let' forget for a while about space/optimization and real device parameters

assume 1000000 records

case 1 LRECL 1000 BLKSIZE 100000 ==> 100 record x block ==> 10000 block to be read
1000000 * 1000 bytes to transfered

case 2 LRECL 10000 BLKSIZE 100000 ==> 10 records x blok ==> 100000 blocks to be read
1000000 * 10000 bytes to be transfered

the data to be transfered and data transfer time is going to be 10 times

do a bit of arithmetics and meditate on Your own