Sort and Merge which one has better performance?

adrianlmk · New User Joined: 02 Jan 2008 Posts: 11 Location: Malaysia

Hi, I have a query on the performance perspective

I have 10 huge file (length 8092 VB and each file got 10 million record)
I need to merge all these file to a single file with a key (e.g. 10,5,CH,A)

I have 2 option to do this.

First
i can concatenate all these file by using SORTIN & SORTOUT SYSIN = SORT FILEDS=(10,5,CH,A)

Second
I can use SORTIN01, SORTIN02........SORTIN10 & SORTOUT SYSIN = MERGE FILEDS=(10,5,CH,A) --- because all this 10 file already pre-sorted with this keys.

May i know which method is better in term of PERFORMANCE.

Thanks

Phrzby Phil · Posted: Wed Jun 01, 2011 6:19 pm

Well, let's think about it.

Your first proposed solution would re-sort a large number of records that had been sorted, except that you strung them together via concatenate, thereby totally unsorting the whole shebang (I'm not sure of the origin of this technical term).

Your 2nd solution takes already sorted files and then just interleaves them, each loop taking the lowest ranking top record from one of the ten files.

Hmmmmmm.

Frank Yaeger · Posted: Wed Jun 01, 2011 10:30 pm

In general, a MERGE is more efficient than a SORT.

However, why not try doing both yourself and see which one is more efficient yourself. That's always the best way to answer a question like that.

Escapa · Posted: Thu Jun 02, 2011 5:29 am

Frank Yaeger · Posted: Thu Jun 02, 2011 10:42 pm

Yes, but a pretty well known one. Sorting is more complex than merging and generally requires the use of more resources. For example, a SORT may require the use of intermediate storage (e.g. hiperspace or work space) whereas a MERGE never does.

adrianlmk · New User Joined: 02 Jan 2008 Posts: 11 Location: Malaysia

Hi everyone, thanks for giving me a lot of comment and suggestion. I think I got the answer that I want.. Thanks for your help