IBM Mainframe Forum Index
 
Log In
 
IBM Mainframe Forum Index Mainframe: Search IBM Mainframe Forum: FAQ Register
 

Being confused by merge performance


IBM Mainframe Forums -> DFSORT/ICETOOL
Post new topic   Reply to topic
View previous topic :: View next topic  
Author Message
giraffe

New User


Joined: 09 Nov 2010
Posts: 27
Location: SHENZHEN CHINA

PostPosted: Wed Jan 05, 2011 6:49 pm
Reply with quote

Hi, friends,

Recently I am engaged in batch performance tuning, but I was really confused by the DFSORT MERGE tool, since it works worse than COBOL program, even worse than sort in some cases(hiperspace is enough for sort).

I can guarantee input files in order individually, will MERGE tool take extra time to check the order of every input file ?

Once I suppose the performance difference between SORT and merge is N*LOGN vs N, is it right?
Back to top
View user's profile Send private message
dick scherrer

Moderator Emeritus


Joined: 23 Nov 2006
Posts: 19244
Location: Inside the Matrix

PostPosted: Wed Jan 05, 2011 9:45 pm
Reply with quote

Hello,

Quote:
since it works worse than COBOL program, even worse than sort in some cases(hiperspace is enough for sort).
Sorry, but this does not make sense. . .

MERGEing 2 already sorted files is extremely fast - unless you have done something to degrade the performance. . .

Suggest you post the JCL and MERGE control statements as well as the informational messages (including message ids) generated by the run.
Back to top
View user's profile Send private message
giraffe

New User


Joined: 09 Nov 2010
Posts: 27
Location: SHENZHEN CHINA

PostPosted: Thu Jan 06, 2011 6:17 am
Reply with quote

Hi, dick, my code list as below:
Code:
//MERGE      EXEC PGM=SORT
//SORTIN01 DD     DSN=...(FILE RECORD LENGTH 1500)
//SORTIN02 DD     DSN=...
...
//SORTIN12 DD     DSN=...
//OUT1        DD     DSN=...
//OUT2        DD     DSN=...
//SYSIN       DD     *
MERGE    FIELDS=(1,2,PD,A,3,9,PD,A)                   
RECORD   TYPE=F                                       
                                                     
OUTFIL   FNAMES=(OUT1),                           
         BUILD=(3,9,C'1',8X,1329,12,6X,3,9,C'1',50X,/,
                3,9,C'2',8X,1341,12,6X,3,9,C'2',50X,/,
                3,9,C'3',1277,26,3,9,C'3',50X,/,     
                3,9,C'4',1303,26,3,9,C'4',50X,/,     
                3,9,C'6',1225,26,3,9,C'6',50X,/,     
                3,9,C'7',1251,26,3,9,C'7',50X)       
                                                     
OUTFIL   FNAMES=(OUT2),                           
         BUILD=(1049,50,3,9,41X,/,                   
                1099,50,3,9,41X,/,                   
                1149,50,3,9,41X)                     
/*


The maximum available resource for me: main storage: 28G, mips: 2600.
and there're 2 million records in each SORTINnn dataset.
the cpu time for this step is 3.9 minutes, excp 301K, elapse time 10 minutes.
with COBOL program read, compare, and write,
the cpu time for this step is 2.5 minutes, excp 680K, elapse time 6 minutes.
Back to top
View user's profile Send private message
dick scherrer

Moderator Emeritus


Joined: 23 Nov 2006
Posts: 19244
Location: Inside the Matrix

PostPosted: Thu Jan 06, 2011 10:37 am
Reply with quote

Hello,

That's a start. . . We also need:
Quote:
as well as the informational messages (including message ids) generated by the run.


There is a COBOL program that actually does a 12 file merge. . .? How does this code accomplish the "merge"?

Unless there is something i am missing, main storage should not be an issue. Why do you believe you have 2600 mips available for this process?
Back to top
View user's profile Send private message
giraffe

New User


Joined: 09 Nov 2010
Posts: 27
Location: SHENZHEN CHINA

PostPosted: Thu Jan 06, 2011 7:26 pm
Reply with quote

Hi, dick,

the MERGE process implemented by COBOL just like the MERGE illustration showed in DFSORT Application Programming Guide (Figure 14),just read and choose the record whose key is the minmum one.

2600 MIPS is the highest limit for my testing environment, there's also some other jobs being executed.

As to storage, I guess the sort performance will be affected by main storage, if sort possess enough main storage, there's no need to swap intermediate sorted result back to disk.

I'll try to copy and post the JCL log messages from my performance tuning environment.

Thanks a lot for your response.
Back to top
View user's profile Send private message
dick scherrer

Moderator Emeritus


Joined: 23 Nov 2006
Posts: 19244
Location: Inside the Matrix

PostPosted: Fri Jan 07, 2011 3:02 am
Reply with quote

Hello,

Quote:
As to storage, I guess the sort performance will be affected by main storage, if sort possess enough main storage, there's no need to swap intermediate sorted result back to disk.
Well, there is some truth here, but this truth should not be as significant when doing a MERGE. . .

Lots of storage can help speed up a SORT because things may be done in-core instead of with i/o (as you mention).

It will still be most helpful if the information from both the MERGE and the COBOL runs are posted.
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic View Bookmarks
All times are GMT + 6 Hours
Forum Index -> DFSORT/ICETOOL

 


Similar Topics
Topic Forum Replies
No new posts Merge two VSAM KSDS files into third ... JCL & VSAM 6
No new posts exploiting Z16 performance PL/I & Assembler 2
This topic is locked: you cannot edit posts or make replies. Merge 2 input files based on the reco... JCL & VSAM 2
No new posts Merge 2 input files after sort SYNCSORT 14
No new posts JOIN STATEMENT PERFORMANCE. DFSORT/ICETOOL 12
Search our Forums:

Back to Top