Portal | Manuals | References | Downloads | Info | Programs | JCLs | Master the Mainframes
IBM Mainframe Computers Forums Index
 
Register
 
IBM Mainframe Computers Forums Index Mainframe: Search IBM Mainframe Forum: FAQ Memberlist Usergroups Profile Log in to check your private messages Log in
 

 

Being confused by merge performance

 
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> DFSORT/ICETOOL
View previous topic :: :: View next topic  
Author Message
giraffe

New User


Joined: 09 Nov 2010
Posts: 28
Location: SHENZHEN CHINA

PostPosted: Wed Jan 05, 2011 6:49 pm    Post subject: Being confused by merge performance
Reply with quote

Hi, friends,

Recently I am engaged in batch performance tuning, but I was really confused by the DFSORT MERGE tool, since it works worse than COBOL program, even worse than sort in some cases(hiperspace is enough for sort).

I can guarantee input files in order individually, will MERGE tool take extra time to check the order of every input file ?

Once I suppose the performance difference between SORT and merge is N*LOGN vs N, is it right?
Back to top
View user's profile Send private message

dick scherrer

Site Director


Joined: 23 Nov 2006
Posts: 19270
Location: Inside the Matrix

PostPosted: Wed Jan 05, 2011 9:45 pm    Post subject:
Reply with quote

Hello,

Quote:
since it works worse than COBOL program, even worse than sort in some cases(hiperspace is enough for sort).
Sorry, but this does not make sense. . .

MERGEing 2 already sorted files is extremely fast - unless you have done something to degrade the performance. . .

Suggest you post the JCL and MERGE control statements as well as the informational messages (including message ids) generated by the run.
Back to top
View user's profile Send private message
giraffe

New User


Joined: 09 Nov 2010
Posts: 28
Location: SHENZHEN CHINA

PostPosted: Thu Jan 06, 2011 6:17 am    Post subject:
Reply with quote

Hi, dick, my code list as below:
Code:
//MERGE      EXEC PGM=SORT
//SORTIN01 DD     DSN=...(FILE RECORD LENGTH 1500)
//SORTIN02 DD     DSN=...
...
//SORTIN12 DD     DSN=...
//OUT1        DD     DSN=...
//OUT2        DD     DSN=...
//SYSIN       DD     *
MERGE    FIELDS=(1,2,PD,A,3,9,PD,A)                   
RECORD   TYPE=F                                       
                                                     
OUTFIL   FNAMES=(OUT1),                           
         BUILD=(3,9,C'1',8X,1329,12,6X,3,9,C'1',50X,/,
                3,9,C'2',8X,1341,12,6X,3,9,C'2',50X,/,
                3,9,C'3',1277,26,3,9,C'3',50X,/,     
                3,9,C'4',1303,26,3,9,C'4',50X,/,     
                3,9,C'6',1225,26,3,9,C'6',50X,/,     
                3,9,C'7',1251,26,3,9,C'7',50X)       
                                                     
OUTFIL   FNAMES=(OUT2),                           
         BUILD=(1049,50,3,9,41X,/,                   
                1099,50,3,9,41X,/,                   
                1149,50,3,9,41X)                     
/*


The maximum available resource for me: main storage: 28G, mips: 2600.
and there're 2 million records in each SORTINnn dataset.
the cpu time for this step is 3.9 minutes, excp 301K, elapse time 10 minutes.
with COBOL program read, compare, and write,
the cpu time for this step is 2.5 minutes, excp 680K, elapse time 6 minutes.
Back to top
View user's profile Send private message
dick scherrer

Site Director


Joined: 23 Nov 2006
Posts: 19270
Location: Inside the Matrix

PostPosted: Thu Jan 06, 2011 10:37 am    Post subject:
Reply with quote

Hello,

That's a start. . . We also need:
Quote:
as well as the informational messages (including message ids) generated by the run.


There is a COBOL program that actually does a 12 file merge. . .? How does this code accomplish the "merge"?

Unless there is something i am missing, main storage should not be an issue. Why do you believe you have 2600 mips available for this process?
Back to top
View user's profile Send private message
giraffe

New User


Joined: 09 Nov 2010
Posts: 28
Location: SHENZHEN CHINA

PostPosted: Thu Jan 06, 2011 7:26 pm    Post subject:
Reply with quote

Hi, dick,

the MERGE process implemented by COBOL just like the MERGE illustration showed in DFSORT Application Programming Guide (Figure 14),just read and choose the record whose key is the minmum one.

2600 MIPS is the highest limit for my testing environment, there's also some other jobs being executed.

As to storage, I guess the sort performance will be affected by main storage, if sort possess enough main storage, there's no need to swap intermediate sorted result back to disk.

I'll try to copy and post the JCL log messages from my performance tuning environment.

Thanks a lot for your response.
Back to top
View user's profile Send private message
dick scherrer

Site Director


Joined: 23 Nov 2006
Posts: 19270
Location: Inside the Matrix

PostPosted: Fri Jan 07, 2011 3:02 am    Post subject:
Reply with quote

Hello,

Quote:
As to storage, I guess the sort performance will be affected by main storage, if sort possess enough main storage, there's no need to swap intermediate sorted result back to disk.
Well, there is some truth here, but this truth should not be as significant when doing a MERGE. . .

Lots of storage can help speed up a SORT because things may be done in-core instead of with i/o (as you mention).

It will still be most helpful if the information from both the MERGE and the COBOL runs are posted.
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> DFSORT/ICETOOL All times are GMT + 6 Hours
Page 1 of 1

 

Search our Forum:

Similar Topics
Topic Author Forum Replies Posted
No new posts BC, BCR, BRC, BRCL performance steve-myers PL/I & Assembler 0 Fri Dec 23, 2016 7:44 am
No new posts PL/I code tuning/Performance improvement Virendra Shambharkar PL/I & Assembler 4 Mon Dec 05, 2016 11:57 am
No new posts What are the way we can improve CPU p... Gunapala CN DB2 10 Mon Oct 24, 2016 2:16 pm
No new posts SQL Merge error -254 Robin Sulsona DB2 8 Wed Jun 01, 2016 8:28 pm
No new posts SAS MERGE to DFSORT Musab Ahmed DFSORT/ICETOOL 6 Mon Feb 01, 2016 5:44 pm


Facebook
Back to Top
 
Mainframe Wiki | Forum Rules | Bookmarks | Subscriptions | FAQ | Tutorials | Contact Us