Portal | Manuals | References | Downloads | Info | Programs | JCLs | Master the Mainframes
IBM Mainframe Computers Forums Index
 
Register
 
IBM Mainframe Computers Forums Index Mainframe: Search IBM Mainframe Forum: FAQ Memberlist Usergroups Profile Log in to check your private messages Log in
 

 

MERGE performance

 
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> DFSORT/ICETOOL
View previous topic :: :: View next topic  
Author Message
Joel

New User


Joined: 07 Feb 2006
Posts: 11

PostPosted: Tue May 09, 2006 3:20 am    Post subject: MERGE performance
Reply with quote

Can anyone explain why the PULL step below takes about 3 times longer
than the PUSH step ? (I was hoping they would take equal elasped time as
the PULL step merges exactly the data created in the PUSH step).
Code:
                     
//*                                                                     
//PUSH     EXEC SORT                                                   
//SORTIN   DD  *                                                       
                                                                       
//S1       DD  DISP=(,PASS),SPACE=(CYL,300),DSN=&&S1                   
//S2       DD  DISP=(,PASS),SPACE=(CYL,300),DSN=&&S2                   
//S3       DD  DISP=(,PASS),SPACE=(CYL,300),DSN=&&S3                   
                                                                       
 OPTION COPY                                                           
 OUTFIL FNAMES=(S1,S2,S3),                                             
 OUTREC=(SEQNUM,16,ZD,27998:X),REPEAT=9000                             
                                                                       
//PULL     EXEC SORT                                                   
//SORTIN01 DD  DISP=SHR,DSN=&&S1                                       
//SORTIN02 DD  DISP=SHR,DSN=&&S2                                       
//SORTIN03 DD  DISP=SHR,DSN=&&S3                                       
//SORTOUT  DD  SYSOUT=*                                                 
                                                                       
 MERGE FIELDS=(1,16,CH,A)                                               
 INCLUDE COND=(14,3,CH,EQ,C'666')
Back to top
View user's profile Send private message

Joel

New User


Joined: 07 Feb 2006
Posts: 11

PostPosted: Wed May 10, 2006 3:27 am    Post subject: Re: MERGE performance
Reply with quote

I tried changing the JCL to use permanent datasets instead of && datasets:

//TTD0397P JOB 0,'PUSHPULL',MSGCLASS=X,CLASS=B
//*
//PUSH EXEC SORT
//SORTIN DD *

//S1 DD DISP=(,CATLG),SPACE=(CYL,300),DSN=TTDX.S1
//S2 DD DISP=(,CATLG),SPACE=(CYL,300),DSN=TTDX.S2
//S3 DD DISP=(,CATLG),SPACE=(CYL,300),DSN=TTDX.S3

OPTION COPY
OUTFIL FNAMES=(S1,S2,S3),
OUTREC=(SEQNUM,16,ZD,27998:X),REPEAT=9000

//PULL EXEC SORT
//SORTIN01 DD DISP=SHR,DSN=TTDX.S1
//SORTIN02 DD DISP=SHR,DSN=TTDX.S2
//SORTIN03 DD DISP=SHR,DSN=TTDX.S3
//SORTOUT DD SYSOUT=*

MERGE FIELDS=(1,16,CH,A)
INCLUDE COND=(14,3,CH,EQ,C'666')

And look what happened:

EXCP ACCESS METHOD USED FOR SORTIN01
EXCP ACCESS METHOD USED FOR SORTIN02
EXCP ACCESS METHOD USED FOR SORTIN03

became:

BSAM ACCESS METHOD USED FOR SORTIN01
BSAM ACCESS METHOD USED FOR SORTIN02
BSAM ACCESS METHOD USED FOR SORTIN03

EXCP's went way up, CPU went up a little, but elapsed time on the PULL
step is 3 times faster than before !
Back to top
View user's profile Send private message
Frank Yaeger

DFSORT Moderator


Joined: 15 Feb 2005
Posts: 7130
Location: San Jose, CA

PostPosted: Wed May 10, 2006 3:51 am    Post subject:
Reply with quote

It's hard to tell anything given that you haven't said which sort product you're using or showed us any of the messages.

The EXCP vs BSAM difference could be caused by the use of extended format for the permanent data set or something else.

I notice you're using INCLUDE for the PULL step, so that could reduce the number of output records.

Also, "repeating" each input record 9000 times to three output data sets for the PUSH step is more costly than just reading the three input data sets for the PULL step.
Back to top
View user's profile Send private message
Joel

New User


Joined: 07 Feb 2006
Posts: 11

PostPosted: Wed May 10, 2006 4:04 am    Post subject: Re: MERGE performance
Reply with quote

Thanks Frank !

I'm using YOUR product, DFSORT.

Your hunch was right about the extended datasets (PS-E). When I change to using permanent, non-extended (PS) datasets I get even better results - the lowest EXCP's, CPU, and elapsed !

Thanks. I knew you'd know !
Back to top
View user's profile Send private message
Frank Yaeger

DFSORT Moderator


Joined: 15 Feb 2005
Posts: 7130
Location: San Jose, CA

PostPosted: Wed May 10, 2006 4:14 am    Post subject:
Reply with quote

A lucky educated guess. icon_lol.gif
Back to top
View user's profile Send private message
Joel

New User


Joined: 07 Feb 2006
Posts: 11

PostPosted: Fri May 12, 2006 3:38 am    Post subject: Re: MERGE performance
Reply with quote

Upon further investigation, this is not a DFSORT issue at all.
Nor is it a BSAM vs EXCP issue.
Nor is it a permanent vs temporary dataset issue.
Nor is it an extended addressability issue.

It is a HARDWARE issue. All the slow read elapsed times were
from volumes on a StorageTek SVA960 box. I tested this with
the MERGE step above and also with simple COPY steps in 3
separate jobs running in parallel. The SVA960 just performs
horribly with this workload.
Back to top
View user's profile Send private message
Frank Yaeger

DFSORT Moderator


Joined: 15 Feb 2005
Posts: 7130
Location: San Jose, CA

PostPosted: Fri May 12, 2006 3:44 am    Post subject:
Reply with quote

Thanks for for the update. Of course I'm happy to hear it's NOT a DFSORT issue.
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> DFSORT/ICETOOL All times are GMT + 6 Hours
Page 1 of 1

 

Search our Forum:

Similar Topics
Topic Author Forum Replies Posted
This topic is locked: you cannot edit posts or make replies. Merge two files and update Trailer re... Yashashri JCL & VSAM 1 Thu May 04, 2017 12:54 pm
No new posts To Merge mutliple records into a sing... anandgbe DFSORT/ICETOOL 6 Wed Feb 22, 2017 8:49 am
No new posts BC, BCR, BRC, BRCL performance steve-myers PL/I & Assembler 0 Fri Dec 23, 2016 7:44 am
No new posts PL/I code tuning/Performance improvement Virendra Shambharkar PL/I & Assembler 4 Mon Dec 05, 2016 11:57 am
No new posts What are the way we can improve CPU p... Gunapala CN DB2 10 Mon Oct 24, 2016 2:16 pm


Facebook
Back to Top
 
Mainframe Wiki | Forum Rules | Bookmarks | Subscriptions | FAQ | Tutorials | Contact Us