IBM Mainframe Forum Index
 
Log In
 
IBM Mainframe Forum Index Mainframe: Search IBM Mainframe Forum: FAQ Register
 

MERGE performance


IBM Mainframe Forums -> DFSORT/ICETOOL
Post new topic   Reply to topic
View previous topic :: View next topic  
Author Message
Joel

New User


Joined: 07 Feb 2006
Posts: 11

PostPosted: Tue May 09, 2006 3:20 am
Reply with quote

Can anyone explain why the PULL step below takes about 3 times longer
than the PUSH step ? (I was hoping they would take equal elasped time as
the PULL step merges exactly the data created in the PUSH step).
Code:
                     
//*                                                                     
//PUSH     EXEC SORT                                                   
//SORTIN   DD  *                                                       
                                                                       
//S1       DD  DISP=(,PASS),SPACE=(CYL,300),DSN=&&S1                   
//S2       DD  DISP=(,PASS),SPACE=(CYL,300),DSN=&&S2                   
//S3       DD  DISP=(,PASS),SPACE=(CYL,300),DSN=&&S3                   
                                                                       
 OPTION COPY                                                           
 OUTFIL FNAMES=(S1,S2,S3),                                             
 OUTREC=(SEQNUM,16,ZD,27998:X),REPEAT=9000                             
                                                                       
//PULL     EXEC SORT                                                   
//SORTIN01 DD  DISP=SHR,DSN=&&S1                                       
//SORTIN02 DD  DISP=SHR,DSN=&&S2                                       
//SORTIN03 DD  DISP=SHR,DSN=&&S3                                       
//SORTOUT  DD  SYSOUT=*                                                 
                                                                       
 MERGE FIELDS=(1,16,CH,A)                                               
 INCLUDE COND=(14,3,CH,EQ,C'666')
Back to top
View user's profile Send private message
Joel

New User


Joined: 07 Feb 2006
Posts: 11

PostPosted: Wed May 10, 2006 3:27 am
Reply with quote

I tried changing the JCL to use permanent datasets instead of && datasets:

//TTD0397P JOB 0,'PUSHPULL',MSGCLASS=X,CLASS=B
//*
//PUSH EXEC SORT
//SORTIN DD *

//S1 DD DISP=(,CATLG),SPACE=(CYL,300),DSN=TTDX.S1
//S2 DD DISP=(,CATLG),SPACE=(CYL,300),DSN=TTDX.S2
//S3 DD DISP=(,CATLG),SPACE=(CYL,300),DSN=TTDX.S3

OPTION COPY
OUTFIL FNAMES=(S1,S2,S3),
OUTREC=(SEQNUM,16,ZD,27998:X),REPEAT=9000

//PULL EXEC SORT
//SORTIN01 DD DISP=SHR,DSN=TTDX.S1
//SORTIN02 DD DISP=SHR,DSN=TTDX.S2
//SORTIN03 DD DISP=SHR,DSN=TTDX.S3
//SORTOUT DD SYSOUT=*

MERGE FIELDS=(1,16,CH,A)
INCLUDE COND=(14,3,CH,EQ,C'666')

And look what happened:

EXCP ACCESS METHOD USED FOR SORTIN01
EXCP ACCESS METHOD USED FOR SORTIN02
EXCP ACCESS METHOD USED FOR SORTIN03

became:

BSAM ACCESS METHOD USED FOR SORTIN01
BSAM ACCESS METHOD USED FOR SORTIN02
BSAM ACCESS METHOD USED FOR SORTIN03

EXCP's went way up, CPU went up a little, but elapsed time on the PULL
step is 3 times faster than before !
Back to top
View user's profile Send private message
Frank Yaeger

DFSORT Developer


Joined: 15 Feb 2005
Posts: 7129
Location: San Jose, CA

PostPosted: Wed May 10, 2006 3:51 am
Reply with quote

It's hard to tell anything given that you haven't said which sort product you're using or showed us any of the messages.

The EXCP vs BSAM difference could be caused by the use of extended format for the permanent data set or something else.

I notice you're using INCLUDE for the PULL step, so that could reduce the number of output records.

Also, "repeating" each input record 9000 times to three output data sets for the PUSH step is more costly than just reading the three input data sets for the PULL step.
Back to top
View user's profile Send private message
Joel

New User


Joined: 07 Feb 2006
Posts: 11

PostPosted: Wed May 10, 2006 4:04 am
Reply with quote

Thanks Frank !

I'm using YOUR product, DFSORT.

Your hunch was right about the extended datasets (PS-E). When I change to using permanent, non-extended (PS) datasets I get even better results - the lowest EXCP's, CPU, and elapsed !

Thanks. I knew you'd know !
Back to top
View user's profile Send private message
Frank Yaeger

DFSORT Developer


Joined: 15 Feb 2005
Posts: 7129
Location: San Jose, CA

PostPosted: Wed May 10, 2006 4:14 am
Reply with quote

A lucky educated guess. icon_lol.gif
Back to top
View user's profile Send private message
Joel

New User


Joined: 07 Feb 2006
Posts: 11

PostPosted: Fri May 12, 2006 3:38 am
Reply with quote

Upon further investigation, this is not a DFSORT issue at all.
Nor is it a BSAM vs EXCP issue.
Nor is it a permanent vs temporary dataset issue.
Nor is it an extended addressability issue.

It is a HARDWARE issue. All the slow read elapsed times were
from volumes on a StorageTek SVA960 box. I tested this with
the MERGE step above and also with simple COPY steps in 3
separate jobs running in parallel. The SVA960 just performs
horribly with this workload.
Back to top
View user's profile Send private message
Frank Yaeger

DFSORT Developer


Joined: 15 Feb 2005
Posts: 7129
Location: San Jose, CA

PostPosted: Fri May 12, 2006 3:44 am
Reply with quote

Thanks for for the update. Of course I'm happy to hear it's NOT a DFSORT issue.
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic View Bookmarks
All times are GMT + 6 Hours
Forum Index -> DFSORT/ICETOOL

 


Similar Topics
Topic Forum Replies
No new posts Merge two VSAM KSDS files into third ... JCL & VSAM 6
No new posts exploiting Z16 performance PL/I & Assembler 2
This topic is locked: you cannot edit posts or make replies. Merge 2 input files based on the reco... JCL & VSAM 2
No new posts Merge 2 input files after sort SYNCSORT 14
No new posts JOIN STATEMENT PERFORMANCE. DFSORT/ICETOOL 12
Search our Forums:

Back to Top