Portal | Manuals | References | Downloads | Info | Programs | JCLs | Master the Mainframes
IBM Mainframe Computers Forums Index
 
Register
 
IBM Mainframe Computers Forums Index Mainframe: Search IBM Mainframe Forum: FAQ Memberlist Usergroups Profile Log in to check your private messages Log in
 

 

Regarding help needed in fine tuning the job step
Goto page 1, 2  Next
 
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> JCL & VSAM
View previous topic :: :: View next topic  
Author Message
shashikant

New User


Joined: 07 Sep 2007
Posts: 6
Location: chennai

PostPosted: Fri Apr 18, 2008 2:30 pm    Post subject: Regarding help needed in fine tuning the job step
Reply with quote

Hi All,

I have one jobstep which merges 10 datasets.
Each of these datasets are having minium of one million records and this step is taking most of the time in merging these dataset itself,
can any one please help me in giving the best solution so that i can reduce the time taken for this step

Thanks,
Shashikanth
Back to top
View user's profile Send private message

expat

Global Moderator


Joined: 14 Mar 2007
Posts: 8593
Location: Back in jolly old England

PostPosted: Fri Apr 18, 2008 2:42 pm    Post subject:
Reply with quote

How do you do it now ?
Back to top
View user's profile Send private message
shashikant

New User


Joined: 07 Sep 2007
Posts: 6
Location: chennai

PostPosted: Fri Apr 18, 2008 3:26 pm    Post subject:
Reply with quote

This is plain concatenation of 10 datasets with in the same jobstep.

//skrao1 dd=dsn1,disp=shr
// dd=dsn2,disp=shr
// dd=dsn3,disp=shr
// dd=dsn4,disp=shr
// dd=dsn5,disp=shr
// dd=dsn6,disp=shr
// dd=dsn7,disp=shr
// dd=dsn8,disp=shr
// dd=dsn9,disp=shr
// dd=dsn10,disp=shr

Thanks,
Shashikanth
Back to top
View user's profile Send private message
stodolas

Active Member


Joined: 13 Jun 2007
Posts: 632
Location: Wisconsin

PostPosted: Fri Apr 18, 2008 6:02 pm    Post subject:
Reply with quote

What is "most of the time" 20 minutes, 10 minutes? Wall time, cpu time? If a job runs for 8 hours and only uses 1 hour of CPU time that could be a good thing. If this step is taking 10 minutes that might not be a bad thing
Back to top
View user's profile Send private message
expat

Global Moderator


Joined: 14 Mar 2007
Posts: 8593
Location: Back in jolly old England

PostPosted: Fri Apr 18, 2008 6:55 pm    Post subject:
Reply with quote

What program is being used to concatenate the data ?
Back to top
View user's profile Send private message
Bill Dennis

Active Member


Joined: 17 Aug 2007
Posts: 562
Location: Iowa, USA

PostPosted: Fri Apr 18, 2008 7:10 pm    Post subject:
Reply with quote

Are the datasets on tape or DASD? If tape, are they multi-volume? What record size/Block size?
Back to top
View user's profile Send private message
Craq Giegerich

Senior Member


Joined: 19 May 2007
Posts: 1512
Location: Virginia, USA

PostPosted: Fri Apr 18, 2008 10:36 pm    Post subject:
Reply with quote

It doesn't even appear to be a merge! Some more detail would help us to help you!
Back to top
View user's profile Send private message
Alissa Margulies

SYNCSORT Support


Joined: 25 Jul 2007
Posts: 500
Location: USA

PostPosted: Sat Apr 19, 2008 12:20 am    Post subject:
Reply with quote

Shashikanth,

I made the following assumptions:

1. All the datasets are FB and have the same LRECL.
2. All the data is already in sorted order.
3. The SORT control field starts in position 1 for the length of 10.

Try the following (modify based on your actual data):
Code:
//STEP1 EXEC PGM=SORT
//SORTIN01 DD DSN=FILE1,DISP=SHR
//SORTIN02 DD DSN=FILE2,DISP=SHR
//SORTIN03 DD DSN=FILE3,DISP=SHR
//SORTIN04 DD DSN=FILE4,DISP=SHR
//SORTIN05 DD DSN=FILE5,DISP=SHR
//SORTIN06 DD DSN=FILE6,DISP=SHR
//SORTIN07 DD DSN=FILE7,DISP=SHR
//SORTIN08 DD DSN=FILE8,DISP=SHR
//SORTIN09 DD DSN=FILE9,DISP=SHR
//SORTIN10 DD DSN=FILE10,DISP=SHR
//SORTOUT  DD DSN=MERGED.OUTPUT,DISP=...
//SYSOUT   DD SYSOUT=*
//SYSIN    DD *
    MERGE FIELDS=(1,10,CH,A)
/*
Back to top
View user's profile Send private message
UmeySan

Active Member


Joined: 22 Aug 2006
Posts: 743
Location: Germany

PostPosted: Mon Apr 21, 2008 11:55 am    Post subject:
Reply with quote

Hi shashikant !

Assuming same as Alissa Margulies, and that every DSN is having minium of one million records i would alloc more space to Sort by using the $ORTPARM Parameter.

Something like this:
VSCORE=5M
VSCORET=30M

Also try using BUFNO Parameter in DD for the Sortin-Files.

I also agree with stodolas !!!

Regards, UmeySan
Back to top
View user's profile Send private message
ramangouda patil

New User


Joined: 09 Apr 2008
Posts: 39
Location: India

PostPosted: Mon Apr 21, 2008 8:16 pm    Post subject:
Reply with quote

Please correct me if I am wrong. Alissa, wouldn't we need to specify work files for the merging done using the SORT utility ?
Back to top
View user's profile Send private message
Alissa Margulies

SYNCSORT Support


Joined: 25 Jul 2007
Posts: 500
Location: USA

PostPosted: Mon Apr 21, 2008 8:22 pm    Post subject:
Reply with quote

Shashikanth,

If you have been testing the various suggestions made to you, but are still not seeing the desired performance improvements, then add the following to your sort step:
Code:
//$ORTPARM DD *
MSG=AP,LIST,GMSG

This will provide additional tuning messages that will help me to better assist you. You can email me the complete job listing offline at alissa.margulies@syncsort.com.

(And as both UmeySan and stodolas pointed out, you still need to identify which you are looking to improve - Wall Clock or CPU time).
Back to top
View user's profile Send private message
Alissa Margulies

SYNCSORT Support


Joined: 25 Jul 2007
Posts: 500
Location: USA

PostPosted: Mon Apr 21, 2008 8:24 pm    Post subject:
Reply with quote

ramangouda patil wrote:
Please correct me if I am wrong. Alissa, wouldn't we need to specify work files for the merging done using the SORT utility ?
Not if dynamic allocation is enabled (either by the DYNALOC installation option or the DYNALLOC runtime parameter).
Back to top
View user's profile Send private message
Alissa Margulies

SYNCSORT Support


Joined: 25 Jul 2007
Posts: 500
Location: USA

PostPosted: Mon Apr 21, 2008 8:27 pm    Post subject:
Reply with quote

UmeySan wrote:
Also try using BUFNO Parameter in DD for the Sortin-Files.

FYI, BUFNO values are ignored by SyncSort under normal circumstances where SyncSort is handling the I/O.
Back to top
View user's profile Send private message
ramangouda patil

New User


Joined: 09 Apr 2008
Posts: 39
Location: India

PostPosted: Mon Apr 21, 2008 8:33 pm    Post subject:
Reply with quote

Alissa

Code:
//$ORTPARM DD *
MSG=AP,LIST,GMSG


Could you elaborate a little on what this piece of code signifies ?
Back to top
View user's profile Send private message
Alissa Margulies

SYNCSORT Support


Joined: 25 Jul 2007
Posts: 500
Location: USA

PostPosted: Mon Apr 21, 2008 8:41 pm    Post subject:
Reply with quote

ramangouda patil wrote:
Code:
//$ORTPARM DD *
MSG=AP,LIST,GMSG
Could you elaborate a little on what this piece of code signifies ?

The $ORTPARM DD statement is used to override PARM or control statement information. In this case, I want to ensure that all the sort related message are written to the message dataset (MSG=AP), that the header lines and user control statements are also listed with the SYSOUT data set (LIST), and that additional tuning messages get posted (GMSG).

Hope this helps.
Back to top
View user's profile Send private message
ramangouda patil

New User


Joined: 09 Apr 2008
Posts: 39
Location: India

PostPosted: Mon Apr 21, 2008 8:58 pm    Post subject:
Reply with quote

One last doubt Alissa icon_confused.gif
How does one look into the message dataset (MSG=AP) and GMSG (What is the path or where are they stored) ?
And how can one fine tune the sort looking into them (you can help a lot by taking examples say values for CU time and explain thus)
Thanks a lot!
Back to top
View user's profile Send private message
Alissa Margulies

SYNCSORT Support


Joined: 25 Jul 2007
Posts: 500
Location: USA

PostPosted: Mon Apr 21, 2008 9:35 pm    Post subject:
Reply with quote

ramangouda patil wrote:
How does one look into the message dataset (MSG=AP) and GMSG (What is the path or where are they stored) ?

Each site controls where they route the SyncSort messages to via the MSGDD installation option. The default name is SYSOUT. However, if you need to verify your current settings, you can run SYNCLIST which produces a report of all your current SyncSort installation options.

ramangouda patil wrote:
And how can one fine tune the sort looking into them

'G' messages contain internal information that are specifically designed for SyncSort technicians to analyze.
Back to top
View user's profile Send private message
gcicchet

Senior Member


Joined: 28 Jul 2006
Posts: 1703
Location: Australia

PostPosted: Tue Apr 22, 2008 4:39 am    Post subject:
Reply with quote

Hi,

Quote:
Please correct me if I am wrong. Alissa, wouldn't we need to specify work files for the merging done using the SORT utility ?


Why do we need work files when MERGE is used, shouldn't the files be already sorted ?


Gerry
Back to top
View user's profile Send private message
Frank Yaeger

DFSORT Moderator


Joined: 15 Feb 2005
Posts: 7130
Location: San Jose, CA

PostPosted: Tue Apr 22, 2008 5:12 am    Post subject:
Reply with quote

Alissa said:

Quote:
Please correct me if I am wrong. Alissa, wouldn't we need to specify work files for the merging done using the SORT utility ?


Quote:
Not if dynamic allocation is enabled (either by the DYNALOC installation option or the DYNALLOC runtime parameter).


Alissa,

Are you saying that Syncsort actually uses dynamic allocation for a MERGE? DFSORT doesn't so I'm surprised that Syncsort does.

I see people using //SORTWKdd DD files for COPY and MERGE quite often and I suggest they remove them to avoid wasting space, so I'd like to clear up the confusion here.
Back to top
View user's profile Send private message
Arun Raj

Moderator


Joined: 17 Oct 2006
Posts: 2161
Location: @my desk

PostPosted: Tue Apr 22, 2008 9:11 am    Post subject:
Reply with quote

Quote:
I see people using //SORTWKdd DD files for COPY and MERGE quite often and I suggest they remove them to avoid wasting space


Frank,

Are you saying that SORTWKdd is not required for COPY and MERGE?

Thanks,
Arun
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> JCL & VSAM All times are GMT + 6 Hours
Goto page 1, 2  Next
Page 1 of 2

 

Search our Forum:

Similar Topics
Topic Author Forum Replies Posted
No new posts PL/I code tuning/Performance improvement Virendra Shambharkar PL/I & Assembler 4 Mon Dec 05, 2016 11:57 am
No new posts Converting multiple VB files to FB fi... Viswanath Reddy JCL & VSAM 6 Mon Aug 08, 2016 11:49 pm
This topic is locked: you cannot edit posts or make replies. JCL MULTIPLE STEP EXECUTION QUERY Susanta JCL & VSAM 18 Sat Jul 30, 2016 1:17 pm
No new posts Help Needed with View Direct shailesh_do CA Products 2 Wed Jul 13, 2016 10:39 am
No new posts Single step utility for compare and u... ramprakashn JCL & VSAM 5 Fri Apr 29, 2016 3:43 pm


Facebook
Back to Top
 
Mainframe Wiki | Forum Rules | Bookmarks | Subscriptions | FAQ | Tutorials | Contact Us