Portal | Manuals | References | Downloads | Info | Programs | JCLs | Master the Mainframes
IBM Mainframe Computers Forums Index
 
Register
 
IBM Mainframe Computers Forums Index Mainframe: Search IBM Mainframe Forum: FAQ Memberlist Usergroups Profile Log in to check your private messages Log in
 

 

Sorting Tape files efficiently than by the ordinary sort

 
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> JCL & VSAM
View previous topic :: :: View next topic  
Author Message
raak

Active User


Joined: 23 May 2006
Posts: 166
Location: chennai

PostPosted: Wed Dec 13, 2006 6:32 pm    Post subject: Sorting Tape files efficiently than by the ordinary sort
Reply with quote

Hi

i want to know whether there is any way in which a tape file can be sorted more efficiently than by the ordinary sort.

i have 10 daily jobs which have sort steps involving Tape files as input. it is consuming a lot of time as the input can have millions of records.

Is there any other efficient way in which the time consumption can be reduced.

I thought i had come across some topic like this. But a search didnt provide me with any useful results..


so hoping that someone will come up with an answer right away
Back to top
View user's profile Send private message

demora

New User


Joined: 12 Dec 2006
Posts: 4
Location: Cleveland, OH

PostPosted: Sat Dec 16, 2006 3:25 am    Post subject:
Reply with quote

Have you considered just copying the tapes to a temp data set (SORT FIELDS=COPY), sorting and then writing back to a tape? This way, the tapes are only accessed twice which should speed up the sort/process time.

That's about the only way I can think of right off to save processing time.
Back to top
View user's profile Send private message
dick scherrer

Site Director


Joined: 23 Nov 2006
Posts: 19270
Location: Inside the Matrix

PostPosted: Sat Dec 16, 2006 4:08 am    Post subject:
Reply with quote

Hello,

If you can change the block sizes of the input and output tape files, make them as large as you can - this will reduce the number of physical i/o's which is where much of the tape time is spent.

Greatly increasing the region size may also help. (Frank?) The more core the sort has to use, the less sort i/o's may be needed. . . .

My own personal preference for very large sorts is to include several SORTWKnn DD statements.
Back to top
View user's profile Send private message
Frank Yaeger

DFSORT Moderator


Joined: 15 Feb 2005
Posts: 7130
Location: San Jose, CA

PostPosted: Sat Dec 16, 2006 4:25 am    Post subject:
Reply with quote

Quote:
Greatly increasing the region size may also help. (Frank?)


I can't comment on this based on the small amount of information given by the poster. I don't know which sort product is being used, I don't know anything specific about the job, and I don't know whether or not the statement about "consuming a lot of time" is true or not.
Back to top
View user's profile Send private message
MFRASHEED

Active User


Joined: 14 Jun 2005
Posts: 186
Location: USA

PostPosted: Sat Dec 16, 2006 6:36 am    Post subject: Re: Sorting Tape files efficiently than by the ordinary sort
Reply with quote

Also add buffers to Tape dataset (12 or 16). The DEFAULT BUFFER setting for a QSAM file is 5 which is bottle neck.

Code:


//TAPEDD1 DD  DSN=TAPE.DATASET(0),       
//             DISP=OLD,                                   
//             DCB=BUFNO=12             



And as Dick mentioned, change BLKSIZE. The BLOCKSIZE will be a multiple of the LRECL, up to 32760 for TAPE.

And here is quote from Sort manual(Read SORT Manual):

Quote:

Optimizing Tape Sort
Three factors are crucial to Tape Sort efficiency: a generous amount of intermediate storage,
a closely estimated input size value on the SORT or EXEC statement, and the freedom
to select the best sorting technique (BALN, OSCL or POLY) based on the nature of the
application and conditions at execution time. Accordingly, the number of SORTWK data
sets should be in excess of those suggested in the chart above, the BALN/OSCL/POLY
PARM should be omitted and an accurate SIZE or FILSZ estimate should be provided.


For more tips provide more details as Frank has asked for.
Back to top
View user's profile Send private message
Frank Yaeger

DFSORT Moderator


Joined: 15 Feb 2005
Posts: 7130
Location: San Jose, CA

PostPosted: Sat Dec 16, 2006 7:04 am    Post subject:
Reply with quote

MFRASHEED,

DFSORT ignores BUFNO since it does its own buffer optimization. I'd guess the same is true for Syncsort.

The discussion of "Optimizing Tape Sort" is I believe from the Syncsort manual. "Tape Sort" refers to using tapes as work data sets which is highly discouraged. The poster is "using tape files as input" which has nothing to do with "tape sort" unless he's also using tape work data sets in which case switching to DASD work data sets would speed things up considerably.
Back to top
View user's profile Send private message
MFRASHEED

Active User


Joined: 14 Jun 2005
Posts: 186
Location: USA

PostPosted: Mon Dec 18, 2006 9:23 pm    Post subject: Re: Sorting Tape files efficiently than by the ordinary sort
Reply with quote

Thanks Frank for correction.
Back to top
View user's profile Send private message
raak

Active User


Joined: 23 May 2006
Posts: 166
Location: chennai

PostPosted: Thu Jan 04, 2007 7:04 pm    Post subject:
Reply with quote

The sort product used in our shop is SYNCSORT.

The output dataset is defined like shown below.

SORTOUT DD DSN=XXXX.XXXX(+1),
UNIT=VTAPE,RETPD=10,
DCB=(MBB.MODEL,RECFM=VB,LRECL=16397,BLKSIZE=0),
DISP=(NEW,CATLG,DELETE)


what can be done to make this sort more efficiently???
Back to top
View user's profile Send private message
dick scherrer

Site Director


Joined: 23 Nov 2006
Posts: 19270
Location: Inside the Matrix

PostPosted: Thu Jan 04, 2007 8:57 pm    Post subject:
Reply with quote

Hello,

How many records are on the tape(s) and how long does it take to do the sort? Does the run time vary depending on the time of day? How many files are input to the sort? If there is concatenated input and the data is already in the proper sequence, use MERGE rather than SORT - it is exponentially faster. The sort may be performing well, but due to time constraints it needs to be sped up.

Given that you have specified VTAPE, it is most likely not "really" a physical tape but rather a virtual tape. Usually virtual tape outperforms physical tape.

Look at (or post here) your sort control statement(s). If you have not specified an estimated number of records to sort, do so.

If you have a BIG sort (i.e. a billion records) and if your site uses high-density tapes, you might run a timing test sending the output to one of these tapes.
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> JCL & VSAM All times are GMT + 6 Hours
Page 1 of 1

 

Search our Forum:

Similar Topics
Topic Author Forum Replies Posted
No new posts TAPE VERIFY utility?? MrExtraordinare JCL & VSAM 3 Fri Dec 09, 2016 10:57 pm
No new posts High CPU consumption Job using IAM fi... aswinir JCL & VSAM 15 Thu Dec 01, 2016 8:28 pm
No new posts Limit duplicate records in the SORT pshongal SYNCSORT 6 Mon Nov 21, 2016 12:54 pm
No new posts How to convert the VBM file to VB or... Sulabh Agrawal JCL & VSAM 4 Fri Nov 18, 2016 1:04 pm
No new posts Match or compare two files in VB Format anatol DFSORT/ICETOOL 14 Thu Nov 03, 2016 7:41 pm


Facebook
Back to Top
 
Mainframe Wiki | Forum Rules | Bookmarks | Subscriptions | FAQ | Tutorials | Contact Us