SORT CAPACITY EXCEEDED

kumar1234 · New User Joined: 06 Nov 2007 Posts: 84 Location: bangalore

Hi,

I have a production job that abended in the sort step due to insufficient space and the abend message is as 'SORT CAPACITY EXCEEDED'.

I tried to increase the Region from REGION=2048K to REGION=0M and run that step but still I am getting the same abend error.

Could anyone please let me know if we need to add any other parameters.

Here is the Jcl,

//STEP03 EXEC PGM=SORT,REGION=2048K
//SYSOUT DD SYSOUT=*
//SORTIN DD DSN=PURS.V600.TRF250.DIRJRNL(0),DISP=SHR
//SORTOUT DD DSN=T0SPSVS.PURS.TRF250.SORTED.LOSSES,
// UNIT=TST1,DISP=(NEW,CATLG,DELETE),VOL=(,,,99),
// SPACE=(CYL,(250,25),RLSE),
// DCB=(LRECL=1039,RECFM=VB)
//SORTWK01 DD UNIT=SYSDA,SPACE=(CYL,(200,75),RLSE)
//SORTWK02 DD UNIT=SYSDA,SPACE=(CYL,(200,75),RLSE)
//SORTWK03 DD UNIT=SYSDA,SPACE=(CYL,(200,75),RLSE)
//SORTWK04 DD UNIT=SYSDA,SPACE=(CYL,(200,75),RLSE)
//SORTWK05 DD UNIT=SYSDA,SPACE=(CYL,(200,75),RLSE)
//SORTWK06 DD UNIT=SYSDA,SPACE=(CYL,(200,75),RLSE)
//SORTWK07 DD UNIT=SYSDA,SPACE=(CYL,(200,75),RLSE)
//SORTWK08 DD UNIT=SYSDA,SPACE=(CYL,(200,75),RLSE)
//SORTWK09 DD UNIT=SYSDA,SPACE=(CYL,(200,75),RLSE)
//SORTWK10 DD UNIT=SYSDA,SPACE=(CYL,(200,75),RLSE)

Thanks.

kumar1234 · New User Joined: 06 Nov 2007 Posts: 84 Location: bangalore

In addition to above query here is the sorting informtion,

SORT FIELDS=(035,294,BI,A,363,038,BI,A,698,016,BI,A,347,008,BI,A,
329,006,BI,A)
INCLUDE COND=(75,1,CH,EQ,C'L')

Bill Woodger · Posted: Mon Jan 09, 2012 3:21 pm

From your message it is workspace which is deficient.

Are you running Syncsort, as it is in the JCL forum?

How many records, can you run the "histogram"? Your sort keys are huge, that will not help. You primary on your work files is not big. Can you use dynamic work allocation?

kumar1234 · New User Joined: 06 Nov 2007 Posts: 84 Location: bangalore

I guess there are more than 30 million records....dynamic work allocation is not used in our system.

Bill Woodger · Posted: Mon Jan 09, 2012 4:12 pm

You didn't confirm Syncsort. If so, what about the histogram program that they supply to help in these sort of situations?

Anuj Dhawan · Posted: Mon Jan 09, 2012 4:33 pm

kumar1234 · New User Joined: 06 Nov 2007 Posts: 84 Location: bangalore

It is a syncsort.

Bill Woodger · Posted: Mon Jan 09, 2012 4:53 pm

You are running out of sortwork space. Specify more. Look up histogrm in your Syncsort documentation. Run it. See if the output helps. Look at the manual section about sizings and performance.

Anuj Dhawan · Posted: Mon Jan 09, 2012 4:56 pm

kumar1234 · New User Joined: 06 Nov 2007 Posts: 84 Location: bangalore

Here is the sysout messages,

enrico-sorichetti · Posted: Mon Jan 09, 2012 5:21 pm

to add one more concern...
do You realize that for a disk dataset
the output blksize is the worst You could have chosen ?
with the optimum blksize of half track ( 27998 ) You could store 55996 bytes per track
VS. 32760 with a loss of 23236 bytes per track

Bill Woodger · Posted: Mon Jan 09, 2012 5:27 pm

So, you allocated 30,000 tracks in the primary space. You are using 450,000+ tracks before it blows up, at about 2/3 of your data.

I'm not going to mention the histogram again, you don't seem interested.

Give it a whole lot more sortwork space. Also explain to your techies that you don't have dynamic allocation, even though Syncsort says it is using it.

I don't know why you have that blocksize either. The worst place to have it is with huge disk datasets. Is your input on tape?

gylbharat · Posted: Mon Jan 09, 2012 5:30 pm

Hi Bill,

What is the Histogram option? Can you post a sample JCL and some links to documents?

Thanks in advance

Bill Woodger · Posted: Mon Jan 09, 2012 5:55 pm

Hi,

I don't have Synsort documentation.

I just googled after your question, "SYNCSORT HISTOGRM". First hit should give you something. I didn't bother with the others.

Not used it for a while, but always used to run it for "big" files, always found it useful. I don't know how fancy it is these days, but I suspect it does more now that it used to. At its very basic, it shows you the counts of records of different lengths across the file.

With a knowledge of the exact amount of data, an estimate from the sort keys (used to be suggestions in the manual for that, I assume there still are) we always managed to get close enough for sortwork allocation. For one-off files, we could specify the exact number of records (don't know if that matters these days, it used to). For routine files, we could apply growth figures and monitor.

I'm sure you'll find references to it here as well.

bodatrinadh · Posted: Mon Jan 09, 2012 6:05 pm

Hi Kumar,

Use "MAXSORT". It worked for me.

Anuj Dhawan · Posted: Mon Jan 09, 2012 6:12 pm

kumar1234,

Unfortunately, I've to say that - You are in a mess. Your shop is way behind in using the product SyncSort. You're using SYNCSORT FOR Z/OS 1.1CR which is, unless I'm mistaken out of supoprt now. SynsSort is running on 1.4 release of its these days.
Apart from that, you've got a good Blocking problem ahead of you. One of them Enrico has mentioned already.
This should be of your interest:

Code:

WER036B G=1500,B=18440,SEGLEN=18680,BIAS=99

If you'll read bout it - you'll understand that - G=1500, is the number of records that can be contained in SyncSort's working virtual storage area. For variable-length records, this number is the number of segments. You've got APROX RCD CNT# of 23416803 so yo ucan guess how much time it's going to take to process all of them.

B=18440 indicates the physical blocking used for intermediate storage. For fixed-length records, this number represents the blocking factor. For variable-length records, it represents the blocksize. This is another place where the block you've used is questionable.
Suggest you work with SyncSort representative at your shop.
In the meantime, you can try adding the following to the sort step in question:

Code:

//$ORTPARM DD *
DYNALLOC=(SYSDA,100,RETRY=(5,3))
With the above post us back what happens - we might then need to consider about using VSCORE and VSCORET. BUT I warn you of not using them just because I mention about them here - they MUST be used after consulting the SyncSort representative at your shop.
What you've posted, I've enclosed that in BBCodes. Please learn to use BBcodes.

kumar1234 · New User Joined: 06 Nov 2007 Posts: 84 Location: bangalore

Yes, input is a tape file.......

Bill Woodger · Posted: Mon Jan 09, 2012 6:29 pm

Well, you are showing output to disk, so there the 32760 is not a blocksize you want to use.

gylbharat · Posted: Mon Jan 09, 2012 6:32 pm

Bill Woodger · Posted: Mon Jan 09, 2012 6:42 pm

Yes. I don't know if it currently provides suggestions, but even doing it "old school" like we used to (pencil and old program listing) gave us a "close enough" answer to correclty allocate the sortwork. The sort keys do/did have to be taken into account, there was a little formula in the manual - no idea if it is still there.

Dynamic allocation of sortwork removes the need for this, generally. The HISTOGRM can also be useful just to understand the data that exists in the file. Like, "what the heck are those three short records?".

gylbharat · Posted: Mon Jan 09, 2012 7:05 pm

Thanks Bill

Bill Woodger · Posted: Mon Jan 09, 2012 8:33 pm

Just found an old SyncSort reference card, circa 1986. More advanced that the one I was thinking of (which would have been pre-1979) but it gives a "rule-of-thumb" of 1.20 * space occupied. No mention of the keys, but I seem to remember something. Only had to look once as all our keys were 40 bytes long :-)

The information may have changed in the intervening years :-)

dick scherrer · Posted: Mon Jan 09, 2012 9:51 pm

Hello,

One way to get around the work space problem might be to split the input file into 10 (or 20 or) files that meet the:
INCLUDE COND=(75,1,CH,EQ,C'L') Rule.

Then sort these individual files by the "key" and at the end Merge them into the big single output file.