IBM Mainframe Forum Index
 
Log In
 
IBM Mainframe Forum Index Mainframe: Search IBM Mainframe Forum: FAQ Register
 

SORT CAPACITY EXCEEDED


IBM Mainframe Forums -> JCL & VSAM
Post new topic   Reply to topic
View previous topic :: View next topic  
Author Message
kumar1234

New User


Joined: 06 Nov 2007
Posts: 84
Location: bangalore

PostPosted: Mon Jan 09, 2012 2:46 pm
Reply with quote

Hi,

I have a production job that abended in the sort step due to insufficient space and the abend message is as 'SORT CAPACITY EXCEEDED'.

I tried to increase the Region from REGION=2048K to REGION=0M and run that step but still I am getting the same abend error.

Could anyone please let me know if we need to add any other parameters.

Here is the Jcl,


//STEP03 EXEC PGM=SORT,REGION=2048K
//SYSOUT DD SYSOUT=*
//SORTIN DD DSN=PURS.V600.TRF250.DIRJRNL(0),DISP=SHR
//SORTOUT DD DSN=T0SPSVS.PURS.TRF250.SORTED.LOSSES,
// UNIT=TST1,DISP=(NEW,CATLG,DELETE),VOL=(,,,99),
// SPACE=(CYL,(250,25),RLSE),
// DCB=(LRECL=1039,RECFM=VB)
//SORTWK01 DD UNIT=SYSDA,SPACE=(CYL,(200,75),RLSE)
//SORTWK02 DD UNIT=SYSDA,SPACE=(CYL,(200,75),RLSE)
//SORTWK03 DD UNIT=SYSDA,SPACE=(CYL,(200,75),RLSE)
//SORTWK04 DD UNIT=SYSDA,SPACE=(CYL,(200,75),RLSE)
//SORTWK05 DD UNIT=SYSDA,SPACE=(CYL,(200,75),RLSE)
//SORTWK06 DD UNIT=SYSDA,SPACE=(CYL,(200,75),RLSE)
//SORTWK07 DD UNIT=SYSDA,SPACE=(CYL,(200,75),RLSE)
//SORTWK08 DD UNIT=SYSDA,SPACE=(CYL,(200,75),RLSE)
//SORTWK09 DD UNIT=SYSDA,SPACE=(CYL,(200,75),RLSE)
//SORTWK10 DD UNIT=SYSDA,SPACE=(CYL,(200,75),RLSE)


Thanks.
Back to top
View user's profile Send private message
kumar1234

New User


Joined: 06 Nov 2007
Posts: 84
Location: bangalore

PostPosted: Mon Jan 09, 2012 3:02 pm
Reply with quote

In addition to above query here is the sorting informtion,

SORT FIELDS=(035,294,BI,A,363,038,BI,A,698,016,BI,A,347,008,BI,A,
329,006,BI,A)
INCLUDE COND=(75,1,CH,EQ,C'L')
Back to top
View user's profile Send private message
Bill Woodger

Moderator Emeritus


Joined: 09 Mar 2011
Posts: 7309
Location: Inside the Matrix

PostPosted: Mon Jan 09, 2012 3:21 pm
Reply with quote

From your message it is workspace which is deficient.

Are you running Syncsort, as it is in the JCL forum?

How many records, can you run the "histogram"? Your sort keys are huge, that will not help. You primary on your work files is not big. Can you use dynamic work allocation?
Back to top
View user's profile Send private message
kumar1234

New User


Joined: 06 Nov 2007
Posts: 84
Location: bangalore

PostPosted: Mon Jan 09, 2012 3:46 pm
Reply with quote

I guess there are more than 30 million records....dynamic work allocation is not used in our system.
Back to top
View user's profile Send private message
Bill Woodger

Moderator Emeritus


Joined: 09 Mar 2011
Posts: 7309
Location: Inside the Matrix

PostPosted: Mon Jan 09, 2012 4:12 pm
Reply with quote

You didn't confirm Syncsort. If so, what about the histogram program that they supply to help in these sort of situations?
Back to top
View user's profile Send private message
Anuj Dhawan

Superior Member


Joined: 22 Apr 2006
Posts: 6250
Location: Mumbai, India

PostPosted: Mon Jan 09, 2012 4:33 pm
Reply with quote

Quote:
tried to increase the Region from REGION=2048K to REGION=0M and run that step but still I am getting the same abend error.
This is not the correct solution for such problems. Depending upon the CLASS in which the Job is executing value on REGION parameter might/might-not be honoured.

Said that, I'll also ask if it's about SyncSort or DFSort; as the solution and the contributors will drastically change based on that answer of yours.
Back to top
View user's profile Send private message
kumar1234

New User


Joined: 06 Nov 2007
Posts: 84
Location: bangalore

PostPosted: Mon Jan 09, 2012 4:33 pm
Reply with quote

It is a syncsort.
Back to top
View user's profile Send private message
Bill Woodger

Moderator Emeritus


Joined: 09 Mar 2011
Posts: 7309
Location: Inside the Matrix

PostPosted: Mon Jan 09, 2012 4:53 pm
Reply with quote

You are running out of sortwork space. Specify more. Look up histogrm in your Syncsort documentation. Run it. See if the output helps. Look at the manual section about sizings and performance.
Back to top
View user's profile Send private message
Anuj Dhawan

Superior Member


Joined: 22 Apr 2006
Posts: 6250
Location: Mumbai, India

PostPosted: Mon Jan 09, 2012 4:56 pm
Reply with quote

Quote:
dynamic work allocation is not used in our system.
Are you sure about it? Most of the sites these days go for DYNALLOC=Y.

What release of SyncSort are you at?

The JCL/Job you showed does not help us, however, I'd say - coding SORTWKs in the JCL is not a SyncSort requirement. Usually, if you allow SyncSort to dynamically allocate necessary SORTWORK space, it's generally considered more efficient, as the SORTWORK space is acquired "as needed" and you avoid scenarios where disk space is potentially over-allocated by the user.

Post us the entire SYSOUT including "B" messages (WERXXXB).
Back to top
View user's profile Send private message
kumar1234

New User


Joined: 06 Nov 2007
Posts: 84
Location: bangalore

PostPosted: Mon Jan 09, 2012 5:04 pm
Reply with quote

Here is the sysout messages,

Code:
SYNCSORT FOR Z/OS  1.1CR   TPF3   U.S. PATENTS: 4210961, 5117495  (C) 2002 SYNC
                                1888A, MODEL 2086 350              LICEN
PARMEXIT : CORE=MAX-250K,BMSG                                                 
SYSIN :                                                                       
 SORT FIELDS=(035,294,BI,A,363,038,BI,A,698,016,BI,A,347,008,BI,A,      0038000
              329,006,BI,A)                                             0039000
 INCLUDE COND=(75,1,CH,EQ,C'L')                                         0040000
WER161B  ALTERNATE PARM USED                                                   
WER164B  34,820K BYTES OF VIRTUAL STORAGE AVAILABLE, MAX-250K BYTES REQUESTED,
WER164B     1M BYTES RESERVE REQUESTED, 34,796K BYTES USED                     
WER146B  20K BYTES OF EMERGENCY SPACE ALLOCATED                               
WER108I  SORTIN   :  RECFM=VB   ; LRECL=  1039; BLKSIZE= 32760                 
WER110I  SORTOUT  :  RECFM=VB   ; LRECL=  1039; BLKSIZE= 32760                 
WER410B  32M BYTES OF VIRTUAL STORAGE AVAILABLE ABOVE THE 16MEG LINE,         
WER410B     0 BYTES RESERVE REQUESTED, 32M BYTES USED                         
WER036B  G=1500,B=18440,SEGLEN=18680,BIAS=99                                   
WER162B  30,000 PREALLOCATED SORTWORK TRACKS, 165,000 DYNAMICALLY ALLOCATED,   
WER162B     260,730 ACQUIRED IN 126 SECONDARY EXTENTS, 0 RELEASED, TOTAL OF 455
WER046A  SORT CAPACITY EXCEEDED               
WER055I  INSERT          0, DELETE     462888
WER211B  SYNCSMF  CALLED BY SYNCSORT; RC=0000
WER066A  APROX RCD CNT        23416803
Back to top
View user's profile Send private message
enrico-sorichetti

Superior Member


Joined: 14 Mar 2007
Posts: 10872
Location: italy

PostPosted: Mon Jan 09, 2012 5:21 pm
Reply with quote

to add one more concern...
do You realize that for a disk dataset
the output blksize is the worst You could have chosen ?
with the optimum blksize of half track ( 27998 ) You could store 55996 bytes per track
VS. 32760 with a loss of 23236 bytes per track
Back to top
View user's profile Send private message
Bill Woodger

Moderator Emeritus


Joined: 09 Mar 2011
Posts: 7309
Location: Inside the Matrix

PostPosted: Mon Jan 09, 2012 5:27 pm
Reply with quote

So, you allocated 30,000 tracks in the primary space. You are using 450,000+ tracks before it blows up, at about 2/3 of your data.

I'm not going to mention the histogram again, you don't seem interested.

Give it a whole lot more sortwork space. Also explain to your techies that you don't have dynamic allocation, even though Syncsort says it is using it.

I don't know why you have that blocksize either. The worst place to have it is with huge disk datasets. Is your input on tape?
Back to top
View user's profile Send private message
gylbharat

Active Member


Joined: 31 Jul 2009
Posts: 565
Location: Bangalore

PostPosted: Mon Jan 09, 2012 5:30 pm
Reply with quote

Hi Bill,

What is the Histogram option? Can you post a sample JCL and some links to documents?

Thanks in advance
Back to top
View user's profile Send private message
Bill Woodger

Moderator Emeritus


Joined: 09 Mar 2011
Posts: 7309
Location: Inside the Matrix

PostPosted: Mon Jan 09, 2012 5:55 pm
Reply with quote

Hi,

I don't have Synsort documentation.

I just googled after your question, "SYNCSORT HISTOGRM". First hit should give you something. I didn't bother with the others.

Not used it for a while, but always used to run it for "big" files, always found it useful. I don't know how fancy it is these days, but I suspect it does more now that it used to. At its very basic, it shows you the counts of records of different lengths across the file.

With a knowledge of the exact amount of data, an estimate from the sort keys (used to be suggestions in the manual for that, I assume there still are) we always managed to get close enough for sortwork allocation. For one-off files, we could specify the exact number of records (don't know if that matters these days, it used to). For routine files, we could apply growth figures and monitor.

I'm sure you'll find references to it here as well.
Back to top
View user's profile Send private message
bodatrinadh

Active User


Joined: 05 Jan 2007
Posts: 101
Location: chennai (India)

PostPosted: Mon Jan 09, 2012 6:05 pm
Reply with quote

Hi Kumar,

Use "MAXSORT". It worked for me.

Code:


//S15U     EXEC PGM=IEFBR14                                         
//O1BKPT   DD DSN=<temp data set>,               
//            DISP=(,CATLG,DELETE),                                 
//            UNIT=SYSDA,SPACE=(CYL,(15000,750),RLSE),             
//            DCB=(RECFM=FB,LRECL=3000,BLKSIZE=0)                   
//SYSOUT   DD SYSOUT=*                                               

//S15SC    EXEC PGM=SORT,REGION=3M,                                 
//    PARM='MAXSORT,MINWKSP=600,MAXWKSP=70000,DYNATAPE,TAPENAME=VTS3'
//SORTBKPT DD DSN=<temp data set>,               
//            DISP=(SHR,DELETE,KEEP)                                 
//SORTOU00 DD DISP=(,KEEP),                                         
//            UNIT=(VTS1,,DEFER),VOL=(PRIVATE,,,6),LABEL=RETPD=1     
//SORTOU01 DD DISP=(,KEEP),                                         
//            UNIT=(VTS1,,DEFER),VOL=(PRIVATE,,,6),LABEL=RETPD=1     
//SORTOU02 DD DISP=(,KEEP),                                         
//            UNIT=(VTS1,,DEFER),VOL=(PRIVATE,,,6),LABEL=RETPD=1     
//SORTOU03 DD DISP=(,KEEP),                                         
//            UNIT=(VTS1,,DEFER),VOL=(PRIVATE,,,6),LABEL=RETPD=1     
//SORTOU04 DD DISP=(,KEEP),                                         
//            UNIT=(VTS1,,DEFER),VOL=(PRIVATE,,,6),LABEL=RETPD=1     
//SORTOU05 DD DISP=(,KEEP),                                         
//            UNIT=(VTS1,,DEFER),VOL=(PRIVATE,,,6),LABEL=RETPD=1 
//SORTOU06 DD DISP=(,KEEP),                                       
//            UNIT=(VTS1,,DEFER),VOL=(PRIVATE,,,6),LABEL=RETPD=1   
//SYSOUT DD SYSOUT=*
//SORTIN DD DSN=PURS.V600.TRF250.DIRJRNL(0),DISP=SHR
//SORTOUT DD DSN=T0SPSVS.PURS.TRF250.SORTED.LOSSES,
// UNIT=TST1,DISP=(NEW,CATLG,DELETE),VOL=(,,,99),
// SPACE=(CYL,(250,25),RLSE),
// DCB=(LRECL=1039,RECFM=VB)


Thanks
-3nadh
Back to top
View user's profile Send private message
Anuj Dhawan

Superior Member


Joined: 22 Apr 2006
Posts: 6250
Location: Mumbai, India

PostPosted: Mon Jan 09, 2012 6:12 pm
Reply with quote

kumar1234,

  • Unfortunately, I've to say that - You are in a mess. Your shop is way behind in using the product SyncSort. You're using SYNCSORT FOR Z/OS 1.1CR which is, unless I'm mistaken out of supoprt now. SynsSort is running on 1.4 release of its these days.

  • Apart from that, you've got a good Blocking problem ahead of you. One of them Enrico has mentioned already.
  • This should be of your interest:
    Code:
    WER036B G=1500,B=18440,SEGLEN=18680,BIAS=99
    If you'll read bout it - you'll understand that - G=1500, is the number of records that can be contained in SyncSort's working virtual storage area. For variable-length records, this number is the number of segments. You've got APROX RCD CNT# of 23416803 so yo ucan guess how much time it's going to take to process all of them.

    B=18440 indicates the physical blocking used for intermediate storage. For fixed-length records, this number represents the blocking factor. For variable-length records, it represents the blocksize. This is another place where the block you've used is questionable.

  • Suggest you work with SyncSort representative at your shop.

  • In the meantime, you can try adding the following to the sort step in question:
    Code:
    //$ORTPARM DD *
     DYNALLOC=(SYSDA,100,RETRY=(5,3))

  • With the above post us back what happens - we might then need to consider about using VSCORE and VSCORET. BUT I warn you of not using them just because I mention about them here - they MUST be used after consulting the SyncSort representative at your shop.

  • What you've posted, I've enclosed that in BBCodes. Please learn to use BBcodes.
Back to top
View user's profile Send private message
kumar1234

New User


Joined: 06 Nov 2007
Posts: 84
Location: bangalore

PostPosted: Mon Jan 09, 2012 6:25 pm
Reply with quote

Yes, input is a tape file.......
Back to top
View user's profile Send private message
Bill Woodger

Moderator Emeritus


Joined: 09 Mar 2011
Posts: 7309
Location: Inside the Matrix

PostPosted: Mon Jan 09, 2012 6:29 pm
Reply with quote

Well, you are showing output to disk, so there the 32760 is not a blocksize you want to use.
Back to top
View user's profile Send private message
gylbharat

Active Member


Joined: 31 Jul 2009
Posts: 565
Location: Bangalore

PostPosted: Mon Jan 09, 2012 6:32 pm
Reply with quote

Bill Woodger wrote:
Hi,

I don't have Synsort documentation.

I just googled after your question, "SYNCSORT HISTOGRM". First hit should give you something. I didn't bother with the others.

Not used it for a while, but always used to run it for "big" files, always found it useful. I don't know how fancy it is these days, but I suspect it does more now that it used to. At its very basic, it shows you the counts of records of different lengths across the file.

With a knowledge of the exact amount of data, an estimate from the sort keys (used to be suggestions in the manual for that, I assume there still are) we always managed to get close enough for sortwork allocation. For one-off files, we could specify the exact number of records (don't know if that matters these days, it used to). For routine files, we could apply growth figures and monitor.

I'm sure you'll find references to it here as well.


Thanks Bill,

I goggled... I got this link as 1st link - ibmmainframes.com/about28241.html

What I understand is , we run the stats to get the statistics of the input data-set and use appropriate number of sort work-files.

Is my understanding correct?
Back to top
View user's profile Send private message
Bill Woodger

Moderator Emeritus


Joined: 09 Mar 2011
Posts: 7309
Location: Inside the Matrix

PostPosted: Mon Jan 09, 2012 6:42 pm
Reply with quote

Yes. I don't know if it currently provides suggestions, but even doing it "old school" like we used to (pencil and old program listing) gave us a "close enough" answer to correclty allocate the sortwork. The sort keys do/did have to be taken into account, there was a little formula in the manual - no idea if it is still there.

Dynamic allocation of sortwork removes the need for this, generally. The HISTOGRM can also be useful just to understand the data that exists in the file. Like, "what the heck are those three short records?".
Back to top
View user's profile Send private message
gylbharat

Active Member


Joined: 31 Jul 2009
Posts: 565
Location: Bangalore

PostPosted: Mon Jan 09, 2012 7:05 pm
Reply with quote

Thanks Bill icon_smile.gif
Back to top
View user's profile Send private message
Bill Woodger

Moderator Emeritus


Joined: 09 Mar 2011
Posts: 7309
Location: Inside the Matrix

PostPosted: Mon Jan 09, 2012 8:33 pm
Reply with quote

Just found an old SyncSort reference card, circa 1986. More advanced that the one I was thinking of (which would have been pre-1979) but it gives a "rule-of-thumb" of 1.20 * space occupied. No mention of the keys, but I seem to remember something. Only had to look once as all our keys were 40 bytes long :-)

The information may have changed in the intervening years :-)
Back to top
View user's profile Send private message
dick scherrer

Moderator Emeritus


Joined: 23 Nov 2006
Posts: 19244
Location: Inside the Matrix

PostPosted: Mon Jan 09, 2012 9:51 pm
Reply with quote

Hello,

One way to get around the work space problem might be to split the input file into 10 (or 20 or) files that meet the:
INCLUDE COND=(75,1,CH,EQ,C'L') Rule.

Then sort these individual files by the "key" and at the end Merge them into the big single output file.
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic View Bookmarks
All times are GMT + 6 Hours
Forum Index -> JCL & VSAM

 


Similar Topics
Topic Forum Replies
No new posts How to split large record length file... DFSORT/ICETOOL 10
No new posts JCL sort card - get first day and las... JCL & VSAM 9
No new posts Sort First/last record of a subset th... DFSORT/ICETOOL 7
No new posts how to calculate SUM value for VB fil... DFSORT/ICETOOL 1
No new posts how to calculate SUM for VB file usin... JCL & VSAM 1
Search our Forums:

Back to Top