Joined: 09 Mar 2011 Posts: 7309 Location: Inside the Matrix
From your message it is workspace which is deficient.
Are you running Syncsort, as it is in the JCL forum?
How many records, can you run the "histogram"? Your sort keys are huge, that will not help. You primary on your work files is not big. Can you use dynamic work allocation?
Joined: 22 Apr 2006 Posts: 6248 Location: Mumbai, India
Quote:
tried to increase the Region from REGION=2048K to REGION=0M and run that step but still I am getting the same abend error.
This is not the correct solution for such problems. Depending upon the CLASS in which the Job is executing value on REGION parameter might/might-not be honoured.
Said that, I'll also ask if it's about SyncSort or DFSort; as the solution and the contributors will drastically change based on that answer of yours.
Joined: 09 Mar 2011 Posts: 7309 Location: Inside the Matrix
You are running out of sortwork space. Specify more. Look up histogrm in your Syncsort documentation. Run it. See if the output helps. Look at the manual section about sizings and performance.
Joined: 22 Apr 2006 Posts: 6248 Location: Mumbai, India
Quote:
dynamic work allocation is not used in our system.
Are you sure about it? Most of the sites these days go for DYNALLOC=Y.
What release of SyncSort are you at?
The JCL/Job you showed does not help us, however, I'd say - coding SORTWKs in the JCL is not a SyncSort requirement. Usually, if you allow SyncSort to dynamically allocate necessary SORTWORK space, it's generally considered more efficient, as the SORTWORK space is acquired "as needed" and you avoid scenarios where disk space is potentially over-allocated by the user.
Post us the entire SYSOUT including "B" messages (WERXXXB).
to add one more concern...
do You realize that for a disk dataset
the output blksize is the worst You could have chosen ?
with the optimum blksize of half track ( 27998 ) You could store 55996 bytes per track
VS. 32760 with a loss of 23236 bytes per track
Joined: 09 Mar 2011 Posts: 7309 Location: Inside the Matrix
So, you allocated 30,000 tracks in the primary space. You are using 450,000+ tracks before it blows up, at about 2/3 of your data.
I'm not going to mention the histogram again, you don't seem interested.
Give it a whole lot more sortwork space. Also explain to your techies that you don't have dynamic allocation, even though Syncsort says it is using it.
I don't know why you have that blocksize either. The worst place to have it is with huge disk datasets. Is your input on tape?
Joined: 09 Mar 2011 Posts: 7309 Location: Inside the Matrix
Hi,
I don't have Synsort documentation.
I just googled after your question, "SYNCSORT HISTOGRM". First hit should give you something. I didn't bother with the others.
Not used it for a while, but always used to run it for "big" files, always found it useful. I don't know how fancy it is these days, but I suspect it does more now that it used to. At its very basic, it shows you the counts of records of different lengths across the file.
With a knowledge of the exact amount of data, an estimate from the sort keys (used to be suggestions in the manual for that, I assume there still are) we always managed to get close enough for sortwork allocation. For one-off files, we could specify the exact number of records (don't know if that matters these days, it used to). For routine files, we could apply growth figures and monitor.
I'm sure you'll find references to it here as well.
Joined: 22 Apr 2006 Posts: 6248 Location: Mumbai, India
kumar1234,
Unfortunately, I've to say that - You are in a mess. Your shop is way behind in using the product SyncSort. You're using SYNCSORT FOR Z/OS 1.1CR which is, unless I'm mistaken out of supoprt now. SynsSort is running on 1.4 release of its these days.
Apart from that, you've got a good Blocking problem ahead of you. One of them Enrico has mentioned already.
This should be of your interest:
Code:
WER036B G=1500,B=18440,SEGLEN=18680,BIAS=99
If you'll read bout it - you'll understand that - G=1500, is the number of records that can be contained in SyncSort's working virtual storage area. For variable-length records, this number is the number of segments. You've got APROX RCD CNT# of 23416803 so yo ucan guess how much time it's going to take to process all of them.
B=18440 indicates the physical blocking used for intermediate storage. For fixed-length records, this number represents the blocking factor. For variable-length records, it represents the blocksize. This is another place where the block you've used is questionable.
Suggest you work with SyncSort representative at your shop.
In the meantime, you can try adding the following to the sort step in question:
Code:
//$ORTPARM DD *
DYNALLOC=(SYSDA,100,RETRY=(5,3))
With the above post us back what happens - we might then need to consider about using VSCORE and VSCORET. BUT I warn you of not using them just because I mention about them here - they MUST be used after consulting the SyncSort representative at your shop.
What you've posted, I've enclosed that in BBCodes. Please learn to use BBcodes.
I just googled after your question, "SYNCSORT HISTOGRM". First hit should give you something. I didn't bother with the others.
Not used it for a while, but always used to run it for "big" files, always found it useful. I don't know how fancy it is these days, but I suspect it does more now that it used to. At its very basic, it shows you the counts of records of different lengths across the file.
With a knowledge of the exact amount of data, an estimate from the sort keys (used to be suggestions in the manual for that, I assume there still are) we always managed to get close enough for sortwork allocation. For one-off files, we could specify the exact number of records (don't know if that matters these days, it used to). For routine files, we could apply growth figures and monitor.
I'm sure you'll find references to it here as well.
Joined: 09 Mar 2011 Posts: 7309 Location: Inside the Matrix
Yes. I don't know if it currently provides suggestions, but even doing it "old school" like we used to (pencil and old program listing) gave us a "close enough" answer to correclty allocate the sortwork. The sort keys do/did have to be taken into account, there was a little formula in the manual - no idea if it is still there.
Dynamic allocation of sortwork removes the need for this, generally. The HISTOGRM can also be useful just to understand the data that exists in the file. Like, "what the heck are those three short records?".
Joined: 09 Mar 2011 Posts: 7309 Location: Inside the Matrix
Just found an old SyncSort reference card, circa 1986. More advanced that the one I was thinking of (which would have been pre-1979) but it gives a "rule-of-thumb" of 1.20 * space occupied. No mention of the keys, but I seem to remember something. Only had to look once as all our keys were 40 bytes long :-)
The information may have changed in the intervening years :-)
Joined: 23 Nov 2006 Posts: 19243 Location: Inside the Matrix
Hello,
One way to get around the work space problem might be to split the input file into 10 (or 20 or) files that meet the:
INCLUDE COND=(75,1,CH,EQ,C'L') Rule.
Then sort these individual files by the "key" and at the end Merge them into the big single output file.
.jpg)
