|
View previous topic :: View next topic
|
| Author |
Message |
kitchu84
New User
Joined: 02 Dec 2006
Posts: 33
Location: chennai
|
|
|
|
I am running SYNCSORT on an i/p file with 350 million recs and LRECL of 4800 , using the DYNALLOC option without any additional SORTWK in the JCL. Below are the parameters that i am passing;
| Code: |
| PARM='DYNALLOC=(SYSDA,230),HIPRMAX=OPTIMAL,VSCORE=16M,VSCORET=64M' |
After an hour of execution i am seeing the following messages in my job spool, though my job is still running. We have addtional DASD added prior to this execution.
Can anyone please let me know what paramters or additional steps needs to be added to sort such a big file .
Does adding additional SORTWK in JCL going to help out?
| Code: |
------------------------------------------------------
SORT FIELDS=(09,30,CH,A,41,04,CH,A,47,30,CH,A)
SYSDIAG= 165571, 3386707, 3386707, 7989825
106,476K BYTES OF VIRTUAL STORAGE AVAILABLE, MAX REQUESTED,
0 BYTES RESERVE REQUESTED, 102,432K BYTES USED
20K BYTES OF EMERGENCY SPACE ALLOCATED
SORTIN : RECFM=VB ; LRECL= 4772; BLKSIZE= 32760
SORTOUT : RECFM=VB ; LRECL= 4772; BLKSIZE= 32760
98,288K BYTES OF VIRTUAL STORAGE AVAILABLE ABOVE THE 16MEG LINE,
0 BYTES RESERVE REQUESTED, 98,264K BYTES USED
G=5029,B=13672,SEGLEN=15728,BIAS=94
0 PREALLOCATED SORTWORK TRACKS, 1,725,000 DYNAMICALLY ALLOCATED,
9,662,400 ACQUIRED IN 3,224 SECONDARY EXTENTS, 0 RELEASED, TOTAL OF 1,370,006 TRACKS USED
SORT INTERNAL ERROR - RECOVERY ATTEMPT IN PROGRESS
SYNCSMF CALLED BY SYNCSORT; RC=0000
SYNCSORT GLOBAL DSM SUBSYSTEM ACTIVE
------------------------------------------------------------------ |
Edited: Please use BBcode when You post some code/error, that's rather readable, Thanks... Anuj |
|
| Back to top |
|
 |
dick scherrer
Moderator Emeritus
Joined: 23 Nov 2006
Posts: 19243
Location: Inside the Matrix
|
|
|
|
Hello,
How did the job end?
How much of the data is in sequence before this sort begins? If this process is to take a smaller amount of unsorted data and sort it with the large amount, it would possibly (probably) be better to sort the small amount of data and then MERGE it with the large amount. |
|
| Back to top |
|
|
kitchu84
New User
Joined: 02 Dec 2006
Posts: 33
Location: chennai
|
|
|
|
The job was still running , with the same above message . Hence i cancelled the job and resubmitted with the below parm and no extra SORTWK
PARM='DYNALLOC=(SYSDA,500),VSCORET=128M'
But after some 30 to 45 mins still getting the same message in the spool. Any thoughts or suggestions which would make this sort work is really appreciated. |
|
| Back to top |
|
|
kitchu84
New User
Joined: 02 Dec 2006
Posts: 33
Location: chennai
|
|
|
|
To add to that , i need to sort this whole file based on the key and then load it back to the data base. Hence splitting and running would be tha last option that i would choose.
Any suggestions/thoughts , |
|
| Back to top |
|
|
PeterHolland
Global Moderator
Joined: 27 Oct 2009
Posts: 2481
Location: Netherlands, Amstelveen
|
|
|
|
Run the HISTOGRM program and try to sort with the information generated by HISTOGRM.
Did you try using MAXSORT? |
|
| Back to top |
|
|
Bill Woodger
Moderator Emeritus
Joined: 09 Mar 2011
Posts: 7309
Location: Inside the Matrix
|
|
|
|
| Split the file. Sort the splits. Merge the sorted splits. |
|
| Back to top |
|
|
PeterHolland
Global Moderator
Joined: 27 Oct 2009
Posts: 2481
Location: Netherlands, Amstelveen
|
|
|
|
| Bill Woodger wrote: |
| Split the file. Sort the splits. Merge the sorted splits. |
Bill,
that is what MAXSORT will do. |
|
| Back to top |
|
|
Bill Woodger
Moderator Emeritus
Joined: 09 Mar 2011
Posts: 7309
Location: Inside the Matrix
|
|
|
|
Ah, without all the fiddly bits like doing the JCL yourself. Nice. Learn something every day. Thanks Peter.
Kitchu84, definitely contact Syncsort.
When posting sort messages here, please keep the message numbers. |
|
| Back to top |
|
|
Anuj Dhawan
Superior Member
Joined: 22 Apr 2006
Posts: 6248
Location: Mumbai, India
|
|
|
|
You are using OPTION HIPRMAX, so what value for FILSZ= being used?
And, when you are dealing with some vendor-product - it's always good to include the message-ID for the error messages, which is the only thing you share with us from your problem per se. |
|
| Back to top |
|
|
Bill Woodger
Moderator Emeritus
Joined: 09 Mar 2011
Posts: 7309
Location: Inside the Matrix
|
|
|
|
| Just as a tease, you're not doing this just to get a count of the records are you? (It is just a tease, I can see you are not) |
|
| Back to top |
|
|
Anuj Dhawan
Superior Member
Joined: 22 Apr 2006
Posts: 6248
Location: Mumbai, India
|
|
|
|
On the other hand, increasing the number in
| Quote: |
| 'DYNALLOC=(SYSDA,500), |
is not going to change anything, on the fly. How much such storage allocation can be secured for the a Job is governed by many factors and before we go in to the details of DATACLAS or ACS routines at your shop, there is a default limit set by SyncSort-vendors at the time of installation itself - beyond which you can't allocate sort work file dynamically. |
|
| Back to top |
|
|
Robert Sample
Global Moderator
Joined: 06 Jun 2008
Posts: 8700
Location: Dubuque, Iowa, USA
|
|
|
|
350 million 4800 byte records requires about 1.6 terabytes (1600 gigabytes) so you absolutely have to work with your site support group on this sort. You probably also need to contact Syncsort to get some help on how to structure this sort.
Even though you are reluctant to do so, I suspect your only option will turn out to be split the file into some number of smaller files, sort each of them seperately, then merge back into one big file so you can load your data base. And don't expect the process to be fast -- I wouldn't be surprised to see the sort elapsed time to be days, not hours, even if you do manage to sort the one big file. |
|
| Back to top |
|
|
Anuj Dhawan
Superior Member
Joined: 22 Apr 2006
Posts: 6248
Location: Mumbai, India
|
|
|
|
| I concur with Robert on the advice of getting in touch with the Support Group. You also show that you are using VSCORE=16M,VSCORET=64M, these are some of those parameters which should not be used without consulting the SyncSort representativesa and might cause more damage than they cure, if not used intelligently. |
|
| Back to top |
|
|
PeterHolland
Global Moderator
Joined: 27 Oct 2009
Posts: 2481
Location: Netherlands, Amstelveen
|
|
|
|
The TS is sorting VB records. So he should run HISTOGRM and use the
HISTOGRM information to run a MAXSORT.
But then im repeating myself. |
|
| Back to top |
|
|
Anuj Dhawan
Superior Member
Joined: 22 Apr 2006
Posts: 6248
Location: Mumbai, India
|
|
|
|
I'm listening, Peter!  |
|
| Back to top |
|
|
expat
Global Moderator
Joined: 14 Mar 2007
Posts: 8796
Location: Welsh Wales
|
|
|
|
Peter, is there an echo in here  |
|
| Back to top |
|
|
PeterHolland
Global Moderator
Joined: 27 Oct 2009
Posts: 2481
Location: Netherlands, Amstelveen
|
|
|
|
Guys, guys,
im not the TS so please stop making jokes on me. Beside that if you count all the times you did repeat answers then there is a lot of echoing here.  |
|
| Back to top |
|
|
Anuj Dhawan
Superior Member
Joined: 22 Apr 2006
Posts: 6248
Location: Mumbai, India
|
|
|
|
| echo! |
|
| Back to top |
|
|
dick scherrer
Moderator Emeritus
Joined: 23 Nov 2006
Posts: 19243
Location: Inside the Matrix
|
|
|
|
Hi Guys,
RedundancyOn
Doesn't Syncsort have a tool called HISTOGRM that would be of use here?
RedundancyOff
heh heh heh 
Have a great weekend!
d |
|
| Back to top |
|
|
PeterHolland
Global Moderator
Joined: 27 Oct 2009
Posts: 2481
Location: Netherlands, Amstelveen
|
|
|
|
| dick scherrer wrote: |
Hi Guys,
RedundancyOn
Doesn't Syncsort have a tool called HISTOGRM that would be of use here?
RedundancyOff
heh heh heh
Have a great weekend!
d |
Not after this input Dick.
Now i have to prolong my friday happy hours the whole week-end.
But whatever, have a nice week-end too. And after some reflection the rest of the (I(C)T) community here too. |
|
| Back to top |
|
|
|
|
