IBM Mainframe Forum Index
 
Log In
 
IBM Mainframe Forum Index Mainframe: Search IBM Mainframe Forum: FAQ Register
 

Sort a file with 30 Milllion records LRECL=5000


IBM Mainframe Forums -> JCL & VSAM
Post new topic   Reply to topic
View previous topic :: View next topic  
Author Message
abin

Active User


Joined: 14 Aug 2006
Posts: 198

PostPosted: Fri Sep 07, 2007 2:16 pm
Reply with quote

Hi,

I am trying to sort a file with 30 Milllion records LRECL=5000.

Sort card is

Code:
SORT FIELDS=(740,20,CH,A, 
             415,3,CH,A,   
             12,02,ZD,D,   
             22,6,PD,D)   


Job abended with following message.
Abend code is U0016
Code:
WER276B    SYSDIAG=   2200954, 3528789, 3528789, 2202593                           
WER164B  102,332K BYTES OF VIRTUAL STORAGE AVAILABLE, MAX REQUESTED,           
WER164B     0 BYTES RESERVE REQUESTED, 102,308K BYTES USED                     
WER146B  20K BYTES OF EMERGENCY SPACE ALLOCATED                               
WER108I  SORTIN   : RECFM=FB   ; LRECL=  5000; BLKSIZE= 25000                 
WER110I  SORTOUT  : RECFM=FB   ; LRECL=  5000; BLKSIZE= 25000                 
WER410B  101,304K BYTES OF VIRTUAL STORAGE AVAILABLE ABOVE THE 16MEG LINE,     
WER410B     0 BYTES RESERVE REQUESTED, 101,304K BYTES USED                     
WER036B  G=10707,B=5,BIAS=97                                                   
WER162B  90,000 PREALLOCATED SORTWORK TRACKS, 210,000 DYNAMICALLY ALLOCATED,   
WER162B     1,207,905 ACQUIRED IN 423 SECONDARY EXTENTS, 0 RELEASED, TOTAL OF 1
WER046A  SORT CAPACITY EXCEEDED                                               
WER211B  SYNCSMF  CALLED BY SYNCSORT; RC=0000                                 
WER449I  SYNCSORT GLOBAL DSM SUBSYSTEM ACTIVE                                 
WER066A  APROX RCD CNT        15074853     


I presume problem is with sort work data sets.

Please help,
Abin
Back to top
View user's profile Send private message
abin

Active User


Joined: 14 Aug 2006
Posts: 198

PostPosted: Fri Sep 07, 2007 2:47 pm
Reply with quote

Is this problem with work data set. I have given work data set as follows.
Code:
//SORTWK01 DD SPACE=(CYL,(1500,2500)),UNIT=(SYSDA,20)       
//SORTWK02 DD SPACE=(CYL,(1500,2500)),UNIT=(SYSDA,20)       
//SORTWK03 DD SPACE=(CYL,(1500,2500)),UNIT=(SYSDA,20)       
//SORTWK04 DD SPACE=(CYL,(1500,2500)),UNIT=(SYSDA,20)   


Thanks,
Abin
Back to top
View user's profile Send private message
abin

Active User


Joined: 14 Aug 2006
Posts: 198

PostPosted: Fri Sep 07, 2007 2:53 pm
Reply with quote

Yes, SYNCSORT FOR Z/OS 1.2.2.2R

Thanks,
abin
Back to top
View user's profile Send private message
abin

Active User


Joined: 14 Aug 2006
Posts: 198

PostPosted: Fri Sep 07, 2007 3:17 pm
Reply with quote

Hi,

Code:
OPTION DYNALLOC=(SYSDA,nn)

Is this the solution?

from my sysout I have
Code:
90,000 PREALLOCATED SORTWORK TRACKS, 210,000 DYNAMICALLY ALLOCATED,
1,165,740   ACQUIRED IN 419 SECONDARY EXTENTS, 0 RELEASED, TOTAL OF 1,525,740 TRACKS   USED
 


So how much can I give in my nn.

Also do I need to provide sortwk data set.

Thanks,
Abin.
Back to top
View user's profile Send private message
abin

Active User


Joined: 14 Aug 2006
Posts: 198

PostPosted: Fri Sep 07, 2007 5:57 pm
Reply with quote

Hi,

I'm trying with OPTION DYNALLOC=(SYSDA,50).

job is running. Looks like it will pass. Is 50 too much.

Hi guptae,

Could you please tell me what is MINCORE. I googled. looks like related to Linux.

Thanks,
Abin
Back to top
View user's profile Send private message
abin

Active User


Joined: 14 Aug 2006
Posts: 198

PostPosted: Fri Sep 07, 2007 6:48 pm
Reply with quote

Hi,

Job abended even after OPTION DYNALLOC=(SYSDA,50).

Sysout says.

Code:
SYSDIAG= 1793350, 3202385, 3202385, 1887846                 
102,332K BYTES OF VIRTUAL STORAGE AVAILABLE, MAX REQUESTED,
   0 BYTES RESERVE REQUESTED, 102,308K BYTES USED           
0 PREALLOCATED SORTWORK TRACKS, 375,000 DYNAMICALLY ALLOCATED,     
   2,130,855 ACQUIRED IN 749 SECONDARY EXTENTS, 0 RELEASED, TOTAL OF 2,505,855 TRACKS USED
SORT CAPACITY EXCEEDED
Back to top
View user's profile Send private message
abin

Active User


Joined: 14 Aug 2006
Posts: 198

PostPosted: Fri Sep 07, 2007 8:20 pm
Reply with quote

sorry for asking,

unable to find MAXSORT in manual

Thanks,
Abin
Back to top
View user's profile Send private message
Alissa Margulies

SYNCSORT Support


Joined: 25 Jul 2007
Posts: 496
Location: USA

PostPosted: Fri Sep 07, 2007 10:50 pm
Reply with quote

abin,

Based on the extremely large filesize and space availability, it looks like you'll need about 250,000 cylinders of sortwork space. If this amount of space is not available, then MAXSORT is definitely the way to go. However, if it is going to be available, then you have several options. The first is code all of the SORTWKs in the JCL. I would recommend 100 SORTWKs, each with 2000 cylinders of primary space, or 200 SORTWKs of 1250 cylinders of primary space each plus secondary space. The second option is to allow DYNALLOC to get the space for you. I would recommend a minimum of DYNALLOC=(SYSDA,100). Although, 125 or 150 would be better. You could also code some SORTWKs in the JCL and let SyncSort get the remaining space for you.

If you are going to do this as a conventional sort there are 2 things you should be careful of. The first thing is the filesize. If the input is being read by the sort and is from either DASD or from tape, SyncSort should be able to get a good estimate on how much data is being sorted. If the input is supplied by an exit it will be important for you to provide and estimate of how much data is going to be sorted. In your example, this can be done by adding the following code to the sort step:
Code:

//$ORTPARM DD *
FILSZ=E30000000

You should also make sure that the DYNALLOC RETRY is set properly. This will allow the sort to WAIT for DASD space to free up in the event of a DASD shortage when a dynamic allocation request is made for additional SORTWK devices. This is documented in both the SyncSort for z/OS 1.2 Programmer's Guide and the Installation Guide.

Hope this is helpful. If you have any further questions, or would like information about how to modify your job to be processed as a MAXSORT application, please feel free to contact SyncSort Product Services directly at 201-930-8260. We would be happy to provide you with the Exploiting MAXSORT Guide along with personnal assistance.

psreddy,
Quote:
How can we decide no of workfiles needed.

The calculation for coding appropriate SORTWK space can be found in the SyncSort for z/OS 1.2 Programmer's Guide on pages 13.7-13.8.

Regards,
Back to top
View user's profile Send private message
abin

Active User


Joined: 14 Aug 2006
Posts: 198

PostPosted: Fri Sep 07, 2007 11:04 pm
Reply with quote

Hi Alissa,

I submitted the job using OPTION DYNALLOC=(SYSDA,150). Job is still running.

Job is now writing to output. looks like it will pass through.

Quote:
"If you are going to do this as a conventional sort there are 2 things you should be careful of. The first thing is the filesize."


Input is TAPE. So, I hope, FILSZ need not be given.

RETRY is not given for the running job. I'll do it in the next test.

I'll let U know the result once job is over.

Thanks for the help,
Abin.
Back to top
View user's profile Send private message
abin

Active User


Joined: 14 Aug 2006
Posts: 198

PostPosted: Fri Sep 07, 2007 11:14 pm
Reply with quote

Sorry,

One more thing. What could be the best values for RETRY. Also, there aint any need to allocate SORTWK explicitly in JCL if using DYNALLOC. right?

Please forgive if I'm bugging, but I cannot take any chance with this job.

Thanks,
Abin.
Back to top
View user's profile Send private message
Alissa Margulies

SYNCSORT Support


Joined: 25 Jul 2007
Posts: 496
Location: USA

PostPosted: Fri Sep 07, 2007 11:36 pm
Reply with quote

Quote:
What could be the best values for RETRY.

I would suggest RETRY=(5,3).
Quote:
any need to allocate SORTWK explicitly in JCL if using DYNALLOC.
It is not a necessity.
Back to top
View user's profile Send private message
abin

Active User


Joined: 14 Aug 2006
Posts: 198

PostPosted: Sat Sep 08, 2007 11:22 am
Reply with quote

Hi,

Job completed finally by giving OPTION DYNALLOC=(SYSDA,150). icon_biggrin.gif

It took 9 hours to complete icon_mad.gif

Thanks,
Abin.
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic View Bookmarks
All times are GMT + 6 Hours
Forum Index -> JCL & VSAM

 


Similar Topics
Topic Forum Replies
No new posts FTP VB File from Mainframe retaining ... JCL & VSAM 2
No new posts Need to set RC4 through JCL SORT DFSORT/ICETOOL 5
No new posts Extract the file name from another fi... DFSORT/ICETOOL 6
No new posts How to split large record length file... DFSORT/ICETOOL 10
No new posts Extracting Variable decimal numbers f... DFSORT/ICETOOL 17
Search our Forums:

Back to Top