Portal | Manuals | References | Downloads | Info | Programs | JCLs | Master the Mainframes
IBM Mainframe Computers Forums Index
 
Register
 
IBM Mainframe Computers Forums Index Mainframe: Search IBM Mainframe Forum: FAQ Memberlist Usergroups Profile Log in to check your private messages Log in
 

 

DYNALLOC - SAS Sort

 
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> DFSORT/ICETOOL
View previous topic :: :: View next topic  
Author Message
vasanthz

Global Moderator


Joined: 28 Aug 2007
Posts: 1445
Location: Azeroth

PostPosted: Thu Nov 21, 2013 1:50 am    Post subject: DYNALLOC - SAS Sort
Reply with quote

Hi,

We have a SAS job, that sorts 80000000(8 followed by 7 0's) observations.

The SORTPARM for the job is
Code:
SORTPARM='DYNALLOC=(SYSALLDA,250),FILSZ=E40000000' (4 followed by 7 0's)

The job sometimes fails, it has the below messages 6 times,
Code:
+ICE097I 1 JOBNAME1.S1       SORT RECOVERING FROM B37 ABEND ON SORTWK DATA SET
IEC030I B37-04,IFG0554A,JOBNAME1,SAS,SORTWKB2,554C,PUB528,0427041D,XXXXX.YYYYY.RA000.JOBNAME1.R0142402
and then fails with
Code:
BPXP018I THREAD 2686630000000002, IN PROCESS 65729, ENDED

The saslog has
Code:
ERROR: THE SAS SYSTEM STOPPED PROCESSING THIS STEP BECAUSE OF INSUFFICIENT MEMORY
Could you please let me know if the failure is related to FILSZ value being much smaller than actual number of records, or due to MEMORY shortage.


Would increasing the FILSZ help.

Thanks & Regards,
Back to top
View user's profile Send private message

Bill Woodger

DFSORT Moderator


Joined: 09 Mar 2011
Posts: 7228

PostPosted: Thu Nov 21, 2013 1:58 am    Post subject: Reply to: DYNALLOC - SAS Sort
Reply with quote

I'd say yes. 100% out is too much. If DFSORT is not able to estimate the records itself (because it is not processing the files itself) then the better the estimate of records (and perhaps average record size) the better it can do the dynamic allocation.
Back to top
View user's profile Send private message
vasanthz

Global Moderator


Joined: 28 Aug 2007
Posts: 1445
Location: Azeroth

PostPosted: Thu Nov 21, 2013 3:26 am    Post subject:
Reply with quote

Hi Bill,

Thanks for your thought on this. I was partially having the same idea as you did. I was at a dilemma since the saslog says "memory",
Code:

ERROR: THE SAS SYSTEM STOPPED PROCESSING THIS STEP BECAUSE OF INSUFFICIENT MEMORY


and i have seen sort jobs that have below messages, but complete successfully.
Code:
+ICE097I 1 JOBNAME1.S1       SORT RECOVERING FROM B37 ABEND ON SORTWK DATA SET
IEC030I B37-04,IFG0554A,JOBNAME1,SAS,SORTWKB2,554C,PUB528,0427041D,XXXXX.YYYYY.RA000.JOBNAME1.R0142402


also increasing the REGION parameter of the step to 0M makes the job run successfully "sometimes".

I was hesitating to experiment anything, since the job runs for like 10 hours and I don't want it to fail at 9th hour cos of my parm change :-)

Thanks & Regards,
Back to top
View user's profile Send private message
vasanthz

Global Moderator


Joined: 28 Aug 2007
Posts: 1445
Location: Azeroth

PostPosted: Thu Nov 21, 2013 3:34 am    Post subject:
Reply with quote

A sudden flash of thought, not sure how far it is true

Since SORT is capable of using HIPERSPACE, memory objects & work datasets, perhaps SAS referred the error generically as "INSUFFICIENT MEMORY" instead of "INSUFFICIENT DISK" icon_rolleyes.gif
Back to top
View user's profile Send private message
Skolusu

Senior Member


Joined: 07 Dec 2007
Posts: 2205
Location: San Jose

PostPosted: Thu Nov 21, 2013 3:57 am    Post subject:
Reply with quote

vasanthz,

Check this link

http://support.sas.com/kb/18/401.html
Back to top
View user's profile Send private message
expat

Global Moderator


Joined: 14 Mar 2007
Posts: 8593
Location: Back in jolly old England

PostPosted: Thu Nov 21, 2013 12:47 pm    Post subject:
Reply with quote

Hi Vasanthz,

I had a similar problem a couple of years ago, so will give you the solution that worked for me with help from Frank Yaeger.
Only a mere 27 million records though icon_biggrin.gif
S400 abends, with memory shortage messages given if the memory (no pun intended) still works correctly.

Code:
//DFSPARM DD *                                             
  DYNALLOC=(,32)                                           
/*                                                         
//SYSIN    DD *                                             
 OPTIONS NOSORTBLKMODE DYNALLOC SORTPGM=SORT;               
//         DD  DISP=SHR,DSN=PDS.NAME(member) 


I found at the time that for some reason SAS would invoke its own SORT routines rather than DFSORT which we have installed here. So the above forced SAS to use DFSORT from the start and ignore the initial usage of the SAS SORT routines.

Can't recall why we came up with NOSORTBLKMODE, but the original thread and responses are on this forum somewhere.
- - - Just read the link posted by Skolusu, so maybe from there.

Hopefully this may be of some use to you. Funny enough I'm back at the very same site where this oproblem arose, and the job is still running just fine using a REGION=64M icon_eek.gif
Back to top
View user's profile Send private message
vasanthz

Global Moderator


Joined: 28 Aug 2007
Posts: 1445
Location: Azeroth

PostPosted: Thu Nov 21, 2013 7:35 pm    Post subject:
Reply with quote

Hi Kolusu,

The link you posted seems to be describing the issue at our shop. I would try out the suggestions mentioned over there along with FILSZ,
Thank you. It is very helpful.

Expat, Thanks to you too, we are planning to experiment with NOSORTBLKMODE as you suggested & the same is mentioned on the link.

Regards,
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> DFSORT/ICETOOL All times are GMT + 6 Hours
Page 1 of 1

 

Search our Forum:

Similar Topics
Topic Author Forum Replies Posted
No new posts Limit duplicate records in the SORT pshongal SYNCSORT 6 Mon Nov 21, 2016 12:54 pm
No new posts How to convert the VBM file to VB or... Sulabh Agrawal JCL & VSAM 4 Fri Nov 18, 2016 1:04 pm
No new posts Sort records based on numeric field. Alks SYNCSORT 2 Wed Oct 19, 2016 10:14 pm
No new posts How to delete second instance from Fl... Gunapala CN DFSORT/ICETOOL 6 Tue Oct 18, 2016 11:42 pm
No new posts abend sort based on count records in ... anatol DFSORT/ICETOOL 5 Mon Oct 17, 2016 10:10 pm


Facebook
Back to Top
 
Mainframe Wiki | Forum Rules | Bookmarks | Subscriptions | FAQ | Tutorials | Contact Us