Problem in executing 100 MILLION RECORDS in SORTWK

harry · New User Joined: 27 Mar 2006 Posts: 31

Hi,

My input file is having more than 100 million records and I am sorting it on some key could any one please let me know how many SORTWORK DATA SETS I NEED TO GIVE FOR EXECUTING THIS JOB:

I have given 16 sortwork data sets each of SPACE=(TRK,(3000,300))

but still it is failing....

Please let me know:

1> How to calculate the Sortwork data sets?

2> If i am giving 10 Sortwork sets then can i give 1 having equal space as 10 datasets. What is the best option and why?

Thanks!!

Regards
Harry P Thomson

Dave Betten · New User Joined: 24 Jan 2006 Posts: 26

Can you tell me what the record length is so I have an idea of how big the sort is?

Generally, the number of sortworks is going to be based on the size of the input file and the amount of freespace you can get on a single volume. If you use DFSORT's dynamic allocation instead of JCL sortworks, we'll calculate the space allocation based on the number of sortworks specified in DYNALLOC and the size of the file. That way you don't need to specify the space parameters. If you'd like to send me your sysout, I'd be happy to take a look and provide some more detailed recommendations.

Dave Betten
DFSORT Development, Performance Lead
IBM Corporation
betten@us.ibm.com

stodolas · Posted: Thu Feb 07, 2008 12:26 am

I was going to say, I thought the concept of SORTWRK statements was superseded by using the DYNALLOC statement

Phrzby Phil · Posted: Thu Feb 07, 2008 1:00 am

Isn't there a sort option that tells it to just sort the key fields, not dragging the entire records along, and then outputs the entire records only at the end of the sort?

Frank Yaeger · Posted: Thu Feb 07, 2008 2:02 am

No, there's no such option. For non-trivial amounts of data, you need to write the records to intermediate storage of some kind (e.g. work data sets), so you must drag the entire record along so you can get it back when you need it. The kind of technique you mentioned only works with in-memory (small) sorts. Internally, we automatically use appropriate sorting techniques based on lots of different factors.