Hi ,
I have a jcl which have sort step running 9 hrs to process 60000 record and its record length 30000.
record format =VB
record length = 30000
sort condition is
SORT FIELDS = (0091,13,CH,A)
is it any other way or sort technic to sort quickly?
Joined: 06 Jun 2008 Posts: 8696 Location: Dubuque, Iowa, USA
It sounds like your problem is not sorting but something else -- 60,000 records in 9 hours is about 2 records per second and that is WAY off. I just looked at one of our SMF jobs and SAS sorted 17,561 records in 5.31 seconds of elapsed time and 3.65 seconds of CPU time. For a sort to run for 9 hours, it should involve hundreds of millions or even billions of records. Record length has little to do with how long a sort takes, so why tell us the records are 30,000 bytes long?
I'd start by looking at:
- WLM policy
- REGION size of the SORT step
- channel / device contention
Quote:
is it any other way or sort technic to sort quickly?
You are asking the WRONG question -- SORT can run much, much, MUCH faster than 60,000 records in 9 hours, so you need to ask what is causing the slowdown at your site? Contact your site support group and work with them because the performance of your sort is nothing less than abysmal.
.... I have a jcl which have sort step running 9 hrs to process 60000 record ........
sort condition is
SORT FIELDS = (0091,13,CH,A)...
Is it possible for you to show the JESMSGLG, JESYSMSG, JESJCL and the DFSORT SYSOUT screenshot for the aforementioned step, that took 9 hours for completion?
If, and that right there is a BIG IF, it is a fault of DFSORT (which again is highly unlikely), Kolusu and his team at IBM DFSORT labs, will be more than happy to look it up, because with modern day machines a 9 hour step doing a fairly simple SORT is a big one, and if it is a genuine issue, they would surely want to take a peek at it.
It sounds like your problem is not sorting but something else -- 60,000 records in 9 hours is about 2 records per second and that is WAY off. I just looked at one of our SMF jobs and SAS sorted 17,561 records in 5.31 seconds of elapsed time and 3.65 seconds of CPU time. For a sort to run for 9 hours, it should involve hundreds of millions or even billions of records. Record length has little to do with how long a sort takes, so why tell us the records are 30,000 bytes long?
I'd start by looking at:
- WLM policy
- REGION size of the SORT step
- channel / device contention
Quote:
is it any other way or sort technic to sort quickly?
You are asking the WRONG question -- SORT can run much, much, MUCH faster than 60,000 records in 9 hours, so you need to ask what is causing the slowdown at your site? Contact your site support group and work with them because the performance of your sort is nothing less than abysmal.
Sorry... I have checked the whole jobs taken 9 hrs and the particular sort step taking 4 hours. I hope sort technic for quickly process.I do not know why it took these much time (4hrs) to process 60000 records.Can we tune sort or any other sort technic we have?
.... I have a jcl which have sort step running 9 hrs to process 60000 record ........
sort condition is
SORT FIELDS = (0091,13,CH,A)...
Is it possible for you to show the JESMSGLG, JESYSMSG, JESJCL and the DFSORT SYSOUT screenshot for the aforementioned step, that took 9 hours for completion?
If, and that right there is a BIG IF, it is a fault of DFSORT (which again is highly unlikely), Kolusu and his team at IBM DFSORT labs, will be more than happy to look it up, because with modern day machines a 9 hour step doing a fairly simple SORT is a big one, and if it is a genuine issue, they would surely want to take a peek at it.
sorry. Unable to share JESMSGLG, JESYSMSG, JESJCL and the DFSORT SYSOUT screenshot .
It is really-really difficult to comment on a tuning approach without looking at the basic elements such as the ones requested.
Also, believe me DFSORT is more than capable of processing the data mentioned by you in a very small time consuming way lesser resources; but one really can not comment on it unless the diagnostics requested are shown.
You have also not commented on the questions Mr. Sample had raised.
Joined: 07 Feb 2009 Posts: 1306 Location: Vilnius, Lithuania
cmsmoon wrote:
Unable to share JESMSGLG, JESYSMSG, JESJCL and the DFSORT SYSOUT screenshot .
Then please get out of here. If you cannot even do something as simple as mask the few "secret" details on the screenshots of those output datasets, you should not further waste our time here!
Joined: 06 Jun 2008 Posts: 8696 Location: Dubuque, Iowa, USA
Quote:
Can we tune sort or any other sort technic we have?
Let me restate things for you:
1. SORT is not tunable. It is already highly optimized and hence you are not going to improve its performance, no matter what you "think".
2. There is NO "sort technic (sic)" -- I assume you mean "technique" here -- that will help you. SORT is not the issue, even though it is a SORT step that is taking so long.
3. You should be able to post the SORTOUT messages (WER or ICE prefix) since there is nothing to hide in them. Without those messages, we will be unable to help you.
4. Your problem lies in something specific to your site (since sites all around the world sort millions of records every day in MUCH less time than your sort is taking), and hence your BEST bet at getting a solution is to work with your site support group.
5. Refusing to post requested information on this, or any, forum tends to leave you without any help -- we don't ask for things just because; there's always a reason for the request (whether or not we tell you that reason, there is still a reason).
Joined: 06 Jul 2010 Posts: 765 Location: Whitby, ON, Canada
<useless speculation> I wonder if this job is even running on a real mainframe? It could be using an emulator running on a less than powerful PC for all we know.
SORT taking 4 hours to sort a small file is one thing, but the rest of the 9 hour job is taking 5 hours. That's a strong hint that *none* of the job steps are running quickly, not just SORT.
</useless speculation>
I have a jcl which have sort step running 9 hrs to process 60000 record and its record length 30000.
Why should we believe you here when you have given no single proof of it?If you need help, then rerun by adding //SORTDIAG DD DUMMY to your JCL and either send it to us or to your site support.
If what you said is happening everyday then best is to ask the producer of the data set to send you records sorted at first place than worrying for sorting later.
cmsmoon, since you didn't even know whether it was 4 hrs or 9 hrs, that shows your research before posting it on a forum.
The online forums are last resorts when you have exhausted all options.
You haven't told whether it is production job or testing. Whether it was only a single instance which took long hours Or is it a regular thing. You never told the things that you have checked to make sure things look good e.g. whether a dataset was held by some other job/user while you were trying to run the job.
You could have told all the above (and possibly more) without even posting the required technical details BUT you chose Not to.