Portal | Manuals | References | Downloads | Info | Programs | JCLs | Master the Mainframes
IBM Mainframe Computers Forums Index
 
Register
 
IBM Mainframe Computers Forums Index Mainframe: Search IBM Mainframe Forum: FAQ Memberlist Usergroups Profile Log in to check your private messages Log in
 

 

Usage of Virtual Storage by a SORT step

 
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> JCL & VSAM
View previous topic :: :: View next topic  
Author Message
Vidhya Kalyanasundaram

New User


Joined: 19 Jul 2007
Posts: 30
Location: chennai

PostPosted: Tue May 24, 2011 2:29 pm    Post subject: Usage of Virtual Storage by a SORT step
Reply with quote

Hi,

I need a clarification regarding a performance issue related to the usage of Virtual Storage consumed by a SORT step.

The below SORT step consumed 44 Minutes of Elapsed time on a day.

Code:

SYSIN :                                                                       
    SORT FIELDS=(0001,0076,CH,A),DYNALLOC=(SYSDA,99)                    004006
WER276B  SYSDIAG= 2654590, 4253089, 4253089, 2083425                         
WER415H  DSM FACILITY ACTIVE:  STORAGE ADJUSTED BY 88,332K BYTES             
WER426G  AVAILABLE STORAGE RESOURCES:  2,097,088K EXCESS CENTRAL, 2,097,088K EXPANDED
WER164B  100,624K BYTES OF VIRTUAL STORAGE AVAILABLE, MAX REQUESTED,         
WER164B     0 BYTES RESERVE REQUESTED, 100,584K BYTES USED                   
WER146B  32K BYTES OF EMERGENCY SPACE ALLOCATED                               
WER108I  SORTIN   : RECFM=FB   ; LRECL=   276; BLKSIZE= 32568                 
WER110I  SORTOUT  : RECFM=FB   ; LRECL=   276; BLKSIZE= 32568                 
WER410B  99,596K BYTES OF VIRTUAL STORAGE AVAILABLE ABOVE THE 16MEG LINE,     
WER410B     0 BYTES RESERVE REQUESTED, 99,620K BYTES USED                     
WER036B  G=144314,B=66,BIAS=91                                               
WER162B  0 PREALLOCATED SORTWORK TRACKS, 480,000 DYNAMICALLY ALLOCATED,       
WER162B     903,810 ACQUIRED IN 426 SECONDARY EXTENTS, 1,230 RELEASED, TOTAL OF 1,382,450 TRACKS USED
WER045C  END SORT PHASE                                                       
WER418I  DATASPACE(S) AND/OR HIPERSPACE(S) USED                               
WER417H  491,624K BYTES OF SHS AREAS CREATED, 424,410K BYTES USED             
WER211B  SYNCSMF  CALLED BY SYNCSORT; RC=0000                                 
WER494I  INPUT PHASE USED MIDAW                                               
WER449I  SYNCSORT GLOBAL DSM SUBSYSTEM ACTIVE                               
WER449I  SYNCSORT GLOBAL DSM SUBSYSTEM ACTIVE                             
WER416B  BSAM WAS USED FOR SORTIN                                         
WER416B  BSAM WAS USED FOR SORTOUT                                       
WER416B  SORTWK01 : EXCP'S=248,UNIT=3390,DEV=655B,CHP=BEBFCBCDEBF2F3FD,VOL=PPU240
WER416B  SORTWK02 : EXCP'S=103,UNIT=3390,DEV=3973,CHP=BBBDC9CADBEAFAFC,VOL=PPUB09
WER416B  SORTWK03 : EXCP'S=19,UNIT=3390,DEV=3958,CHP=BBBDC9CADBEAFAFC,VOL=PPUBAF
WER416B  SORTWK04 : EXCP'S=129,UNIT=3390,DEV=3B5D,CHP=BBBDC9CADBEAFAFC,VOL=PPUBFE
WER416B  SORTWK05 : EXCP'S=119,UNIT=3390,DEV=399A,CHP=BBBDC9CADBEAFAFC,VOL=PPUB74
WER416B  SORTWK06 : EXCP'S=15,UNIT=3390,DEV=3984,CHP=BBBDC9CADBEAFAFC,VOL=PPUB23
.......
WER416B  SORTWK61 : EXCP'S=19,UNIT=3390,DEV=3B69,CHP=BBBDC9CADBEAFAFC,VOL=PPU101
WER416B  SORTWK62 : EXCP'S=13,UNIT=3390,DEV=6793,CHP=BEBFCBCDEBF2F3FD,VOL=PPU224
WER416B  SORTWK63 : EXCP'S=14,UNIT=3390,DEV=6796,CHP=BEBFCBCDEBF2F3FD,VOL=PPU227
WER416B  SORTWK64 : EXCP'S=11,UNIT=3390,DEV=678A,CHP=BEBFCBCDEBF2F3FD,VOL=PPU203
WER416B  TOTAL OF 3,157 EXCP'S ISSUED FOR SORTWORKS
WER246I  FILESIZE 75,530,232,864 BYTES             
WER054I  RCD IN  273660264, OUT  273660264         
WER072I  EQUALS, BALANCE IN EFFECT                 
WER169I  RELEASE 1.3 BATCH 0506 TPF LEVEL 2.1     
WER419G                                                                 
WER419G   0 00000000021 00000000005 00000000000 00000000003 00000000000 00000000486 00000000000 00000001400 00000000000
WER419G   1 00000129685 00000027284 00000000000 00000001266 00000000000 00002376220 00000000000 00001795996 00000000000
WER419G   3 00000487248 00000004455 00000000000 00000002130 00000000000 00001784519 00000000000 00000845705 00000000000
WER419G   4 00000000089 00000000002 00000000000 00000000001 00000000000 00000000184 00000000000 00000000673 00000000000
WER419G   A 00000617043 00000031746 00000000000 00000003400 00000000000 00004161409 00000000000 00002643776 00000000000
WER052I  END SYNCSORT - JOB1,PROC1,R020,DIAG=E200,F8DD,CA37,B057,BBDA,4CE3,0260,2E62


Run Time Stats -
START TIME 03:59:31.38
STOP TIME 04:43:35.75
ELAP. TIME 00:44:04.37
VIRT SYS USED 696K
VIRT CORE USED 1,032K


The same SORT step consumed 1 Hour 40 Minutes on another day. The spool message is shown below for reference -

Code:

SYSIN :                                                                       
    SORT FIELDS=(0001,0076,CH,A),DYNALLOC=(SYSDA,99)                    00400600
WER276B  SYSDIAG= 2325692, 4743244, 4743244, 2919450                         
WER164B  100,592K BYTES OF VIRTUAL STORAGE AVAILABLE, MAX REQUESTED,         
WER164B     0 BYTES RESERVE REQUESTED, 100,552K BYTES USED                   
WER146B  32K BYTES OF EMERGENCY SPACE ALLOCATED                               
WER108I  SORTIN   : RECFM=FB   ; LRECL=   276; BLKSIZE= 32568                 
WER110I  SORTOUT  : RECFM=FB   ; LRECL=   276; BLKSIZE= 32568                 
WER410B  99,564K BYTES OF VIRTUAL STORAGE AVAILABLE ABOVE THE 16MEG LINE,     
WER410B     0 BYTES RESERVE REQUESTED, 99,588K BYTES USED                     
WER036B  G=144314,B=66,BIAS=91                                               
WER162B  0 PREALLOCATED SORTWORK TRACKS, 292,500 DYNAMICALLY ALLOCATED,       
WER162B     1,101,900 ACQUIRED IN 538 SECONDARY EXTENTS, 4,695 RELEASED, TOTAL OF 1,389,477 TRACKS USED
WER045C  END SORT PHASE                                                       
WER211B  SYNCSMF  CALLED BY SYNCSORT; RC=0000                                 
WER416B  BSAM WAS USED FOR SORTIN                                             
WER416B  BSAM WAS USED FOR SORTOUT                                           

WER416B  SORTWK01 : EXCP'S=1094,UNIT=3390,DEV=6780,CHP=BEBFCBCDEBF2F3FD,VOL=PPU209
WER416B  SORTWK02 : EXCP'S=733,UNIT=3390,DEV=65B5,CHP=BEBFCBCDEBF2F3FD,VOL=PPUB0D
WER416B  SORTWK03 : EXCP'S=686,UNIT=3390,DEV=3942,CHP=BBBDC9CADBEAFAFC,VOL=PPU152
WER416B  SORTWK04 : EXCP'S=683,UNIT=3390,DEV=679B,CHP=BEBFCBCDEBF2F3FD,VOL=PPU220
WER416B  SORTWK05 : EXCP'S=380,UNIT=3390,DEV=6767,CHP=BEBFCBCDEBF2F3FD,VOL=PPU184
.......
WER416B  SORTWK36 : EXCP'S=885,UNIT=3390,DEV=393A,CHP=BBBDC9CADBEAFAFC,VOL=PPUBAA
WER416B  SORTWK37 : EXCP'S=763,UNIT=3390,DEV=393A,CHP=BBBDC9CADBEAFAFC,VOL=PPUBAA
WER416B  SORTWK38 : EXCP'S=440,UNIT=3390,DEV=65B1,CHP=BEBFCBCDEBF2F3FD,VOL=PPUB84
WER416B  SORTWK39 : EXCP'S=100,UNIT=3390,DEV=3B72,CHP=BBBDC9CADBEAFAFC,VOL=PPU110
WER416B  TOTAL OF 26,058 EXCP'S ISSUED FOR SORTWORKS                         
WER246I  FILESIZE 75,480,052,200 BYTES                                       
WER054I  RCD IN  273478450, OUT  273478450                                   
WER072I  EQUALS, BALANCE IN EFFECT                                           
WER169I  RELEASE 1.3 BATCH 0506 TPF LEVEL 2.1                               
WER052I  END SYNCSORT - JOB1,PROC1,R020,DIAG=8E00,D853,A29D,3C7F,D762,68EB,2640,A460


Run Time Stats -
START TIME 03:25:47.35
STOP TIME 05:06:44.87
ELAP. TIME 01:40:57.52
VIRT SYS USED 620K
VIRT CORE USED 1,032K


We are totally unaware of the reason for this difference. Hence, we contacted our Storage team who came up with the below explanation -

Quote:

The JOB1 normally runs around 3-4AM. Today it ran at around 5:45AM. Due to other workloads in the system at that time, the sort step could not use the virtual storage that it normally has available when it runs earlier.


Is there any way to avoid this type of unpredictable run time issue because it impacts the file delivery to the users at a wide range.

--
Thanks,
Vidhya

"If you have knowledge, let others light their candles with it. "
Back to top
View user's profile Send private message

gylbharat

Active Member


Joined: 31 Jul 2009
Posts: 565
Location: Bangalore

PostPosted: Tue May 24, 2011 2:38 pm    Post subject:
Reply with quote

Hi,

You can use ELAP Parameter.

ELAP
minimizes the sort elapsed time of each sort at the expense of
CPU time and I/O activity.
Back to top
View user's profile Send private message
PeterHolland

Global Moderator


Joined: 27 Oct 2009
Posts: 2422
Location: Netherlands, Amstelveen

PostPosted: Tue May 24, 2011 2:58 pm    Post subject:
Reply with quote

Quote:

Is there any way to avoid this type of unpredictable run time issue because it impacts the file delivery to the users at a wide range.


Schedule your job to run at around 3-4AM.
Back to top
View user's profile Send private message
Bill Woodger

DFSORT Moderator


Joined: 09 Mar 2011
Posts: 7223

PostPosted: Tue May 24, 2011 3:03 pm    Post subject: Reply to: Usage of Virtual Storage by a SORT step
Reply with quote

It may well be "resources" more generally rather than unavailable virtual storage per se. If there is heavy CPU/IO use later in the morning, then your job is going to "elapse" for longer. Talk to your "Production Control" about scheduling, priorities, etc, to keep it out of the heavier-load time.
Back to top
View user's profile Send private message
Vidhya Kalyanasundaram

New User


Joined: 19 Jul 2007
Posts: 30
Location: chennai

PostPosted: Tue May 24, 2011 3:11 pm    Post subject: Reply to: Usage of Virtual Storage by a SORT step
Reply with quote

Bharat,

Thank you. I ll try using that ELAP option.

Peter,

This job cannot be scheduled to run around 3-4AM because it needs to wait for its predecessor jobs for input. Not all the days, the stream gets delayed. Incase of huge input volume or delayed predecessor jobs, this job is getting delayed.

Already the job is kick started late and in addition to it, this Virtual Storage issue contributes additional elapsed time which further delays the completion of the job stream impacting the file delivery time.

--
Thanks,
Vidhya

"If you have knowledge, let others light their candles with it. "
Back to top
View user's profile Send private message
Bill Woodger

DFSORT Moderator


Joined: 09 Mar 2011
Posts: 7223

PostPosted: Tue May 24, 2011 3:37 pm    Post subject: Reply to: Usage of Virtual Storage by a SORT step
Reply with quote

I'm not so sure it can be just a lack of "virtual storage".

A lack of any resource you require will increase the time the job takes to run. If there is little free CPU time for your priority, that will slow you down. If little free IO capacity for your priority, that will slow you down. The lack of either of these is going to tend to indicate a lack of available virtual storage (if lots of other things are "doing something" then they have to have somewhere to do it).

You have the same problem everyone has. Availability of resources. If your job slips into a period of heavier use, then it will run for longer. Talk to your "Production Control" people. If your job is more important, in relative terms, than some of the others, they may be able to help you out.

Review your system for performance. Maybe you'll find something there.
Back to top
View user's profile Send private message
Robert Sample

Global Moderator


Joined: 06 Jun 2008
Posts: 7904
Location: Bellevue, IA

PostPosted: Tue May 24, 2011 4:07 pm    Post subject:
Reply with quote

Quote:
Is there any way to avoid this type of unpredictable run time issue because it impacts the file delivery to the users at a wide range.
Absolutely not. The elapsed time a job takes is a function not only of what resources the job itself needs, but also a function of a wide variety of system factors -- such as number of jobs, the amount of CPU those jobs needs, the amount of I/O those jobs are doing, whether or not the I/O being done hits the same disks and channels as other jobs, etc, etc, etc.

Bottom line: there is no way to remove variations in elapsed time on a z/OS system unless you dedicate one machine to running one job.
Back to top
View user's profile Send private message
kratos86

Active User


Joined: 17 Mar 2008
Posts: 148
Location: Anna NGR

PostPosted: Tue May 24, 2011 4:52 pm    Post subject:
Reply with quote

As bill suggested, talk to your 'production control' people to prioritize your job so that your job don't have to worry about other jobs being tied to CPU and IO time.

In most of the shops using the ELAP parameter is strictly prohibitied as it delays other jobs/process.
Back to top
View user's profile Send private message
Anuj Dhawan

Senior Member


Joined: 22 Apr 2006
Posts: 6258
Location: Mumbai, India

PostPosted: Tue May 24, 2011 7:54 pm    Post subject: Re: Reply to: Usage of Virtual Storage by a SORT step
Reply with quote

Vidhya Kalyanasundaram wrote:
Thank you. I ll try using that ELAP option.
If you are Application developer, suggest, don't try such things at home. Get in contact with your SyncSort representative and the Storage admins and decide the approach. Parameters like ELAP are installation dependent and must be used with the help of site-support group.
Back to top
View user's profile Send private message
enrico-sorichetti

Global Moderator


Joined: 14 Mar 2007
Posts: 10201
Location: italy

PostPosted: Tue May 24, 2011 8:02 pm    Post subject: Reply to: Usage of Virtual Storage by a SORT step
Reply with quote

You should have noticed the small difference in the two runs
Quote:
WER418I DATASPACE(S) AND/OR HIPERSPACE(S) USED
WER417H 491,624K BYTES OF SHS AREAS CREATED, 424,410K BYTES USED
WER211B SYNCSMF CALLED BY SYNCSORT; RC=0000
WER494I INPUT PHASE USED MIDAW
WER449I SYNCSORT GLOBAL DSM SUBSYSTEM ACTIVE
WER449I SYNCSORT GLOBAL DSM SUBSYSTEM ACTIVE


I am not a SYNCSORT expert, but it would be wise to
investigate with Your support why in the second run
the SYNCSORT GLOBAL DSM was not active and why SYNCSORT was not able to use DATA/HYPERspaces

and what about the
Quote:
WER494I INPUT PHASE USED MIDAW
Back to top
View user's profile Send private message
dick scherrer

Site Director


Joined: 23 Nov 2006
Posts: 19270
Location: Inside the Matrix

PostPosted: Tue May 24, 2011 8:04 pm    Post subject: Reply to: Usage of Virtual Storage by a SORT step
Reply with quote

Hello,

How is the input for this sort built? Is there a smaller amount of data that is not in sequence that is being sorted together with a large amount of data that is already in sequence to create a complete file?

If so, suggest you consider sorting the small amount of data and then MERGing this with the larger, already sorted, file. The elapsed time improvement (as well as other resources required) will be greatly improved.
Back to top
View user's profile Send private message
Vidhya Kalyanasundaram

New User


Joined: 19 Jul 2007
Posts: 30
Location: chennai

PostPosted: Wed Jun 01, 2011 1:55 pm    Post subject: Reply to: Usage of Virtual Storage by a SORT step
Reply with quote

Thanks a lot for all your suggestions. I would look for the right schedule changes for my job checking with the Production Control team and also would look how to simplify the SORT better with the MERGE option.

--
Thanks,
Vidhya

"If you have knowledge, let others light their candles with it. "
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> JCL & VSAM All times are GMT + 6 Hours
Page 1 of 1

 

Search our Forum:

Similar Topics
Topic Author Forum Replies Posted
No new posts Limit duplicate records in the SORT pshongal SYNCSORT 6 Mon Nov 21, 2016 12:54 pm
No new posts How to convert the VBM file to VB or... Sulabh Agrawal JCL & VSAM 4 Fri Nov 18, 2016 1:04 pm
No new posts Sort records based on numeric field. Alks SYNCSORT 2 Wed Oct 19, 2016 10:14 pm
No new posts How to delete second instance from Fl... Gunapala CN DFSORT/ICETOOL 6 Tue Oct 18, 2016 11:42 pm
No new posts abend sort based on count records in ... anatol DFSORT/ICETOOL 5 Mon Oct 17, 2016 10:10 pm


Facebook
Back to Top
 
Mainframe Wiki | Forum Rules | Bookmarks | Subscriptions | FAQ | Tutorials | Contact Us