General Data Current Allocation
Management class . . : LQUARTR Allocated cylinders : 4,396
Storage class . . . : BASE Allocated extents . : 4
Volume serial . . . : BDSE28 +
Device type . . . . : 3390
Data class . . . . . : 無指定
Organization . . . : PS Current Utilization
Record format . . . : VBS Used cylinders . . : 4,396
Record length . . . : 256 Used extents . . . : 4
Block size . . . . : 27998
1st extent cylinders: 693
Secondary cylinders : 100 Dates
Data set name type : Creation date . . . : 2013/01/23
SMS Compressible. . : NO Referenced date . . : 2013/05/23
Expiration date . . : ** なし **
Code:
Data Set Name . . . . : XXXX.DS.DATASET.D04
General Data Current Allocation
Management class . . : LQUARTR Allocated cylinders : 4,638
Storage class . . . : BASE Allocated extents . : 2
Volume serial . . . : BDSEI5 +
Device type . . . . : 3390
Data class . . . . . : 無指定
Organization . . . : PS Current Utilization
Record format . . . : VBS Used cylinders . . : 4,638
Record length . . . : 256 Used extents . . . : 2
Block size . . . . : 27998
1st extent cylinders: 3331
Secondary cylinders : 100 Dates
Data set name type : Creation date . . . : 2013/04/19
SMS Compressible. . : NO Referenced date . . : 2013/05/23
Expiration date . . : ** なし **
My question is : while the primary space has been assigned to be 300 cylinder, how it can range from 693 to 3331 cylinder. Meanwhile the second space has been assinged to be 100, how it can be 4,396 cylinder after 4 times extent?
Joined: 14 Mar 2007 Posts: 8797 Location: Welsh Wales
Also bear in mind that the SMS allocation routines may override your JCL and make a completely different space allocation based on the DATACLAS definition.
As Bill has said, this is a multi volume dataset. Did you press enter and see just how many volumes have been used.
Joined: 27 May 2010 Posts: 47 Location: da lian.china
expat wrote:
Also bear in mind that the SMS allocation routines may override your JCL and make a completely different space allocation based on the DATACLAS definition.
As Bill has said, this is a multi volume dataset. Did you press enter and see just how many volumes have been used.
as it had been defined [UNIT=(DISK2,6)],6 volume have been used
Code:
Data Set Information
C Esssssssssss Volume Information sssssssssssN
e e
D e Command ===> e
e e
G e All allocated volumes: e Allocation
e 続く + e ed cylinders : 4,396
e Number of volumes allocated: 6 e ed extents . : 4
e e
e BDSE28 BDSN68 * * * e
e * e
e e Utilization
e F1=Help F2=Split F3=Exit e linders . . : 4,396
e F7=Backward F8=Forward F9=Swap e tents . . . : 4
Joined: 27 May 2010 Posts: 47 Location: da lian.china
expat wrote:
No, only two of the possible 6 volumes have been used.
but the information shows that there were 6 volums have been allocated.
whatever, how can the 1st extent and 2nd extend far more large than the number I have assigned
Joined: 09 Mar 2011 Posts: 7309 Location: Inside the Matrix
If you want answers from us, you have to give answers and information from all our questions.
If you want to know why your extents have been "consolidated", ask your storage-management people, they know what you have, with what options, and knowing that you will have an explanation for that.
Joined: 14 Mar 2007 Posts: 8797 Location: Welsh Wales
The display shows that only 2 of the possible 6 volumes have been used. The asterisk indicates that there are 4 more volumes available for the dataset to expand onto if required.
Did you notice the difference between the creation date and the last referenced date.
Possibilities are -
HSM has processed these datasets during the HSM housekeeping cycle and has used the extent reduction process to consolidate the dataset into the minimum number of extents that it could.
Has the dataset been migrated and then recalled - HSM will reallocate the dataset in as few extents as possible.
Joined: 27 May 2010 Posts: 47 Location: da lian.china
Bill Woodger wrote:
If you want answers from us, you have to give answers and information from all our questions.
If you want to know why your extents have been "consolidated", ask your storage-management people, they know what you have, with what options, and knowing that you will have an explanation for that.
Sorry, I put the all system output in the following. VBS stand for file's format is Variable Blocked Spanned.
Code:
VPW//PRT010 DD SYSOUT=*,DCB=(RECFM=VBA)
1 OPTION SORTINS=2
RECORD TYPE=F,LENGTH=(250)
SORT FIELDS=(1,6,CH,A)
0 * PHASE 1 INPUT 100,569,450
DELETE 0
USED 100,569,450
* PHASE 3 SUMMARY 0
OUTPUT 100,569,450
VPW//PRT011 DD SYSOUT=*,DCB=(RECFM=VBA)
1ICE201I H RECORD TYPE IS F - DATA STARTS IN POSITION 1
ICE118I 0 UNKNOWN FILE SIZE - FILSZ=EN MAY IMPROVE RESOURCE USAGE AND PERFORMANCE
ICE751I 0 C5-K76982 C6-K90026 C7-K82419 C8-K67572 E4-K58148 C9-BASE E5-K80744 E7-K79990
ICE143I 0 BLOCKSET SORT TECHNIQUE SELECTED
ICE250I 0 VISIT クホホニ://ムムム.ケイテ.ウナテ/ヘホナネアキオ/エカヘナネホ FOR DFSORT PAPERS, EXAMPLES AND MORE
ICE000I 0 - CONTROL STATEMENTS FOR 5694-A01, Z/OS DFSORT V1R12 - 03:03 ON THU MAY 23, 2013 -
0 SORT FIELDS=(1,6,CH,A)
RECORD TYPE=F,LENGTH=(250)
ICE201I H RECORD TYPE IS F - DATA STARTS IN POSITION 1
ICE118I 0 UNKNOWN FILE SIZE - FILSZ=EN MAY IMPROVE RESOURCE USAGE AND PERFORMANCE
ICE751I 0 C5-K76982 C6-K90026 C7-K82419 C8-K67572 E4-K58148 C9-BASE E5-K80744 E7-K79990
ICE193I 0 ICEAM2 INVOCATION ENVIRONMENT IN EFFECT - ICEAM2 ENVIRONMENT SELECTED
ICE252I 1 PARMLIB OPTIONS WERE MERGED WITH INSTALLATION MODULE DEFAULTS
ICE089I 1 OS03QM10.SORT20 .A , INPUT LRECL = 250, TYPE = F
ICE093I 0 MAIN STORAGE = (MAX,6291456,6291456)
ICE156I 0 MAIN STORAGE ABOVE 16MB = (6234096,6234096)
ICE127I 0 OPTIONS: OVFLO=RC0 ,PAD=RC0 ,TRUNC=RC0 ,SPANINC=RC16,VLSCMP=N,SZERO=Y,RESET=Y,VSAMEMT=Y,DYNSPC=32
ICE128I 0 OPTIONS: SIZE=6291456,MAXLIM=2097152,MINLIM=450560,EQUALS=Y,LIST=Y,ERET=ABEND,MSGDDN=SYSOUT
ICE129I 0 OPTIONS: VIO=N,RESDNT=ALL ,SMF=FULL ,WRKSEC=Y,OUTSEC=Y,VERIFY=N,CHALT=N,DYNALOC=N ,ABCODE=MSG
ICE130I 0 OPTIONS: RESALL=4096,RESINV=0,SVC=109 ,CHECK=Y,WRKREL=Y,OUTREL=Y,CKPT=N,COBEXIT=COB2
ICE131I 0 OPTIONS: TMAXLIM=6291456,ARESALL=0,ARESINV=0,OVERRGN=16384,CINV=Y,CFW=Y,DSA=0
ICE132I 0 OPTIONS: VLSHRT=N,ZDPRINT=N,IEXIT=N,TEXIT=N,LISTX=N,EFS=NONE ,EXITCK=S,PARMDDN=DFSPARM ,FSZEST=N
ICE133I 0 OPTIONS: HIPRMAX=8 ,DSPSIZE=MAX ,ODMAXBF=0,SOLRF=Y,VLLONG=N,VSAMIO=N,MOSIZE=MAX
ICE235I 0 OPTIONS: NULLOUT=RC0
ICE236I 0 OPTIONS: DYNAPCT=10 ,MOWRK=Y
ICE750I 0 DC 0 TC 0 CS DSVVV KSZ 10 VSZ 10
ICE752I 0 FSZ=0 RE IGN=0 C AVG=256 0 WSP=0 E DYN=0 0
ICE751I 1 DE-K83743 D5-K58148 D3-K83080 D7-K58148 E8-K79990
ICE091I 0 OUTPUT LRECL = 250, TYPE = F
ICE055I 0 INSERT 100569450, DELETE 100569450
ICE054I 0 RECORDS - IN: 0, OUT: 0
ICE134I 0 NUMBER OF BYTES SORTED: 25142362500
ICE253I 0 RECORDS SORTED - PROCESSED: 100569450, EXPECTED: 0
ICE165I 0 TOTAL WORK DATA SET TRACKS ALLOCATED: 505710 , TRACKS USED: 500055
ICE199I 0 MEMORY OBJECT USED AS MAIN STORAGE = 0M BYTES
ICE299I 0 MEMORY OBJECT USED AS WORK STORAGE = 1937M BYTES
ICE180I 0 HIPERSPACE STORAGE USED = 0K BYTES
ICE188I 0 DATA SPACE STORAGE USED = 0K BYTES
ICE052I 0 END OF DFSORT
This is quite possible for a number of reasons, and I'd suggest you don't get to worried about it. As long as the file is big enough and you're not getting space failures what is the issue?
If you allocate 300,100 cyls space and unit count of 6, then a standard dataset can have the primary allocation plus 15 secondary on each of the volumes, or if it is Extended Format (as determined by Dataclas) it can have up to 123 extents per volume. If it has a guaranteed-Space storclas will also allocate primary first on each of the volumes before taking secondary extents.
When you view it in 3.4 the '1st extent cylinders' field shows the size of the first extent of the primary allocation only. This can vary because of things like:
a) extent conslidation where SMS will join extents together if there are tracks free next to one extent when it takes another extent.
b) how fragmented the volume is at allocation time
c) if you run some form of defrag of the volume which may combine extents together
d) if it's been migrated by DFHSM it will try and allocate the full size of the dataset as primary on the recall.
c) Space Constraint Relief (a Dataclas field) can cut allocations down to make it fit into available gaps so the value for primary in 3.4 could look different every time it's allocated.
Joined: 27 May 2010 Posts: 47 Location: da lian.china
Pete Wilson wrote:
This is quite possible for a number of reasons, and I'd suggest you don't get to worried about it. As long as the file is big enough and you're not getting space failures what is the issue?
If you allocate 300,100 cyls space and unit count of 6, then a standard dataset can have the primary allocation plus 15 secondary on each of the volumes, or if it is Extended Format (as determined by Dataclas) it can have up to 123 extents per volume. If it has a guaranteed-Space storclas will also allocate primary first on each of the volumes before taking secondary extents.
When you view it in 3.4 the '1st extent cylinders' field shows the size of the first extent of the primary allocation only. This can vary because of things like:
a) extent conslidation where SMS will join extents together if there are tracks free next to one extent when it takes another extent.
b) how fragmented the volume is at allocation time
c) if you run some form of defrag of the volume which may combine extents together
d) if it's been migrated by DFHSM it will try and allocate the full size of the dataset as primary on the recall.
c) Space Constraint Relief (a Dataclas field) can cut allocations down to make it fit into available gaps so the value for primary in 3.4 could look different every time it's allocated.
perfect , thank you so much.
Would kindly please tell me where you have got this information? Is there any reference or guides?
Joined: 09 Mar 2011 Posts: 7309 Location: Inside the Matrix
I know what VBS is, I was wondering how you were managing to use it :-)
From the sysout, both your input and output are being treated a F, not even FB. I'm not sure what you are getting on your output, I've never tried that :-)
You may have some performance issues.
What is the DCB of the input?
Again, knowing what VBS is, why are you using it for what you have told SORT is fixed-length records?
Usually Spanned records are larger than the blocksize. If they are not, there is not much point in using Spanned over Variable, that I can think of, anyway. Even if variable, why have them all as fixed-length - that is an extra four bytes per record, not isiginificant when you have 100,000,000+ records :-)
Joined: 06 Jun 2008 Posts: 8696 Location: Dubuque, Iowa, USA
Quote:
Usually Spanned records are larger than the blocksize. If they are not, there is not much point in using Spanned over Variable, that I can think of, anyway.
Bill: I had a case (many, many years ago) where spanned helped for an application. I had a fixed set of employee data, followed by a variable number of job histories. The fixed data was about 1000 bytes and the variable portion pushed the LRECL up over 22,000. At the time, the operating system wrote variable-length blocks when the remaining space in the block was not large enough to hold a complete record, so effectively I was leaving thousands and tens of thousands of bytes unused in each block. By using spanned, the system would leave at most 4 bytes unused in the block and vastly decreased the number of blocks used.
Joined: 09 Mar 2011 Posts: 7309 Location: Inside the Matrix
Thanks Robert.
For readers, these days APPLY WRITE-ONLY deals with that problem from COBOL, the block is only written when the current record written is too long for the remaining space in the block (not the maximum-length record (LRECL)).
In this topic I can't 1) see what VBS gets (LRECL of 256), 2) see why to treat them in SORT as F, 3) see why override the output dataset as VBS again (again 256 LRECL, whereas F 250 in the SORT) - and I don't even know what that will do :-)
Joined: 06 Jun 2008 Posts: 8696 Location: Dubuque, Iowa, USA
The data posted so far is definitely inconsistent, and I'm surprised the application can use the output of the SORT anyway. VBS with LRECL of 256 implies a record length of 252, yet SORT is outputting 250 bytes fixed length records. And yet the only concern is about the allocation of extents?
Would kindly please tell me where you have got this information? Is there any reference or guides?
Look at IBM manuals:
- Using Datasets
- DFSMS Storage Administration Reference (and the Guide)
I figured out what the Primary space value was in 3.4 by looking at a phsyical volume map of the volume a file was allocated on (IEHLIST or Fileaid 3.7 for example), because I was mystified how it was showing such numbers.
Joined: 27 May 2010 Posts: 47 Location: da lian.china
Bill Woodger wrote:
I know what VBS is, I was wondering how you were managing to use it :-)
From the sysout, both your input and output are being treated a F, not even FB. I'm not sure what you are getting on your output, I've never tried that :-)
You may have some performance issues.
What is the DCB of the input?
Again, knowing what VBS is, why are you using it for what you have told SORT is fixed-length records?
Usually Spanned records are larger than the blocksize. If they are not, there is not much point in using Spanned over Variable, that I can think of, anyway. Even if variable, why have them all as fixed-length - that is an extra four bytes per record, not isiginificant when you have 100,000,000+ records :-)
Thank you for your patient explanation. The reason to use VBS is that, the file in FB format is too large, and there are many blanks in the content of the dataset, therefore use VBS dataset can save huge space for me.(In face, in my application, VBS dataset have been offen used to storage data which can save a lot of space, you can have a try that use DCPVV1 to transfrom a dataset in FB format which have many blanks to VBS format, it may only use less than 20% storage of the original FB dataset, in my case, the record lenght was 250, and the recoreds number are 100,569,450, and in FB dataset it needs almost 30,000 cylinders whereas VBS file is only 4700 cylinders)
Joined: 27 May 2010 Posts: 47 Location: da lian.china
Bill Woodger wrote:
Thanks Robert.
For readers, these days APPLY WRITE-ONLY deals with that problem from COBOL, the block is only written when the current record written is too long for the remaining space in the block (not the maximum-length record (LRECL)).
In this topic I can't 1) see what VBS gets (LRECL of 256), 2) see why to treat them in SORT as F, 3) see why override the output dataset as VBS again (again 256 LRECL, whereas F 250 in the SORT) - and I don't even know what that will do :-)
1) input file is VBS 256,
2) because there are keys to be sorted has been specified, I am not sure the result is right to treat them in SORT as V
3)to overrige the output from F to VBS is to save the storage.
Joined: 09 Mar 2011 Posts: 7309 Location: Inside the Matrix
But why not just make it VB? Why the "spanned"?
You are getting this:
Quote:
ICE118I 0 UNKNOWN FILE SIZE - FILSZ=EN MAY IMPROVE RESOURCE USAGE AND PERFORMANCE
On top of that, you used 33,337 cylinders of sortwork space. If by using variable-length records you get that down to 4,700 cylinders, that alone is going to improve things greatly for this step.
So, why VBS, not VB, why F not VB, why VBS in-and-out but treat it as FB in the SORT?
Joined: 27 May 2010 Posts: 47 Location: da lian.china
Bill Woodger wrote:
But why not just make it VB? Why the "spanned"?
You are getting this:
Quote:
ICE118I 0 UNKNOWN FILE SIZE - FILSZ=EN MAY IMPROVE RESOURCE USAGE AND PERFORMANCE
On top of that, you used 33,337 cylinders of sortwork space. If by using variable-length records you get that down to 4,700 cylinders, that alone is going to improve things greatly for this step.
So, why VBS, not VB, why F not VB, why VBS in-and-out but treat it as FB in the SORT?
Thank you very much, I will do some test based on your suggestion. :-)
Joined: 27 May 2010 Posts: 47 Location: da lian.china
expat wrote:
The display shows that only 2 of the possible 6 volumes have been used. The asterisk indicates that there are 4 more volumes available for the dataset to expand onto if required.
Did you notice the difference between the creation date and the last referenced date.
Possibilities are -
HSM has processed these datasets during the HSM housekeeping cycle and has used the extent reduction process to consolidate the dataset into the minimum number of extents that it could.
Has the dataset been migrated and then recalled - HSM will reallocate the dataset in as few extents as possible.
Yes, the dave was created days ago, and sure it have been in MIGRAT1.
I think this is the right answer to my question. Thank you very much!
Joined: 27 May 2010 Posts: 47 Location: da lian.china
Bill Woodger wrote:
I'm moving this to the DFSORT part of the forum so that Kolusu can comment on it.
No pressure, Kolusu... :-)
And is there any benefit in knowing you've got about 4,700 cylinders of data yet only specifying 300,100 for the space?
Because there is an specify of [UNIT=(DISK2,6)], which means it will extent in 6 volumns.
For one volumn, specify 300,100 means the max space can allocate is 300 + 100 * 15 = 1800
but in the case of 6 volumns, it can specify max 4800 cylinders data
the calculation way is that:
first volume:300 + 100 * 15
second to six volme: 100 * 16
the total max space = 300 + 100 * 15 + (100 * 6) * 5 = 4800 cylinder
By the way, I want to know more specificly about DCPSORT, is there any reference about it?
Joined: 09 Mar 2011 Posts: 7309 Location: Inside the Matrix
:-)
Well, bear in mind what Pete has already posted about "extended".
Bear in mind also, the 300 cylinders of the primary space may require more than one extent to satisfy. Then you are at least 100 cylinders short of your 4,800. If this happens also on another volume, you are at least 200 cylinders short. Etc. If there is not space for a secondary allocation of 100, this might also take one than one extent...
You may have lots of free space, but find that it is "fractured" (lots of small extents). This happens readily when space for a dataset is allocated... in exactly the way in which you are doing it :-)
However, it is your step consuming a lot of resources, if you want to let it roll like that, so be it.
If you want to use 4,800 as a rock-solid maximum (what you seem to be implying now), why not something like a primary of 2,400 on two volumes with no secondary? Or something. Anything but 300,100 for a 4,700-cylinder dataset, perhaps...
Joined: 09 Mar 2011 Posts: 7309 Location: Inside the Matrix
You might want to consider the Subject you gave to your post, and describe your concerns about Sortwork, or whether those concerns will be satisfied if work space is reduced to reflect the size of the input/output datasets.