IBM Mainframe Forum Index
 
Log In
 
IBM Mainframe Forum Index Mainframe: Search IBM Mainframe Forum: FAQ Register
 

Sortwork and file space allocation mechanism


IBM Mainframe Forums -> DFSORT/ICETOOL
Post new topic   Reply to topic
View previous topic :: View next topic  
Author Message
liying

New User


Joined: 27 May 2010
Posts: 47
Location: da lian.china

PostPosted: Thu May 23, 2013 7:07 pm
Reply with quote

expat wrote:
The display shows that only 2 of the possible 6 volumes have been used. The asterisk indicates that there are 4 more volumes available for the dataset to expand onto if required.

Did you notice the difference between the creation date and the last referenced date.

Possibilities are -
HSM has processed these datasets during the HSM housekeeping cycle and has used the extent reduction process to consolidate the dataset into the minimum number of extents that it could.

Has the dataset been migrated and then recalled - HSM will reallocate the dataset in as few extents as possible.


Yes, the dave was created days ago, and sure it have been in MIGRAT1.
I think this is the right answer to my question. Thank you very much!
Back to top
View user's profile Send private message
Bill Woodger

Moderator Emeritus


Joined: 09 Mar 2011
Posts: 7311
Location: Inside the Matrix

PostPosted: Thu May 23, 2013 7:09 pm
Reply with quote

I'm moving this to the DFSORT part of the forum so that Kolusu can comment on it.

No pressure, Kolusu... :-)

And is there any benefit in knowing you've got about 4,700 cylinders of data yet only specifying 300,100 for the space?
Back to top
View user's profile Send private message
liying

New User


Joined: 27 May 2010
Posts: 47
Location: da lian.china

PostPosted: Thu May 23, 2013 7:29 pm
Reply with quote

Bill Woodger wrote:
I'm moving this to the DFSORT part of the forum so that Kolusu can comment on it.

No pressure, Kolusu... :-)

And is there any benefit in knowing you've got about 4,700 cylinders of data yet only specifying 300,100 for the space?


Because there is an specify of [UNIT=(DISK2,6)], which means it will extent in 6 volumns.

For one volumn, specify 300,100 means the max space can allocate is 300 + 100 * 15 = 1800
but in the case of 6 volumns, it can specify max 4800 cylinders data

the calculation way is that:

first volume:300 + 100 * 15
second to six volme: 100 * 16

the total max space = 300 + 100 * 15 + (100 * 6) * 5 = 4800 cylinder



By the way, I want to know more specificly about DCPSORT, is there any reference about it?
Back to top
View user's profile Send private message
Bill Woodger

Moderator Emeritus


Joined: 09 Mar 2011
Posts: 7311
Location: Inside the Matrix

PostPosted: Thu May 23, 2013 8:12 pm
Reply with quote

:-)

Well, bear in mind what Pete has already posted about "extended".

Bear in mind also, the 300 cylinders of the primary space may require more than one extent to satisfy. Then you are at least 100 cylinders short of your 4,800. If this happens also on another volume, you are at least 200 cylinders short. Etc. If there is not space for a secondary allocation of 100, this might also take one than one extent...

You may have lots of free space, but find that it is "fractured" (lots of small extents). This happens readily when space for a dataset is allocated... in exactly the way in which you are doing it :-)

However, it is your step consuming a lot of resources, if you want to let it roll like that, so be it.

If you want to use 4,800 as a rock-solid maximum (what you seem to be implying now), why not something like a primary of 2,400 on two volumes with no secondary? Or something. Anything but 300,100 for a 4,700-cylinder dataset, perhaps...

And I assume you mean DFSORT, not DCPSORT?
Back to top
View user's profile Send private message
Bill Woodger

Moderator Emeritus


Joined: 09 Mar 2011
Posts: 7311
Location: Inside the Matrix

PostPosted: Thu May 23, 2013 8:24 pm
Reply with quote

You might want to consider the Subject you gave to your post, and describe your concerns about Sortwork, or whether those concerns will be satisfied if work space is reduced to reflect the size of the input/output datasets.
Back to top
View user's profile Send private message
Skolusu

Senior Member


Joined: 07 Dec 2007
Posts: 2205
Location: San Jose

PostPosted: Thu May 23, 2013 10:51 pm
Reply with quote

Bill Woodger wrote:
I'm moving this to the DFSORT part of the forum so that Kolusu can comment on it.

No pressure, Kolusu... :-)


Not sure what to comment on here as the DSN info is quite different from the DFSORT sysout. I am guessing that this is program invoked sort. Is the program modifying the records to F and passing to DFSORT?

DFSORT messages
Code:

ICE089I 1 OS03QM10.SORT20  .A       , INPUT LRECL = 250, TYPE = F
ICE091I 0 OUTPUT LRECL = 250, TYPE = F



Code:

Data Set Name . . . . : XXXX.DS.DATASET.D01
  Record format . . . : VBS             Used cylinders  . . : 4,396           
  Record length . . . : 256             Used extents  . . . : 4   

Data Set Name . . . . : XXXX.DS.DATASET.D04         
  Record format . . . : VBS             Used cylinders  . . : 4,638     
  Record length . . . : 256             Used extents  . . . : 2         
Back to top
View user's profile Send private message
Bill Woodger

Moderator Emeritus


Joined: 09 Mar 2011
Posts: 7311
Location: Inside the Matrix

PostPosted: Fri May 24, 2013 4:22 am
Reply with quote

Yes, this is an old one, perhaps even from the same site. DCPSORT PROC.

Further down is:

Code:
//DCPSORT PROC SP='10,10',SOUT='*',REG=2048K                           
//*                                                                     
//*       DCP UTILITY ----- SORT/MERGE OF COMPRESSION FILE             
//*                                                                     
//*           SYSIN CONTROL CARD STATEMENT                             
//*                   OPTION SORTINS=1                                 
//*                   SORT FIELDS=(N,N,CH,A)    (SAME AS NEWSORT)       
//*                   RECORD TYPE=F,LENGTH=(X)  (LOGICAL LENGTH)       
//*                                                                     
//*           IF MULTI SORTIN EXIST THEN DO                             
//*                   OPTION SORTINS=N ----- NO. OF SORTINXX           
//*                 //  EXEC DCPSORT                                   
//*                 //SORTIN01 DD DSN=-----                             
//*                 //SORTIN02 DD DSN=-----                             
//*                 //SORTIN03 DD DSN=-----                             
//*                 //SORTOUT  DD DSN=-----                             
//*                 //SYSIN    DD *                                     
//*                   OPTION SORTINS=N ----- NO. OF SORTINXX           
//*                                                                     
//*                 WHERE   NN        -----  MAX 16                     
//A       EXEC PGM=SORTX,REGION=&REG                                   
//STEPLIB  DD  DSN=DCP.DCPLIB,DISP=SHR          /* DCP LIBRARY */       
//SYSPRINT DD  SYSOUT=*                                                 
//SYSOUT   DD  SYSOUT=&SOUT                                             
//SORTIN01 DD  DDNAME=SORTIN                                   


Also got to here, in much the same way as enrico once found this :-)

There is a reference to NEWSORT in the above, which links the whole thing to this one. And this one.

So, I guess "DCP" is "DeComPress", a "wrapper" which can be used around DFSORT to turn a "compressed" file (specified as VBS) into its original fixed-length format, and compress it again on the way out.

From the difference in the known LRECL example, the 256 VBS vs 250 F, and connecting it to what the current TS has said, I'd guess that the perhaps only trailing blanks are "compressed", with the two bytes being the "count" of trailing blanks.

IF only trailing blanks are compressed, I can't help but think that VB is better. It has the length of the data, and once it has a control card to know the length of the de-compressed record, it would be no problem to re-create the correct number of trailing blanks and saving two further bytes per record.

However, it seems to have been in use for eight years or more, so maybe they're happy with it.

If VB would work, the records could be used in the SORT without decompression. Even, if only trailing blanks are dropped, the VBS records could be used directly as well.

I wonder if they'll hire me to check everything out? Looks like there could be a mass of savings there...

Let's hope DCP does a bit more than the TS thinks, eh? :-)
Back to top
View user's profile Send private message
Skolusu

Senior Member


Joined: 07 Dec 2007
Posts: 2205
Location: San Jose

PostPosted: Fri May 24, 2013 5:10 am
Reply with quote

Bill,

Thanks for the explanation.

liying,

I would like to see how the allocations work if you invoke DFSORT directly instead of the wrapper.

Can you run this Job and show us the complete sysout?

Code:

//STEP0100 EXEC PGM=ICEMAN
//SYSOUT   DD SYSOUT=*
//SORTDIAG DD DUMMY
//SORTIN   DD DISP=SHR,DSN=Your Input VBS file
//SORTOUT  DD DSN=Your Output VBS file,
//            DISP=(NEW,CATLG,DELETE),
//            SPACE=(CYL,(400,100),RLSE)
//*
//SYSIN    DD *
  OPTION VLSHRT,DYNALLOC=(,32),FILSZ=E101000000
  SORT FIELDS=(5,6,CH,A)
//*
Back to top
View user's profile Send private message
Bill Woodger

Moderator Emeritus


Joined: 09 Mar 2011
Posts: 7311
Location: Inside the Matrix

PostPosted: Fri May 24, 2013 5:56 am
Reply with quote

Have a look at a couple of the records first. You might need the key to start at 7, if there is some binary up front.
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic View Bookmarks
All times are GMT + 6 Hours
Forum Index -> DFSORT/ICETOOL Goto page 1, 2  Next

 


Similar Topics
Topic Forum Replies
No new posts Compare the file record count percent... DFSORT/ICETOOL 4
No new posts compare two file to find position/dat... SYNCSORT 2
No new posts Converting a file from PD to display ... SYNCSORT 4
No new posts Record count of RECFM=U tape file DFSORT/ICETOOL 17
This topic is locked: you cannot edit posts or make replies. Write Character to Hexadecimal file w... COBOL Programming 8
Search our Forums:

Back to Top