IBM Mainframe Forum Index
 
Log In
 
IBM Mainframe Forum Index Mainframe: Search IBM Mainframe Forum: FAQ Register
 

performance improvement through LRECL


IBM Mainframe Forums -> JCL & VSAM
Post new topic   Reply to topic
View previous topic :: View next topic  
Author Message
Rahul_kumar
Warnings : 2

New User


Joined: 24 Jun 2006
Posts: 50

PostPosted: Wed Apr 27, 2011 10:10 am
Reply with quote

Hi,

my job was taking a lot of time and perfomance improvement was required.I noticed that the o/p file being created through that job was having a RECFM =VB and LRECL was very high as compared to the actual length of records in that file.Therby,I reduced the LRECL and the perffomance was reduced significantly.I would like to know how through LRECL; the execution time reduced significantly.My O/P file contains millions of records.

Thanks in advance
Rahul
Back to top
View user's profile Send private message
Garry Carroll

Senior Member


Joined: 08 May 2006
Posts: 1193
Location: Dublin, Ireland

PostPosted: Wed Apr 27, 2011 11:24 am
Reply with quote

First reaction is that, by reducing the, the blocking factor increased - i.e each EXCP (execute channel program) returned more records than previously. Are you sure, though, that you won't have records larger than the LRECL you have now specified?

Garry.
Back to top
View user's profile Send private message
Rahul_kumar
Warnings : 2

New User


Joined: 24 Jun 2006
Posts: 50

PostPosted: Wed Apr 27, 2011 11:36 am
Reply with quote

Hi Garry,

Thanks for the reply.

Yes,we are sure that the maximum record length(REVFM=VB) won't increase more than LRECL.
Back to top
View user's profile Send private message
enrico-sorichetti

Superior Member


Joined: 14 Mar 2007
Posts: 10872
Location: italy

PostPosted: Wed Apr 27, 2011 11:55 am
Reply with quote

utter nonsense! and wrong conlusion... or wrong premises
review Your understanding of VB...
the lrecl in the dscb has nothing to do with the number of records stored in a block
it simply tells how large i the buffer that contains the data record for move mode access!

the number of records stored in a block depends only on the real record length

<bdw><rdw>data<rdw>data<rdw>data<rdw>data<rdw>data

the disk occupancy of datasets with the same record length pattern is the same period and does not depend on the maximum record length, only on the blocksize
test it if You do not believe it

on the other side given the poor understanding of VB around here
most probably there is some confusion
quite too often the posters talk about VB and FB without understanding what is going on


what was the disk occupancy before and after ???
Back to top
View user's profile Send private message
Rahul_kumar
Warnings : 2

New User


Joined: 24 Jun 2006
Posts: 50

PostPosted: Wed Apr 27, 2011 1:07 pm
Reply with quote

Hi enrico,
The cobol module splices 2 i/p files into a VB file.The data in the i/p files is same before and after the run.just we reduced the LRECL of the O/P VB file and the CPU execution time reduced drastically.
Back to top
View user's profile Send private message
dbzTHEdinosauer

Global Moderator


Joined: 20 Oct 2006
Posts: 6966
Location: porcelain throne

PostPosted: Wed Apr 27, 2011 1:19 pm
Reply with quote

let us try to determine the o/p file attributes:
  • What was the original LRECL, DCB BLOCKSIZE, COBOL Module Blocksice for the output file
  • what were these values after the performance change?
Back to top
View user's profile Send private message
enrico-sorichetti

Superior Member


Joined: 14 Mar 2007
Posts: 10872
Location: italy

PostPosted: Wed Apr 27, 2011 1:19 pm
Reply with quote

if You are happy I am happy ( or better I do not care )
still Your understanding of data management is just plain wrong
and the conclusions You are drawing are wrong
You did not show any technical evidence of the fact
recfm , blksize, lrecl , space used, record count of the before
the same of the after ...
I/O counts of the before and the after

agin... show scientific evidence with all the data necessary to investigate

I am not telling that the elapsed and cpu observations are wrong,
I am just telling that the changes cannot depend on just a different LRECL allocation specification!
Back to top
View user's profile Send private message
dbzTHEdinosauer

Global Moderator


Joined: 20 Oct 2006
Posts: 6966
Location: porcelain throne

PostPosted: Wed Apr 27, 2011 1:36 pm
Reply with quote

Rahul_kumar,
how about answering my question if you want any help.
Back to top
View user's profile Send private message
Bill Woodger

Moderator Emeritus


Joined: 09 Mar 2011
Posts: 7309
Location: Inside the Matrix

PostPosted: Wed Apr 27, 2011 1:59 pm
Reply with quote

Rahul_kumar wrote:
The data in the i/p files is same before and after the run.


Quite a normal situation.



I'd guess if you ever answer Dick, you'd have RECORD CONTAINS 0 in your Cobol program, thereby taking the LRECL from the JCL.
Back to top
View user's profile Send private message
enrico-sorichetti

Superior Member


Joined: 14 Mar 2007
Posts: 10872
Location: italy

PostPosted: Wed Apr 27, 2011 2:03 pm
Reply with quote

the TS talks about the OP ( output file )
Back to top
View user's profile Send private message
enrico-sorichetti

Superior Member


Joined: 14 Mar 2007
Posts: 10872
Location: italy

PostPosted: Wed Apr 27, 2011 2:23 pm
Reply with quote

Quote:
The cobol module splices 2 i/p files into a VB file.The data in the i/p files is same before and after the run.just we reduced the LRECL of the O/P VB file and the CPU execution time reduced drastically.


You posted the same message already twice and it was deleted
instead of blabbering, reply with the info we asked You.

if You are not willing to reply to what we ask You stop bothering

You ask for help we spend time on Your issue,
so show respect and good manners by replying to our questions
do You understand icon_evil.gif

if You do not come up with the info requested the topic will be locked and after a while deleted !
Back to top
View user's profile Send private message
dbzTHEdinosauer

Global Moderator


Joined: 20 Oct 2006
Posts: 6966
Location: porcelain throne

PostPosted: Wed Apr 27, 2011 2:24 pm
Reply with quote

Rahul_kumar wrote:
The cobol module splices 2 i/p files into a VB file.The data in the i/p files is same before and after the run.just we reduced the LRECL of the O/P VB file and the CPU execution time reduced drastically.


repeating that useless comment is not helping, and three times is not a charm!

changing the lrecl of the output file could have had the following effects:
  • original lrecl=14,000 new lrecl = 6,000 - impact - instead of 1 record per block, potentially 4
  • originally your code could have produced a max length vb every for every write, and the new code for the reduced lrecl managed to write a valid length'd vb record
  • you changed the block size
  • you did something else that you are not telling us
we are not in the habit of substantiating unfounded comments
based on the observance of someone who can not read and respond to questions.
Back to top
View user's profile Send private message
enrico-sorichetti

Superior Member


Joined: 14 Mar 2007
Posts: 10872
Location: italy

PostPosted: Wed Apr 27, 2011 2:27 pm
Reply with quote

at this point we can only guess that the program is just wrongly writing all the times a record with the maximum record length allowable

Quote:
I am just telling that the changes cannot depend on just a different LRECL allocation specification!
given/assuming that the program is correctly coded icon_biggrin.gif
but since the program is certainly badly coded then it is reasonable to expect a result according to the observation

as usually I make the mistake of assuming that the TS has done the due diligence and coded properly icon_evil.gif

certainly the storage salesdroid will be quite happy with his overachievement icon_biggrin.gif
Back to top
View user's profile Send private message
Rahul_kumar
Warnings : 2

New User


Joined: 24 Jun 2006
Posts: 50

PostPosted: Wed Apr 27, 2011 3:06 pm
Reply with quote

Hi all,
Apologies for the late reply.actually I was refreshing the page for the reply and the same msg was being posted again & again.

The original LRECL was around 32000 and I reduced it to around 1000 bytes.Also,the same length was specified in the cobol program under FD section thereby that was also changed.I did'nt change the blk size or any other DCB parameter.

Also,I had also made a in the program to calculate the record length of VB file for each record.Since,previously the record length was 32K thereby the module was counting backward from 32K but now that was changed to 1000 bytes.But,this change did'nt made much difference and only after changing the LRECL,the time was reduced.
I want to know how by only changing the LRECL the performance can reduce significantly.
Back to top
View user's profile Send private message
dbzTHEdinosauer

Global Moderator


Joined: 20 Oct 2006
Posts: 6966
Location: porcelain throne

PostPosted: Wed Apr 27, 2011 4:33 pm
Reply with quote

you changed the lrecl to a point where there were more records per block.
at 32000, you only get 1.
at 1000, you get 27.

so you have reduced your read requirement by a factor of 27.

it was not the reducing the lrecl that had an affect.
if was significantly reducing the number of blocks that was effective in reducing cpu.
Back to top
View user's profile Send private message
enrico-sorichetti

Superior Member


Joined: 14 Mar 2007
Posts: 10872
Location: italy

PostPosted: Wed Apr 27, 2011 4:35 pm
Reply with quote

Quote:
I want to know how by only changing the LRECL the performance can reduce significantly.


for what friendly reason You keep refusing to post what we asked You to ...

allocation info for the <poor performing> dataset
allocation info for the <good performing> dataset

the space the space the space for the umpteen time the space used

if You are not able to reply to simple questions it would be wise to meditate on a career switch

the reason for the odd behavior is that the program was and probably is still wrong

it was and still takes the LRECL from the allocation rather than calculating it for each record based on the data layout

POST WHAT WE ASKED YOU FOR NOT YOUR HORSE MANURE


and also it would be a good idea to look at a physical dump of the block
if the RDW are all the same the program is wrong!

edited to change the f word to a less disturbing one icon_cool.gif
Back to top
View user's profile Send private message
enrico-sorichetti

Superior Member


Joined: 14 Mar 2007
Posts: 10872
Location: italy

PostPosted: Wed Apr 27, 2011 4:36 pm
Reply with quote

36_2_18.gif
Back to top
View user's profile Send private message
Robert Sample

Global Moderator


Joined: 06 Jun 2008
Posts: 8696
Location: Dubuque, Iowa, USA

PostPosted: Wed Apr 27, 2011 4:51 pm
Reply with quote

Quote:
,I reduced the LRECL and the perffomance was reduced significantly.
Uh, no it did not. Something changed and improved performance -- but merely changing the LRECL pretty much had no impact of the overall performance and job throughput. Since LRECL is a LOGICAL concept and not a PHYSICAL contruct, there is no way that merely changing the LRECL would impact performance. Changing LRECL and thereby affecting block size, for example, could have a performance impact -- but it's not the LRECL change, per se, that is making the difference but the block size change.
Back to top
View user's profile Send private message
gcicchet

Senior Member


Joined: 28 Jul 2006
Posts: 1702
Location: Australia

PostPosted: Wed Apr 27, 2011 4:56 pm
Reply with quote

Hi,

run this across both files and show the results

Code:
//STEP0100 EXEC PGM=ICETOOL                                 
//TOOLMSG  DD SYSOUT=*                                     
//DFSMSG   DD SYSOUT=*                                     
//IN       DD DSN=input-file,DISP=SHR                       
//RPT      DD SYSOUT=*                                     
//TOOLIN   DD *                                             
  STATS FROM(IN) ON(VLEN)                                   
  OCCUR FROM(IN) LIST(RPT) BLANK -                         
   HEADER('LENGTH') ON(VLEN) -                             
   HEADER('COUNT') ON(VALCNT)                               
/*                                                         


That is if you have DFSORT.

Gerry
Back to top
View user's profile Send private message
vasanthz

Global Moderator


Joined: 28 Aug 2007
Posts: 1742
Location: Tirupur, India

PostPosted: Wed Apr 27, 2011 5:21 pm
Reply with quote

Im not the TS or nor do i know him anyway, but Enrico's choice of words is are offending..

I have seen people advice others about not using 'ur' 'u' 'thx' saying that this is a professional forum and not SMS service, but which category does this belong to?
Quote:

for what fucking reason You keep refusing to post what we asked You to ...

Quote:
POST WHAT WE ASKED YOU FOR NOT YOUR HORSE MANURE




icon_rolleyes.gif
Back to top
View user's profile Send private message
enrico-sorichetti

Superior Member


Joined: 14 Mar 2007
Posts: 10872
Location: italy

PostPosted: Wed Apr 27, 2011 5:24 pm
Reply with quote

did You read the whole topic ... looks like not icon_evil.gif
count how many times we have asked for the proper info
hoe many time the ts has posted useless info
( I deleted three posts with the same garbage content)
Quote:
The cobol module splices 2 i/p files into a VB file.The data in the i/p files is same before and after the run.just we reduced the LRECL of the O/P VB file and the CPU execution time reduced drastically.


it' s the TS who is offending/disrespecting the people who try to help him
on our own time, free of charge

the category/tone belong to the natural reaction of somebody sharing its experience
and spending ( rather wasting) time to help somebody who does not care to provide proper info
and keeps posting/venting the same useless information

but maybe it' s the free of charge that make people ask without any commitment ti properly pursue the issue
Back to top
View user's profile Send private message
Ronald Burr

Active User


Joined: 22 Oct 2009
Posts: 293
Location: U.S.A.

PostPosted: Wed Apr 27, 2011 5:52 pm
Reply with quote

Perhaps the TS did NOT use the APPLY WRITE-ONLY clause for the file (or specify compiler option AWO), and thus a simple change in the LRECL (with NO change to the BLKSIZE) WOULD have made a significant impact on performance.
The Programmer's Guide says:


"Use the APPLY WRITE-ONLY clause to make optimum use of buffer and device space when you create a sequential file with blocked variable-length records. With APPLY WRITE-ONLY specified, a buffer is truncated only when the next record does not fit in the unused portion of the buffer. Without APPLY WRITE-ONLY specified, a buffer is truncated when it does not have enough space for a maximum-size record. The APPLY WRITE-ONLY clause has meaning only for sequential files that have variable-length records and are blocked. The AWO compiler option applies an implicit APPLY WRITE-ONLY clause to all eligible files. The NOAWO compiler option has no effect on files that have the APPLY WRITE-ONLY clause specified. The APPLY WRITE-ONLY clause takes precedence over the NOAWO compiler option. The APPLY-WRITE ONLY clause can cause input files to use a record area rather than process the data in the buffer. This use might affect the processing of both input files and output files."


With NOAWO and a 32K LRECL (with the REQUIRED 32K+4 BLKSIZE), one is guaranteed that there will be only 1 record per physical block regardless of actual record lengths.
With NOAWO and a 1K LRECL (with the SAME 32K+4 BLKSIZE), one is guaranteed that there will be at least 31 records per physical block, and perhaps a lot more).
With AWO (and the SAME 32K+4 BLKSIZE), the actual number of records per block would be totally independent of the STATED LRECL (which only represents the MAXIMUM possible LRECL), but would depend solely on the ACTUAL record lengths being passed to the I/O routines.
Back to top
View user's profile Send private message
dbzTHEdinosauer

Global Moderator


Joined: 20 Oct 2006
Posts: 6966
Location: porcelain throne

PostPosted: Wed Apr 27, 2011 5:53 pm
Reply with quote

The TS's continual disregard of facts/knowledge
is further exemplified by: ibmmainframes.com/viewtopic.php?t=54283
Back to top
View user's profile Send private message
Bill Woodger

Moderator Emeritus


Joined: 09 Mar 2011
Posts: 7309
Location: Inside the Matrix

PostPosted: Wed Apr 27, 2011 6:34 pm
Reply with quote

Rahul_kumar wrote:
...

Also,I [...] made a [change] in the program to calculate the record length [...] for each record.
...


Why did you do this?

How did you do this?

How do think you were able to affect the record-length?
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic View Bookmarks
All times are GMT + 6 Hours
Forum Index -> JCL & VSAM

 


Similar Topics
Topic Forum Replies
No new posts VB to FB - Finding LRECL SYNCSORT 4
No new posts exploiting Z16 performance PL/I & Assembler 2
No new posts JOIN STATEMENT PERFORMANCE. DFSORT/ICETOOL 12
No new posts Which SORT utility can improve the Pe... DFSORT/ICETOOL 16
No new posts A command to change LRECL of an exist... JCL & VSAM 7
Search our Forums:

Back to Top