Portal | Manuals | References | Downloads | Info | Programs | JCLs | Master the Mainframes
IBM Mainframe Computers Forums Index
 
Register
 
IBM Mainframe Computers Forums Index Mainframe: Search IBM Mainframe Forum: FAQ Memberlist Usergroups Profile Log in to check your private messages Log in
 

 

IEBGENER vs SORT
Goto page 1, 2  Next
 
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> JCL & VSAM
View previous topic :: :: View next topic  
Author Message
fazilbe

New User


Joined: 03 Dec 2008
Posts: 25
Location: INDIA

PostPosted: Tue Jan 20, 2009 6:31 pm    Post subject: IEBGENER vs SORT
Reply with quote

Hi,

I have a job which runs for close to 10 hours each month. When analyzed found that one particular PROC step consuming all the run time. The step just does a copy of a huge file to a GDG using IEBGENER. Input file has a more than 100gb of record. Can we make this step more faster using any other technique. Will SORT be faster compared to IEBGENER. Please help me solve this issue. Please find the JCL below.
Code:

//JS010    EXEC AIP04000,             
//             RETPD='720',         
//             UNIT=CART,           
//             UNIT1='SYSDA',       
//             SPACEMY='100,50',           
//PS040    EXEC PGM=IEBGENER,                           
//             COND=(0,LT)                               
//SYSUT1   DD  DSN=temp1,       
//             DISP=SHR                                                     
//SYSUT2   DD  DSN=temp2(+1),
//             VOL=(,,,99),                                 
//             FREE=CLOSE,                                 
//             DISP=(NEW,CATLG,DELETE),                     
//             UNIT=&UNIT,                                 
//             LABEL=RETPD=&RETPD,                         
//             DCB=(&DSCB)                                 
//SYSIN    DD  DUMMY                                       
//SYSPRINT DD  SYSOUT=*                                 
//SYSUDUMP DD  SYSOUT=Y

Regards,
Fazil
Back to top
View user's profile Send private message

Robert Sample

Global Moderator


Joined: 06 Jun 2008
Posts: 7912
Location: Bellevue, IA

PostPosted: Tue Jan 20, 2009 6:38 pm    Post subject:
Reply with quote

Add buffers to input and output file; with large files you should never use the z/OS default.
Back to top
View user's profile Send private message
expat

Global Moderator


Joined: 14 Mar 2007
Posts: 8593
Location: Back in jolly old England

PostPosted: Tue Jan 20, 2009 6:50 pm    Post subject:
Reply with quote

Quote:
Will SORT be faster compared to IEBGENER. Please help me solve this issue. Please find the JCL below.

May I suggest that you try it for yourself. The workload of your machine can not be replicated on ours, and this is one of the major factors involved.

Code:
//             DCB=(&DSCB)

What are the specified paramters for &DSCB and to they allow for SDB to be used.
Back to top
View user's profile Send private message
fazilbe

New User


Joined: 03 Dec 2008
Posts: 25
Location: INDIA

PostPosted: Tue Jan 20, 2009 7:04 pm    Post subject: Reply to: IEBGENER vs SORT
Reply with quote

Hi,

DCB=(796 27,860 0 FB) is what i could find in the spool log. Does this help?. Please let me know if more information is needed.

Regards,
Fazil
Back to top
View user's profile Send private message
expat

Global Moderator


Joined: 14 Mar 2007
Posts: 8593
Location: Back in jolly old England

PostPosted: Tue Jan 20, 2009 7:29 pm    Post subject:
Reply with quote

I would replace that line with
Code:
//      RECFM=FB,LRECL=796


I assume that the output is tape and the code above will allow the system to determine the best output blocksize. This would allow more records per block to be written so less writes are required.

Is the input file tape also ? If so, is the blocksize optimum for tape or also defined for DASD ?

Also look at the suggestion by Robert to increase the I/O buffering, say 20 buffers per DD may significantly improve performance.
Back to top
View user's profile Send private message
Bill Dennis

Active Member


Joined: 17 Aug 2007
Posts: 562
Location: Iowa, USA

PostPosted: Tue Jan 20, 2009 8:02 pm    Post subject:
Reply with quote

Another option would be to allocate multiple tape drives on input and output as
Code:
//  UNIT=(&UNIT,2)


This is less effective on newer tape technology where the "end" is back at the "start" but if your tapes are virtual and are being recalled it might help.
Back to top
View user's profile Send private message
Douglas Wilder

Active User


Joined: 28 Nov 2006
Posts: 305
Location: Deerfield IL

PostPosted: Tue Jan 20, 2009 9:01 pm    Post subject:
Reply with quote

Check how long it was between the time the system requested the tape to be mounted and the time the operator informed the system that the tape was mounted. If the operators do not mount the tapes in a timely manor no system changes can improve the time that much.
Do allow the system to choose the best block size and increate the buffers as stated by others, and change IEBGENER to ICEGENER (If it has been installed this way on your system). This will use sort to copy the file and my experience is that this will help quite a bit.
Back to top
View user's profile Send private message
Alissa Margulies

SYNCSORT Support


Joined: 25 Jul 2007
Posts: 500
Location: USA

PostPosted: Tue Jan 20, 2009 9:44 pm    Post subject:
Reply with quote

If you are running SyncSort for z/OS, verify that the BetterGener function has been installed. The BetterGener utility program offers improved performance over IEBGENER in straight copy tasks and is transparent to users. Implementing BetterGener will provide significant savings in system resources such as elapsed time, CPU time, and channel utilization through improved I/O handling. Once BetterGener is activated, eligible IEBGENER job streams are automatically processed by SyncSort. There is no need to replace working IEBGENER job streams with explicit requests for SyncSort's more efficient copying.
Back to top
View user's profile Send private message
Frank Yaeger

DFSORT Moderator


Joined: 15 Feb 2005
Posts: 7130
Location: San Jose, CA

PostPosted: Tue Jan 20, 2009 10:11 pm    Post subject:
Reply with quote

Quote:
Do allow the system to choose the best block size and increate the buffers as stated by others, and change IEBGENER to ICEGENER (If it has been installed this way on your system). This will use sort to copy the file and my experience is that this will help quite a bit.


If DFSORT's ICEGENER has been installed as a replacement for IEBGENER, then you don't need to change PGM=IEBGENER to PGM=ICEGENER. ICEGENER will be used automatically. If DFSORT's ICEGENER has not been installed as a replacement for IEBGENER, then using PGM=ICEGENER would be necessary.

You don't need to increase the buffers for ICEGENER - it does its own optimal buffering.

For more information on DFSORT's ICEGENER, see:

http://publibz.boulder.ibm.com/cgi-bin/bookmgr_OS390/BOOKS/ICE1CA30/9.2.16?DT=20080528171007
Back to top
View user's profile Send private message
gcicchet

Senior Member


Joined: 28 Jul 2006
Posts: 1703
Location: Australia

PostPosted: Wed Jan 21, 2009 5:21 am    Post subject:
Reply with quote

Hi,

I would also check the BLOCKSIZE of the input file.


Gerry
Back to top
View user's profile Send private message
fazilbe

New User


Joined: 03 Dec 2008
Posts: 25
Location: INDIA

PostPosted: Wed Jan 21, 2009 1:06 pm    Post subject: Reply to: IEBGENER vs SORT
Reply with quote

Hi,

I will try the valuable suggestions given by everybody and let you all know the outcome of it. Thanks a lot.

Regards,
Fazil
Back to top
View user's profile Send private message
fazilbe

New User


Joined: 03 Dec 2008
Posts: 25
Location: INDIA

PostPosted: Wed Jan 21, 2009 3:14 pm    Post subject: Reply to: IEBGENER vs SORT
Reply with quote

Hi,

Actually input is in DASD and the output GDG has been written to TAPE.
Back to top
View user's profile Send private message
skkp2006

New User


Joined: 14 Jul 2006
Posts: 93
Location: Chennai,India

PostPosted: Fri Jan 23, 2009 8:17 am    Post subject: Reply to: IEBGENER vs SORT
Reply with quote

Alissa,

Quote:
"If you are running SyncSort for z/OS, verify that the BetterGener function has been installed. "


How can we possibly check whether BetterGener has been installed in our shop? Does it come along with SyncSort or is it a pay and use product?


Syam
Back to top
View user's profile Send private message
Arun Raj

Moderator


Joined: 17 Oct 2006
Posts: 2158
Location: @my desk

PostPosted: Fri Jan 23, 2009 9:59 am    Post subject:
Reply with quote

Quote:
Once BetterGener is activated, eligible IEBGENER job streams are automatically processed by SyncSort
skkp2006,

As pointed out by Alissa, you can run a small IEBGENER copy and check if SyncSort gets invoked.
Back to top
View user's profile Send private message
skkp2006

New User


Joined: 14 Jul 2006
Posts: 93
Location: Chennai,India

PostPosted: Fri Jan 23, 2009 10:22 am    Post subject: Reply to: IEBGENER vs SORT
Reply with quote

Arun,


I ran a simple test using the below JCL

Code:
//IEB      EXEC PGM=IEBGENER
//SYSOUT   DD SYSOUT=*
//SYSPRINT DD SYSOUT=*
//SYSUDUMP DD SYSOUT=*
//SYSUT1   DD DSN=input.file,DISP=SHR
//SYSUT2   DD DSN=output.file,DISP=OLD
//SYSIN    DD DUMMY




what i have in sysprint is

Code:
DATA SET UTILITY - GENERATE
PROCESSING ENDED AT EOD



and no indication that syncsort is getting invoked.... :-(



i ran the same job with icegener and got the following in sysout


Code:
DATA SET UTILITY - GENERATE
SYNCSORT COPY FEATURE CALLED - RC = 0




i am rather confused now icon_rolleyes.gif .... what does this indicate? is BetterGener getting invoked when i use ICEGENER or is this a simple sort copy job? and how do i differenciate a BetterGener job with others ?



Syam
Back to top
View user's profile Send private message
fazilbe

New User


Joined: 03 Dec 2008
Posts: 25
Location: INDIA

PostPosted: Fri Jan 23, 2009 7:06 pm    Post subject:
Reply with quote

Hi,

I used the RECFM=FB,LRECL=796 and coded BUFNO=50 in the input and output files. I could see a improvement of over 90% in the CPU time , But very less improvements in the elapsed time. Sort ran for 3 hours and IEBGENER ran for 3hours and 20 minutes. Also the FASTGENER is being used in our system which makes the IEBGENER copy very fast and performance is equivalent to SORT copy.

But the point is, In the original prod run, same job ran for 10 hours. But while testing it with SORT and IEBGENER job it just ran for 3 hours with all the records properly sorted as that of production machine. Not sure why such a difference even while using IEBGENER. Even if the production machine was busy and had so many running at the same time, I think 7 hours variance as some what odd. Does RECFM=FB,LRECL=796 would have contributed to this variance in test regions. Please advise.

Also please find the code below which i used for testing.

//SORTIN DD DSN=TEMP1,
// DISP=SHR,
// BUFNO=50,
// DCB=(LRECL=796,RECFM=FB)
// DD DSN=TEMP2,
// DISP=SHR,
// BUFNO=50,
// DCB=(LRECL=796,RECFM=FB)
//SORTOUT DD DSN=OUTPUT11,
// DISP=(NEW,CATLG,DELETE),
// VOL=(,,,99),
// BUFNO=50,
// UNIT=CART,
// DCB=(LRECL=796,RECFM=FB)
//SYSIN DD *
SORT FIELDS=COPY

Regards,
Fazil
Back to top
View user's profile Send private message
Alissa Margulies

SYNCSORT Support


Joined: 25 Jul 2007
Posts: 500
Location: USA

PostPosted: Fri Jan 23, 2009 9:41 pm    Post subject: Re: Reply to: IEBGENER vs SORT
Reply with quote

skkp2006,

Without knowing which of the 2 methods for installing BetterGener was done at your site, it appears that the LINKLIST library containing the entry IEBGENER comes before the IEBGENER alias/ICEGENER alias/SYNCGENR module. Therefore, when you execute IEBGENER, you are getting true IEBGENER. However, when you execute ICEGENER, you are getting the ICEGENER alias for SYNCGENR, which can use SyncSort's COPY feature.
Back to top
View user's profile Send private message
Douglas Wilder

Active User


Joined: 28 Nov 2006
Posts: 305
Location: Deerfield IL

PostPosted: Fri Jan 23, 2009 9:47 pm    Post subject:
Reply with quote

Did you check how long it took the operator to mount the tapes?
Back to top
View user's profile Send private message
Mistermind
Warnings : 2

New User


Joined: 08 Feb 2008
Posts: 46
Location: Dublin

PostPosted: Fri Jan 30, 2009 7:56 pm    Post subject:
Reply with quote

What is the blocksize in each situation,
the EXCP count,
the TCB count,
the SRC count on the SDSF joblog?
The MIPs of the test and Production cpu's?

Presumably your input is on disk. Try switch to UNIT=SYSDA instead of =CART on the output and watch the difference in peripheral performance.
Back to top
View user's profile Send private message
fazilbe

New User


Joined: 03 Dec 2008
Posts: 25
Location: INDIA

PostPosted: Wed Feb 04, 2009 12:24 pm    Post subject: Reply to: IEBGENER vs SORT
Reply with quote

Hi,

I found that difference in the run time between the test machine and prod machine is bound to happen because during the month end, prod machine seems to be very busy and necessary resource would not be available for the jobs at all the time. I learnt this issue on heavy traffic from one of the SME's.

Also the encouraging thing while using SORT is the reduction in CPU time. I saw 90% reduction in CPU time while using SORT. So i have got the approval to use the SORT copy technique for the job i was analysing.

Also could someone please let me know how to find whether the tapes being mounted quickly or being done with some delay?.

I will let you all know the updates on this issue very soon.

Regards,
Fazil
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> JCL & VSAM All times are GMT + 6 Hours
Goto page 1, 2  Next
Page 1 of 2

 

Search our Forum:

Similar Topics
Topic Author Forum Replies Posted
No new posts Limit duplicate records in the SORT pshongal SYNCSORT 6 Mon Nov 21, 2016 12:54 pm
No new posts How to convert the VBM file to VB or... Sulabh Agrawal JCL & VSAM 4 Fri Nov 18, 2016 1:04 pm
No new posts Sort records based on numeric field. Alks SYNCSORT 2 Wed Oct 19, 2016 10:14 pm
No new posts How to delete second instance from Fl... Gunapala CN DFSORT/ICETOOL 6 Tue Oct 18, 2016 11:42 pm
No new posts abend sort based on count records in ... anatol DFSORT/ICETOOL 5 Mon Oct 17, 2016 10:10 pm


Facebook
Back to Top
 
Mainframe Wiki | Forum Rules | Bookmarks | Subscriptions | FAQ | Tutorials | Contact Us