Portal | Manuals | References | Downloads | Info | Programs | JCLs | Master the Mainframes
IBM Mainframe Computers Forums Index
 
Register
 
IBM Mainframe Computers Forums Index Mainframe: Search IBM Mainframe Forum: FAQ Memberlist Usergroups Profile Log in to check your private messages Log in
 

 

idcams repro vs using syncsort

 
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> JCL & VSAM
View previous topic :: :: View next topic  
Author Message
raghavmcs

Active User


Joined: 14 Jul 2005
Posts: 105

PostPosted: Fri Jul 31, 2009 2:34 am    Post subject: idcams repro vs using syncsort
Reply with quote

Dear Experts,

I see there around more than 300 production file copy steps being used in production with my installation.
They all are using IDCAMS repro.most of them are unloading the VSAM file data to a flat file(mostly gdg).
Arounn 100 of the copy steps run during the critical prodcuction nightly update time.

Do you suggest of using the copy with syncsort(we have this with the current installation)

I gave a trail for copying using iceman.theere were 2899510 records in vsam file.The CPU time statsictics comparasion from idcams repro and sort were same!

Just thought to double check with you expert as I have read that SYNCSORt should preferable be used than to IDCAMS REPRO,thanks
Back to top
View user's profile Send private message

dick scherrer

Site Director


Joined: 23 Nov 2006
Posts: 19270
Location: Inside the Matrix

PostPosted: Fri Jul 31, 2009 7:00 am    Post subject:
Reply with quote

Hello,

Neither would use much cpu.

Check the performance of the 2 jobs. I suspect that Syncsort copy runs much better than the repro. . .
Back to top
View user's profile Send private message
expat

Global Moderator


Joined: 14 Mar 2007
Posts: 8593
Location: Back in jolly old England

PostPosted: Fri Jul 31, 2009 11:32 am    Post subject:
Reply with quote

You may wish to see if IDCAMS EXPORT is any different, and if your site has FAVER installed to see what that has to offer.
Back to top
View user's profile Send private message
Robert Sample

Global Moderator


Joined: 06 Jun 2008
Posts: 7995
Location: Bellevue, IA

PostPosted: Fri Jul 31, 2009 4:58 pm    Post subject:
Reply with quote

How do the EXCP counts compare? How many buffers are you using for each run? How do the service units compare for the two jobs?
Back to top
View user's profile Send private message
raghavmcs

Active User


Joined: 14 Jul 2005
Posts: 105

PostPosted: Fri Jul 31, 2009 10:04 pm    Post subject:
Reply with quote

EXCP COUNT FOR THE JOB WITH IDCAMS REPRO:4797
SERV UNITS FOR THE JOBS WITH IDCAMS REPORO:112K

FOR THE JOB WITH SORT COPY STEP

EXCP COUNT FOR THE JOB WITH IDCAMS REPRO:520
SERV UNITS FOR THE JOBS WITH IDCAMS REPORO:92089


THE OUTPUT GDG DETAILS ARE AS FOLLOWS
Record length: 80
Block size: 27920
Current Allocation:
Allocated cylinder 101
Allocated extents: 2


Current Utilization:
Used cylinders: 26
Used extents: 2

THE ABOVE STATISTICS IS FOR ONE FILE COPY THERE ARE 10 FILES IN TOTAL WHICH ARE GETTING COPIED USING IDCAMS REPRO.
Back to top
View user's profile Send private message
Douglas Wilder

Active User


Joined: 28 Nov 2006
Posts: 305
Location: Deerfield IL

PostPosted: Sat Aug 01, 2009 1:19 am    Post subject:
Reply with quote

CPU time the same.

EXCP COUNT FOR THE JOB WITH IDCAMS REPRO: 4797 Sort: 520
SERV UNITS FOR THE JOBS WITH IDCAMS REPRO:112K Sort: 92K

It looks like sort wins again.
Back to top
View user's profile Send private message
raghavmcs

Active User


Joined: 14 Jul 2005
Posts: 105

PostPosted: Sat Aug 01, 2009 2:38 am    Post subject:
Reply with quote

My choice of declaring winner would be based on the CPU time saving!!

But in this case the CPU time do not encourage me!!

But yes the EXCP and SERV are surely decreased!

The shop is not having FEVER

Please suggest!!
Thanks!!
Back to top
View user's profile Send private message
Robert Sample

Global Moderator


Joined: 06 Jun 2008
Posts: 7995
Location: Bellevue, IA

PostPosted: Sat Aug 01, 2009 6:20 am    Post subject:
Reply with quote

Copying files is not a high-CPU operation. It depends more upon the I/O and you reduced the EXCP count by 90% by using Sort over IDCAMS while the measure of total resources used, the service units, dropped by 18% using Sort over IDCAMS.

Not all workloads can be measured the same. IBM benchmarks machines by using different workloads because they realize a strongly I/O oriented workload may not look the same as a strongly computational workload. Measuring a copy job by the CPU used can be very misleading.
Back to top
View user's profile Send private message
dick scherrer

Site Director


Joined: 23 Nov 2006
Posts: 19270
Location: Inside the Matrix

PostPosted: Sat Aug 01, 2009 7:10 am    Post subject:
Reply with quote

Hello,

Quote:
My choice of declaring winner would be based on the CPU time saving!!
Suggest you come to a bit better understanding of which resource usage needs attention in different situations.

Quote:
Arounn 100 of the copy steps run during the critical prodcuction nightly update time.
In the case you posted, it sounds like the wall-time (elapsed) may be far more significant than cpu cycles. The process that most quickly copies the files may be the better alternative. I suspect that not only did the sort use only a fraction of the excp's, but it probably ran much more quickly as well. Keep in mind that not only did the sort use less resources, it caused less contention for i/o to other processes running at the same time.

On the other side, let's say there was a way to only use 25% of the current cpu usage, but the number of i/o's went up to 47,970 and the wall-time went to 2 hours for each copy. I suggest that this would be far less desirable even though it saved 75% of the current cpu usage.
Back to top
View user's profile Send private message
raghavmcs

Active User


Joined: 14 Jul 2005
Posts: 105

PostPosted: Mon Aug 03, 2009 9:04 pm    Post subject:
Reply with quote

Dear Experts,

Thanks to you alll for putting me on right track!!
I would still like to have your input for the following before actually going to proceed for the change
As I said there are more than 100 copy jobs using IDCAMS repro and all running during very critical time of nightly batch processing.

Will it be any benefitial for if they all use sort copy wherevr possible?

As of now the batch cycle runs for around 6 hours on and off depending on load and volume.

Is there any way by which the benefit can be measured/calculated/projected?
Thanks and Regards
Back to top
View user's profile Send private message
dick scherrer

Site Director


Joined: 23 Nov 2006
Posts: 19270
Location: Inside the Matrix

PostPosted: Mon Aug 03, 2009 11:37 pm    Post subject:
Reply with quote

Hello,

Quote:
Will it be any benefitial for if they all use sort copy wherevr possible?
Probably. . .
Quote:
Is there any way by which the benefit can be measured/calculated/projected?
Modify a few of the largest, run for a few cycles, and compare against the current usage.

If there are 100 of these and many of them run concurrently and they use the same volumes, they are quite possibly interfering with each other.
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> JCL & VSAM All times are GMT + 6 Hours
Page 1 of 1

 

Search our Forum:

Similar Topics
Topic Author Forum Replies Posted
No new posts VSAM RLS=NRI while doing IDCAMS sheersh JCL & VSAM 1 Tue Mar 07, 2017 1:55 pm
No new posts SYNCSORT SEQNUM and FI fields. nartcr SYNCSORT 7 Sat Jan 21, 2017 4:02 am
No new posts Syncsort Help to group fields sudhakarraju SYNCSORT 6 Thu Dec 29, 2016 1:38 am
No new posts syncsort: copy lines after the keyword shreya19 SYNCSORT 7 Fri Dec 02, 2016 9:47 am
No new posts Syncsort - Hard coded Value in output PORYES SYNCSORT 1 Thu Nov 03, 2016 9:23 am


Facebook
Back to Top
 
Mainframe Wiki | Forum Rules | Bookmarks | Subscriptions | FAQ | Tutorials | Contact Us