IBM Mainframe Forum Index
 
Log In
 
IBM Mainframe Forum Index Mainframe: Search IBM Mainframe Forum: FAQ Register
 

Improve performance of file comparison


IBM Mainframe Forums -> JCL & VSAM
Post new topic   This topic is locked: you cannot edit posts or make replies.
View previous topic :: View next topic  
Author Message
dick scherrer

Moderator Emeritus


Joined: 23 Nov 2006
Posts: 19244
Location: Inside the Matrix

PostPosted: Mon Oct 03, 2011 10:29 pm
Reply with quote

Hello,

Quote:
but as I mentioned earlier I cannot use SuperC
You simply must start paying attention. I have NOT suggested you use superc for what you need. Please stop repeating this . . . icon_sad.gif

You need to understand why either of your processes takes so very long. Obviously, the problem is on your system.

Why does your compare take so long? You seem be unwilling or unable determine this. How long does it take to simply copy the 2 files (no comparison at all)?
Back to top
View user's profile Send private message
Akatsukami

Global Moderator


Joined: 03 Oct 2009
Posts: 1788
Location: Bloomington, IL

PostPosted: Mon Oct 03, 2011 10:54 pm
Reply with quote

Now, I must come to Priyanka's defense here. Back on September 29, she said:
Quote:
SuperC also taking almost same amount of time

She's also posted her JCL and control cards.

I suspect that there's some problem with the data set DCBs, but I don't have the data sets, so I can't be sure. I'm not a File Manager wizard; can anyone suggest a situation (data sets are VBS? VSAM? VTOL?) which neither File Mangler nor SuperC handle well?

Priyanka, what is the record format and block size of the data sets? What are the EXCP counts on the input file DDs?
Back to top
View user's profile Send private message
dick scherrer

Moderator Emeritus


Joined: 23 Nov 2006
Posts: 19244
Location: Inside the Matrix

PostPosted: Mon Oct 03, 2011 11:07 pm
Reply with quote

Hi Akatsumi,

Quote:
Now, I must come to Priyanka's defense here.
I don't believe defending is necessary. . .

I do believe that pretty much all of the answers are on that system, to which we have no access. For us to help, Pryanka must be our ears, eyes, and hands icon_smile.gif

Once again i wonder how/why these 2 runs take almost the same amount of time on that system. I only posted my stats to show that superc runs exponentially more data in considerable less time. . .

I also believe a keyed 2-file match (with the copybook used as the basis for showing mis-matches) would perform better.
Back to top
View user's profile Send private message
Bill Woodger

Moderator Emeritus


Joined: 09 Mar 2011
Posts: 7309
Location: Inside the Matrix

PostPosted: Mon Oct 03, 2011 11:36 pm
Reply with quote

If both the comparisons are slow (FileManage and SurperC), as has been stated, then there must be something "odd" about the datasets or some big conflict between the control cards and the datasets.

dbz queried the backward reference to the sort step. Can you run with the actual known dcb info as a test, not do the backwards reference?

If that makes no difference, can you strip the FileManager cards down to the basics and see how that runs? If that is different, add the cards back a little at a time so you can locate what did it.

As has been requested, list DBB info for both files, and show EXCP info from the messages output.

Also, as dbz asked, how is the copybook getting into the comparison?

This thread is becoming typical of your queries. Long on length, short on... most things except length.
Back to top
View user's profile Send private message
dick scherrer

Moderator Emeritus


Joined: 23 Nov 2006
Posts: 19244
Location: Inside the Matrix

PostPosted: Tue Oct 04, 2011 12:11 am
Reply with quote

Hello,

The more i think about this, the more i'm convincing myself that the reason for the similarity in time used is because there are rather few mis-matched records.

Hopefully, the extra overhead is only used when there is a mis-match and the fomatted output is generated.

Which leads me to further believe that this job/class/media/etc is at the bottom of the barrel and only gets minimal resources. . .
Back to top
View user's profile Send private message
dbzTHEdinosauer

Global Moderator


Joined: 20 Oct 2006
Posts: 6966
Location: porcelain throne

PostPosted: Tue Oct 04, 2011 1:10 am
Reply with quote

unfortunately, the answer for the copybook question was given:

$$FILEM TCOLD=SYS2.xxxx.COPYLIB(zzzz),
$$FILEM TCNEW=SYS2.xxxx.COPYLIB(zzzz),

I don"t like the potental small blocksize on the output.
if there are many, many records that could have an effect.

boosting the AMP count on the vsam files could also help.
(but maybe file mananger is smart enough to buffer itself up as dfsort does)
and then again, a region size parm maybe holding back
the allocation of sufficient buffers to run,
what should be a compute bound process,
and making it an i/o bound processes.
i realize that we are sending the TS in several directions,
but as stated, we have no idea where to start.

having knowledge of the vsam process jesmsgs,
as well as a few important file attributes,
could possibly solve this problem, easily.
Back to top
View user's profile Send private message
sandeep.kumar

New User


Joined: 16 Aug 2021
Posts: 11
Location: India

PostPosted: Mon Aug 16, 2021 10:50 am
Reply with quote

Priyanka Pyne wrote:
Hi,

I wanted to understand the below, how did you implement it? Could you please share the complete code?

I have created one tool using REXX and JCL which will compare two mainframe files in copybook layout.I am using filemanager to compare the files. But itis taking long time to give the output result. Is there any way which can expedite the comparison?
Back to top
View user's profile Send private message
Rohit Umarjikar

Global Moderator


Joined: 21 Sep 2010
Posts: 3051
Location: NYC,USA

PostPosted: Mon Aug 16, 2021 5:50 pm
Reply with quote

Please don’t tailgate any older posts .. start your own new post .
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   This topic is locked: you cannot edit posts or make replies. View Bookmarks
All times are GMT + 6 Hours
Forum Index -> JCL & VSAM Goto page 1, 2  Next

 


Similar Topics
Topic Forum Replies
No new posts FTP VB File from Mainframe retaining ... JCL & VSAM 8
No new posts Extract the file name from another fi... DFSORT/ICETOOL 6
No new posts How to split large record length file... DFSORT/ICETOOL 10
No new posts Extracting Variable decimal numbers f... DFSORT/ICETOOL 17
No new posts SFTP Issue - destination file record ... All Other Mainframe Topics 2
Search our Forums:

Back to Top