|
View previous topic :: View next topic
|
| Author |
Message |
Biswajit D
New User
Joined: 17 Apr 2012
Posts: 50
Location: India
|
|
|
|
Hi,
I searched the forum but could not get anything (not sure if I searched properly). So I would need a little help in running File Manager compare for a VB file.
Here is my problem:
I have two datasets I want to compare using File Manager compare utility. But no matter what option I give during comparison, the utility does not compare the datasets. After wasting almost a day on this I thought about putting this up in the forum.
Here are the details:
Below is the dataset attribute for the files being compared:
| Code: |
General Data Current Allocation
Management class . . : L0015XX1 Allocated megabytes : 380
Storage class . . . : STANDARD Allocated extents . : 5
Volume serial . . . : DTD010
Device type . . . . : 3390
Data class . . . . . : DLTDATA
Organization . . . : PS Current Utilization
Record format . . . : VB Used megabytes . . : 380
Record length . . . : 17000 Used extents . . . : 5
Block size . . . . : 27998
1st extent megabytes: 45
Secondary megabytes : 90 Dates
Data set name type : Creation date . . . : 2012/05/28
SMS Compressible. . : NO Referenced date . . : 2012/05/29 |
Both files contain 147097 records
Below is how my JCL looks like:
| Code: |
//FILEMGR EXEC PGM=FMNMAIN
//SYSPRINT DD SYSOUT=*
//FMNTSPRT DD SYSOUT=*
//SYSTERM DD SYSOUT=*
//SYSIN DD *
$$FILEM SET HEADERPG=YES,PAGESIZE=60,PRTTRANS=ON
$$FILEM DSCMP TYPE=FORMATTED,
$$FILEM PACK=UNPACK,
$$FILEM SYNCH=ONETOONE,
$$FILEM LIST=DELTA,
$$FILEM HILIGHT=YES,
$$FILEM CHNGDFLD=YES,
$$FILEM IGNLEN=NO,
$$FILEM NUMDIFF=ALL,
$$FILEM PBK=LAYOUTS,
$$FILEM DSNOLD=FileOld,
$$FILEM TCOLD=LayoutA,
$$FILEM LANG=COBOL,
$$FILEM CBLMAXRC=04,
$$FILEM SKIPOLD=0,
$$FILEM CMPOLD=ALL,
$$FILEM TCNEW=LayoutA,
$$FILEM SKIPNEW=0,
$$FILEM CMPNEW=ALL,
$$FILEM DSNNEW=FileNew
/*
|
This job gives a return code 4 with the below output
| Code: |
IBM File Manager for z/OS
Comparison summary:
Old data set records processed: 0
New data set records processed: 0
Matching records found: 0
Changed records: 0
Old records not selected: 147097
New records not selected: 147097
Old records not compared: 0
New records not compared: 0
Old data set records deleted: 0
New data set records inserted: 0
Synchronization: One-to-one
Comparison type: Formatted
Listing type: Delta
Listing options: Highlight changes
Show changed fields only
Template reporting options: Show template layouts
IBM File Manager for z/OS
|
I ran a 3.13 and it shows difference between the two files but since I want the field which has a change in value (coz I dont know what position data has changed), 3.13 does not serve me a purpose. Here is what 3.13 has to say:
| Code: |
LINE COMPARE SUMMARY AND STATISTICS
147095 NUMBER OF LINE MATCHES 2 TOTAL CHANGES (PAIRED+NONPAIRED
0 REFORMATTED LINES 2 PAIRED CHANGES (REFM+PAIRED INS
2 NEW FILE LINE INSERTIONS 0 NON-PAIRED INSERTS
2 OLD FILE LINE DELETIONS 0 NON-PAIRED DELETES
147097 NEW FILE LINES PROCESSED
147097 OLD FILE LINES PROCESSED
LISTING-TYPE = CHNG COMPARE-COLUMNS = 1:16996 LONGEST-LINE = 8044
PROCESS OPTIONS USED: NONE
THE FOLLOWING PROCESS STATEMENTS (USING COLUMNS 1:72) WERE PROCESSED:
CMPCOLM 1:27000
ISRS004I LISTING LINES MAY BE TRUNCATED DUE TO LIMITING OUTPUT LINE WIDTH.
|
Any help or a tutorial on how to use file manager compare utility would be deeply appreciated.
Thanks,
Biswajit |
|
| Back to top |
|
 |
Anuj Dhawan
Superior Member
Joined: 22 Apr 2006
Posts: 6250
Location: Mumbai, India
|
|
|
|
| Do you get only the summry after 3.13? What "Listing Type" are you using while comparison? |
|
| Back to top |
|
|
Biswajit D
New User
Joined: 17 Apr 2012
Posts: 50
Location: India
|
|
|
|
| Listing Type is CHNG. |
|
| Back to top |
|
|
Biswajit D
New User
Joined: 17 Apr 2012
Posts: 50
Location: India
|
|
|
|
To add to my confusion, I ran the same job but now on another output file with the below characteristics and it gave me the correct output:
Characteristics of files being compared:
| Code: |
General Data Current Allocation
Management class . . : L0015XX1 Allocated megabytes : 1
Storage class . . . : STANDARD Allocated extents . : 1
Volume serial . . . : DTD001
Device type . . . . : 3390
Data class . . . . . : DLTDATA
Organization . . . : PS Current Utilization
Record format . . . : VB Used megabytes . . : 1
Record length . . . : 16390 Used extents . . . : 1
Block size . . . . : 27998
1st extent megabytes: 1
Secondary megabytes : 90 Dates
Data set name type : Creation date . . . : 2012/05/28
SMS Compressible. . : NO Referenced date . . : 2012/05/29
Expiration date . . : ***None*** |
Number of records in file: 1
Output of File Manager compare JCL:
| Code: |
IBM File Manager for z/OS
* * * * Formatted comparison with one-to-one synchronization
* * New data set: FileNew
* * using template "Template"
* * Old data set: FileOld
* * using template "Template"
C New data set record 000001 O Old data record 000001
*I-NEXT-RUN-DT 1120703 I-NEXT-RUN-DT 1120628
IBM File Manager for z/OS 5/29/2012 (2012.150) 02:42 Page 4
Comparison summary:
Old data set records processed: 1
New data set records processed: 1
Matching records found: 0
Changed records: 1
Old records not selected: 0
New records not selected: 0
Old records not compared: 0
New records not compared: 0
Old data set records deleted: 0
New data set records inserted: 0
Synchronization: One-to-one
Comparison type: Formatted
Listing type: Delta
Listing options: Highlight changes
Show changed fields only
Template reporting options: Show template layouts
IBM File Manager for z/OS |
Summary of 3.13:
| Code: |
LINE COMPARE SUMMARY AND STATISTICS
0 NUMBER OF LINE MATCHES 1 TOTAL CHANGES (PAIRED+NONPAIRED
0 REFORMATTED LINES 1 PAIRED CHANGES (REFM+PAIRED INS
1 NEW FILE LINE INSERTIONS 0 NON-PAIRED INSERTS
1 OLD FILE LINE DELETIONS 0 NON-PAIRED DELETES
1 NEW FILE LINES PROCESSED
1 OLD FILE LINES PROCESSED
LISTING-TYPE = CHNG COMPARE-COLUMNS = 1:16386 LONGEST-LINE = 4086
PROCESS OPTIONS USED: NONE
THE FOLLOWING PROCESS STATEMENTS (USING COLUMNS 1:72) WERE PROCESSED:
CMPCOLM 1:27000
ISRS004I LISTING LINES MAY BE TRUNCATED DUE TO LIMITING OUTPUT LINE WIDTH. |
Only thing I can think of is the LRECL difference... does FM compare have anything to do with LRECL? |
|
| Back to top |
|
|
Biswajit D
New User
Joined: 17 Apr 2012
Posts: 50
Location: India
|
|
|
|
I also tried to break the 147097 records in 3 files and compared the lesser number of records...still no success...
any help... please... |
|
| Back to top |
|
|
dick scherrer
Moderator Emeritus
Joined: 23 Nov 2006
Posts: 19244
Location: Inside the Matrix
|
|
|
|
Hello,
How would splitting the file into 3 help resolve the problem?
| Quote: |
| Only thing I can think of is the LRECL difference... does FM compare have anything to do with LRECL? |
If the files have unequal lrecls, they should always compare unequal (as far as i know - we don't compare using file manager).
Getting different results by changing files shows the process may be ok but the data may be inconsistent.
What happens if the compare is done using SuperC rather than file manager? |
|
| Back to top |
|
|
Akatsukami
Global Moderator
Joined: 03 Oct 2009
Posts: 1788
Location: Bloomington, IL
|
|
|
|
| I don't know from File Manager (yet, although my shop is planning to replace File-AID with it by year's-end. Now if someone here would give me pointers to training materials...). However, I note that whilst your control cards seem to refer to layouts, those layouts don't seem to be available through any DD statement or control card. Is that significant? |
|
| Back to top |
|
|
Bill Woodger
Moderator Emeritus
Joined: 09 Mar 2011
Posts: 7309
Location: Inside the Matrix
|
|
|
|
| There must be something in the Filemanager documentation which explains the output, and specifically what "not selected" is telling you. Presumably you are going to need to arrange to select them. |
|
| Back to top |
|
|
Ed Goodman
Active Member
Joined: 08 Jun 2011
Posts: 556
Location: USA
|
|
|
|
Oh geez, I got lost in this stuff for a while too.
Make sure the template you are using does NOT have any selection criteria. You can get those stuck in there accidentally just by going to that screen while setting up the template.
Also, make SURE the layout that was used to make the template/view is the right record length for the file. If it doesn't match, FM will skip it.
The best part is that once you get it all straightened out, FM is a great program. We switched over from another product, and we all hated it at first, but then I came to like it better than the old old one.
The biggest problem I've had with it is what you're seeing now: The templates are pre-compiled and NOT dynamic. So if you change a copybook, the template/view is just wrong. You can't tell it to "go out and grab that copybook at run time." |
|
| Back to top |
|
|
Biswajit D
New User
Joined: 17 Apr 2012
Posts: 50
Location: India
|
|
|
|
Dick,
I split the file into a few parts just to make sure that FM does not behave the way it is due to number of records to be compared. Since I could successfully compare another file having lesser number of records, I thought splitting and then comparing might give me the comparison.
Akatsukami,
When FM creates the JCL for comparison, it puts in the member path against TCOLD and TCNEW. I had mentioned LayoutA in my first post but in real it is something like this:
| Code: |
$$FILEM TCOLD=XX.Y.ABCDEF.PDS(MEMBER),
$$FILEM TCNEW=XX.Y.ABCDEF.PDS(MEMBER),
|
FM compare step has all the info it needs available in SYSIN and does not refer to any control card or DD name. 
Bill,
I just got hold of a document on FM. It says:
| Quote: |
Comparing data sets using templates
When you choose to use templates in your comparisons, you can specify an “Old” template only, a “New” template only or both “Old” and “New” templates.
"Old" template only or "New" template only
This excludes records not-selected in either the “Old” or the “New” data set.
During the compare, the Compare Utility uses the “Old” or “New” template to provide information about the selected record types, record identification criteria and record selection criteria for the “Old” and “New” data sets. Any field selection and field resequencing information in the templates is ignored, however, you can specify fields to be used in a multi-segment key that can be used with the Keyed Synchronization comparison option.
You determine which records are selected or excluded by editing the template and:
1 Selecting record types
2 Specifying record identification criteria
3 Specifying record selection criteria
Not-selected records:
1. do not match a record type in the template, due to either record identification criteria or record length;
2. match a record type, but belong to an unselected record type; or
3. do not meet the record selection criteria for the record type.
|
I am still trying to make out what the above means. 
Ed,
1. The template does not have any selection criteria.
2. The file I am comparing is written out of a COBOL program with the same layout that I am using. So, record length of layout = record length of the file.
3. I always compile the layout taking a fresh copy from production environment just to make sure that there are no selection criterias left behind.
Anyways, Thank You everyone for your inputs. I am still looking out for a solution though  |
|
| Back to top |
|
|
dick scherrer
Moderator Emeritus
Joined: 23 Nov 2006
Posts: 19244
Location: Inside the Matrix
|
|
|
|
Hello,
Did you try SuperC just to see what happens? |
|
| Back to top |
|
|
Ed Goodman
Active Member
Joined: 08 Jun 2011
Posts: 556
Location: USA
|
|
|
|
"So, record length of layout = record length of the file."
Not quite.
The FD has to match, the copybook does not. IF the copybook is in the FD, then it is correct, but if the copybook is in working storage, they can be different.
If the copybook is longer, records will be truncated. If the copybook is shorter, then the WRITE x from y will pad spaces during the implicit move.
You said you ran the "same job" on a smaller file, but that file also had a different LRECL.
To me, the biggest clue is that the output says "not selected". Try using IGNLEN=YES, that will ignore length mismatches. |
|
| Back to top |
|
|
Biswajit D
New User
Joined: 17 Apr 2012
Posts: 50
Location: India
|
|
|
|
| dick scherrer wrote: |
Hello,
Did you try SuperC just to see what happens? |
I already tried superc. I mentioned it in my first post too..
| Quote: |
I ran a 3.13 and it shows difference between the two files but since I want the field which has a change in value (coz I dont know what position data has changed), 3.13 does not serve me a purpose. Here is what 3.13 has to say:
| Code: |
LINE COMPARE SUMMARY AND STATISTICS
147095 NUMBER OF LINE MATCHES 2 TOTAL CHANGES (PAIRED+NONPAIRED
0 REFORMATTED LINES 2 PAIRED CHANGES (REFM+PAIRED INS
2 NEW FILE LINE INSERTIONS 0 NON-PAIRED INSERTS
2 OLD FILE LINE DELETIONS 0 NON-PAIRED DELETES
147097 NEW FILE LINES PROCESSED
147097 OLD FILE LINES PROCESSED
LISTING-TYPE = CHNG COMPARE-COLUMNS = 1:16996 LONGEST-LINE = 8044
PROCESS OPTIONS USED: NONE
THE FOLLOWING PROCESS STATEMENTS (USING COLUMNS 1:72) WERE PROCESSED:
CMPCOLM 1:27000
ISRS004I LISTING LINES MAY BE TRUNCATED DUE TO LIMITING OUTPUT LINE WIDTH.
|
|
SuperC does not help me in my case so i need to get the FM job going.
Ed,
Yes. The layout is used in FD so length should not be an issue. I will give a try with IGNLEN=YES. |
|
| Back to top |
|
|
Ed Goodman
Active Member
Joined: 08 Jun 2011
Posts: 556
Location: USA
|
|
|
|
In your last post, you can see the LRECL mismatch again.
"COMPARE-COLUMNS = 1:16996" vs
"COMPARE-COLUMNS = 1:16386" on the job that "worked" |
|
| Back to top |
|
|
Biswajit D
New User
Joined: 17 Apr 2012
Posts: 50
Location: India
|
|
|
|
Thats because I compared two different files... lemme break down what i did...
There is a program X which I have made changes to. This program writes 2 output files.. FileA and FileB.
LRECL of FileA=16996
LRECL of FileB=16386
Changes in FileB were intended and comparison of FileB before and after code changes showed me the changed field. But due to some reason , some field (i do not know which field or byte position) in FileA also got modified due my program changes. SuperC gives me the line in FileA where the unwanted changes are there but it does not give me the field. |
|
| Back to top |
|
|
dick scherrer
Moderator Emeritus
Joined: 23 Nov 2006
Posts: 19244
Location: Inside the Matrix
|
|
|
|
Hello,
I misread your first post - sorry 'bout that . . . 
Once the problem records (by key value - which is usually the first part of the record) have been identified, can they not be copied to a separate file and individually looked at with the editor (maybe HEX ON)?
If you have a set of these records from both files and split your screen so the same records are visable on both screens, you can scroll up/down (not so much as there are only 2<g>) and right/left and visually spot any differences. |
|
| Back to top |
|
|
Ed Goodman
Active Member
Joined: 08 Jun 2011
Posts: 556
Location: USA
|
|
|
|
So, when you tried with IGNLEN=Y the records were all still not selected?
This also means that it was NOT the "same" job as the one that worked.
I sense your frustration, I really do. I've been there, done that. I recommend starting from scratch and setting up a new template/view and using that. Don't use anything that you already have in place, just start over.
Setting up a new compare should take about fifteen minutes max. That time is TOTALLY justified by the amount of time some little mistake is costing you. |
|
| Back to top |
|
|
Biswajit D
New User
Joined: 17 Apr 2012
Posts: 50
Location: India
|
|
|
|
Dick,
I have already done that and i got the location where there is a change in data. But I would have preferred the FM job to have given me the info. I would have made life simpler.
Ed,
I repeated the FM compare process n number of times with all the permutation and combination. When I run with IGNLEN=Y, the comparison says that there is change in 2 records but it does not give me the field (same info that SuperC gave me).
I strongly feel that my layout needs to be looked into. It has some redefines in it and I suppose FM is not able to resolve the redefines. I will remove all the redefines and run the job tomorrow. I will keep you posted if it works.
Thanks everyone for all your inputs. |
|
| Back to top |
|
|
Biswajit D
New User
Joined: 17 Apr 2012
Posts: 50
Location: India
|
|
|
|
Sorry everyone for the late reply....
I compared my file using the old school method... 3.13 + manual compare of data in files...
file 1 with HEX ON(note the date in column 163 : 1120528)
| Code: |
Menu Utilities Compilers Help
sssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss
BROWSE XX.X.XXXXXXX.XXXXXXXX.XXXX.DT0528.V1 Line 00072236 Col 161 240
Command ===> Scroll ===> CSR
----+----7----+----8----+----9----+----0----+----1----+----2----+----3----+----4
----+----F----+----F----+----F----+----F----+----F----+----F----+----F----+----F
----+----7----+----8----+----9----+----0----+----1----+----2----+----3----+----4
------------------------------------------------------------------------------
....êð
00125844444444444444444444444444444444444444444444444444444444444444444444444444
CC102C00000000000000000000000000000000000000000000000000000000000000000000000000
------------------------------------------------------------------------------
....
44000044444444444444444444444444444444444444444444444444444444444444444444444444
00000C00000000000000000000000000000000000000000000000000000000000000000000000000 |
file 2 - data at column 163 = 1120627
| Code: |
Menu Utilities Compilers Help
sssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss
BROWSE XX.X.XXXXXXX.XXXXXXXX.XXXX.DT0528.V2 Line 00072236 Col 161 240
Command ===> Scroll ===> CSR
----+----7----+----8----+----9----+----0----+----1----+----2----+----3----+----4
----+----F----+----F----+----F----+----F----+----F----+----F----+----F----+----F
----+----7----+----8----+----9----+----0----+----1----+----2----+----3----+----4
------------------------------------------------------------------------------
....Â@
00126744444444444444444444444444444444444444444444444444444444444444444444444444
CC102C00000000000000000000000000000000000000000000000000000000000000000000000000
------------------------------------------------------------------------------ |
to conclude... I could not compare this file using FM with my layout...  |
|
| Back to top |
|
|
Biswajit D
New User
Joined: 17 Apr 2012
Posts: 50
Location: India
|
|
|
|
| is there any option in FM (or any other utility) that would compare 2 files and give the byte position at which it detects data mismatch??? |
|
| Back to top |
|
|
Bill Woodger
Moderator Emeritus
Joined: 09 Mar 2011
Posts: 7309
Location: Inside the Matrix
|
|
|
|
| Biswajit D wrote: |
[...]
file 1 with HEX ON(note the date in column 163 : 1120528)
[...]
file 2 - data at column 163 = 1120627
[...]
to conclude... I could not compare this file using FM with my layout... |
The data (date) is different, so it is nothing to to with using your record-layout. You need to check why there is a difference, the point of doing the compare, isn't it, to look for differences?
Isn't there any "masking" available? |
|
| Back to top |
|
|
Biswajit D
New User
Joined: 17 Apr 2012
Posts: 50
Location: India
|
|
|
|
The whole point of starting this post was to find out this difference using FM. but since FM did not give me the comparison (i mean the field that has change in data), i compared using SuperC. I knew that there is difference somewhere in the record but was not sure of the byte position where there is change. .
it is for this reason (I suspect this would be a very common scenario that people would be facing), that I was looking out for any utility that would give me the byte position where data changes in the datasets being compared. |
|
| Back to top |
|
|
dick scherrer
Moderator Emeritus
Joined: 23 Nov 2006
Posts: 19244
Location: Inside the Matrix
|
|
|
|
Hello,
It is fairly easy to code a program do the byte-by-byte compare and mark the differences. All you need is the input files defined as 1-byte arrays (in WS) and a third array of the same length for the "markers".
First, compare the entire record and if equal get the next records.
If not equal, loop thru both files byte-by-byte and moving a "marker" to the 3rd array at that position (i use an asterisk).
When the loop concludes, print all 3 lines (in hex preferably) and i'd suggest the output be written into a file or have the process terminate after some small number of differences found. |
|
| Back to top |
|
|
Ed Goodman
Active Member
Joined: 08 Jun 2011
Posts: 556
Location: USA
|
|
|
|
I know FM can do exactly what you are trying to do. I know because I use it for the same thing.
The newest piece of info you stated was that there is a redefines in the layout. Are you saying the difference that you know exists is in the redefined area?
Here's where you are losing me: You insisted that the lengths were correct between the file and the copybook. Then you got "not selected" for the entire file. When you turned on the "ignore length" option, suddenly you see two records changed. THIS IS PROOF THAT THE COPYBOOKS ARE NOT THE SAME AS THE FILE.
With the PROOF in mind, take a closer look at the copybooks and the views you have created with them. Maybe the redefined area is smaller than the full record?? Maybe??
If you have different record layouts for different record types, you'll have to tell FM how to match them up with the copybook areas. |
|
| Back to top |
|
|
Biswajit D
New User
Joined: 17 Apr 2012
Posts: 50
Location: India
|
|
|
|
FM compare is a really cool functionality...really really like it... although this time I did nt get my desired result (verified that the layout I used had too many redefines and also not correct in the sense that the byte that had changes was a FILLER  )...
but still... if there would have been an option in FM that would only tell the byte where difference is....
whenever time permits...will code the byte-by-byte comparison program... Thanks Dick for the suggestion... |
|
| Back to top |
|
|
|
|
