For comparison of files which utility is best

obulisankar · New User Joined: 03 May 2007 Posts: 20 Location: bangalore

Hi all,

I have a doubt here, for comparing 2 or more files, which utility is best in terms of performance, space.....

I have a confusion here to use either ICETOOL or SORT utility.

Any suggestions / points could be of great help...

Thanks in Advance....

Skolusu · Posted: Wed Sep 10, 2008 10:26 pm

obulisankar,

It depends. Various factors like LRECL, RECFM, volume of records, no: of duplicates, the length of the fields to be compared etc.. influence the performance. Please provide us with a sample input and desired output with the DCB properties of all files involved along with the rules for comparison

Kolusu

obulisankar · New User Joined: 03 May 2007 Posts: 20 Location: bangalore

Kolusu,
Here is the information you asked for:

LRECL --> 160
RECFM --> FB
Volume of records --> will be in millions
No. of duplicates --> keep varying
No. of keys used
for comparision --> 1
Lenghth of
key field --> 10
Position of Key --> 147

The DCB properties are same for all the input and output files.

I have 3 files: File1 , File2 , File3 . All of these files contain the records of the same type.

I need to compare these 3 files and get 2 output files.

Output File1 : Should contain the matching records from File1 and File2 and non-matching records from File3.

Output File2 : Should contain all the matching records from 3 files.

Example:
--------

File1:
-----
AAA BBB 100 DDD
EEE FFF 200 ccc
DDD NNN 300 AAA
ZZZ NNN 300 AAA

File2:
------
AAA BBB 100 DDD
CCC FFF 600 EEE
XXX JJJ 400 VVV
ZZZ NNN 300 AAA

File3:
------
AAA BBB 100 DDD
CCC FFF 200 GGG
EEE JJJ 500 KKK

Output File1:
--------------
DDD NNN 300 AAA
ZZZ NNN 300 AAA

Output File2:
-------------
AAA BBB 100 DDD

Thanks in advance

Skolusu · Posted: Thu Sep 11, 2008 9:49 pm

obuli sankar,

How do you want to handle duplicates? Do you need the duplicate records in the output when a match is found? Lets say file 1 has 5 duplicates for the key AAA BBB 100 DDD . It is a matching key in all the 3 files. Do you need to see the 5 records in output or just one record?

Kolusu

obulisankar · New User Joined: 03 May 2007 Posts: 20 Location: bangalore

Yes. Its like we are doing File1 - File2. When comparing 2 files File1 and File2 only the duplicates from the first file should be retained.

Thanks

obulisankar · New User Joined: 03 May 2007 Posts: 20 Location: bangalore

I want all the duplicates from the first file to be in the output file.

Skolusu · Posted: Fri Sep 12, 2008 9:26 pm

obulisankar,

The following DFSORT/ICETOOL JCL will give you the desired results.

obulisankar · New User Joined: 03 May 2007 Posts: 20 Location: bangalore

Kolusu,

Thanks for your reply, but still according to our requirements the volume of records in the datasets is very huge.. My intention is to reduce Temprary files as much as possible..

Will it be possible to have only one "Temporary Dataset" for comparison of 3 files.

By copying records from File1 to T1 and marking as "AAA"
By copying records from File2 to T1 and marking as "BBB"
By copying records from File3 to T1 and marking as "CCC"

Something of this will be really helpful for me. Actually i'm able to do comparison for 2 Files using only 1 Temporary file.

Thanks in Advance

Skolusu · Posted: Fri Sep 12, 2008 11:03 pm

ap_mainframes · Posted: Mon Sep 15, 2008 10:53 pm

obulisankar,

Do you have any other option than ICETOOL ?
I think normal DFSORT cant handle two input files anytime ! And in your example you are doing of file balancing.

Any corrections are more than welcome !

Skolusu · Posted: Mon Sep 15, 2008 11:29 pm

Craq Giegerich · Posted: Mon Sep 15, 2008 11:34 pm

ap_mainframes · Posted: Mon Sep 15, 2008 11:45 pm

Skolusu,

Thats what I meant that there is no other option than ICETOOL.
To my knowledge, Sort cant handle two input files.

Frank Yaeger · Posted: Wed Sep 17, 2008 2:23 am

ap_mainframes · Posted: Mon Sep 22, 2008 6:15 am

Frank,

I am under impression that when we do PGM=SORT, DFSORT is invoked.
I may be wrong though. If I am, Can you expain what is a diffrence between PGM=SORT and PGM=ICEMAN ?
I think, when you do PGM=SORT you cant handle two files. thats what I meant. Correct me if I am worng.

Thanks,

dick scherrer · Posted: Mon Sep 22, 2008 6:27 am

Hello,

obulisankar · New User Joined: 03 May 2007 Posts: 20 Location: bangalore

Frank,

I'm new to ICEMAN, My intention is to compare two or more files using ICEMAN (the constraint is only one temporary file should be used).

Below is the information:

LRECL --> 160
RECFM --> FB
Volume of records --> will be in millions
No. of duplicates --> keep varying
No. of keys used for comparision --> 1
Lenghth of key field --> 10
Position of Key --> 147

The DCB properties are same for all the input and output files.

I have 3 files: File1 , File2 , File3 . All of these files contain the records of the same type.

I need to compare these 3 files and get 2 output files.

Output File1 : Should contain the matching records from File1 and File2 and non-matching records from File3. [(File1 & File2) - File3]

Output File2 : Should contain all the matching records from 3 files.
[File1 & File2 & File3]

Example:
--------

File1:
-----
AAA BBB 100 DDD
AAA BBB 100 DDD
EEE FFF 200 CCC
DDD NNN 300 AAA
ZZZ NNN 300 AAA

File2:
------
AAA BBB 100 DDD
CCC FFF 600 EEE
DDD NNN 300 AAA
ZZZ NNN 300 AAA

File3:
------
AAA BBB 100 DDD
CCC FFF 200 GGG
EEE JJJ 500 KKK

Output File1:
--------------
DDD NNN 300 AAA
ZZZ NNN 300 AAA

Output File2:
-------------
AAA BBB 100 DDD
AAA BBB 100 DDD

Thanks in advance

dick scherrer · Posted: Mon Sep 22, 2008 7:42 pm

Hello,

Is there some reason you posted all of this again?

When you asked last week, you were given an answer from a DFSORT developer:

Frank Yaeger · Posted: Mon Sep 22, 2008 9:04 pm

PGM=ICEMAN and PGM=SORT are identical - both invoke DFSORT. Read my previous comments as "If you're talking about PGM=ICEMAN or PGM=SORT (DFSORT) ...".