IBM Mainframe Forum Index
 
Log In
 
IBM Mainframe Forum Index Mainframe: Search IBM Mainframe Forum: FAQ Register
 

File Comparison in ICETOOL vs Cobol


IBM Mainframe Forums -> DFSORT/ICETOOL
Post new topic   Reply to topic
View previous topic :: View next topic  
Author Message
shaktiprasad

New User


Joined: 20 Aug 2006
Posts: 34
Location: chennai

PostPosted: Mon Aug 01, 2011 8:41 pm
Reply with quote

Hi ,

Just a clarification. When we do some file comparison based on some keys. If we have billions of records then which will be better in performnce ICETOOL or Cobol ?
I know ICETOOL will easier to code but performnce wise need to decide also. I will be using SPLICE as I need to overwrite few fields based on some condition.


Thanks
Shakti
Back to top
View user's profile Send private message
dick scherrer

Moderator Emeritus


Joined: 23 Nov 2006
Posts: 19244
Location: Inside the Matrix

PostPosted: Mon Aug 01, 2011 8:57 pm
Reply with quote

Hello,

Suggest you look at DFSORT's JOINKEYS function.
Back to top
View user's profile Send private message
sqlcode1

Active Member


Joined: 08 Apr 2010
Posts: 577
Location: USA

PostPosted: Mon Aug 01, 2011 9:10 pm
Reply with quote

shaktiprasad,
Other than looking at JOINKEYS, try with few million records first and see which one is more efficient. You should get answer by yourself.

If you do, cobol file comparison,wouldn't Cobol need both files "pre-sorted" on key fields?

Thanks,
Back to top
View user's profile Send private message
dick scherrer

Moderator Emeritus


Joined: 23 Nov 2006
Posts: 19244
Location: Inside the Matrix

PostPosted: Mon Aug 01, 2011 9:16 pm
Reply with quote

Hello,

No matter how you do this, you need to make sure the input is sorted first. . .

If the volume is too great, the inputs can be split by ranges and then compared. Depending on just how much data there is (lrecl * nbr-recs) you may not be able to find enough work dasd. . .
Back to top
View user's profile Send private message
PeterHolland

Global Moderator


Joined: 27 Oct 2009
Posts: 2481
Location: Netherlands, Amstelveen

PostPosted: Mon Aug 01, 2011 9:41 pm
Reply with quote

Quote:

billions of records


Really?
Back to top
View user's profile Send private message
Bill Woodger

Moderator Emeritus


Joined: 09 Mar 2011
Posts: 7309
Location: Inside the Matrix

PostPosted: Mon Aug 01, 2011 9:46 pm
Reply with quote

I'd say that for the I/O DFSORT/ICETOOL for a few billion records would knock Cobol into a cocked hat. Hare vs Tortoise, without any danger of the Hare going to sleep.

If you look at the Cobol manual for compiler option FASTSRT (or similar name), you get the idea if you want the faster processing of the files, let Sort do the IO.

For such volumes, you really want to get it right first time, and be nice and stable, to avoid lost time due to silly failures.

If you give us the information about the rest of the processing, I'm sure you can get some very good advice.

As sqlcode1 has said, get yourself some decent-sized, and representative, test files. For this task, performance is going to be important to you.

If some of the processing is too complex, there is always the option of "Sort Exit"s, modules which you write, can be in Cobol, and are called by the sort itself (saving Cobol reading the file(s) again).
Back to top
View user's profile Send private message
sqlcode1

Active Member


Joined: 08 Apr 2010
Posts: 577
Location: USA

PostPosted: Mon Aug 01, 2011 9:47 pm
Reply with quote

shaktiprasad,
JOINKEYS has been available for DFSort for quite a some time now. Read JOINKEYS for DFSort here

Quote:
No matter how you do this, you need to make sure the input is sorted first. . .

Dick,
DFSORT can sort and join the files in the same step. Input files doesn't have to pre-sorted as an input to DFSort JOINKEYS.

With Cobol, unless OP is using internal sort for this huge volume, he would need separate steps to sort both the input files on key fields first and then feed these 2 files in cobol, whereas with DFSort Joinkey, he doesn't have to do this in separate steps as JOINKEYS would sort the keys and match the files in a single step.

However, if I am to process files with "billions of records" and using JOINKEYS for match, I would go with separate sort steps each files and follow it up with SORTED,NOSEQCK in JOINKEYS.

Thanks,
Back to top
View user's profile Send private message
dick scherrer

Moderator Emeritus


Joined: 23 Nov 2006
Posts: 19244
Location: Inside the Matrix

PostPosted: Mon Aug 01, 2011 10:07 pm
Reply with quote

Hi Dick,

I was somewhat surprised that the organization would even allow someone with so little experience to implement this process that has the potential to dim the lights in the data center. . .

My personal approach to monsters like this is to break them into more managable pieces - both for my sanity and to be a considerate corporate citizen on the system. . .

d
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic View Bookmarks
All times are GMT + 6 Hours
Forum Index -> DFSORT/ICETOOL

 


Similar Topics
Topic Forum Replies
No new posts How to split large record length file... DFSORT/ICETOOL 10
No new posts Replace each space in cobol string wi... COBOL Programming 3
No new posts Extracting Variable decimal numbers f... DFSORT/ICETOOL 17
No new posts COBOL -Linkage Section-Case Sensitive COBOL Programming 1
No new posts SFTP Issue - destination file record ... All Other Mainframe Topics 2
Search our Forums:

Back to Top