Portal | Manuals | References | Downloads | Info | Programs | JCLs | Master the Mainframes
IBM Mainframe Computers Forums Index
 
Register
 
IBM Mainframe Computers Forums Index Mainframe: Search IBM Mainframe Forum: FAQ Memberlist Usergroups Profile Log in to check your private messages Log in
 

 

Choice of data loading - problem

 
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> DB2
View previous topic :: :: View next topic  
Author Message
kanisha_prabha

New User


Joined: 10 Mar 2006
Posts: 26

PostPosted: Sat Dec 05, 2009 6:52 pm    Post subject: Choice of data loading - problem
Reply with quote

Hi,
A particular DB2 table has around 20 million records. This table needs to be updated with around 10 million records which exist in a flat file. Some of the data in the file will be updates to the table (against inserts). Note that in case of updates, the file records must be updated to the existing table data. There are 2 ways to accomplish this -

1. Unload the table data to another flat file. Perform a file matching logic to check updates against inserts. Create one file containing, original un-affected table records, update records and new inserts. Then load replace or resume no the table.

2. Use fileaid with SQL INSERT option to directly update/insert into the table.

I also have to keep tab of how many records got updated and how many were inserted after the load is done.

Given the above scenario, which method should I use? Which method will be faster? If I am using method 2, can I capture the record count for updates and inserts and report the same? Are there any other method that is more suitable and I am ignoring?

Thanks.
Back to top
View user's profile Send private message

enrico-sorichetti

Global Moderator


Joined: 14 Mar 2007
Posts: 10211
Location: italy

PostPosted: Sat Dec 05, 2009 8:40 pm    Post subject: Reply to: Choice of data loading - problem
Reply with quote

use method one...
as a rule of the thumb when more than 30% of the <things> are processed
a sequential match approach will be faster

apart the issues related to locking, commits, backouts and so on
Back to top
View user's profile Send private message
dick scherrer

Site Director


Joined: 23 Nov 2006
Posts: 19270
Location: Inside the Matrix

PostPosted: Sat Dec 05, 2009 11:02 pm    Post subject:
Reply with quote

Hello,

Individual inserts are high-overhead activity. Every row of data and all of the indexes have to be handled for each insert.

If unload/reload is used (in addition to the potentially substantial processing improvement), the completely reloaded data will usually provide better performance when subequently used.
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> DB2 All times are GMT + 6 Hours
Page 1 of 1

 

Search our Forum:

Similar Topics
Topic Author Forum Replies Posted
No new posts Storing huge volume of data, compare ... Pradeep K M All Other Mainframe Topics 3 Mon Jan 16, 2017 5:08 pm
No new posts how to recover an uncataloged VSAM da... archanamuthukrishnan All Other Mainframe Topics 3 Wed Jan 11, 2017 6:18 pm
No new posts HALDB data refresh/copy from producti... vineetanand2007 IMS DB/DC 1 Mon Jan 02, 2017 11:16 am
No new posts SYMNAMES problem jacobdng DFSORT/ICETOOL 7 Thu Dec 22, 2016 7:47 am
No new posts JES2 JEC: Use UNIX Pipes to Pass Data... Virendra Shambharkar JCL & VSAM 21 Tue Dec 20, 2016 6:55 pm


Facebook
Back to Top
 
Mainframe Wiki | Forum Rules | Bookmarks | Subscriptions | FAQ | Tutorials | Contact Us