Portal | Manuals | References | Downloads | Info | Programs | JCLs | Master the Mainframes
IBM Mainframe Computers Forums Index
 
Register
 
IBM Mainframe Computers Forums Index Mainframe: Search IBM Mainframe Forum: FAQ Memberlist Usergroups Profile Log in to check your private messages Log in
 

 

Query regarding Data Archival & Retrieval ?

 
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> DB2
View previous topic :: :: View next topic  
Author Message
leo_sangha

New User


Joined: 11 Aug 2005
Posts: 85
Location: England

PostPosted: Wed May 23, 2007 6:29 pm    Post subject: Query regarding Data Archival & Retrieval ?
Reply with quote

Hi,

I have a requirement of creating new DB2 Tables(Offcourse this will be created by the DBA). Now in these DB2 tables around 10 million records will come on a daily basis.
We have a requirement originating because of this mentioned below.
1) The number of tables to be archived is less(around 10) but the volume is VERY HIGH.
2) We want to keep this archived DATA for 2 YEARS(OFFLINE) and keep the last 6 months data (ONLINE on DASD's ).
3) Now business wants to retrieve this data ON DEMAND basis for security reasons.

Questions:
------------
1) Can i implement some Archival/Retrieval strategy MVS environment, if YES, Can you please let me know which TOOL/UTILITY can i use to ARCHIVE the Data?
2) What sort of Architecture can i propose which will help in retrieval of historical data rapidly(In terms of where the Data Archived should reside)?

Any sort of Inputs will be helpful. Please do share your experiences.

Thank
Back to top
View user's profile Send private message

dick scherrer

Site Director


Joined: 23 Nov 2006
Posts: 19270
Location: Inside the Matrix

PostPosted: Wed May 23, 2007 7:24 pm    Post subject:
Reply with quote

Hello,

Before you begin design, i'd suggest you get a definition of just what is "rapidly". That may mean quite different things to different people in your organization.

I'm unsure how the "on demand" requirement relates to security. Clarification would help.

How much of the archived data might be needed for an "on demand" process? Something to consider is that the smaller 6 months of data will be about 1,800,000,000 rows. The larger (2 years or 1.5 years) will be 3-4 times that many rows. I suspect that there is no really rapid (again, depending on the definition of rapid) way to reload over a billion rows - and that is the smaller case.

Your organization may have a different theory, but most of the places i support operate on the theory that their databases are just as (if not more) secure than flat files.

If your organization is willing to invest in a large volume of historical data, using one or more read-only databases may accomplish what you want. Keep in mind that the read-only databases could be defined with many alternate indexes to support a variety of "on demand" requests for data.
Back to top
View user's profile Send private message
leo_sangha

New User


Joined: 11 Aug 2005
Posts: 85
Location: England

PostPosted: Thu May 24, 2007 12:42 pm    Post subject: Re: Query regarding Data Archival & Retrieval ?
Reply with quote

Hello Dick,

Thank you for your response. I have tried to provide some explanations to your queries, please do read.
First of all 'rapidly' is not the correct word(thanks for pointing it out)... they are more intersted in architecture part and then looking for more cost effective solution to this problem.
We can push the request for Archived Data by business into Batch process also.

There are some transactions being carried out by the Back End staff, and due to recent SOX compliance directives, my organization want to track those(infact all) trasactions now. I hope this clarifies the word security reasons. Just to prevent frauds etc..

In the worst case, the request for online+offline data could even be for 2.5 years(maximum). But i guess even in this case the resultset will not be billion rows, infact i suppose it is going to be significantly less
(around 1000) as it is based on some conditional inputs from business.

Yes you are correct we don't need to update the archived DB, could you please clarify what is meant by READ ONLY Database?

Could you share with me what sort of Architecture can i propose for this situation? And you mentioned something about the Index to be built on the Archived Data, though i have some understanding about indexes, Could you please throw more light on how multiple indexes could be used in this case?

Thanks
Back to top
View user's profile Send private message
View previous topic :: :: View next topic  
Post new topic   Reply to topic    IBMMAINFRAMES.com Support Forums -> DB2 All times are GMT + 6 Hours
Page 1 of 1

 

Search our Forum:

Similar Topics
Topic Author Forum Replies Posted
No new posts copying data without knowing location arunsoods DFSORT/ICETOOL 6 Thu Jul 20, 2017 1:03 pm
This topic is locked: you cannot edit posts or make replies. Fetching data from BAI File arunsoods JCL & VSAM 1 Wed Jul 19, 2017 4:28 pm
No new posts Loading data to table gives wrong for... Raghu navaikulam DB2 19 Thu Jul 13, 2017 2:11 pm
No new posts SQL - select data available in index Nileshkul DB2 3 Mon Jun 26, 2017 1:30 am
No new posts NDM syntax checking without actually ... GAPX1 All Other Mainframe Topics 0 Wed Jun 07, 2017 2:36 am


Facebook
Back to Top
 
Mainframe Wiki | Forum Rules | Bookmarks | Subscriptions | FAQ | Tutorials | Contact Us