Joined: 23 Nov 2006 Posts: 19270 Location: Inside the Matrix
In one word - everything.
Every line of code, every field on a map, every entry on a report, every field in every file and database table, every bit of data to be downloaded, every message that will ever be presented by the system needs to be tested. Every interface to other systems must be tested. Every backup/restore/recovery procedure must be verified.
There is nothng built as part of a project that should not be thoroughly, successfully tested.
Joined: 06 Jun 2008 Posts: 8165 Location: East Dubuque, Illinois, USA
I like black box testing, where you take somebody who knows nothing about the system, set them in front of the screen, and see how long it takes them to blow up the system using the documentation. For an online system, under a minute isn't that rare.
With white box testing the problem is ensuring you're testing all possible data paths. I did automatic test case generation for a batch system one time and wound up generating over 22,000 test records -- but did they ever test that system completely!
In the best possible world, the tester and developer are both given the specs and independently do their thing, one creating test cases and the other creating the system. The test cases may not match what the system does, but the discussions about why the differences are always illuminating.
Joined: 22 Apr 2006 Posts: 6258 Location: Mumbai, India
Probably it's a never ending discussion. Testing would differ from phase to phase - I had a "Regression Testing' in one project & the testing reached to 12000+ test conditions, there were three systems involved in that project & 12000+ were only for my system..! As robets has said - but did they ever test that system completely..!
I belieave there should already be a "policy" in use at your shop for this, but if this is first time you/your shop thought aboput this - I would suggest start with general scenarios of your production so that you are sure at least "BAU" (Business As usual) is not distrubed, once that is done move on to more rare (& more complex) production scenarios..& yes as PeD said in case you have experience with other technologies- aprroach doesn't differ with COBOL.