Tuesday, August 13, 2019

A Successful Test Finds An Error

I have often heard a programmer or tester gleefully declare, "Great news! My test was successful. The program ran correctly." This is the wrong attitude to have when running a test. It also supports the position that programmers should never be the sole testers of their own software. A more constructive attitude is that one is testing to find errors. Thus, a successful test is one that detects an error. Look at the analogous situation with a medical test. Suppose you are feeling ill. The doctor sends a sample of your blood to a laboratory. A few days later, the doctor calls to tell you, "Great news! Your blood was normal." That is not great news. You are sick or you wouldn't have gone to the doctor. A successful blood test reports what's wrong with you. The software has [the potential for] bugs. A successful test reports how these bugs manifest themselves. When generating test plans, you should first select tests based on the likelihood that they will find faults. Only after those tests are written and ran should you write tests for the sake of code coverage.


Reference
Goodenough, J., and Gerhard, S., "Toward a Theory of Test Data Selection," IEEE Transactions on Software Engineering, June 1975.