100% code coverage? Why not?
In a recent post entitled “What I Believe About Testing” I wrote “It is rare that 100% code coverage is necessary”. This may have caught people off guard — that a test professional doesn’t believe in 100% code coverage. Understand why…
Cost vs Benefit
I’m a strong proponent of Test Driven Development. Not every one is. Lots of code has no tests. The effort in these cases to get to 100% code coverage is extremely high for even the simplest application. With greenfield applications, teams tie testing into the process giving a higher percentage of code coverage. Even when that is the case, there will be code that is low risk. There will be code that is extremely difficult to write tests for. There will be code that is both low risk and difficult to test. In those situations we must weigh the cost of coverage vs the benefit. If not, we’re doing the business a disservice.
My work with clients has shown me over the years that this line differs from one team to the next and necessarily so.
Much of today’s code is generated or boilerplate. Do we really need tests for simple getters and setters that do nothing more than get or assign values of properties?In ORMs that generate code like Hibernate and Entity Framework, do we need to test the code generated by the framework? My personal rule is, unless I have a reason to distrust boilerplate or generated code, I don’t write tests to cover them.
There comes a time in many projects where we have to look at opportunity cost — the cost of missed opportunities due to our attention on current actions. This is a real thing, not just biz dude lingo. It’s easy to think as a tester or developer that quality is the most important aspect of a project or that refactoring a major subsystem is priority. The truth is for most organizations their goals are tied tightly to the market and what the competition is doing. That tie is about survival. Remember, if the company doesn’t survive, you’re looking for a job. What is the opportunity cost for seeking 100% coverage?
Much like the maintenance of software applications, it’s rare that teams plan for the labor required to maintain tests over time. The closer we get to 100% coverage the closer we get to unaffordable maintenance costs.
Don’t get me wrong. I love coverage. I love high coverage. It works, it makes sense as a mechanism to insure against regressions. But like most endeavors, thoughtful approaches tend to generate better results.