Pros and Cons of Coverage

Most executives in software building companies stress the need for code coverage during testing. The reasoning is simple; the more the code is exercised before deployment, the less chance of customers finding issues and complaining. Having this metric is a great thing to have in any software development that will help teams become as efficient as possible to get to a quality product for customers.

As this is something that has extremely high value to an organization, how could I put “Pros and Cons” in my blog title? You’re right the concept of coverage is highly valuable when I talk about “Pros and Cons” it is more to do with what is needed to get done to get that understanding of coverage.

The reason I am bringing this up stems from a vendor cold call I got for third party testing services. In my position, I get a lot of those calls, and for the most part, I do not have an issue to start a conversation to see what they provide as long as they can differentiate from all other third party vendors. In this one call, there was one statement that came out that piqued my interest, and I pushed back to get more details. That comment was, “We can have 100% code coverage through automation.” Sounds too good to be true, right? Well, in my experience, it is. I have never seen any software shop that had 100% code coverage through automation. There are too many variables out there for that statement to be reliable; it would take a lot of work.

Before I continue with the above comments, let us go through what I see as the pros and cons.

Pro

• Automation coverage can target specific components

• No need for full regression testing with a change

• Better estimation to changes, more accurate

• Shortened test timeframes

There are more, and there is plenty of literature out there that will go into detail. Here I just wanted to list a few and foster some conversation, as always.

Cons

• A lot of effort needed

• Tedious

• May not be completely accurate

• May not have a full understanding of code

Again the list of the cons can be a lot longer. The good news is the cons are issues that can be worked on and resolved within the company to get to what is needed. There is no technical reason or roadblock. It is about getting the work culture changed to tackle these cons.

There are software packages out there that will give you the code coverage an organization desires. Like all tools, it is all about the correct use of the tool to get it right

Do I think an organization can have 100% coverage? Yes. In automation? Maybe. Technology is getting better and better over time that it is something that could be possible. There are a lot of other variables that need to fall into place that will allow it to happen. Consistent coding practices, full knowledge sharing within the teams, and full cooperation between everyone within the organization, to name a few.