Root Cause Analysis – True QA

Within a organization’s Quality Assurance (QA) program, there is a need to continuously improve on processes throughout the Software Delivery Life Cycle (SDLC). One key process improvement initiative is the use of Root Cause Analysis (RCA); however, a common concern is achieving buy-in from the Business end to pay for these initiatives. The organization must recognize that metrics from this process are precious to the future success of an organization. Previous research supports the importance of a communicative relationship between QA and Business groups (Berriault, 2012). During QUEST 2014, discussion occurred regarding recognition of both disciplines’ needs and consensus of high-level points to encourage a communication focus shift for far-reaching improvements within any organization. The QA Value Chain is one of the critical processes within the Root Cause Analysis process to assist with justification.

RCA is a critical component within any SDLC whether it is Waterfall, Iterative or Agile delivery. There must be a form of tracking and identifying consistent issues that will allow for the development of solutions to improve delivery quality and speed. A functional Root Cause Analysis will impact each of the support activities of Innovation, Technology Efficiency, output and efficiency. The RCA process should not be taken lightly, viewed as an afterthought or considered a vehicle that is quick and easy or a reason to place blame. It is a slow and methodical process that uses statistics to drive out solutions within a collaborative team environment.

Some groups see the end product of this process, basic defect counts and severity.

This negative outcome is one of the issues some organizations see with Root Cause Analysis like QA. Sometimes viewed as a useful metric; however, it is counterintuitive to RCA. Organizations usually expand resources to deal with defects faster.

The three alternatives that come out of this type of RCA is:

• Expand capacity – more staff

• Work harder – overtime

• Keep fixing – postpone for future releases

The first two are the easiest to accomplish. Here is the kicker, they are not. Each will increase the overall costs and decrease the organization’s Return on Investment (ROI), while not resolving the repetitive problematic issue. This a keep fixing solution, which adds more costs and gives the organization’s customers a faulty product. The ROI is impacted through continued associated costs and tucking it away in a scheduled release is only fooling the organization that they are doing the right thing. Achieve quality output; a clear definition of needed data is required before the RCA process is to start.

Defects, variances or bugs are terms used to identify process issues. Ann Hunngate (Quest 2014) coined the term “saves” a more positive and forward term.

A good definition of a save should be Any situation that adversely affects the project’s ability to deliver a product within the project’s expectations. To drill down on the definition, consider any condition that requires unplanned investigation, causing delays. Anything from a coding issue to printer jams will get a save documented against it. Consideration of this definition could increase the number of saves, creating anxiety due to elevated statistics. A high number of saves could be an indication of a profoundly flawed product, which would be further from the truth. There needs to be an organizational change in mindset, where saves only happen during test case execution. The outcome provides valuable data on the inner workings of the SDLC. Think of it as the computer centre within a car. It is a central location that is watching all aspects of the vehicle and providing warnings when something occurs.

As stated earlier, the RCA process is more than collecting and displaying final save numbers. The actual analysis is the critical aspect of providing the organization’s decision-makers with the needed metrics to make positive assessments for future improvements.

RCA requires a focus on processes and process tools, eliminating the human aspect, where emotions could add an unwanted hurdle, and zeroes in on what is indeed causing the issues.

“If you don’t ask the right questions, you don’t get the right answers. Asking questions is the ABC of diagnosis. Only the inquiring mind solves problems” – Edward Hodnett, American poet.

Utilizing a change management process is the best way to ensure stakeholders are comfortable with change. The Why, How, and What are components recognized in most models. Communication is the link between the disciplines. Begin with why the change is needed. Once that is clear, the how and what of the process are the next explanations to be shared, and will allow for more straightforward utilization and acceptance (Sinek, 2009). A side effect of this route of action is the improved overall perception of the QA team, highlighting the group as more than test executioners. There is plenty of change management techniques that will provide additional suggestions to introduce change.

Appropriate use of this process prevents laying blame or placing the failure of some or all of the components on any individual or group creating conflict. Conflict decreases employee morale and creates a barrier for communication between groups and a breakdown of trust.

RCA has two parts:

• Collecting Data

• Asking question

The data collection is a little more complicated than identifying a save, such as a code, requirements or environment issue. It begins early within the SDLC and not in just the execution phases. The following are some points to use when having these discussions.

Process focused

RCA will help determine what methods need adjusting or possible required revamping. Much like an assembly line, the movement of data and work products follow a similar design where information from one area feeds into another. Quality throughout could be improved upon to increase efficiency and the bottom line for the organization. RCA will provide that data to make informed decisions on improvement initiatives (Slack, Chambers, & Johnston, 2010).

One of the key elements to find a process issue where stakeholders will take notice is to have a dollar figure attached to it. Here is where some additional work is needed when documenting saves. Depending on how the organization is set up from a time tracking perspective, the data could already be present and formulated to provide the information needed. If not, there is a simple workaround to use with any toolset that is used to calculate the financial impact. Three documented components are Investigation, Fixing and Retest. As stated earlier, this process occurs throughout the entire SDLC.

Using these terms is not solely for the use of test execution. Investigation and Fixing are universal statements for any work that is done throughout while Retest would be new in some areas. The easiest way to think about retest is confirmation the Fix meets expectations. For example, ensuring a reference in a requirement that was not there before is there.

For each component, there must be a financial component. With most large organizations, there would be a time tracking system where data can be pulled off, such as hourly rates. Defect tracking tools can be customized to allow for each group working on the save to key in the amount of time spent on it. With these two sets of data, you now have a cost associated with each issue.

Here is an example using a base day cost of an issue:

How many hours each person worked on the issue

E.g. Lead 1hr, Tester 2 hrs, Developer 3 hours (all rough estimates and keyed into the comment section of the save)

Then determine the number of days – 7.5 hours = one day

Then multiply by average “person day” cost. (for this example use $450)

Using the model above 6 hours would be 0.8 Person day X 450.00 =360.00.

Therefore the cost associated with the Fix is $360.00. Next, a process or non-employee issue must be related to it, as well. This component can become difficult due to the amount of analysis that is needed. Frequently it is much simpler to put a quick label on it and forget about it.

Here are some guidelines to determine if time is needed to spend on more detailed analysis:

• Do I have an idea of how to avoid this save in the first place?

• Is the save significant? (i.e. Was there a lot of rework time spent on the defect?)

• Could small saves turn into monsters?

  • Some view a saves as a single brick. On its own, in one project, it could be nothing.

  • If repeated in other projects, it will turn into a wall.

  • Potential monitoring unresolved saves throughout the year to catch repeats.

One of the most straightforward RCA tools that would help determine the root of the issue is the why-why Analysis (Slack, Chambers, & Johnston, 2010). At the end of this type of analysis, the human component will not be present.

Here is an example:

A coding issue caused the deletion of all reports.

1 – Why did the reports get deleted?

Developer incorrectly coded the module

2 – Why did the Developer incorrectly code the module?

The Developer did not follow the coding standards

3 – Why did the Developer not follow the coding standards?

The Developer is new to the organization

4 – Why was the Developer not trained on the new standards?

The Developer has access to the share-point where the standards reside

5 – Why didn’t anyone confirm with the Developer that the rules were understood?

There was no time

As per the above example, the broken process link here could is inadequate training procedures. With this information now, it can be tracked to see if it will continuously occur. Also, now that there is a cost associated with it and using cost-benefit analysis suggests a regular similar process break can be applied, and solution costs are available.

The cost-benefit analysis could be as follows:

Each occurrence costs $360.00

It occurred eight times in the past 12 months

Total costs over the year $4320.00

Potential solution a two-hour job shadow session for new developers with senior developers with a daily average per-person cost of $450 for 8 hours; the total solution costs are $225.00.

If the department hires four developers a year, the total yearly costs would be approximately $900. That would be a cost savings of $3420.00 a year. Senior management would find this information purposeful due to the ROI.

RCA is a potent tool to help organizations improve SDLC, and it can be adapted to use in all forms of development from Agile to Waterfall. These metrics will assist the QA department in providing true quality assurance.

Check out the Root Cause Analysis: True QA book for more details on applying RCA.

WORKS CITED

Berriault, J. (2012, September 16). Breaking down the root cause for the knowledge gap between testing and business. Retrieved February 5, 2013, from Athabasca Library: http://dtpr.lib.athabascau.ca/action/download.php?filename=mba-12/open/BerriaultJeremy.pdf

TED (Producer). (2009). How great leaders inspire action [Motion Picture].

Slack, N., Chambers, S., & Johnston, R. (2010). Operations Management (6th edition ed.). Edinborough Gate: Prentice-Hall.