What is legal and yet has a potentially better ROI than drug dealing? 

Debugging requirements.

Several studies have tried to measure the cost of non quality and have come up with ratios between the cost of fixing an issue at the requirements definition phase and the cost of fixing that same issue in production ranging from 10 to 1000. For instance, an IDC research indicated that ratio to be 100+ in a 2008 study [Evaluating Poor Quality and Defect Costs in a Difficult Economy, IDC, 2008]. Even using that conservative value, it still represents an incredible ROI for a perfectly legal activity.

The purpose of this document is to describe a simple yet effective framework to debug your requirements.

The Triangle of Confusion

The problems that continue to beleaguer testing and therefore software development can be represented by what I call the triangle of confusion: stakeholders’ expectations, product requirements and test cases are not aligned.

What this really means is that stakeholders find out when the software is in production that it does not quite do what they expected and that on top of that there are some annoying bugs. In other words, it inconsistently solves a problem the users don’t really have.

Sometimes the surface of the triangle can become so large that projects can get lost in it just like planes in the Bermuda triangle.

The first factor that makes the surface of the triangle vary is the quality of the communication between the project actors. Thus, global delivery virtual teams are more exposed to this challenge than collocated agile teams.

The second factor is the ambiguity of the term ‘Requirement’ itself. It is used to represent very different things: what the user wants, what the project must achieve, what the product should do… It is often used with a qualifier to somewhat reduce the ambiguity. Here is a list of expressions I have come across:

• Product requirements
• Project requirements
• Business requirements
• Functional requirements
• Non functional requirements
• Technical requirements
• Quality requirements
• High level requirements
• Support and training requirements

The problem is that nobody adheres to the strict rule of using a qualifier every time, so that system fails and confusion soon rears its ugly head. Therefore I use a different system that relies on using different words. Here it is:

• Users have expectations
• Projects have constraints
• Products have requirements
• Product requirements can be functional or technical (same as non functional)
• Technical requirements can be further broken down into different categories:
  • Quality
  • Security
  • Performance
  • Compliance
  • Availability
  • …..

Testing methods have long attempted to flatten the evil triangle.

Requirements-based testing enforces alignment between requirements and test cases, therefore focusing on the lower side of the triangle. The resulting new triangle now has a flat bottom.

The situation is slightly improved; the software now meets its requirements. Unfortunately, it still does not do what the stakeholders expected. What is the use of having the best chefs in the kitchen if the waiter cannot take the orders properly?

The left side of the triangle is the one that needs to be taken care of first in order to remove all confusion and to enjoy perfect alignment. Achieving alignment between the stakeholders’ expectations and the product requirements ensures that no bug will appear later in the lifecycle because of a misunderstanding at this stage, in essence it is debugging the requirements.

In order to debug your requirements you need to have them properly identified, structured and formulated first.

Discover your requirements

That’s right, do not gather or collect them like some old clothes people want to get rid of. That’s not enough, go and get them!

It is not easy for people to express what they want, what they really, really want.

You need to take a step back and understand what the business need to be addressed is and to help them realize how that can be mapped to proper software requirements.

Besides, leveraging both his experience and his domain knowledge, the business analyst can complete the requirements by uncovering those that are implicit. For example, business objects often have a more complex lifecycle than the one stakeholders are focusing on and it’s the business analyst’s job to prod the stakeholders with such questions as: «When is it archived?», «When is it purged?».

This process of working with the stakeholders to express their business needs focusing on the «what» and then to link them to products requirements focusing on the «how» brings clarity and alignment to the project.

Clarity is achieved by separating and explicitly managing the relations between stakeholders’ expectations and product requirements. Understanding why something is required provides the context that helps the whole project team see the big picture. This improved awareness not only benefits the current phase of the project it also positively impacts all the downstream activities. As such it is in itself a very worthwhile investment. It is also a good project management practice that will provide you with a tool to measure whether or not your project meets or exceeds the stakeholders’ expectations.

Moreover, this reformulation exercise ensures that the business analyst has properly captured what the business wants. It’s the first checkpoint of this framework.

For this activity to be successful, several factors come into play: the availability of the stakeholders, the experience and domain knowledge of the business analyst and the time devoted to the process.

Make sure you have enough of everything.

Test your requirements

As soon as your list of well formulated and documented requirements starts to take shape, bring in the Test Analyst. For the framework to be successful, it is important that the Business Analyst and Test Analyst be two different people.

The Test Analyst’s job is to derive test cases from the requirements. By doing so, the Test Analyst will not only challenge the requirements to check that they can be verified, he will also provide confirmation that the requirements can be passed on to a third party without loss of information or misinterpretation.

The result of the dialogue between the Business Analyst and the Test Analyst will be an improvement in the precision of the requirements and in the relevance of the test cases.

This is the second checkpoint of the framework.

At this stage the requirements should be complete, consistent, unambiguous, traceable to stakeholders’ expectations and acceptable by stakeholders: in other words, bug free.

The Revenge of the Necessary Evil

What prevents organizations from adopting this kind of framework and enjoying the kind of ROI I mentioned in the introduction?

Two things: awareness and skills.

Although awareness of the problem is growing, some organizations do not still realize how much it cripples them. In order to quantify the problem, it helps to track the effort involved in fixing bugs in production and to analyze the origin of the defect: is it something that could have been caught during the requirements definition stage?

Once awareness is achieved and there is a will to change, the next obstacle may very well be the lack of the required skills. It’s a logical consequence of the previous unawareness: we don’t have a problem with software quality so why hire the best and brightest testers?

The skills gap needs to be measured and a plan to fill it must be undertaken using hiring and training.

Furthermore for the change to be sustainable, organizations must also undergo additional transformations:

• the testing career track needs to be acknowledged and valued by HR;
• the testing function needs to be made independent from development.

Organizations today have to adapt to a lot changes at the same time in order to keep delivering more innovation, comply with an ever increasing amount of regulations and make the necessary transition to the Global Economy.

All those changes heighten the need for new and more efficient software development practices. In that quest for improvement, testing has great potential and is finally getting the attention it deserves.

Translating what the user wants into what the software does has always been the major stumbling block of software development. Using tests to clarify requirements early in the software development lifecycle solves this problem and, as a consequence, reduces time to market and improves quality.