Chapter 14: Constantly Measurable Quality – Agile Productivity Unleashed, 2nd Edition


How much does quality cost?

Although how much does quality cost? is not a trick question, it does have three different answers – particularly where the cost of quality is considered against both the value that high-quality outputs can bring to the organization, and the issues that producing low-quality outputs can create for the organization:

  • Benefits for the organization: high-quality outputs can deliver strong external benefits for an organization, including a positive public image, repeat customers and competitive advantage in the marketplace. However, it can also deliver significant internal benefits, such as reduced overheads, more satisfied employees and fewer last-minute ‘fire-fighting’ activities that create unnecessary stress in the workplace.
  • Risks to the organization: low-quality outputs, on the other hand, can represent a significant liability for the organization, particularly if people outside the organization (e.g. customers, competitors) become aware of them. These external exposures are compounded by the internal impacts of low-quality outputs, including the overhead costs of rework, defect handling and damage control, as well as the effects that poor outputs can have on staff motivation, stress levels and camaraderie (e.g. pitting employees against each other in the ‘blame game’).
  • Implementation costs: these are the overhead costs for the organization in establishing quality processes and practices, and ensuring that there are sufficient resources (e.g. staff, equipment, education, management tools) for employees to use them.

The Doing it right the first time section of Chapter 13: Waste Management explained that resolving defects in a product or service at the end of the process is often far more costly to the organization overall than resolving the issues throughout the process that caused the defect in the first place. Issues within a process can include:

  • Ineffective business processes.
  • Inefficient (or insufficient) work practices.
  • Miscommunication that causes errors and rework.
  • Outputs that do not meet the needs of the internal or external customer.

Although identifying a bad quality output before it leaves the organization can protect the organization from exposure and liability, it does nothing to stop the same problem from occurring again the next time, and the time after that ...

Organizations will often weigh the costs of implementing quality management and control processes against the potential internal and external risks for the organization. For example, a company which produces clothing may be willing to absorb the cost of an occasional faulty product leaving the organization, against the expenditure that would be needed to install better quality production equipment in the factory. A company that produces baby formula, however, will invest significant corporate funds in infrastructure to ensure that every product the organization ships meets stringent industry standards. The cost (and time) investment that an organization is willing to make to ensure quality outputs is in direct correlation with the potential exposure (and financial liability) that bad quality outputs can create for the organization.

For many organizations, ‘quality control’ is a series of checkpoints that occur towards the end of a process to confirm whether or not the outcomes match expectations. Where physical outputs are produced, these checkpoints can be physical measurements, visual inspections and stress tests; where intellectual outputs are produced (such as documents) these checkpoints can be quality reviews by other employees. The intent of the checkpoints is to catch problems in the outputs before they reach their intended recipients, particularly when those recipients are external to the organization. There is, however, a distinction between:

  • Passive quality checkpoints that check a completed (or near completed) deliverable at the end of a process.
  • Active quality checkpoints that review a deliverable early enough in the process to be able to impact (and improve) its quality.

Using passive quality checkpoints to catch a faulty output at the end of a process does not resolve the underlying issues that caused the quality problem in the first place (in the same way that treating a symptom is not the same as curing the disease). Organizations often spend countless resource hours ‘chasing their tails’ trying to ensure better quality by instituting more frequent (and more stringent) passive quality checks at the end of a process. However, if the quality issue is in the process itself, then no amount of passive checks will stop the problem from occurring (and recurring) indefinitely.

Weight control and the bathroom scale

Steve McConnell wrote a guide for software developers that included an exceptionally powerful statement about how employees can ensure ongoing quality in their work:50

If you want to lose weight, don’t buy a new scale; change your diet.

At the heart of McConnell’s statement was the critical premise that quality management is not a series of measurements at the end of the process to see how well the work was done; it is the establishment of a work environment (and corresponding business processes), which ensure that quality is a consideration in every activity along the way. McConnell was not telling readers to ignore the bathroom scale (as weight loss, like quality, needs to be measured in order to be managed effectively); he was telling readers not to assume that measurement alone will improve a situation if the underlying causes of the problem are not addressed.

This chapter identifies a number of approaches that organizations can use to build quality into their work environment and business processes, and explains how Agile practices use active quality checkpoints to maximize the quality of outputs throughout the process.

True quality requires a culture change

Quality is not a coincidence. It is the result of a work environment that:

  • Promotes high communication and information sharing within (and outside) the organization.
  • Creates tools for knowledge capture and knowledge transfer that de-centralize expertise in the organization – and equip less experienced staff to deliver more reliable and consistent outcomes.
  • Encourages employees to look for and recommend areas of improvement throughout the organization (including management procedures that are designed to elicit this information).
  • Structures work to be done in pairs and teams for greater accountability and cross-training.
  • Recognizes and rewards employees for ‘getting it right the first time’.
  • Supports skills development and continued education, so that employees are able to introduce and implement best practices in their work.

No amount of business process improvement is going to significantly change an organization where employees are rewarded for building silos of knowledge, are individually recognized (i.e. singled out) for the work that they did as part of a team, or are encouraged to do their work ‘the way things have always been done around here,’ instead of regularly looking for ways to improve the organization.

The impact of high communication

Agile approaches, such as the ACTION plan detailed in Chapter 5: Responsive Planning, are designed to create high communication, team-based environments. Chapter 11: ‘Just-in-time’ Communication identified a range of techniques that Agile approaches use to encourage communication within and between areas of the organization, including:

  • iteration planning sessions
  • outcomes review sessions
  • daily stand-up meetings
  • pairing of delivery team members
  • co-location of delivery team members
  • cross-training of delivery team members

All of these techniques are designed to promote information sharing and knowledge transfer, not just within a team, but across the organization.

One of the key outcomes of this multi-faceted communication approach is the establishment – and ongoing confirmation – of a shared vision and shared expectations for the work that is being done. This means that delivery teams can use the business owner’s ongoing input to guide and shape the work that they are doing – and business owners can be confident that the outcomes at the end of the process will be as close as possible to what the organization requires. It also means that one key measurement of quality (customer satisfaction) is built directly into the process.

Each outcomes review session is an active checkpoint, where business owners assess the ongoing work of the delivery team against both qualitative and quantitative measurements. In some cases, the measurement of outputs is a subjective assessment by the business owner regarding whether the outputs align with their initial expectations (their ‘vision’). In other cases, the measurement of outputs is based on defined metrics, such as increases in sales orders or reduced overhead costs.

The intent of the review session is not for business owners to accept or reject the outputs presented by the delivery team based on these quality measurements – the intent is for both teams to use the session as an opportunity to communicate with each other, so that they can refine the outputs together.

This is not to say that the use of high-communication practices in one area of the organization is going to address significant communication deficiencies across an organization. A deeply-embedded culture of knowledge silos and ‘business as usual’ mindsets is not going to change overnight, but the adoption of Agile practices in organizations has historically been the result of successful outcomes getting the attention of upper management. Even the most steadfast organizations are strategic enough to leverage approaches – even dramatically different approaches – as long as they can deliver proven results.

Quality by design

Having a work environment that encourages and promotes effective work practices is half the battle for building quality within the organization; the other half is designing the business processes within the organization to have active quality checkpoints throughout.

The lean techniques described in Chapter 13: Waste Management do not just enable business processes to be run more efficiently, they can also result in higher quality outputs by:

  • Reducing the amount of unnecessary work (including ‘just-in-case’ work), so that staff can focus on their core business activities (i.e. the value stream).
  • Eliminating excess movement within the process, so that there are fewer hands involved in each step of the process – and, therefore, less opportunity for work to get lost between physical locations – or in the stack of papers on an employee’s desk.
  • Allowing employees sufficient time to focus properly on their work by minimizing task switching.

The fewer complexities there are in a business process (e.g. decision points, unnecessary tasks), the less potential there is for things to go wrong at each step of the process. This does not mean that a complex business process should be over-simplified, just to reduce the potential for error – but it does encourage organizations to look closely at what activities are core to the value stream, and which activities can be pared down (or eliminated) to reduce both costs and complexity in the process.

The IT industry uses an Agile approach called refactoring to continually review and, where required, restructure software solutions to be as simple as possible to meet the required business objectives. In some cases, this means discarding most (if not all) of the work that they have currently done, in order to establish a ‘more elegant’ solution that will be easier for the organization to manage and extend upon in the future.

For some organizations, the thought of ‘throwing away’ an existing business process would be impossible to sell to upper management. This is, however, not wholly different to a homeowner’s decision to tear down and rebuild a house on their property, instead of extending the existing one.

There are times when it is more cost-effective for an organization to achieve its longer-term objectives by architecting an environment that is specifically designed for that vision, rather than by trying to retrofit an existing process – especially if that process was established 10 years ago to meet the needs of the organization at that time.

So, once the process itself is as simple as it can reasonably be to achieve its intended business objective, how does the organization create active quality checkpoints throughout the process – and avoid the expense and exposure of only finding issues at the end? The key to implementing active quality checkpoints in a business process is making the measurements of success an intrinsic part of the process.

Fit-for-purpose outputs

Agile practitioners in the IT industry have made both a science and an art form of building quality into their business processes.

One of the Agile techniques that the IT industry uses to manage the quality of software while it is being developed is a practice called test-driven development (TDD). The basic premise of TDD is simple (and readily transferable to any business activity):

Identify your measurements for success upfront – and design your work around these measurements.

In the IT industry, this involves having the software development team (literally) build all of the tests that they are going to measure their software against before they begin writing the first line of programming code for the solution. This enables the delivery team to both design their work around these measurements – and regularly check their ongoing work to confirm that they are delivering outputs that will achieve the required results.

Other industries can achieve an equivalent outcome for their business activities by identifying and structuring their end outputs (e.g. products, reports, services) against their measurements for success before beginning the work required to create these deliverables. For example, if a delivery team is required to put together a report that identifies changing trends in customer demand over the past 24 months, the team members would first confirm the measurements for a successful report with the business owners:

  • The customer demand report will achieve its objectives if it can accurately document:
    • the historical and current quantity of customer orders
    • upward and downward trends in customer orders over the past 24 months
    • external factors that might influence fluctuations in customer orders, such as seasonal variations.

The delivery team may even do some background research and preparation with stakeholders before confirming these measurements for success with the business owners, such as:

  • Identifying what specific information is required to accurately capture customer orders (e.g. number of orders per month, products being ordered) and confirming that this information is available within the organization.
  • Laying out the structure of the proposed report in a draft form, including all of the information that they believe will be needed.

Presenting these proposed measurements for success to the business owners (before any significant work has been done) results in the following feedback:

  • Customer orders within the organization need to be benchmarked against overall industry trends, in order to isolate variations in market behavior that are specific to the organization (critical priority).
  • Identifying the ordering trends of individual customers (particularly the ones with the largest orders) would help to identify the behavior of repeat customers – and to determine whether repeat orders can be reasonably predicted in sales forecast reports (high priority).

This initial feedback from the business owners alone has enabled the delivery team to extend their initial measurements for success to also include accurately documenting comparable industry figures and individual customer behavior.

With these measurements for success in hand, the delivery team endeavors to collect the required information and put together a report with real production data by the next outcomes review session. In putting together the report, the delivery team finds that the organization’s internal systems also track the number of times that a customer cancelled an order before it was fulfilled. Before the team members spend time gathering and formatting this new information, they assess it against the original measurements of success that were agreed with the business owners: will knowing how many orders were cancelled assist the organization in accurately determining trends in customer demand?

In this situation, the delivery team realizes that they are not in a position to make this decision on their own. They contact the business owners to confirm whether this added detail will add value to the report. This discussion with the business owners identifies that cancelled orders are not a good indicator of customer demand, as their experience indicates that most customers who cancel an order subsequently resubmit an equivalent order soon after. This means that including this data in the report could artificially inflate the customer demand trends.

Instead of including this additional information in the report simply because it was available, the delivery team assessed the work that would be required to include the information against their originally agreed measurements for success. The subsequent decision not to include this new information in the report resulted in additional time that the delivery team is able to spend focusing on the true criteria for success (e.g. the behavior of repeat customers), which is likely to result in a higher quality output for the organization overall.

The interesting thing about designing business processes around measurements of success is that it is one of the triggers for the delivery team reviewing and restructuring their current work (i.e. refactoring) to better align the activities that they are doing with the business owner’s objectives.

For the customer demand report, the delivery team knowing upfront that the organization may need to track individual customer orders as part of their analysis, means that they can request (and prepare for) this level of detail from the beginning; and knowing that the organization does not require detail about cancelled orders means that they can simplify the report to only include the required information for completed orders. Realigning the report to better suit the needs of the organization may mean that the delivery team needs to discard (or revise) some of the draft report layouts that they were working on (which had included cancelled customer orders in the overall totals). However, the delivery team also realizes that making this change will result in significantly less work in the report development process overall than if they were to restructure the report (and report data) at the end of the process.

The (almost) real-time measuring stick

In the IT industry, monitoring and measuring the quality of outputs is a much more straightforward (and quantifiable) activity than it may be for other industries. Software developers have the benefit of tests which clearly identify when a defined outcome has been achieved (e.g. when a website feedback form has been sent to the customer service department) and when it has failed (e.g. the website feedback form was not sent because the phone number field was left blank). Software developers even have the benefit of automated testing harnesses, which enable them to run all of their quantifiable tests every day as a constant measure of the quality and progress of their work. Any non-quantifiable quality measurements that they have (e.g. usability) can be measured as part of the outcomes review sessions with the business owners. The fact is that daily quality checking is one of the few circumstances in which the IT industry may be better positioned than other industries in using Agile approaches.

For other industries, the degree to which an organization can monitor and measure the quality of work through regular (e.g. daily) quality checking is often more limited. In Chapter 12: Immediate Status Tracking, several reporting tools were identified for regularly monitoring and managing Agile work, including:

  • requirements backlogs
  • delivery backlogs
  • velocity trackers
  • executive dashboards

These tools are primarily designed to track the progress of work completed and the effort remaining to achieve the agreed objectives; they are not specifically designed to monitor the quality of the work that has been done by the delivery team.

Unless you are in an industry (like the IT industry) where work is so quantifiable that quality checking can be automated, it is unlikely that there will be tools available for business owners to easily monitor the quality of outputs on a daily basis. Therefore, the best quality management tools that an organization can use in their Agile work are the communication methods detailed in Chapter 11: ‘Just-in-time’ Communication, particularly pairing and co-location of delivery teams, daily stand-up meetings, and outcomes review sessions at the end of each iteration.

Pairing and co-location of the delivery team establishes a high-communication environment where delivery team members are encouraged to work together, check (and critique) each other’s work, and jointly overcome challenges. In addition, the daily stand-up meetings provide a forum for the delivery team to step back and assess the work that they are doing as a group, as well as raise any issues that they have encountered for the Agile facilitator to resolve. These communication tools help the delivery team to regularly monitor the work that they are doing against the objectives (and measurements) agreed with the business owners at the start of each iteration – and to continually assess whether they are delivering high-quality outputs based on these measurements. They also provide a mechanism for escalating exceptions and problems when they arise, which allows the core work of the delivery team to be progressed without interruption.

Equally, the outcomes review session is a dedicated opportunity for business owners to regularly assess the quality of the delivery team’s work. When the ACTION plan is done in four-week iterations, the organization has at least one time each month where key stakeholders can get a hands-on review of the delivery team’s work – and track the completed work against the originally agreed objectives. For organizations that require more stringent monitoring of the quality of the delivery team’s ongoing work, the ACTION plan can be reduced to two-week iterations. This allows the business owners to get a hands-on review of completed work every other week – and to request rework if the quality of the outputs does not meet their expectations.

The communication tools in Agile approaches provide both the business owners and the delivery team with mechanisms for including active quality checkpoints in their ongoing work. These active checkpoints position the organization to respond more quickly (and more cost-effectively) to issues that arise than traditional quality reviews at the end of the process. They allow the delivery team to focus its efforts on producing high business-value outcomes, instead of rushing at the end of the process to fix the problems that were found just before the deadline.

Exponential returns on your quality investment

This chapter began by identifying the costs of quality, including the benefits that high-quality outputs can bring to the organization – and the protection that these high-quality outputs can provide for the organization against internal issues and external liabilities. It also identified that there are overhead costs in implementing quality processes, practices and tools within the organization – and that organizations need to weigh these costs against the potential internal and external risks for the organization. So, is the investment in high quality simply a way for the organization to avoid litigation? Or, is there a return on investment that makes investing in quality a sound business decision, beyond risk aversion?

An organization that truly implements high-quality practices and processes is likely to receive the following returns on their investment:

  • Market position:
    • more reliable products and services
    • quicker time to market (due to less rework)
    • more positive public image
    • competitive advantage over less stringent organizations
    • more satisfied customers
  • Financial:
    • reduced ‘total cost of production’ overheads (costs, time and staff) – including the ongoing benefits of having more simplified, fit-for-purpose business processes
    • increased sales
    • less work required to win customers
    • greater likelihood of repeat customers
  • Human resources:
    • more satisfied employees
    • more motivated employees
    • greater employee confidence and pride in their work
    • better employee retention rates
    • a working environment with less stress and negativity (due to the minimized need for ‘blame game’ assignment and last-minute ‘fire-fighting’ activities).

In addition, organizations that institute high-quality practices and processes are well-positioned to be formally certified to industry quality management standards (such as ISO9001), which can significantly strengthen their credibility and competitive position in the marketplace – including making them eligible to undertake work that can only be done by quality certified organizations.

Most importantly, the high-communication tools and active quality checkpoints in Agile approaches can position an organization to achieve high-quality outputs without a significant upfront investment. This means that the ROI equation is resolved quickly, so that the benefits listed above can become pure gain for the organization.


50 Code Complete, McConnell S, Microsoft Press (1993) ISBN 1556154844, 9781556154843.