Although it might be tempting for you to want to ‘dive right in’ and start using an Agile methodology (e.g. Scrum) to salvage your struggling software projects, it is valuable for you to step back for a moment and consider the specific needs – and constraints – of your organisation.
Choosing the right project(s)
Deciding where to begin using Agile methodologies in your organisation is as important as deciding which methodology you will use.
Is your objective to do a ‘safe’ trial of Agile before committing to its broader use across the organisation? If so, you may want to select a self-contained new software development project which will not be unduly burdened by the constraints of legacy systems or pre-existing project management techniques. Provide the team with a clean slate to start from, and then use the ROI metrics described in the Monitoring your investment section of this chapter to measure the results.
Is your objective to create some ‘runs on the board’ to influence others in the organisation to consider using Agile methodologies? If so, then you need to choose projects that are important enough so that their success will be meaningful to the organisation. Achieving great results on trivial projects will only serve to fuel resistance from others in the organisation who may believe that Agile methodologies are not applicable to their IT projects.
Is your objective to mandate the immediate use of Agile methodologies for all software development work across the organisation? If so, you will need to review the information about organisations who have successfully achieved this in the Choosing the right method of introduction section, below.
Is your objective to use Agile methodologies to begin to shift the organisational culture towards a more open and collaborative environment? If so, you may want to start with the path of least cultural resistance by choosing an area of the organisation which (or a particular manager who) is most likely to be amenable to working closely with stakeholders, trusting and empowering technical teams, and measuring success through the production and demonstration of tangible results.
Once you have selected the project(s) that align with your core objectives, the next step is determining which Agile methodologies and practices are best suited to the needs of the selected project(s).
Choosing the right methodologies and practices
Agile methodologies are not a ‘one size fits all’ proposition. For these methodologies to deliver genuine business value to your organisation, it is important to find the right Agile methodology (or combination of methodologies and practices) to address your specific challenges, your KPIs and your culture. It is equally important to work with the selected project teams in making this decision.
The Agile methodologies selection workflow tool in Figure 2 takes you through some of the key questions that you need to ask in order to select the most appropriate Agile methodologies21 and practices for each of your selected projects.
It is important to note that the Agile methodologies selection workflow tool is only a guideline for you to use as a starting point in selecting the methodologies and practices that are best suited to the unique requirements of your organisation. Your ongoing use of these approaches is the definitive indicator of how well they meet your needs.
New Software Development or Maintenance?
If the project that you have selected is a new software development, then go to the Stakeholders Available? question below.
If the project is predominantly a maintenance activity, then the next question to consider is whether or not the maintenance work will include the development of Enhancements or Bug Fixes?
If the project is a maintenance activity for bug fixes, you may be best off starting with Kanban to manage this work.
If the project is a maintenance activity that includes enhancements, you may decide to treat the enhancements as new software development; in which case, you will want to go to the Stakeholders Available? question in the following section.
Almost every aspect of Agile methodologies requires involvement from the stakeholders (e.g. business areas, customers) who will be using the delivered software. If these stakeholders are unavailable (and your organisation is not in a position to make them available, or to organise for other resources who can adequately represent their interests), then your options for using Agile methodologies are extremely limited. Therefore, if the answer to the Stakeholders Available? question is No, then the project team may be able to utilise some selected Agile practices, such as daily stand-up meetings, pair programming or test-driven development, to enhance the quality of their development work; but the substantial benefits of using Agile methodologies to deliver usable software solutions that directly align to customer needs will not be achieved.
Teams of at least four to eight staff?
Although this is not a hard-and-fast rule, it is generally believed that the ideal team size for Agile methodologies, such as Scrum, is four to eight team members. If you have fewer than four members on your technical team for the selected project, it may be better for your organisation to consider using elements of XP (noting that at least two team members will be required for pair programming). If you have more than eight team members, it may be better for you to break down the teams into groups of four to eight, and then scale the project work within the selected Agile methodologies.
Need for substantial documentation?
The final question to ask yourself is whether or not the organisation (or you personally) prefer for the work undertaken by technical teams to be substantially documented throughout the process. If so, it may be better for you to use Agile methodologies such as DSDM or FDD which mandate the generation of work products, such as requirements specifications and domain models, as part of the process. Otherwise, it may be better for you to start with methodologies such as Scrum or Lean, which are more flexible, to accommodate each team’s preferred work practices.
One final note regarding the Agile methodologies selection workflow tool: on the right-hand side of the diagram is a series of dotted lines connecting each of the suggested Agile methodologies to the others. These dotted lines indicate that your organisation may opt to combine selected Agile methodologies, e.g. Scrum and XP, Lean and Kanban, in order to receive the cumulative benefits available from each approach.
Also, if you have selected more than one project as your starting point, it may be valuable for you to trial two or three different Agile methodologies at the same time to see which is the best fit for your organisation.
Last, in selecting Agile methodologies for your organisation to trial, you need to be prepared to allow staff to adjust and modify their use of these methodologies to achieve their optimal levels of productivity as their work progresses. For example, technical teams may not, at first, be comfortable with committing to the two- to four-week delivery cycle of Scrum; in which case, the team may be better off using a six- to eight-week cycle as a starting point. Equally, the technical team members may not have enough information at hand to do the value stream mapping that Lean requires, but they may be able to identify and address immediate areas of waste in the current software development processes. The key is to give technical teams the power to adapt the practices of Agile without jeopardising the underlying principles. (See the Avoiding common traps section below for further detail on the perils of the misapplication of Agile methodologies.)
In a perfect world, the introduction of Agile methodologies within your organisation would be achieved through an organisation-wide mandate where staff readily embraced and adopted these approaches. In reality, however, the opportunity to achieve organisation-wide Agile adoption as a starting point may be rare, but it is not impossible.
BT Innovate & Design provides a classic example of an organisation which successfully achieved organisation-wide Agile adoption courtesy of a forward-thinking executive who, in 2004, established a mandate that the organisation produce business value every 90 days.22
It should be noted that the BT example is an exceptional situation. It is far more common for Agile methodologies and practices to be introduced to an organisation by the technical team members themselves, often without the active awareness (or support) of senior management. (You may, in fact, be surprised to learn that there are IT project teams within your organisation which are already using Agile.) This ‘Agile by Stealth’ approach is generally not the team’s preferred way of introducing Agile within the organisation; they would much rather have your support and backing for this work. Instead, the ‘Agile by Stealth’ approach is likely to be perceived by the technical team members as a necessary way to ensure that they can progress with these practices ‘under the radar’ without provoking management resistance or challenging the traditional approaches of the organisation.
If, in your discussions with prospective Agile teams, you find that they have already been utilising these methodologies, you are likely to be one step ahead of the game. In most cases, these teams will not have definitive ROI metrics for the work that they have done to date (see Establishing your baseline and Monitoring your investment for further detail on ROI metrics). They are, however, likely to have a number of work products, including delivered software and stakeholder testimonials, which can indicate how well their selected Agile methodologies worked. They are also likely to have acquired a substantial amount of knowledge about what does – and does not – work within your organisational culture. All of this information can be applied to selecting the best ongoing approaches for your organisation to use, including the expansion of the technical team’s Agile work with the benefit of your support.
If your organisation is not planning to issue organisation-wide mandates to adopt Agile, and you have not been fortunate enough to unearth Agile success stories within your organisation, the next best thing is the selection of one or more trial projects, using the guidelines in Choosing the right project(s) above. Tracking and documenting the outcomes of these trial projects can be used as the basis for persuading other areas of the organisation to consider Agile methodologies for their projects. (See the Expanding Agile section for further details.)
You may also choose to position yourself as an Agile champion within the organisation, using industry case studies (such as Microsoft and Yahoo!) and industry research (such as Forrester) to support your decision to trial these methodologies. Your awareness of Agile methodologies, coupled with your influence, can help to position Agile teams to avoid the most common traps that organisations encounter in their implementation of Agile methodologies.
The simplicity and low overheads that make using Agile methodologies so appealing, also make them highly susceptible to misapplication. The following identifies some of the most common traps that organisations have fallen into when implementing Agile.
Undermining Agile principles
There is a difference between adapting Agile to suit the preferred work practices of your organisation (e.g. adjusting iteration time-frames, using videoconferencing instead of face-to-face meetings), and adapting Agile in a way that contravenes the underlying principles that drive its effectiveness.
For example, an organisation that moves to an ‘Agile’ iteration-based project management model, but still requires all of the work to be signed-off in an up-front specification. Agile methodologies are only valuable when the organisation is in a position to adapt ongoing work as it progresses. Otherwise, iterative work just becomes shorter delivery cycles that are limited by the same core constraints; and Agile methodologies get an unjustified bad reputation when this pre-constrained process inevitably fails.
Similarly dangerous is applying a handful of Agile practices (e.g. daily stand-up meetings, test-driven development) without the benefit of the core elements of each Agile methodology, such as stakeholder confirmation of requirements, or the ongoing adjustment of work to accommodate emerging information. In the Agile methodologies selection workflow tool, it was identified that, for projects without stakeholder availability, the use of selected Agile practices may be the only option available to the technical team. In this situation, however, the outcome is a somewhat more efficient traditional software development process, not an Agile project.
Insufficient communication and/or training
In order for Agile to be effective, participants (stakeholders and technical team members) need to be educated on the principles and practices of the selected methodology. Otherwise, they (and the organisation) are likely to fall victim to the areas of misapplication identified in the previous section.
In Choosing the right method of introduction, the Agile work undertaken at BT Innovate & Design was referenced as a classic example of the successful organisation-wide adoption of Agile based on a top-down executive mandate. What was not mentioned, however, was how this success was achieved.
When the CIO of the organisation established the mandate for the use of Agile throughout the organisation, he coupled that mandate with a series of training and communication initiatives to educate staff on Agile principles and practices. These initiatives included training and learning events (e.g. roadshows), assigning Agile coaches, and creating The BT Agile Cookbook as an online reference guide for staff. BT’s continued use of these practices in the organisation several years later is a testament to the power of both initial and ongoing communication – as well as the importance of genuine management commitment – in the successful implementation of Agile.
When introducing Agile in your organisation, it is important to consider how Agile principles and practices will be communicated to staff. One cost-effective way to achieve this is by including an Agile resources page on your corporate intranet that provides links to relevant sites, and allows staff to exchange their questions, concerns and ideas about the use of these methodologies before work begins.
Alternatively, your organisation may benefit from more formal guidance on adopting and applying Agile methodologies. There are a number of formal training courses available to teach people how to more effectively apply Agile methodologies. In some cases (e.g. Scrum) there are even certification courses that staff can attend.
All of these communication approaches help to assure you that your selected Agile methodologies are being utilised to the greatest advantage of the organisation.
Using Agile as a doctrine instead of a tool
This last common area of misapplication was alluded to in Choosing the right kick-off point: following Agile as a strict doctrine without adapting it to the needs of your organisation. To receive the greatest benefit from using Agile methodologies, it is important to allow your staff to adjust and modify their use of these methodologies in a way that fits in with their preferred work practices. That is, so long as the underlying principles of Agile are not compromised.
As you are introducing (and trialling) Agile methodologies within your organisation, it is valuable to consider the advice provided by the author of Kanban and Scrum: making the most of both23 who noted that:
Scrum [Agile] is just a tool. You choose when and how to use it. Don’t be a slave to it!
However you and your staff choose to introduce Agile within your organisation, you can (and should) adapt it to suit your specific needs. By its very nature, Agile methodologies are meant to be, well, agile.
Calculating the net business value of current project work
Throughout the previous chapters, the term ROI metrics has been used as a mechanism for quantifying the business value that Agile methodologies can deliver to your organisation. In order to measure comparative ROI value from the implementation of Agile, your organisation needs to be able to assess and record the net business value of equivalent work activities prior to its introduction. For example, a software project, which was delivered using your current software development processes, and which is similar in scope and complexity to the software that is being proposed for your intended Agile project (i.e. your ‘comparison project’). The business value of your comparison project can then represent the baseline for future ROI comparison.
Identifying the potential quantitative net business value return on your Agile investment is, from my experience, contingent upon four primary factors:
- the business value that your software solutions generate
- your current costs to develop and implement software solutions
- your current costs to support, maintain and extend the software solutions that you deliver
- your current costs in managing staff turnover.
- a quantifiable business value for that project’s delivered software features, based on revenue generated and/or reductions in operational overheads (software value)
- the overhead costs for software development and implementation on that project, e.g. staff, accommodation, equipment (delivery overhead)
- the non-recoverable24 overhead costs for software support, maintenance and extension of that project (non-recoverable support overhead)
- the average cost for replacing an IT staff member who has left the organisation (IT staff turnover cost)
- the percentage of IT staff turnover that can be reasonably aligned to the resources allocated to that project (IT staff turnover percentage).
Then apply these metrics, using the following software project net business value formula:
The resulting figure represents the quantitative net business value for your comparison project.
To provide you with the most complete assessment of business value, these quantitative metrics should be coupled with qualitative metrics on:
- how satisfied your stakeholders (external customers, internal business areas) are with delivered software
- how satisfied your IT staff members are with their current work environment
- how often technical teams are working overtime or ‘fire-fighting’ to address software problems.
Compensating for insufficient metrics
The purest ROI comparison scenario would involve the establishment of two identical projects, each using the same objectives, the same budget allocation and the same number of employees, in the same time-frame: one would be undertaken using your organisation’s current software development processes; the other, using your selected Agile methodology. In order to minimise the impact of external factors, this scenario would also involve the isolation of staff from all other commitments during this time. In my 20 years of working in the industry, however, I have never encountered a situation that supported this scenario.
More realistically, organisations tend to find themselves with a combination of the following:
- high-level budget reports (e.g. department or project expenditure breakdowns)
- incremental project status reports (with minimal quantitative metrics)
- sales figures and customer surveys (for commercial organisations with released software)
- issues registers and support logs (where software is in production use)
- anecdotal evidence of the success or failure of software development projects.
Therefore, in order to baseline the quantitative net business value for your comparison project (i.e. business value; costs for development, implementation, non-recoverable support and maintenance; and staff turnover), you need to determine whether the metrics that your organisation currently record are sufficient.
If you can confidently assess (or reasonably estimate) your comparison project costs, then you should be well positioned to calculate your Agile ROI on an equivalent project undertaken using Agile methodologies.
If you are not confident that your comparison project costs can be sufficiently assessed, then you need to decide if:
- you are comfortable proceeding with partial baseline information
- you would like to track the business value of your proposed Agile implementation without a baseline comparison
- you would prefer to gather baseline metrics on a current (or planned) software project before proceeding with the Agile implementation.
If you do decide to proceed with the information at hand, it may be valuable to establish a framework for tracking the quantitative net business value of Agile projects on an ongoing basis, so that you can measure the relative effectiveness of selected methodologies, as well as any changes in the levels of productivity as staff become more familiar with these approaches.
Tracking the progress of your Agile projects
Monitoring the business value and progress of your Agile projects can begin from the moment the project starts, and continue well before the whole-of-life quantitative net business value is evident.
Agile methodologies provide a number of mechanisms for tracking progress, including formal reports (e.g. executive dashboards), status update tools (e.g. WIP boards, product backlogs), and ongoing communication with stakeholders. Arguably, however, the most valuable measure of the Agile team’s progress is their delivery of tangible outputs. For iterative Agile methodologies, such as Scrum and FDD, the presentation of working software occurs at the end of each iteration. This means that, at fixed time-frames (generally once a month), you are able to see firsthand whether these approaches are delivering their anticipated business value.
Equally important, the technical teams themselves are able to use Agile tools to monitor their own progress during each iteration, and to adjust their ongoing work as needed to meet their agreed commitments.
It is interesting to note that the timing of four-week iterations aligns closely with the timing of monthly reports. This means that Agile teams are also able to use the presentation of working software at the end of each iteration to report on their progress in conjunction with the standard reporting cycles for the organisation overall (where required).
Comparing Agile to your current software development processes
If you have baselined the quantitative net business value for your comparison project (see Establishing your baseline), then comparing that project with the quantitative net business value of an equivalent Agile project is a straightforward ROI calculation – well, almost.
Retrospective comparison between two software solutions that are both in a production environment is reasonably achievable, as long as the variables (e.g. length of time in production) can be normalised.
The challenge (or opportunity) arises when comparing traditional software development projects that are in progress with equivalent Agile projects. The whole-of-life costs for software (including ongoing maintenance, support and extension costs) are not able to be measured until after the software has been released. For traditional software development projects, this is generally only available at the end of the process, i.e. when the complete software solution is implemented in a live production environment. For Agile projects, particularly ones using iterative methodologies, software features can be released for production use on an ongoing basis. This means that you are in a position to track quantitative net business value of your equivalent Agile project well before that information is available for your comparison project. For this reason, it is recommended that your selected comparison project be a historical project (i.e. one that is already in production).
A cautionary note: as explained in the Your Agile ROI section of Chapter 2, the most relevant ROI calculation for your organisation is not likely to be a side-by-side comparison of software development costs for projects using traditional processes versus Agile methodologies; it is a side-by-side comparison of the whole-of-life costs of developing, maintaining, supporting and extending each of these software solutions.
Thus far, the focus of the comparison between traditional and Agile software development projects has been focused on the relative quantitative benefits between these approaches. It is just as (if not more) important to compare the qualitative benefits as well, including:
- the impact that each approach has had on employee motivation, camaraderie and job satisfaction, particularly for the technical team members
- stakeholder response to the usability and relevance of the delivered software
- the impact that each approach has had on management.
Many of the benefits of Agile go well beyond what can be tracked on a balance sheet.
Comparing business value across Agile methodologies
If you have established a framework for tracking the quantitative net business value of Agile projects, then comparing your ROI across different Agile methodologies is also a relatively straightforward calculation. The important distinction is that you need to find a reasonable ‘apples-to-apples’ comparison between equivalent Agile methodologies. For example, Scrum and DSDM are both centred around iterative project delivery based on stakeholder feedback, so it is reasonable to compare outputs from these two methodologies side-by-side. Conversely, XP and Kanban are Agile methodologies with different functions (and outputs) to iterative project delivery. Therefore, it is more difficult to establish a meaningful ROI comparison between these Agile methodologies and iterative project delivery methodologies.
In comparing different Agile methodologies, it is especially important to compare the qualitative benefits as well, particularly to identify technical team and stakeholder reactions to each methodology. Were staff happier with the formal documentation structure of DSDM, or did they find that Scrum was sufficient to communicate requirements? Were they comfortable using the pair programming practices of XP? What were the biggest challenges in each methodology? Are there any areas where they would like more formal training or expert advice (e.g. how to manage a product backlog)? Staff input is critical to undertaking the Agile evolution tracking activities in the following section.
Tracking Agile evolution
As identified in Your Agile ROI, when you first introduce Agile methodologies within your organisation, there are likely to be additional overheads associated with training and knowledge sharing; establishing supporting technologies (e.g. automated testing environments); and establishing centralised locations for technical teams to work and collaborate. As Agile work progresses, staff members will become more comfortable with (and proficient in) these practices, and the methodologies that they use will most likely be adapted to work more effectively in your organisational climate. All of these factors contribute to the ability of Agile methodologies to provide the organisation with even greater business value as they evolve.
The framework that you have devised for tracking the quantitative net business value of Agile projects becomes a toolset that you can use to track the relative net business value of initial Agile projects against projects run by established Agile teams. This not only enables you to see progress in your Agile work, it provides you with a mechanism for undertaking predictive analysis if you decide to extend the use of Agile in your organisation. (See Expanding Agile for further details.)
In addition, the qualitative metrics that you have been gathering for each Agile methodology (e.g. what challenges the team faced) can be transformed into guidelines and customised training materials for staff to use in other Agile projects.
Don’t be fooled by the numbers
Although this section focused primarily on identifying baseline ROI metrics for quantitative comparison, monitoring your Agile investment is much more than a numbers game. Comparative whole-of-life software development costs will give you an indication of the bottom-line business value of Agile, but they will not help you to find the root cause of software errors, usability issues and misalignment with business requirements; nor will they help you to understand the technical team’s dynamics or frustrations. These indicators tend to emerge in more insidious ways, as emergency software patches, lost customers or valued members of your IT staff deciding to leave the organisation. Successfully addressing these issues is the immeasurable benefit of Agile.
After your organisation has had the opportunity to trial Agile methodologies for a few months, it is valuable for you to step back and ask yourself the following questions:
- Is work being done more efficiently?
- Are the stakeholders getting the outcomes that they need?
- Are employees happier to be working in a high-communication environment, rather than in a documentation-centric one?
- Is the quality of their work better than before?
The answers to these questions – in addition to the ROI comparisons described in the previous section – should provide you with sufficient information to consider broadening the use of Agile methodologies to other areas of the organisation. (Or, conversely, to decide that Agile methodologies are not suited to your organisation and should not be extended further; in which case, the low start-up costs of Agile have enabled you to make that decision without foregoing a huge up-front investment.)
If you decide to expand the use of Agile methodologies to other areas of the organisation, the next step is to establish a strategy for broadening awareness of the value of Agile methodologies across the organisation – and for encouraging other areas to trial these methodologies.25
- Information: educating the organisation on the quantitative and qualitative business value of Agile methodologies (using the ROI metrics, team feedback, etc.). This can be done through:
- internal ‘roadshow’ events to show people the tangible outcomes from your Agile work
- your corporate intranet (as described in Avoiding common traps)
- documenting the outcomes of your trial project as a case study for other groups in the organisation.
- Motivation: using the information above, along with your influence, to encourage specific people in the organisation to trial these methodologies in their area (e.g. those who are more open to trying new approaches, or those who have had historical problems with their software development projects).
- Selection: helping interested areas of the organisation in selecting the Agile methodology(ies) that are best suited to their activities (using the Agile methodologies selection workflow tool). The metrics gathered in the previous section can also help these areas to undertake predictive analysis on initial and ongoing ROI benefits for each methodology.
- Collaboration: providing assistance (and, where appropriate, experienced Agile team members) to help each area in their initial application of these methodologies. This includes educating the area on the principles and practices of their selected Agile methodology via:
- an easy-to-use guide that explains the basics of Agile methodologies (such as the ‘Agile Cookbook’ that was created by BT)
- internal training sessions to walk through and demonstrate these methodologies.
Proper understanding of these principles and practices will significantly reduce the potential for the misapplication of Agile methodologies, which could eliminate the possibility of wider organisational support altogether.
As the adoption of Agile methodologies grows and matures in your organisation, you can further refine your use of Agile by enlisting qualified consultants, attending training courses and reading industry resources, such as those listed in More Information on Agile.
One final thought: every time staff members work on an Agile project, the organisation is likely to grow more confident in these methodologies. Management’s initial gut reaction to resist empowering the technical team is replaced by the proven knowledge that this is an extremely effective way to achieve successful outcomes. Stakeholders’ inclination to want everything delivered at once is replaced by an appreciation for prioritising outputs by the business value that they can bring to the organisation.
That is why it is so valuable for people who have been through the process before to work with new areas in the organisation which are trialling Agile methodologies. These experienced Agile resources can act as advisers and facilitators in the process, ensuring that practices are followed correctly, and allaying concerns that staff, managers or stakeholders might have as they move away from their traditional ways of working. Furthermore, once these areas have been through a couple of Agile initiatives, they can begin to take on the adviser role for others in the organisation. This not only creates a larger (and stronger) network of Agile practitioners within the organisation, it decentralises the responsibility for any one area to be involved in each Agile initiative.
The bottom line is that your organisation can achieve real productivity gains using Agile methodologies. The challenge is to implement Agile in a way that aligns with the specific needs, constraints and dynamics of your organisation.
21 The Agile methodologies selection workflow tool only addresses the six Agile methodologies described in the Common Agile methodologies section of Chapter 1. It does not include RUP, AUP, Crystal or other methodologies which may be equally valuable to your organisation.
22 Details on the approach that BT successfully used is provided in the Avoiding common traps section of this chapter, with further details available at Agile Coaching in British Telecom, Meadows L and Hanly S (2006): http://www.agilejournal.com/articles/columns/column-articles/144-agile-coaching-in-british-telecom.
24 As many IT projects pre-allocate funding for warranty period work and/or charge back annual support costs, non-recoverable costs refer to the portion of software maintenance work that cannot be recovered from allocated funds. There is, however, an argument to say that pre-allocated funding for software support should be focused on upgraded features and enhancements; in which case, the cost of bug fixes within these funds should also be considered non-recoverable.
25 That is, unless you decide to issue an organisation-wide Agile mandate, as BT did.