Chapter 4 Project Selection and Approval – Core Concepts of Project Management

CHAPTER 4

Project Selection and Approval

Key Points

  • Factors important in project selection
  • Project selection methods
  • Cost–benefit analysis and net present value
  • Multiple objective analysis

The project process begins with a project proposal, generated by users or management. The first step is an initial selection of projects, either on a case-by-case basis, or periodic selection by committees. A related decision of great importance is to keep track of the progress of a project, so that those that are not going to provide value to the organization can be canceled.

Analyzing the expected financial impact of a software project is referred to as a business case. Business cases can be accomplished in a number of different ways—some financial, others more subjective; some widely used, some offered as potential improvements. This chapter discusses the decision problem of project selection, and shows how the most commonly used methods work. Analytic methods support this selection process in two ways. First, they provide decision makers with analysis of expected outcomes from adopting specific projects. Second, they provide a basis for communication, so that the reasoning behind a selection decision can be explained to others.

Measurement of Project Impact

Information systems projects typically involve benefits that are difficult to measure in terms of concrete monetary benefits. This vastly complicates sound management, because cost–benefit analysis, the ideal tool for evaluating project proposals, will not always accurately reflect project benefits.

Intangible factors: Both costs and benefits tend to have intangible features. The tangible costs and benefits tend to be historical, backed by data or solid price quotations from vendors. But many of the benefits are expected in the future, and are very difficult to measure. These include expected increases in market share, improved customer service, and better corporate image. These would all have a significant impact on the corporation’s bottom line, but guessing exactly what that impact would be is challenging at best.

Hidden outcomes: Other aspects of information technology projects often involve unexpected results. Information technology projects can impact organizational power. New projects can change the power-specific groups may have held in the past, which can have a negative impact on the teamwork of the organization. Information technology also includes components of the organization’s communications network. Often different elements of the organization can adopt projects that impact the organizational communications network without this impact being considered. This can result in duplication of efforts, or development of barriers between groups within the organization. Computers can make work more productive and more attractive. But they also can change work roles to emphasize skills in which specific employees have no training, making them feel less productive.

Failure to identify the impact of projects often is not noticed until project implementation. At that stage, the problems created are more difficult to deal with. It is important to consider the systems aspects of projects, and try to predict how the project will change when people do their jobs. Thorough user involvement can make project impact more obvious, as well as easier to reconcile and convince users of the project’s benefits.

The changing nature of information technology: There are many excellent applications of computer technology to aid businesses. But a major problem is that technology is highly dynamic. Some information systems projects take years to implement. This can result—and indeed has often resulted—in the installation of a new system after it is outdated by even newer technology.

Selection Practice

Information technology tended to be treated as a capital investment. The following financial techniques have often been used.

  1. Payback
  2. Discounted cash flow
  3. Cost–benefit analysis

Each of these methods will be demonstrated in this chapter. Decision makers treat information technology investment more like operations projects with measurable profitability requirements than the revenue-based appraisals appropriate for training and marketing. Operations investments focus on efficiency-related measures. Conversely, marketing investments are usually viewed in terms of their expected impact on improving competitive position, or increasing market share. The capital approach was very appropriate for hardware proposals, but difficulties were encountered when justifying software. Intangible benefits such as competitive advantage or improved practices tended to be disregarded because they are hard to quantify. The most commonly cited justification was reduction in expenses, usually in payroll. The second most common justification involved the subjective aspect of accomplishing some strategic objectives. Intangible benefits cited were enhanced patient care and satisfaction for a health care organization, better speed of responding to customers, and the need to satisfy governmental regulations.

Net present value masks some of the true value of information technology proposals. On the other hand, some projects that have a low impact on corporate performance often appear attractive to cost–benefit analyses. This is because cost–benefit analysis emphasizes those features most easily measured. The value of information technology projects is in making organizations more competitive, increasing customer satisfaction, and operating more effectively. These are sometimes the intangible strategic benefits that are often disregarded because they were not measurable.

Cost–benefit analysis should consider costs over the entire life cycle of the project. Simms stated that life cycle costs are roughly four times those of development costs for most information systems projects. But these long-range costs are much less predictable, and therefore often not included in cost–benefit analyses.

Another issue involves who should do the estimating. In construction, a common practice is to have those who will be responsible for implementing the project do the estimation. This is in a competitive bidding environment, where there is great danger in estimating too low. The motivation is for these estimators to be very cautious. This tends to result in safe estimates considering every possible risk, which is another way of saying that it results in inflated estimates. (Inflating estimates is how you cope with risk at the estimation stage.) In the information systems field, there is far more work to do than the capacity to do it. Therefore, the environment is less competitive. While information system consulting is rapidly growing, in the private sector, competition is based more on negotiation and perceived quality than competitive bidding on price. Therefore, estimators do not have the motivation to consider all risks. Software developers who estimate their own performance tend to be optimistic.

Project Evaluation Techniques

A number of methods exist to evaluate project proposals, either from the perspective of selecting the best option available, designing an ideal option, or rank-ordering options. This section demonstrates some of the most widely used methods, and shows how other methods can be used to consider other factors describing expected project performance.

General Project Selection

Economic and financial analyses include payback (determining the expected time until investment is recovered) and cost–benefit analysis. Net present value and internal rate of return extend cost–benefit analysis to consider the time value of money, appropriate when projects are lengthy (3 years or more).

Checklists describe the criteria of importance and their minimum acceptable levels of requirement. Screening methods are a variant of checklists, eliminating projects that do not have minimum estimated performance on specific measures. Project profiles describe the performance of each project on criteria, so that the decision maker can see each project’s strengths and weaknesses. Scoring and rating models are a simple form of multicriteria analysis where measures are obtained on each criterion of importance, and combined in some fashion. Multicriteria decision models are in general more formal than scoring and rating models, but operate on essentially the same principle—identify factors that are important, measure how well each project does on each factor, and combine these into some value score that can be used for ranking.

The rest of this chapter will demonstrate most of the basic methods. The first method demonstrated is screening, a common form of the checklist method that can be and often is combined with other criteria.

Screening

Screening is a process that is very useful in cutting down the dimensions of the decision problem. The way in which screening operates can vary widely in detail, but essentially involves identifying those factors that are important, establishing a minimum level of importance, and eliminating those projects that fail on any one of these minimum standards. Obviously, if the standards are set too high, the decision problem disappears as no projects survive the screening. This is appropriate if the minimum standards reflect what management demands in return for their investment.

To demonstrate screening, assume that 100 information systems project proposals are received this month. All of the projects were evidently worthwhile in someone’s mind, but management must consider budgets and other resource limitations. Assume that the criteria and minimum performance levels given in Table 4.1 are required:

Table 4.1 Screening example

Expected return on investment

At least 30%

Qualified project team leadership

Available

Company expertise in this type of work

Either company has experience or desires to gain it

Project completion time

Within 48 months

If any of the 100 proposed projects failed to meet all four of these standards, they would be rejected preemptively. This reduces the number of proposed projects for a more detailed analysis. This approach can be implemented by checklists, which give clearly defined standards on those areas of importance to management.

Screening is good at quickly weeding out those projects with unacceptable features. The negative side of screening is that trade-offs between very good features and these unacceptable features are disregarded. The willingness of decision makers to accept lower return on investment (ROI) for projects with strategic importance is disregarded. For those projects for which such trade-offs are not important, screening is a very efficient way to reduce the number of proposals to a more manageable number.

In the prior section, we gave a list of risk factors for information system projects. These could be implemented as a checklist by management specifying minimum acceptable measures that can be used to screen individual projects. Not all risk elements might apply for a given organization’s checklist. An example checklist is presented in Table 4.2:

Table 4.2 Example checklist

Factor

Minimum standard

Project manager ability

Qualified manager available

Experience with this type of application

Have experience, or application is a strategically key new technology

Experience with system or language

Personnel with experience can be obtained

Familiarity with modern programming practices

If not, training is available

Availability of critical equipment, software, and programming language

Each critical component available

Project team complete

Key personnel identified and agreed to work;

Support personnel easily available

Checklists ensure implementation of policy limits. Checklists are a way of implementing screening from the perspective of what features management feels are important. The next step in analysis is to more directly compare alternative project proposals.

The intent of a project profile is to display how the project proposal compares to standards as well as how the project compares to other proposals. Profiles have a benefit over screening limits, because poor performance on one factor can be compensated for by strong performance on another factor. For instance, matching with company strategic programs can be an important factor. There could be other project proposals that contribute nothing to the firm’s strategic program, yet have an outstanding cost improvement for administrative work. This would be reflected in very strong performance on ROI. Conversely, another project may have a slightly negative ROI calculation, but may involve entering a new field in which the firm wants to gain experience.

To demonstrate project profiles, assume a firm has a number of information projects proposed. This is generally a large list, because of the many beneficial things information technology can do for organizations. Here we give a short list of six proposals in Table 4.3, measured for resources used, as well as benefits expected.

Table 4.3 Project profiles

Project identifier

Estimated cost

Systems analysts

Cash flow this period

NPV/Cost ratio

Key to strategy

A265

230,000

3

100,000

0.43

No

A801

370,000

4

−190,000

0.51

Yes

A921

790,000

5

360,000

0.46

No

B622

480,000

3

−52,000

0.11

Yes

B837

910,000

7

−200,000

0.22

Yes

C219

410,000

3

170,000

0.41

No

A profile displays the characteristics of individual projects. Estimated cost is needed to determine if available budget can support a project. The same is true for other scarce resources, such as systems analysts in this case. A tabular form is given above. Graphical displays and ratios are often valuable to give a measure with which relative performance can be measured. For measures such as NPV/cost ratio, cutoff levels can be used to screen out projects. For instance, a 40 percent return on estimated cost in net present terms might be desired. Project B622 and B837 are both below this limit, and might be screened out. However, both of these projects are listed as key to the organization’s strategy, and management might be willing to accept a lower return for the potential for advancing organizational strategy.

Traditional Analysis

Assume that a company is proposing an information technology project to improve their operations. They can build the system in-house, or outsource (in effect, installing an over-the-counter, or OTC software product). One year has been allocated to develop and install the system (either in-house or OTC). The benefits of the proposed project are expected to be obtained for 3 years after project completion. Staff will be required to operate the new system. There is some money budgeted to train this staff in the initial year. Staff payroll expenses for this operation include inflation reflected in operating expenses. The company has a marginal value of capital of 15 percent per year.

Cost–Benefit Analysis

The cost–benefit calculation for a new project requires identification of benefits in monetary units. Use of net present value requires identification of the timing of monetary exchanges. The benefit from the new project consists of lowered personnel costs. Calculation of costs and benefits is given in Table 4.4:

Table 4.4 Simple cost–benefit calculation

Year

Develop in-house

Operating cost

Benefit

Net

0

$300,000

$80,000

($380,000)

1

$180,000

$250,000

$70,000

2

$200,000

$350,000

$150,000

3

$225,000

$450,000

$225,000

TOTALS

$300,000

$685,000

$1,050,000

$65,000

Year

Outsource

Operating cost

Benefit

Net

0

$500,000

$20,000

($520,000)

1

$60,000

$250,000

$190,000

2

$80,000

$350,000

$270,000

3

$110,000

$450,000

$340,000

TOTALS

$500,000

$270,000

$1,050,000

$280,000

The outsourced project appears to have a strong advantage in this case. The cost–benefit ratio requires some convention to describe just what costs are assigned to investment and which ones to benefits. Operating expenses seem more appropriately combined with savings, but note that different views might yield opposing conclusions.

Ratio: The nominal cost–benefit ratio (disregarding the time value of money) for the in-house project is $365,000/$300,000 = 1.22. This indicates that the project is worthwhile, in that the extra initial expenses of $300,000 would be exceeded by expected benefits by 22 percent. For the outsourced project, this ratio is $780,000/$500,000 = 1.56. By these measures, the outsourced project appears to have a significant cost– benefit advantage.

Return on investment (ROI) is defined as net project benefits divided by investment (times 100 if you want to view ROI in percentage return). It can be applied to non-discounted cash flows as with cost–benefit analysis. For long-term projects, discounted ROI should be used, which we will demonstrate in the net present value section.

Cost–benefit analysis should consider costs over the entire life cycle of the project. Life cycle costs are roughly four times development costs for most information systems projects, but these long-range costs are much less predictable, and therefore often not included in cost–benefit analyses.

Payback

Payback is a rough estimate of the time required to recover investment. While being simple, payback presents a view of the transaction that is very understandable and important to managers. One alternative may be superior to another on the net present value of the total life cycle of the project. However, cost–benefit analysis does not consider the impact of negative cash flow in early periods. For instance, in our process reengineering example, the cash flow would be as given in Table 4.5. The net benefit column is calculated by subtracting the cost of the new system from the cost of the old system by year. In the first year, this is negative, due to the high investment cost of the new system. In years 2 through 6, the new system provides a positive net benefit relative to the old system.

Table 4.5 Payback

Year

In-house net

Cumulative

Outsource net

Cumulative

0

($380,000)

($380,000)

($520,000)

($520,000)

1

$70,000

($310,000)

$190,000

($330,000)

2

$150,000

($160,000)

$270,000

($60,000)

3

$225,000

$65,000

$340,000

$280,000

Both alternatives gain a nominal advantage by the end of year 3 (payback is about 2.7 years for the in-house alternative, about 2.2 years for the outsourced alternative). However, $380,000 has been sacrificed at the beginning for the in-house alternative and $520,000 for the outsourced alternative. One of the most common reasons for company failure in the United States is lack of cash flow. In this case, if the firm has cash-flow difficulties, the outsourced option would be less attractive than if they had adequate cash reserves.

Net Present Value

We can modify the cost–benefit ratio by considering the time value of money. In this project, for instance, the nominal expected gains of $1,050,000 are spread out over 3 years, while the development costs are incurred at the beginning. Having the $380,000 (or $520,000 for the outsourced alternative) would mean that the company would not be able to adopt some other investments (and maybe even force the firm to borrow money). The marginal value of money for the firm is 15 percent per year. Net present value converts a time stream of money back to its worth in today’s terms (or in terms of the project’s start, or any other specific time of reference).

Table 4.6 shows the changes in cash flow between the two systems (shown in the net difference column). Discounting each year’s net change in cash flow by the discount rate of 1.15 per year to the t-th power, where t is the time period, we get the following. Note that initial expenses are treated as occurring during year 1.

Table 4.6 Net present value

Year (t)

In-house net

Divide by 1.15t

Outsource net

Divide by 1.15t

0

($380,000)

($380,000)

($520,000)

($520,000)

1

$70,000

$60,870

$190,000

$165,217

2

$150,000

$113,422

$270,000

$204,159

3

$225,000

$147,941

$340,000

$223,556

NPV

($57,768)

$72,932

Viewed in this light, relative to obtaining a return of 15 percent per year on alternative investments, adopting the in-house alternative would be like writing a check initially for over $57,000, while the outsourced alternative would be equivalent to a gain of over $72,000.

A related concept is internal rate of return (IRR), which is the marginal value of capital for which the net present value of a stream of cash flow would break even, or equal zero. Note that IRR can equate to ROI in that ROI is viewed with discounted cash flow. In this case, the IRR amounts to 1.07 for the in-house alternative, for 7 percent average return, while the IRR of the outsourced alternative is 1.225, or an average return of 22.5 percent. By these measures, the outsourced alternative is attractive and the in-house alternative is not.

Cost–benefit analysis seeks to identify accurate measures of benefits and costs in monetary terms, and uses the ratio benefits/costs (the term benefit–cost ratio seems more appropriate, and is sometimes used, but most people refer to cost–benefit analysis). For projects involving long time frames, considering the net present value of benefits and costs is important.

Other Factors

There are a number of complications that can be brought into the calculation of cost–benefit ratios. One of the most obvious limitations of the method is that benefits, and even costs, can involve high levels of uncertainty. The element of chance can be included in cost–benefit calculations by using expected values. For instance, the demand for production output appears to be increasing. Therefore, using an expected demand of 30,000 units per year is probably conservative. Demand could very well continue to increase. The expected value calculation can be quite complicated in its purest form, consisting of identifying all possible demands for a given year and associating accurate probabilities to each outcome. Instead of getting involved in such a speculative and detailed exercise, most managers do what we did—assume a conservative value. However, it should be recognized that there is added benefit to the new machine in its ability to expand. If this expansion capacity is not considered in the cost–benefit calculation, the new machine option is not accurately evaluated.

For instance, if growth in demand was expected to increase at the rate of 1,000 units per year, this form of benefit for the new machine could be reflected as follows, where an extra ($2 − $1.20) × 1,000 units/year = +$800 in gain is obtained each year. As before, net difference is calculated as the new machine cash flow minus the old machine cash flow. Table 4.7 shows the calculations.

Table 4.7 Net present value calculation

Year (t)

Old machine

New machine

Net difference

Divide by 1.15t

0

0

−$100,000

−$100,000

−$100,000

1

$6,000

$24,000

+$18,000

+$15,652

2

$6,000

$24,800

+$18,800

+$14,216

3

$6,000

$25,600

+$19,600

+$12,887

4

$6,000

$26,400

+$20,400

+$11,664

5

$6,000

$27,200

+$21,200

+$10,540

6

$6,000

$28,000

+$22,000

+$9,511

7

$6,000

$28,800

+$22,800

+$8,571

8

$6,000

$29,600

+$23,600

+$7,715

9

$6,000

$30,400

+$24,400

+$6,936

10

$6,000

$31,200

+$25,200

+$6,229

NPV

+$3,921

This would involve a cost–benefit ratio in net present value terms of $103,921/$100,000 = 1.04, and an ROI of 1.160, greater than the company cost-of-capital of 15 percent.

The cost–benefit ratio does not reflect intangible benefits unless they are presented in monetary terms. Cost–benefit analyses have included measurements for intangible items, but they tend to be given lower values because of the uncertainty involved in their estimates. Detailed analyses of the decision maker’s willingness to pay for intangible factors have been conducted, but can be time-consuming and less than convincing. Governments have encountered some problems in applying cost–benefit analysis to public works, to include (1) evaluating the benefit of recreational facilities and (2) placing a dollar value on human life. When a dam is built, there clearly is benefit obtained from providing many citizens much improved fishing and water sports. (There is also added cost in depriving citizens of the opportunity to view some flooded historical sites.) The approach usually taken has been to place some dollar value on recreation, based on some very insubstantial measures. The evaluation of human life has also been tackled by economists, who have applied things like the net present value of the expected earnings of those whose lives are expected to be lost in some proposed investment project. This of course involves high levels of speculation as well, because the calculation assumes certain ages, assumes that the only value of a human is what they earn, and disregards who pays and who benefits.

If a firm was threatened with a severe monetary penalty for not complying with a governmental regulation with respect to environmental pollution or safe working conditions, a net present value analysis might well lead to the conclusion that it would be rational to pay the penalty and avoid improving operations. For instance, assume that a blast furnace is pouring out black matter at a phenomenal rate that the government finds terribly offensive. Governmental regulations call for a penalty of $10,000,000 if the pollution source is not cleaned up within 1 year. Hard-core cost–benefit analysis would identify the cost of cleaning up the facility, which might involve an expense of $12,000,000 in equipment and installation, and an added cost of operations of $5,000,000 per year over the next 8 years, the remaining life of the equipment. At a discount rate of 12 percent per year, the net present value of benefits and costs would be as shown in Table 4.8. The net column shows discounted values for benefits and costs. Totals are presented in the end.

Table 4.8 Net present value calculation

Year

Benefit

Net

Costs

Net

1

$10,000,000

$8,928,571

$17,000,000

$15,178,571

2

$5,000,000

$3,985,969

3

$5,000,000

$3,558,901

4

$5,000,000

$3,177,590

5

$5,000,000

$2,837,134

6

$5,000,000

$2,533,156

7

$5,000,000

$2,261,746

8

$5,000,000

$2,019,416

Total

$8,928,571

$35,552,483

Here the ratio of net present benefits to net present costs is $8,928,571/$35,552,483 = 0.25, well below 1.0, indicating that the rational decision maker would pay the fine and keep operating as is. But the government did not impose the fine limit for the purpose of raising money. They imposed the fine as a means to coerce polluters to clean up operations. The U.S. Congress has no trouble adding extra zeroes to penalties. If the firm continued to pollute, it is not too hard to imagine the penalty being raised in the future to a much larger figure. There have been actual cases similar to this scenario, where within 3 years the penalty was raised to much larger values, providing a much different cost–benefit ratio. Benefits are often difficult to forecast.

To demonstrate intangible benefits in our machine investment case, one of the clear advantages the new system has is in its higher level of quality output. The loss of discarded products is included in the analysis. But poor production quality results in more than just the identification of products not passing testing limits and rejecting them. Quality comes on a continuous scale. The older machine will in all likelihood include the production of a number of products that barely pass test limits, and do not contain desired levels of quality. The customer will accept them, but slowly over time a reputation for inferior workmanship will result. The new machine should improve the company’s image with respect to quality. Placing a dollar value on image is pure speculation. Rigorous proponents of cost–benefit analysis would disregard such benefits that are immeasurable as “soft,” and not worthy of hard-core analysis. Managers with more vision would recognize that there was a relative advantage for the new machine that was not reflected in the cost–benefit analysis.

An additional benefit of the new machine is that it is more flexible. It has the capacity to respond to larger markets because it has added capacity. At the moment of analysis, there is only one customer. We have just considered the impact of increased demand on the part of this customer, but there might be additional sources of sales in the future. This again would be highly speculative. A hard approach would require the decision maker to identify a specific expected increase in demand. A soft approach would list flexibility as a measure of importance. There could also be other advantages that are intangible, such as the safety of workers, impact on market share, or replenishing capital equipment, so that old facilities do not fall apart, disrupting the ability of the firm to conduct business.

Multiple Objectives

Profit has long been viewed as the determining objective of a business. However, as society becomes more complex, and as the competitive environment develops, businesses are finding that they need to consider multiple objectives. Although short-run profit remains important, long-run factors such as market maintenance, product quality, and development of productive capacity often conflict with measurable short-run profit.

Conflicts

Conflicts are inherent in most interesting decisions. In business, profit is a valuable concentration point for many decision makers because it has the apparent advantage of providing a measure of worth. Minimizing risk becomes a second dimension for decision making. There are cash flow needs, which become important in some circumstances. Businesses need developed markets to survive. The impact of advertising expenditure is often very difficult to forecast. Yet decision makers must consider advertising impact. Capital replenishment is another decision factor which requires consideration of trade-offs. The greatest short-run profit will normally be obtained by delaying reinvestment in capital equipment. Many U.S. companies have been known to cut back capital investment to appear reasonably profitable to investors. Labor policies can also have an impact on long-range profit. In the short run, profit will generally be improved by holding the line on wage rates and risking a high labor turnover. There are costs that are not obvious, however, in such a policy. First, there is training expense involved with a high turnover environment. The experience of the members of an organization can be one of its most valuable assets. Second, it is difficult for employees to maintain a positive attitude when their experience is that short-run profit is always placed ahead of employee welfare. And innovative ideas are probably best found from those people who are involved with the grass roots of an organization—the workforce.

This variety of objectives presents decision makers with the need to balance conflicting objectives. We will present the simple multi-attribute rating technique (SMART), an easy to use method to aid selection decisions with multiple objectives.

The simple multi-attribute rating technique (SMART) method offers a simple way to quantify subjective elements. There are two elements to the value function in SMART: factor weight (wi) and score (sij) of each alternative j on each factor. Overall value is then:

Weights wi can be obtained in a number of ways, but a simple estimate with some reliability involves ranking the factors, considering the range of likely performances, and then estimating the relative value of moving from the worst to the best performance relative to such a swing from either the most important or the least important. We will do both, and use a rough average of these two estimates. In the example used earlier in this paper, assume that the ranges and rankings for the five criteria are as given in Table 4.9.

Table 4.9 Criteria weight development

Criterion

Worst

Best

Rank

Best = 100

Worst = 10

Risk

Very high

Very low

2

70

120

Market share

Very negative

Very positive

1

100

150

Control

Low

High

5

5

10

Investment

$1,000,000

$100,000

4

25

40

Return (NPV)

0%

100%

3

50

80

First, the extreme expected measures for each criterion are identified, for the decision maker to understand the range of possibilities to compare. Then, the decision maker ranks the importance of swinging from the worst measure to the best measure for each criterion. In this case, the decision maker considered it most important to move market share impact from very negative to very positive, followed by moving risk from very high to very low, moving expected return from 0 percent to 100 percent, investment from $1 million to $100,000, and finally (and least important) moving control from low to high. Next the decision maker is asked to quantify these changes. Viewed from the perspective of the most important criterion, if changing market share impact from very negative to very positive were worth 100, the decision maker assigned a value of 70 to risk, 50 to return, 25 to investment, and 5 to control. To get a second estimate, with 10 assigned to the least important criterion (control), the assigned values were 40 for investment, 80 for return, 120 for risk, and 150 for market share. This information is then used to generate the set of weights wi. Each of the sets of assigned weights is totaled, and each entry divided by the appropriate total. This gives two estimates of weights, which the decision maker can use to compromise (preferably using rounded numbers) as in Table 4.10.

Table 4.10 SMART calculation of weights

Criterion

Best = 100

Divide by sum

Worst = 10

Divide by sum

Compromise

Risk

70

0.28

120

0.30

0.29

Market share

100

0.40

150

0.375

0.39

Control

5

0.02

10

0.025

0.02

Investment

25

0.10

40

0.10

0.10

Return (NPV)

50

0.20

80

0.20

0.20

The scores (sij) for each alternative on each criterion are now needed. These can be assigned directly, to reflect any kind of utility function for each criterion. Better performance should always receive a higher score than an inferior measure. A possible set of assigned scores are given in Table 4.11.

Table 4.11 SMART value calculation

Weights

0.29

0.39

0.02

0.10

0.20

1.00

Criteria

Risk

Mkt Share

Control

Invest

NPV

Values

In-house

0.1

0.5

0.9

0.8

0.1

0.342

Outsource

0.8

1.0

0.3

0.6

0.9

0.868

In this case, the outsourced alternative has a clear advantage.

Summary

We have reviewed some of the primary methods used to evaluate project proposals. Screening provides a way to simplify the decision problem by focusing on those projects that are acceptable on all measures. Profiles provide information that display trade-offs on different measures of importance. Cost–benefit analysis (with net present value used if the time dimension is present) is the ideal approach from the theoretical perspective, but has a number of limitations. It is very difficult to measure benefits, and also difficult to measure some aspects of costs accurately. One way of dealing with this problem is to measure more accurately. Economists have developed ways to estimate the value of a life and the value of scenic beauty. However, these measures are difficult to sell to everybody.

A more common view is that it is wasted effort to spend inordinate time seeking a highly unstable and inaccurate dollar estimate for many intangible factors. Value analysis is one such alternative method. Value analysis isolates intangible benefits from those benefits and costs that are more accurately measurable in monetary terms, and relies upon the decision maker’s judgment to come to a more informed decision. The SMART method, one of a family of multiple criteria decision analysis techniques, provides a way to quantify these intangible factors to allow decision makers to trade-off values.

Cost–benefit provides an ideal way to proceed if there are no intangible factors (or at least no important intangible factors). However, usually such factors are present. Intermediate approaches, such as payback analysis and value analysis, exist to deal with some cases. More complex cases are better supported by multiple criteria analysis. In cases of constraints, such as budgets, it is sometimes appropriate to optimize over some objective. Linear programming provides a means of generating the best portfolio of funded projects subject to constraint limits given that accurate measures of project performance are available.

Glossary

Business case. Financial analysis of a proposed information systems software project.

Checklist. A means of evaluating initial project proposals in terms of a list of expected characteristics.

Costbenefit analysis. Economic analysis pricing all factors, dividing benefits by investment.

Discounted cash flow. Conversion of cash flows into their worth as of some base period, usually the present.

Intangible factors. Items of value that are difficult to quantify in monetary terms.

Internal rate of return. The marginal value of capital that yields a net present value of zero for a stream of cash flow.

Multiple criteria analysis. Analysis considering not only one measure of value (such as profit), but the value trade-offs over multiple measurement scales.

Net present value. The worth, in today’s terms, of a stream of cash flow over time, assuming a given marginal cost-of-capital.

Optimization models. Models of decisions allowing the identification of the decision that is optimal with respect to selected criterion (or criteria).

Payback. Financial analysis estimating the time required to recover investment.

Project Profile. A display of expected project performance on important criteria, allowing a comparison of alternative projects.

Return on investment (ROI). Rate of financial return on investment, defined as net profit divided by investment.

Screening. A method of implementing a checklist for project evaluation by listing the worst acceptable performance on a list of criteria.

SMART. Simple multi-attribute rating theory, a means of objectively comparing the values of projects considering multiple criteria.

Tangible benefits. Benefits that are measureable; in cost–benefit terms, measurable specifically in monetary terms.

Time value of money. The current worth of future flows of cash flow.

PMBOK Items Relating to Chapter 4

Business value is defined as the entire value of the business.

All projects can be mapped to the generic project life cycle structure of start, organize and prepare, carry out, and close.

A project phase is a collection of logically related project activities that culminates in the completion of one or more deliverables.

4.2 Develop Project Management Plan—map out how the project is to be accomplished.

7.1 Plan Cost Management—process that establishes the policies; procedures; and documentation for planning, managing, expending, and controlling project costs.

12.1 Plan Procurement Management—process of documenting project procurement decisions, specifying the approach, and identifying potential sellers.

12.2 Conduct Procurements—process of obtaining seller responses, selecting a seller, and awarding a contract.

Thought Questions

  1. Compare the ease of estimating costs and benefits of projects with estimation of costs and benefits in repetitive operations. Why might they be different?
  2. Cost–benefit analysis is cited as the ideal method for economic analysis in some circles. What are the limitations and/or issues with cost–benefit analysis?
  3. Describe a few multiple criteria that typically are involved in projects.