Financial metrics for IT: the holy grail of ROI, and how it misses the point: Part 1

Let’s talk some more about one of my favorite topics, project portfolio management (PPM). A lot of literature on PPM tends to focus on evaluating risks and returns. An excellent article on IT governance last week in The Wall Street Journal had the following sage advice:

“Create an IT portfolio by evaluating risks and returns. Just as an investor balances risk and returns in constructing a portfolio of investments, management should analyze the costs, benefits and risks of all IT projects to determine how to get the most benefit from the dollars invested in technology.”

I can’t argue with that. But I also like to talk about another major part of IT portfolio management, which focuses on juggling which projects can actually be resourced. It’s unfortunately easy to come up with ten distinct projects with positive return on investment (ROI), for example, in a situation where it’s really only feasible to do one or two of these a year. In some companies, the pressure to do any positive-ROI project becomes enormous, even if it means the company is biting off too much at once. So what to do?

One way to think about it is that there are two major categories of “disciplined decisions” about projects: CAN we do it, and SHOULD we do it? Both disciplines are needed; organizations tend to struggle with both, and decisions on both are often made by the seat of the pants.

• “CAN we” has to do with resources and capabilities, and whether the ones needed for the project fit in the scheme of everything else that’s going on. The results are usually expressed in manpower loading charts, as well as needs analysis papers that cover the trade-offs involved in bringing the proposed project to fruition.
• “SHOULD we” has to do with strategy, risk, cost, and expected return. The results are often expressed in financial metrics such as ROI, NPV, IRR, and so on. Note that there can often be projects that we “should” do (in a metrics-based view), but which we can’t (resource-based). That seems obvious? Well, believe me, it often isn’t.

If your organization is just starting out along these lines, it doesn’t make much sense to try to instill both the “Can we“/”Should we” disciplines all at once from scratch. If you have neither discipline well-entrenched in your organization, I recommend trying to establish the “Can we” discipline first: figure out ways to estimate manpower loading, assess project needs, and conduct executive-level project prioritization, so that you can collectively “right-size” what you take on as an organization.

What if you’re lucky enough to be in an organization that clearly understands that there’s a need for both disciplines, and already has some semblance of this “Can we” kind of careful resource allocation? Then I recommend for all envisioned projects (of sufficient size), the “Should we” aspects deserve to be evaluated first, simply to determine project eligibility (and assistance in prioritization) for further consideration in the portfolio. Evaluating the “Should we” aspects without a firm grasp of the “Can we” side of things? That’s a recipe for frustration and eventual failure. So, this post and (especially) the next will cover the “Should we” aspects, in detail and with specific tools recommended, while a later post will go into more detail on the numerical sides of “Can we“.

Just to get quantitative for a moment: here are some sample financial metrics for a project; by all means, you want to get to the point where you can reliably calculate, present, and make project choices based on such concrete metrics:

For detailed explanations of the meaning and suggested interpretation of each of these financial metrics, please see some of the references contained in the Lagniappe section at the bottom of this post. (NOTE: the spreadsheet itself is supplied in a link at the bottom of Part Two of this post, here.)

In this day of spreadsheets and canned financial functions, though, the basic calculation of all those metrics is almost too easy, really just a few formulas away. My belief is that metrics are hugely important for decision-making, but that the data that goes into calculating the metric is what matters most of all. Specifically, what are the assumptions? What’s the analysis behind the numbers? Are you taking all the project factors into account? Have you brainstormed through all the costs and hammered on what benefits are truly achievable?

The point is to get as complete a picture as possible by soberly laying out the costs and the benefits, over time, taking everything into account. This is what I’ve seen few organizations do at all, and even fewer do well. In fact, many organizations will simply spout that “we base all project selection on demonstrated ROI,” without actually having a rigorous process in place for ensuring that they capture all the costs and all the benefits that go into calculating that ROI. A nice-looking ROI number comes out of the process, but the assumptions behind it aren’t well thought-out, so it’s fairly meaningless.

So as I discuss ROI, NPV, and other “Should we” metrics, I want to deemphasize the metrics themselves, and focus on what should be meticulously and methodically gathered in terms of those core assumptions.

Hard benefits tend to boil down to two major categories:

• increased revenue
• decreased costs (less personnel required, hardware or software that can be decommissioned, etc.)

Similarly, costs boil down to two categories:

• One-time costs (cost for acquisition of software and/or hardware, implementation costs)
• Recurring costs (annual maintenance fees, dedicated personnel needed for support, etc)

Each of these major areas should be the topic of intense “devil’s advocate” brainstorming with your team and with business stakeholders, as appropriate.

• What really are the prospects for additional revenue? Are you perhaps fooling yourselves and being overoptimistic? Is the business stakeholder willing to stand behind these estimates, on the record? And, if you think you can reduce costs and/or increase productivity, would you actually be willing to have those dollars removed from your budget at a defined point once the project is implemented? A dose of conservatism should reign here.
• What really are all the costs, both one-time and recurring? Have you thought of everything? Are you counting in costs such as shipping, or sales tax on equipment or software (an instant 8.8% budget jolt, depending on what state you’re in)? What about the full time frame? Are you thinking out the costs and the benefits for five years, and taking into account possible cost increases during that time?

In a subsequent post, we’ll examine a specific project example, showing the assumptions, the careful analysis, and then (almost secondarily) the “Should we” metrics that result.

Lagniappe:

• Gary Anthes, ComputerWorld, “Money’s tight? ROI to the rescue“
• Scott McCready, CIOView.com, “TCO, NPV, EVA, IRR, ROI: Getting the Terms Right“
• “Project Evaluation and Selection Analysis Techniques“