Far fewer soft benefits

Recently I was presented with an ROI for some project.  It included some costs, which weren’t that interesting and it included one hard benefit which was a time savings.  And then it included a series of soft benefits including improved quality, improved customer satisfaction, etc.

Now, as far as I understand it, a hard benefit is one which you can associate a dollar value with and a soft benefit is a benefit which has no dollar value.  But looking above, these benefits aren’t soft benefits.  They’re (potentially) hard benefits.  Quality is worth something since it costs money to fix defects.  Customer satisfaction is worth something because unhappy customers don’t spend their money with you.

These aren’t soft benefits at all.  They’re hard benefits that you didn’t bother to figure out!  But don’t confuse them.  Just because you haven’t figured out the value is very different from having no value.  I’ve run into just one benefit of a process change which was truly a soft benefit.

It was the benefit of feeling more comfortable about what we were delivering.  For example, doing regression testing and finding no bugs has no hard benefit.  Peace of mind that the code is correct is a soft benefit.  Your customer doesn’t care how well you sleep at night and you didn’t prevent any bugs so you had no impact on quality or customer satisfaction.  That peace of mind is a soft benefit.  Everything else is a hard benefit that you don’t know the value of.  If you’re going to mention it in an ROI, figure out what it is worth.  If the change is so small you can’t measure it, then don’t list it as a benefit.  But lets not pretend here you are getting a “soft” benefit when you are truly getting is a minuscule hard benefit.

Return on Investment

I’m sure there are a fair number of places which cover what exactly a return on investment is, so I’m going to keep that part of my post brief. In short, a return on investment (ROI, hereafter) is a calculation of how long it will take you to make your money back.

For example, let’s say you spend $100 to tune up your car. Each month thereafter, you use $10 less in gasoline because your car runs more efficiently. Therefore, your ROI is $100 / $10 = 10 months until you make your money back. There are some subtleties that businesses add to that. Typically you discount future dollars saved because a dollar in the future is worth less than a dollar now.

Ultimately, the goal of doing an ROI is to decide whether to invest or not. Businesses typically look to make their money back in 3 to 5 years on an investment. The best ROIs are selected to decide where to spend your money first among many competing initiatives which want budget money. Just like in your personal life, businesses have to make decisions about what to spend your limited funds on. One alternative to any investment is always to sock the money away in some investment and just grow it. There’s no obligation to spend the money since it must always be weighed against whether it’d just be worth it to invest the cash instead.

Anyway, I digress. Having said all that, the way most businesses present an ROI is as some set of expenses divided by some amount of savings. People brainstorm what the expense factors are (labor, materials, etc.) in order to complete the project and then estimate the benefits. I don’t have any issue with the basic math that people are doing for an ROI. For the most part, people can handle the addition, subtraction, multiplication and division necessary.

Where people go wrong on an ROI is filling in the variables. Let’s take a simple example. I believe that if I create comprehensive system documentation that I can save time in design and development of new code because people won’t have to read the code to figure out how to integrate their new features. It’s pretty logical sounding, right? So, let’s say I estimate it’ll cost $100,000 to create the documentation and I estimate that it’ll save $25,000 in time…. Wait. What?

Where did my numbers come from? I made them up. And that’s what most people are doing when they do an ROI. They estimate some costs (which we tend to be more capable at doing) and then we make up some savings we’ll realize to make the ROI work. Rather than saying “well, based on past performance data projects cost us between X and Y and based on industry research we’ll probably realize between A and B savings” people just make up the numbers to make the ROI work. We already know what the goal is – an ROI in 3 to 5 years.

The problem shifts from an honest analysis of what the costs and benefits are to what numbers I need to plug in to get into that 3 to 5 year range. Microsoft Excel has a handy “goal seek” function you can even use for this. Don’t do it!

Everyone wants their idea to be invested in. Everyone can handle basic math (more or less). The problem with ROIs is all in the assumptions people make about the value that a certain variable will take on. Those values should be based on some sort of real data whether it’s neutral industry research (don’t EVER trust a company selling something), expert opinions (you should always get several opinions and account for the uncertainty), or researched data. Furthermore, you should ALWAYS be modeling the fact that results are rarely exactly always the same. Savings on a per experience basis (per project you run, for example) will vary. Many projects are not an automatic home run – there’s a good chance you’ll lose money on the deal – and you have to be honest about the possibility.

We shouldn’t be doing an ROI because we’re trying just to get past the annoyance of an investment committee. We should be doing an ROI to provide an informed view of the opportunity. We all have good ideas, but that doesn’t mean they’re the BEST idea that the company should invest in.