An estimate IS a prediction

… at least in your customers mind it is. One of the true joys of my job is the constant learning, and therefore the constant need to attend webinars and presentations to get new information. Sadly, a lot of what people present at a webinar isn’t new information but a rehash of old stuff. Still, you have to keep attending just in case you ever discover something new. At any rate, I was attending a webinar today about what measurements your organization should have. One of the things the presenter said, and that I blanched at, was “an estimate is not a prediction.”

Yeah right. I understand where he’s coming from which is “there are lots of unknowns and things change and complications come up so estimates aren’t perfect” but that’s not the same as “an estimate is not a prediction.” It most certainly is!

If I were a contractor and estimated a job, let’s say to install a new bathroom in your house, and I told you it’d cost $15,000 you would expect the final price to come in around $15,000. You then look at your bank account, your other upcoming expenses, the amount of the loan the bank will give you, etc. and make a decision (informed or not) about whether you should do this project. If the estimate is $30k you make a different decision possibly. I’m sure there are people out there who shrug and say “well, they said $15k but it ended up being $30k. C’est la vie.” I’m not one of those people and I suspect most people (whether they act on their frustration about the final result or not) aren’t either.

No reasonable person can do business if you make an estimate and then consistently don’t achieve that goal. Why bother estimating if you can’t predict the cost? That’s not to say you won’t some times screw up the estimate, but the goal of the estimate is to provide a realistic picture of what the final cost of the project will be when it is complete. What person wants to do business with someone who “estimates” X dollars but then might actually cost 2X or 10X that “estimate”? Money is valuable because it is a scarce resource, and that means I don’t have an unlimited amount of it to continue throwing at you because you couldn’t predict your costs.

If anyone tells you that you shouldn’t rely on the estimate they gave you, DON’T DO BUSINESS WITH THEM. With the exception of items whose input costs are already known (an item that has already been created such as a can of beans or a television), much of what we do is agree to trade money for labor/expertise in a familiar but unique situation. We understand that there are some unknowns in the request that need to be accounted for in the estimate, but you can’t just throw the estimate out the window having made it. The PREDICTION of cost is exactly what we are looking for with an estimate to make a business decision. What the hell else is an estimate for?

Apparently there’s been just one study ever done

If the material I’ve been reading lately is any indicator, there’s only ever been one study on human behavior done (or two studies if you want to count the variant) called The Ultimatum Game or The Dictator Game.  It has appeared in books such as The Social Atom, Cheap, Superfreakonomics, and How We Decide.  It has been used as proof that we are altruistic, cheap, cheaters, and sympathetic to our peers all at the same time.

If you read any of these works one would start to think that The Ultimatum Game was essentially the unifying experiment of all human behavior.  How can any one experiment take on so many forms?

First of all, if you’re not familiar, let me explain the psychology of the game… player A is given an amount (typically $10 or $20).  Player A is instructed that he must offer Player B some amount of his money greater than $0.  Player B may accept or reject the offer.  If Player B rejects the offer, neither party gets any money at all.  If Player B accepts the offer, then both parties keep their share.  Variants of the game say the increment must be whole dollars, others allow partial dollar amounts and so on.  In the laboratory environment, players tend to offer amounts of money that are greater than a “rational” person would offer.  That is, if Player B is offered $0.01, it is still greater than $0, and so a rational player would accept that amount since it is better than nothing.  But they don’t.  Offers of (typically) $3 or less are declined by Player B, they are insulted apparently by this lowball offer.  Thus showing that people aren’t rational decision makers.

Variant after variant of the game has been played.  The Dictator Game changes the rules so that Player B just gets whatever he is given.  This is the variant that people claim shows people are altruistic.  In this variant, Player A still often offers far more than a rational person ought to offer.

Superfreakonomics shows an example where the game was moved into the real world and all that generosity broke down totally, thus either showing that people are really rational or at least that we are cheaters.

It’s an interesting study, no doubt, but if you were to read any of the books I was reading you’d begin to think that nobody has ever conducted another study in the history of man.  It shows up seemingly everywhere as evidence for whatever unifying theory the author has.

Why do I care?  Because finding support for your pet theory in whatever scientific study is available is no way to run process change.  Studies are relatively narrowly focused and often don’t hold up in real world scenarios.  Studies are designed to help us understand more (not everything, but more) about human behavior (or whatever they’re studying).  They don’t necessarily have the kind of broad applicability needed to be practical in the real world.

If you want to know what’s going on in your process, study your process in the real world.  Don’t study someone else’s work and say “this justifies my actions.”

Too much information

It’s always important to take in new information, regardless of how much we think we know about anything.  Life seems to have very few absolute truths – death and taxes come to mind.

So I was reading “How We Decide” which turns out to be a well written book on our brain function without getting too technical.  One of the things the author discusses is the various failure modes of our brain function – how our emotional brain makes bad decisions where simple calculations can be performed and how our rational brain makes bad decisions where there are simply too many characteristics to consciously consider.

And it was that second chapter that struck a chord with me.  Though we think of the brain as an elegantly evolved (or designed… I’d hate to offend you creationists who are trying to take in new information … is that an oxymoron?) machine, the author points out that the pre-frontal cortex is a relatively recent evolutionary trait and therefore not all that well world-tested.  It appears to get confused when there is simply too much input and makes worse, not better, decisions.

What does this mean for collaborative approaches to software development?  Do JAR sessions introduce too much data at once for us to make good decisions?  Is there simply too much information coming in for any one person to make a good choice?  Do we destroy the instincts gained by years of programming experience by forcing developers to over-think their design choices?

Yes, I’m serious.  Recent trends in Agile have pushed to bring more people into the room at once so that everyone can have a say and bring to bear their collective thought power on a problem.  But that assumes that we can handle the amount of data created by collective thought.  Indeed, the author goes on to show how there are advantages to taking in a lot of information and then being distracted away from the problem to discover the solution, to allow the less accessible emotional brain to mull on the data and return a response that literally feels good to us and is more likely to be the correct one.

Does it turn out that the silly Demotivator poster is right?  Is “none of us is as dumb as all of us”?

Is there really such a thing as a company?

I don’t mean the title in some sort of legal way, it’s really more of a philosophical thing. A company, in theory, is some collection of individuals and equipment coming together to serve some common purpose – namely making money by supplying a good or service to the economy. Companies arise because the collective power of many individuals working together is greater than the output created if the same number of people work alone. There are, in essence, efficiencies of scale.

That said, machines and other equipment don’t have an opinion on anything, so the collective behavior of all the individuals creates the corporate culture. Companies, of course, do change, if through expansion or through self-destructing and being reformed anew.

In the past, at least in the US economy, when you entered the work force it was typical to work for a company for your entire working life and then retire on a pension. You were in it for the long haul with the company and therefore doing things in the best interest of the company was a natural motivation. If your action (or inaction) was going to destroy your livelihood, would you be so inclined to do it?

Today, however, loyalty to a company is quite low. With the advent of the 401(k), the main driver for staying put just became very portable. There is no need to commit to a company and grow within the ranks; it’s often (as I’ve personally experienced) a better deal to just jump ship. (Make no mistake, the various rules about vesting of 401(k) matches encourage portability as well, with companies having to meet certain short graded or cliff vesting schedules that don’t tie employees to the company for all that long.)

But this creates an interesting question for me. If loyalty to a company is so fleeting, can we assume that the decisions being made by individuals are now more focused on the near term than the long term? If in the near term I can slack off, take in a good wage, and then jump ship before any ill befalls me (who cares what happens to the company), why wouldn’t I?

One of the LEAN principles is that companies form long-term relationships with their suppliers. Why? Because it creates a commitment. The company commits to doing regular business with the supplier, thus ensuring a steady stream of work and the supplier in turn commits to delivering a good quality, reasonable cost product in return. We’ve seen (through Toyota’s recent disasters) that chasing the lowest price around has resulted in disasters. When a supplier can’t be sure of continued business, there isn’t a lot of incentive to do a great job… especially if they’ll be dropped the minute someone else offers the “same” product for a penny less per unit.

So why then do we think an appropriate relationship to have with our employees is a short term one? Why are we making it so easy for them to walk away? If we won’t commit to a long-term relationship (think of Toyota’s no-layoff policy during slower business conditions) with our employees, why should we expect a quality job from them?

If the people are thinking short term, then we can be assured the company thinking is short term as well. After all, the company can’t think longer than their employees do. The company is the employees, not some living, breathing, thinking entity unto itself.

Don’t get me wrong, I’m not necessarily proposing that we go back to the old pension system, but what corrective actions do we take when the company can no longer see past the end of its nose? Philosophies like LEAN won’t survive in this kind of environment, they’re too long-term thinking.

Are conservatives the wrong people to improve process?

Ok, I know that it’s a title which might incite people, but I had this crazy thought while shoveling my walkway today. Yes, I do have too much time to think while shoveling all this snow.

In my job at a large corporation, it’s typical for people to be mostly Republicans. I’m an unapologetic liberal, mostly on social fronts, though I thoroughly believe that there at things the government does far better than the free market.

What was an odd observation, that I never could quite find a place for in my blog until now, was that the process team I used to be on was a liberal bastion in a really conservative company. Why was that? Well, I kind of knew what it was supposed to mean to be a conservative, but I wanted to get some third party information on what it was supposed to mean. From Wikipedia (I know, I know, everyone complains about the source): “Conservatism (Latin: conservare, to “save” or “preserve”) is a political attitude and philosophy that advocates institutions and traditional practices that have developed organically.” Ah ha!

The laissez-faire approach to the markets and the strong bond to tradition because traditions evolved “organically” are conservative values. They’re also disastrously bad for process change. Process change allows no sacred cows. Tradition is not to be respected if it gets in the way of even higher performance. Indeed, “the way we’ve always done things” is a disaster for process change.

Suddenly, it isn’t all that surprising to see why the less conservative individuals in the company gravitated towards the process team. It’s a place where change is embraced and traditions are to be questioned, not necessarily respected. It’s a place where we look at the organically evolved processes and say “hey, that might be how it evolved, but that’s just dumb.”

Is there something inherent in a person who calls himself “conservative” which makes that person just no good for change? Is “today’s liberal is tomorrow’s conservative” a truism simply because those who cannot or will not adapt to the world before them are sure to die out?

Cheap

I just started in on reading Cheap by Ellen Ruppel Shell. It was given to me by a friend who thought I’d enjoy the read. Honestly, I’m not much of a reader, but I felt somewhat obligated to read it. I’m about 50 pages in and from the author’s tone I can already tell her angle.

Her point is essentially that a discount culture has caused us to want low-cost products and the easiest way for businesses to deliver low cost goods is to cut corners on quality. Thus, low-cost comes at a high price, both in terms of destroying manufacturing here in the US and providing an inferior item.

The thing is, I’m not sure I agree with her premise. Yes, these things have and are happening – manufacturing is declining in the US and we are trading quality for lower cost. But what I’m missing, so far, from her book is the true relationship between cost and value. Let’s say I could theoretically put a handle on the value of something to me. Economic utility is challenging to measure and not uniform from person to person. A nice shirt doesn’t have the same “value” to everyone. But, in theory, the relationship she posits is that you are receiving disproportionately less value for the dollar you spend.

Let’s say for example that we could measure automobile value solely in the amount of years it runs for before needing to be replaced. If you spend $20,000 on a car, it’ll run for 10 years. But, if you spend $10,000 on a car, it doesn’t run for 5 years, it runs for 4. Thus, the relationship is worse off with the cheaper car because your total cost of ownership is higher. You’ll need to replace it sooner. Now, of course, it totally ignores the time-value of money, as the saved $10,000 is worth something as well. But let’s ignore that for the moment.

The part I disagree with is her assessment of “value.” Taking her own example, people were willing to buy cheap, static-y radios produced in Japan rather than the expensive vacuum-tube models make in the US. The tradeoff is the introduction of static, but otherwise the two radios delivered the exact same thing – you could hear a radio broadcast. So the question becomes, is the device, which sells for far less than the American made item really of lower “value?” Could you listen to the broadcast still? Yes. Did it have some static in it? Yes. Was it possible that depending on where you lived that even an expensive radio might have some static? Yes. The reality was that delta between the cost of the “cheap” radio and the expensive one was a worthwhile tradeoff. If I had to choose between buying nothing at all (because I couldn’t afford the US-made radio) and buying the inferior model, which met my needs better? Even if I could afford the higher cost item, the alternative is the lower cost item and left over money in my pocket I can spend on something else. Would I like to have a product of infinite quality for no cost? Sure, but I don’t actually need it.

Were discount items so vastly inferior that they didn’t have any value to the consumer any more, nobody would have bought them. If you took the radio out of the box and it didn’t get a single broadcast you’d stop buying it. Further, she argues that discounters, by keeping a limited range of items on the shelf, prevented you from seeing the difference between the $1.49 shirt and the $14+ shirt that the high end retailers sold. Yes, it’s true that the quality was worse, but if the $1.49 shirt met your needs, why would you buy a $14 shirt? Am I to believe that a $14 shirt would last ten times as long? Guess what, both shirts suffer their end at the hands of a spilled glass of wine or a large splotch of tomato sauce. The “value” isn’t there with a $14 shirt.

And I think part of the reason by which we choose low cost items are we can’t discern value. I can measure monetary differences easily – this costs $1 and that costs $14. They look the same to me, so why should I buy the one that has so much greater cost?

My real life example is going out to dinner. If I go to a decent dinner out (which I mean, a restaurant that isn’t a chain) and decide to order a glass of wine, I will always choose the cheapest glass possible. Wine, by the glass, might cost $6, to $12 dollars a glass. The problem is, regardless of what I order, I can’t actually tell the difference other than some glasses of wine I like and some I don’t. So, why order a $12 glass of wine? They could easily pour me a glass from the $6 a glass bottle, charge me for the $12 glass and I’d never know. And that’s the problem with “value.” As a consumer, I can’t really tell the difference in quality so I shouldn’t spend more than the bare minimum on it. Were it something that I truly understood then buying the best value is possible, but lacking that information what other data do I have to go on but price? The word of a salesman – someone who stands to make more money by selling me something more expensive possibly with a better markup? Definitely not!

By the way, the same holds true of lots of stuff I buy in the supermarket. I deliberately test out the cheapest brand of many items (like canned vegetables, snack foods, etc.) and find them just as good, so where’s the “value” in buying Del Monte brand instead of store brand? I can’t tell the difference.

No, I say, the discount culture got this way because it did and continues to meet the needs of the consumer. A minimum standard (for safety at least) is set by the government and recourse is available to me if you run afoul of that. Other than that, I totally understand why lowest price wins, even if it doesn’t necessarily have the absolute highest quality. It has “enough” quality.

The journey as well as the destination

As I understand it, one of the key tenets of process orientation is that the journey is as important as the destination. That is, how you produce a product is as important as the end result. If you manage to ship the huge order you placed, but you had to rework it 3 times, that’s a problem. Having delivered the shipment, even if on time, is not good enough.

So, while catching up on my RSS feeds (yes, I have a bunch of lean blogs in my RSS feeds), I thought it was a little funny to see this post from Evolving Excellence titled “Congratulations, Boeing”. I have to say that I really like the Evolving Excellence blog; they take a good look at organizations that are both leading and trailing in the lean arena. Boeing has been a fairly constant topic on many blogs for their disastrous journey to produce the Dreamliner. I’m definitely not an expert on the topic, but I’m aware of their woes.

But I do find it a bit odd for a blog concerned about the journey to then congratulate a company on the result despite the harrowing path to get there. Isn’t this a point where we stop, look back and say “that could’ve gone better” instead of “good for you.” Oddest of all are the two words that wrapped up the entry – “Nice job.” My knowledge of the situation suggests the wrap-up should be “nice plane, horrid job getting there.”

I know, I know, why even call out my peers on this? Because we should never forget that how we got there is important. That is, to some degree what lean is about – getting there with the highest quality and the least waste, not just putting a product out the door. Once the thing is delivered, passing out the kudos sends the message that the result is what mattered. Results are important, no doubt. A “lean process” that delivers nothing that the customer wants isn’t lean at all. Just don’t forget the other part.

Oil Usage – The Reprise

A while back I wrote about my oil usage and the various choices I had to make when it came to an investment. In fact, given my obsession, I’ve written about my oil use here and here. Anyway, in regards to my second post, I have continued to collect data about my oil use. In fact, I made a handy individuals control chart that I use to track my energy use. You can actually see the effect of the Tekmar Boiler Control 260 in the data.

As a reminder, K-Factor is a measure of your efficiency. It is the number of degree days elapsed divided by the number of gallons of oil you use. Therefore, higher numbers are better. I installed the Tekmar in August of 2008, so I had to wait most of 2009 to actually see if there was going to be any result. Take a look at the data. You can see the data is cyclic with each summer resulting in a low point in the summer of 2005, 2006, 2007 and 2008. Remember, with higher K-Factors being better, there’s some serious evidence that my furnace wastes a lot of energy during the warm months. And then we see 2009, instead of the normal drop in K-Factor we see it go through the roof, well above the upper control limit!

What’s interesting about the Tekmar controller is that it doesn’t seem to do much of anything for the actual heating season. The K-Factor remains right around the mean. But, during the warmer months (as soon as it hits 64 degrees) it does save quite a bit of energy. It’s great when you see something show up in the data like this. Very heartening indeed.

How many defects is that?

I got a, I’m embarrassed to say, question about counting defects that I never considered before. One of the things I’ve tried hard to force myself to do is to perform measurement system analysis on anything I count before using the data. I at least take a look at it to see if it is reasonable. In my line of work, I typically don’t care how many defects there are, for example, as much as I care that the long term trend of defects is improving. Yes, that allows for some amount of error in the measurement, but as long as the error isn’t greater than the signal, I don’t care that much.

Typically I think about error in these measurement systems as defects that aren’t really defects but are getting counted. No, I’m not talking about defects which are really “enhancements” since I’d argue that anything that doesn’t meet a customer need is a defect. I don’t care how it got introduced. What typically mucks up our measurement systems are duplicates and cancellations – places where we either double report the same issue or report something that isn’t an issue at all. We’ve evaluated our system and know that this type of error introduces about +/-5% noise to the system. So, if I measure 100 defects, it likely falls somewhere between 95 and 105 bugs.

Anyway, someone emails me the other day and says “how would you recommend I write this up?” and then goes on to explain that they were testing feature X, which was similar to feature Y and found something wrong. They then examined feature Y because their system knowledge told them something might be wrong there too and indeed it was. Is this one defect or two?

I said two. Here’s my thinking. The fact that person A examined two things means they essentially ran two tests. Now, it is true that test 2 was prompted by the findings of test 1, but it is not unreasonable to conclude that the alternative is that instead of person A running test 1 and test 2 that both bugs could have been found independently if person A ran test 1 while person B ran test 2 in parallel. Assuming person A and B don’t talk to each other, two bugs would be produced.

Now, if while resolving the first bug the second bug also gets resolved – say, shared code caused both bugs, for example – then the second bug would be cancelled since it was “fixed by another code change.” But I think you have to go for the lowest common denominator – you attempted to validate one thing via a test and therefore should write one bug. If a test prompts you to find the problem in more places then you should write up each time you find it. It isn’t your job to ascertain whether or not a shared component caused both issues, just that two issues were detected with the system.

What concerns me is not my proposed resolution, since I feel pretty comfortable that the proposal would play to the least common denominator. What worries me is that absent of the question, what are other testers doing? Are they packaging many bugs together into one convenient deliverable for development to fix? What do they do when the get the bug back and it isn’t fully fixed (ie. We fixed part A of the bug but not part B)? How much noise is this grouping up of bugs into one report inserting into the system? I guess I’ll have to go off and find the answer via yet another measurement system analysis.

We don’t want your money

Today I was trying to order Quicken 2010 from Intuit. I failed. I didn’t fail because I didn’t want the software. I failed because they wouldn’t let me order it. They, essentially, refused to take my money. How? Well, they have a known issue with their website which prevents you from completing your order.

To be fair, they actually did charge my credit card, but I assume the temporary charge will clear soon. Anyway, I tried to call them to place my order instead. They didn’t make it too hard to find their phone number, but it wasn’t exactly right on the page. I mean, if you did want my money, you’d make it as easy as possible for me to give it to you, right?

And while this would otherwise be an irrelevant topic for discussion, I’d like to talk about it in the process framework. When I tried to call Intuit, they let me call them, let me get 2-3 selections in through their annoying phone system and then told me that if I wanted to talk to someone about Quicken that “they’d call me!” They touted this as an obvious advantage since I wouldn’t have to wait on hold.

So, I go to the website and have to fill out a contact form in which they tell me that a) it’ll be up to 3 hours that they’ll contact me and b) a $24.95 charge may apply. I have no idea for what purpose a charge would apply, and hopefully I’ll never find out. Anyway, I happen to have to be somewhere, so I don’t wait around at home and of course I miss their call.

This evening, I try to use their online chat to get help. The person I am chatting with points me to an article on their website that tells me that they’re having technical issues taking orders and if I “click here” I can be added to the list of people to be informed when they fix their issues so that they can take my money.

I tell the chat person “if you have a means to register a complaint, please let them know that Intuit just lost a sale.” His reply? “I’ll send a copy of this chat transcript to your email address.”

Oh thank god! Now I’ll know, in writing, that you couldn’t help me.

So, how is this process relevant? Why do you think they’ll call me? They’re not doing me a favor, although they couch it as such. The likely reason they want to call me is to control call volume. They are shaping their demand for phone support by allowing themselves up the 3 hours to call you back. In that manner, since they don’t have to staff for peak hours (which are probably 9am, noon and maybe 5pm – you know, the times when many people are not working or are on break and have time to call) they can use fewer people overall and spread out the callbacks. No peak volume = less people. I almost applaud their creativity.

The part that they are missing is that when I want to buy a product or service, I don’t want to be tied to my house for 3 hours waiting for a call back. I already hate that feature about the telephone and cable company – does anyone recall the jokes about “we’ll be there sometime between 8am and never”? But sure, it somehow makes sense for a company to take this approach even though it’s a customer relationship disaster in other industries.

And, of course, worst of all, my guess is that they can’t actually take my order over the phone. So why call me anyway? So, how about some alternate solutions… how about if I agree to get out of the queue and let you call me back that you offer me a $5 (or $1) discount? Or I can call you at off peak hours for the same type of deal? How about if the order fails to go through you offer me a direct # right then and there to take my money instead? How about you leverage some sort of overflow system that allows you to share call center staff with other companies to level out demand across many companies if you can’t reasonably level your own demand.

All in all, I think this “don’t call us, we’ll call you” design is stupid. People aren’t parts on a factory conveyor belt and therefore you can’t treat them like they are. You need a system to serve them when they want to be served.