No more RACIs

I’ve decided that I hate RACIs. There’s just too much wrong with them to make them worth doing. A RACI, in case you are unfamiliar (or RASCI sometimes) is a table which lists tasks along one axis, people (or job roles) along another axis and at the intersection marks someone one or more of R,A,S,C,I (responsible, accountable, supporting, consulted, informed).

Now, depending on the school of thought you subscribe to, the S option may or may not exist. A simple RACI might look something like:

	Mommy	Daddy	Daughter
Wake up	R	R, A	R
Get dressed	R, A	R	R
Go to school	A	I	R
Ask about school day	I	R	C
Make dinner (at least in my house)	C	R, A	I
Have dessert	C, A	I	R

One can thus determine what my day is like. In the morning, we are all responsible for waking up, while if someone doesn’t get up on time it’s Daddy’s head who is going to roll. Then, we’re all responsible for getting dressed, but since Daddy has no fashion sense, it’s Mommy who is accountable for this task. I’m simply informed about our daughter going to school since I’m going to work. Technically, though my daughter is too young to drive herself, she’s the one responsible for going to school while Mommy is the one who gets in trouble if my 3 year old doesn’t show up. I’m responsible for asking about her school day, Mommy is simply informed. She overhears my daughter say (who is only 3 but clearly has the kid role figure out) in response to “what’d you do at school today?” almost always replies “nothing.” Obviously, my daughter being the expert on her school day is consulted on this task. One might also choose Supportive I suppose since I can’t complete the task without her. Dinner is strictly a Daddy job with Mommy having some say about what I make for dinner and the little one just eating what is presented. If we consulted her every night would be chicken nuggets, pizza or noodles. And finally for dessert, the little one is responsible for the eating while Daddy is simply informed. Mommy has the go/no-go decision on this one.

Phew. Now that is one heck of a lot of data all from that little table, and this is a joke of a RACI. No real job that I do is so basic as to have just 3 players and 6 tasks to it. You can easily see that many more tasks might be appropriate even in my daily plan – including showers, dishes, bedtime for my daughter, etc. We actually follow a pretty predictable pattern each day – a process you might say – and yet there’s enormous detail that could go into it. The fact of the matter is that any realistic RACI would get unmanageable and quick.

The second issue I have with RACIs is that they’re a punt for figuring out a good process. If you have to go to all this trouble to figure out who does what in the process, isn’t it a reasonable question to ask if you might have a really un-lean process that simply needs fewer actors or individuals who actually have the authority to both be responsible and accountable for the task to get done? The mere fact that R (the responsible person who does the work) can be separated from the A (the accountable person who gets in trouble if the work doesn’t get done) is crazy! The idea that we’d consult all kinds of people suggests that we don’t have enough knowledge (or the wherewithal to gather it) about the job we’re tasked with doing. And informing? Look, if the person isn’t going to act on the data you are provided (ie, give feedback, which makes the consulted not informed) then why are you giving them the information!?!?

Third, nobody gets these things right anyway. Can more than one person be accountable? Responsible? I think not. And yet, they’re always a mishmash of several people being responsible for the same task. Someone has to be running the show and the moment you allow everyone to be accountable then nobody is accountable and the same goes for responsibility. It just becomes a giant circle of finger-pointing when the task doesn’t get done.

Finally, these things are usually more of a suggestion than a reality. Putting that much effort into something that isn’t going to be followed is insane. Even in my own house, with the six silly tasks that I have, if I’m working late then Mommy becomes responsible for all the tasks that I would normally be. Life marches on, regardless of what my RACI says ought to happen.

Don’t do RACIs. Do a nice swimlane process flow and leave it at that.

Far fewer soft benefits

Recently I was presented with an ROI for some project.  It included some costs, which weren’t that interesting and it included one hard benefit which was a time savings.  And then it included a series of soft benefits including improved quality, improved customer satisfaction, etc.

Now, as far as I understand it, a hard benefit is one which you can associate a dollar value with and a soft benefit is a benefit which has no dollar value.  But looking above, these benefits aren’t soft benefits.  They’re (potentially) hard benefits.  Quality is worth something since it costs money to fix defects.  Customer satisfaction is worth something because unhappy customers don’t spend their money with you.

These aren’t soft benefits at all.  They’re hard benefits that you didn’t bother to figure out!  But don’t confuse them.  Just because you haven’t figured out the value is very different from having no value.  I’ve run into just one benefit of a process change which was truly a soft benefit.

It was the benefit of feeling more comfortable about what we were delivering.  For example, doing regression testing and finding no bugs has no hard benefit.  Peace of mind that the code is correct is a soft benefit.  Your customer doesn’t care how well you sleep at night and you didn’t prevent any bugs so you had no impact on quality or customer satisfaction.  That peace of mind is a soft benefit.  Everything else is a hard benefit that you don’t know the value of.  If you’re going to mention it in an ROI, figure out what it is worth.  If the change is so small you can’t measure it, then don’t list it as a benefit.  But lets not pretend here you are getting a “soft” benefit when you are truly getting is a minuscule hard benefit.

The consumer is always right

I was sitting through a meeting the other day and we were talking about the new test case management system that we were installing. It’s nothing special, but one of the features it offers is the ability to track defects along with the test cases. The thing is, as a company we already have 3 other defect tracking systems from different vendors, none of which are integrated with our test case management system. True, it is an annoyance and certainly adds some cost to quality assurance to record the defect when you have to go into a different system.

However, the issue I have is that quality assurance designed the workflow and fields that would go into the defect tracking system. Quality assurance is a (sort-of) user of the tool. They have to enter data into it, so making it convenient for them to enter data is a good thing. However, they aren’t the primary consumer of the data – development is. The reason we enter bugs into the defect tracking system is (besides metrics) so that development gets a detailed account of what went wrong so they can fix it. Although we supply defects into the process, it is development that has to consume our input and use it.

So why then would you have QA define the input for the team who needs the data? It is illogical that QA should determine what information and in what format they’ll be giving it to development. Development, who needs adequate input to fix the bug, should be defining what inputs they need. QA could input more data than development needs if they felt they needed some data recorded too, provided it isn’t interfering with what development needs to get their job done.

Now, maybe development requests something that is ridiculous and QA ought to negotiate for a more acceptable request, but generally I believe that it is the consumer who gets to define what they need from their supplier. In our case, we didn’t ask development at all what they needed, we just built something. We didn’t ask development if they’d be willing to use our new defect tracking system.

I think, if it hasn’t been set out as a universal truth somewhere else before, that it ought to be done so now – the consumer defines the required input, not the supplier.

Why measurement is necessary

This evening, my daughter, in an attempt to be helpful offered to put our dog’s food into his bowl for dinner. She does this every now and again, and usually the mess is kept to a minimum. Regardless, I can hear the difference from across the room when his food goes into our dog’s metal bowl or on the floor.

It isn’t atypical to hear the clatter of a few pieces of his food hit the wood floor on the way and this night was no exception. As my daughter raced back to me with the now empty food scoop in hand, she says to me “I spilled some.” Now, I don’t know what some means to you, but it doesn’t mean the vast majority of the food, right?

Apparently “some” meant exactly that to my daughter. Certainly, as I looked down at my dog’s bowl there was an amount of food in the bowl and another amount outside the bowl on the floor. But “some” is not the word I would have used to describe the amount on the floor. “Most” is the word, if I have to be inexact about it, is the word that I would have used.

I know it’s a dumb example, but this is exactly why we need to measure things. All the words we have to describe portions are inexact. What does a few mean? More than 2 certainly, but is a “few’ deaths as the result of the millions of products you sold 3 people or 300 people or 3000 people? Compared to the whole, even 3000 out of a million might be a few!

The “majority” suffers this issue in news reporting all the time. When the majority of people approve of the President’s performance, it means some number greater than 50% approve. 50.0000001% is a majority. It’s not an overwhelming majority, but it’s a majority technically. And I’ve seen “most” used to mean a simple majority as well, which is crazy, since most clearly means something higher than that. Is 75% most? 80%? 90%? Who knows? The definition is variable.

What about “some”? Officially, it’s just a number greater than 0. Some of the food was outside the bowl. Indeed, not all of the food was outside the bowl, so some is a fair statement. But my version of some and my daughter’s version are really different in this case.

And it’s for this reason that English is an inexact language that true measurement is needed. A proportion would have told a much better story. Not that I would have expected my daughter to say “daddy, I spilled seventy-five percent of the food” but I can expect that from an adult.

Let’s talk real number in business. Put a scale to it – a proportion that is a problem, a count that is a problem, but some real measure of how much is really wrong. “We have some issues with code quality,” after seeing my daughter’s definition of “some” tonight, has a whole new meaning for me.

No good data? Make it up

A friend of mine just recently came back from an Agile software development conference in Florida. Did you ever find it at least slightly odd that all conferences seem to be held in really nice places? Nobody ever says “come to a conference in Idaho.” Some part of me suspects that people don’t go to conferences to actually learn, but for the sightseeing/vacation opportunity.

Anyway, apparently Alistair Cockburn presented at the conference, and (I’m getting this second hand now) he showed a chart about how Agile improves knowledge gain…

It looked something like this:

Notice that it is unit-less on both axes. The cost axis you could almost forgive, since we have some intrinsic knowledge about what it means for something to cost more. Knowledge on the other hand… I don’t know how you measure that. I don’t think he means IQ in this case. His point was something along the lines of “with Agile, you gain knowledge about your work sooner.” His argument is essentially, with Waterfall you integrate and learn very late that things aren’t working.

The problem is, his chart is garbage. This isn’t scientific. He drew a picture. Anyone can draw a picture. There’s no study behind this. There’s no data about the impact to the customer about “learning late.” Even if his hypothesis was true and backed up by data, it has to have bearing on what your customer experiences to matter. Does late learning mean worse quality? Worse costs? Who knows… Not only is this chart imaginary, so is the impact. I drew a chart too (notice the flat line).

The difference between knowing and thinking you know

I was talking with one of my managers the other day when she presented an interesting point that I thought I’d share. There is a difference between knowing something and thinking you know something. Sure, there is pure ignorance, but it’s not the type of issue we were talking about.

The type of intellectual leap you shouldn’t make is the taking of incomplete knowledge and representing it as complete knowledge. It’s the overextension of a specific situation to being the general problem. For example, we know that some defects were introduced when the code was written years ago and are just appearing now, but that doesn’t mean that our problem is all latent bugs. This is inductive fallacy. All we’ve seen (for those that we can confirm at all) is that some bugs exist since the creation of the code, but this does not make the general statement “our issue is latent bugs” necessarily true.

The list of bad extensions is long, and yet easy to remedy. We should be talking about odds and probabilities and uncertainty and recognizing that it exists with much of what we deal with. For example, Stephen Kan, a researcher from IBM, recognized that test execution doesn’t happen linearly but instead follows and S-shaped curve. When we explored our patterns of test execution we discovered the same thing held true of us. And so, knowing that, we could set forth a realistic pattern of what would have to happen (for a given set of tests and duration to complete the testing) in order for us to be done one time. We tried it out, and found out that we don’t always perfectly follow the curve.

It wasn’t about perfectly following the curve. It was about more-or-less following it. There are essentially bands, plus or minus, around this idealized curve which represent the uncertain area which is normal variation from the perfect shape. As long as you stay within those, we feel comfortable that things are on track. But we represent the uncertainty, not ignore it, because it is there. The knowledge says “testing follows and s-shape”, but we don’t make that an absolute truth, just a general pattern that includes room for representing that we don’t understand everything that may cause it to deviate from that path.

"A little learning is a dangerous thing; drink deep, or taste not the Pierian spring: there shallow draughts intoxicate the brain, and drinking largely sobers us again."

- Alexander Pope, An Essay on Criticism

His quote is not just a warning that knowing a little causes you to do bad things, but that it is necessary for us drink deeply, acknowledge that we do not know everything and seek to figure out what it is we do not know.