Just silly

I love cheese.  Mostly soft cheeses like brie, but pretty much anything with character.  So when we venture away from home to my in-laws, we take the opportunity to frequent one of the several cheese shops in town.  Better than buying a delicious cheese, we get to try a vast array of cheeses and then pick out three or four small samples to take home and enjoy with a glass of wine and some crackers.

As I was standing at one of these cheese counters the other day, I noted a gentleman to my right was ordering something from the prepared foods counter.  It wasn’t a cheesemonger that I usually go to, but variety is the spice of life, right?  Anyway, as this person is talking to the woman behind the counter he says “I’ll have some of this, and some of that” and so on.  Then, at last, he says “and a pound of sliced turkey.”

“Oh, I can’t help you with that, ” says the woman. “You’ll have to go over to the deli counter for that.”

“Ok, ” he replies, and takes two steps to his immediate left and turns ninety degrees counterclockwise.

Behind the same continuous counter, the SAME woman pops round the corner of the counter and says “ok, how can I help you?”

WHAT!?!  In the words of the villain in Zoolander “I feel like I’m taking crazy pills!”

Look… if you are capable of serving the person two steps to the left, then you are capable of serving them right where you are standing.  Regardless of what your established process might be, having to restart the transaction because someone isn’t in quite the right place is just silly.  Build processes that meet your customers needs.  This can’t be the first person who has ordered both prepared foods and stuff from a deli counter at the same time.

Velocity

Yes, it’s about due time for another anti-Agile post. In this case, it’s a story about velocity. In theory, during the first few iterations the team isn’t that good at estimating. You don’t understand relative size of stories, so you don’t get a whole lot done. Over the iterations you learn more about your team’s capability and your estimating gets better and better… or does it? What does it mean for an estimate to get better?

I think a lot of people believe a good estimate is one with low variance between estimated effort and actual effort. That’s an ok definition, except I don’t think it is what your customers necessarily mean. When I ask, say a home repair person for an estimate, not only do I want the estimate to be accurate (don’t tell me it’ll be $500 and then charge me $2000 at the end), but I also want it to be cheap (don’t tell me $2000 when you could reasonably do it for $1000). A “good” estimate in my mind, isn’t one that’s just right, it’s also one which meets my needs (both in terms of requirements that will be delivered AND what it costs to deliver those requirements).

The former definition (low variance), creates a problem which I observed on a pseudo-agile project we’ve been working on. The team is trying to estimate stories and (like all early estimations) didn’t do so well. We didn’t deliver very much. However, what’s not clear (because we didn’t study it) is what our issue was. Were we really underestimating our work or were we just not working? In fact, most people on the team are part time allocated to the project (I know, bad no-no for Agile projects). Anyway, we got dinged for not delivering, and not knowing what was really wrong, we just decided the (more or less) halve our velocity for the next go-round (or I should say, double our estimates). Again, we’re really not sure what was wrong with our estimates, but adding more time to do work seemed like a reasonable thing to do.

So now, in order to get a story completed, we estimate twice as much work as we once did. And the stories aren’t getting done still. So maybe twice again is the right number? The thing is, that it is very easy for us to just say “oh, we estimated badly, let’s allow more time for this work.” And while at some point we’ll have so much effort allocated to a single story that Parkinson’s Law will take over and make sure our variance between estimates and actuals is low. The work will simply expand to fill the time allotted.

Velocity, just like any form of estimation, does nothing to account for sand-bagging. A happy customer is not one who just has low variance, but good anticipated value from the estimate (read: the estimate has to be cheap enough to be appealing). In this case, like with waterfall projects, the fact that we got called out for missing estimates created an undesirable behavior where we compensated (or I should say over-compensated) in a less than ideal way. Rather than eliciting a work-smarter reaction, it created a make more time to work the same way reaction.

You can’t just change the way you estimate and expect it to solve human nature. Calling it velocity doesn’t eliminate the issue at hand; only changing the way an estimate miss is viewed does. It has to become an opportunity to learn how to work smarter, not an opportunity to ding someone for not delivering.

Not one of the critical few

Ingrained in the Six Sigma school of thought is the critical few – the 80/20 rule. It is an important rule. In practice, there are a handful of things which often allow you to make big leaps from an incapable process to a capable one. There are more subtle characteristics of the process which can be refined to continually improve the performance, but this isn’t step change, it is refinement. And then there’s a class of things that just don’t matter.

So as I sat today through a long, long meeting trying to define a process, I spent a lot of time thinking about the things that don’t matter. That may have been because that’s all anyone spent their time talking about. And as facilitators, we were enablers of this dragging on. Having been instructed to drive to a single standard process and toolset, we discussed every little one-off thing that people wanted to allow for in the process to see if we could squeeze them out. A day’s worth of 25 people’s time to design a process spent talking about the equivalent of the carpet color.

We wanted perfect compliance to the standard, and that meant a standard which was not necessarily all-inclusive (because some of these one-off requests were truly ridiculous by any standard). This is where I believe we got off track with process work. Process design is about controlling the critical few things which will make the difference in process performance.

But that is not what we were discussing. We were discussing nuances, oddball cases, odd uses of the process, and data elements that some teams wanted and others didn’t. We talked about the 1% and largely ignored the 99%. We talked about things that weren’t going to make the difference, whether they were one way or another.

To begin with, we didn’t know what was going to make the difference. We hadn’t studied the existing processes to understand what made them work – what really mattered and what didn’t. This created unnecessary room for debate because we were unable to bring adequate materials to the table to help the team work through their differences. We had little to no information on what mattered and what didn’t.

Instead of define-measure-analyze-improve-control we just went right into improve. And there we got bogged down discussing every little quirk, because we didn’t know what else we ought to be talking about. Or more importantly, what we shouldn’t be talking about.

Instead of a conversation that was “do we really need that? How many of our teams use that process step?” we could have instead said “sure, it doesn’t matter to me if you allow for that.” And we’d be saying that not because we didn’t care but because we actually knew what did matter. Everything else, the little things that we debated with the teams could have instead been bargaining chips that we could dole out in heaps and have given up basically nothing that really mattered. We could have had a strong position, not because we won all the arguments but because we knew which battles were worth fighting and which were worth conceding.

Had we known what things were not one of the critical few things, we could have appeared very agreeable and allowed the teams as much “leeway” in the process as they claimed they needed. All along we’d be giving up nothing. Nothing that really mattered anyway.

It’s a reminder why a thorough measurement and analysis of a process is important. It isn’t just discovering what the current state is (measurement), but it also understanding why it works (analysis). And from there, narrowing down the bits of process that really do matter, and just letting the rest go. Some things just don’t matter.

It’s a net, people

Ah nets, the greatest safety device possibly ever invented. And also very handy for catching fish. Or is it? In software development, we always talk about nets. Testing is a net to catch all the bugs that developers write. Peer review is a net to catch all the design flaws that analysts create (and possibly a net to catch code bugs as well). But a net it truly is, and nets, by design, have holes in them.

Nets catch big things, like big fish, and let the little ones through. That’s fine if you don’t mind the little fish escaping, but what if instead of being fish the little things getting by are defects? Sure, the really enormous fish, er defect, gets caught, but the little ones, the hundreds of thousands of little ones slip right through.

Nets serve a purpose, but they are imperfect and will always allow some things that are small enough to fit between the spaces to slip through. There are finer nets than others. General purpose nets, like peer review, tend to catch more than special purpose nets like regression testing. After all, peer review (in theory) looks for a broad range of things while regression testing only looks for what it already knows about.

But the purpose of my post is not to talk about the fineness of any net but instead to remind you that no matter how fine your net is, it is still a NET. And you really don’t want nets when it comes to software development, you want walls – barriers that allow NOTHING undesirable by. The barrier that we most often overlook in software development is error proofing the process.

Sure, for new code that’s hard to do, but many systems are configuration driven, and require changes to be made to many parts of the system to enable new functionality. We often need entries in the tables to create the foreign keys, entries in tables to join values together. Multiple rows of configuration working in concert make the right behavior. But what do we do? We write instructions on how to configure the system. Rather than do that, if there’s a limited number of possible results you want to support (say a bunch of client features), then either write the system with a single item of configuration (so you don’t have to worry about updating the right thing in every place) to make it work OR write a tool that makes sure you can configure all those settings with a single click. Error proof. Make it easy to do the right thing.

Walls, not nets, is where the answer lies.

Involve early or Just In Time?

Here’s something that keeps coming up at work that I’m not sure how to deal with. The Agile philosophy has the team entirely together all the time, including the business, developers, testers, etc. As a quality assurance organization, we constantly push to be involved earlier in the process (so much so that I think one day we’ll want to have an opinion on whether the business should invest in a given project or not). We all push to be involved early and often.

However, as far as I can tell, this is somewhat counter to LEAN production, which strives for effort as close to the point of demand (just in time manufacturing) as possible. Now, maybe I’m making a bad overextension, it’s entirely possible, but if we think of quality assurance as manufacturing test strategies, test plans, test cases, etc. then having them involved really early is the creation of inventory.

The earlier they are involved in the project the more likely they are to create their work product which is inadequate due to changes in the requirements, design, etc. later on. It is, in theory, wasteful to create anything earlier than absolutely necessary.

So, for my few readers bold enough to comment… what’s the best answer? Are the benefits of early involvement justified or do dedicated quality assurance activities (like testing) just need to be prepare for a Just In Time approach to manufacturing and executing testing?