I love my tech spec, cause that’s the way I am

I was reading an article in MassHighTech by an acquaintance of mine regarding the death of the tech spec in web development (http://www.bizjournals.com/masshightech/stories/2007/11/26/newscolumn3.html).  Generally speaking, I agree with Richard that advances in technology and methodologies have found better ways to do UI design.  He’s a bright guy, clearly, and has found a successful model in rapid prototyping.

That said, I still love my tech spec.  I don’t do UI design.  I build batches mostly to process massive amounts of corporate meta-data in order to figure out how well we’re performing as an organization.  Frankly, nothing is worse for batch design than just sitting down to code it and nothing better than taking a bit of effort to draft it on paper first.  But that may just be the way I am.  I love doing design; I hate coding it to completion.  At heart, I’m an architect or analyst, not a developer.  You’d be impressed with how many cool (I think they’re cool) applications I’ve started developing – neural networks for image recognition, an application to analyze stock market data, new interfaces for my windows UI (I always admired the ridiculously fake interface the little girl was using in Jurassic Park when she exclaimed “I know this, it’s UNIX!”  Ok, I wanted to beat her with a velociraptor, but the UI was cool).  The fact is, I’m not a developer.  I never finish the programs I start because I am happiest when I am designing an idea.  I love building the constructs, proving that it could basically work and then I get bored.  But the thing is, the tech spec works for me, and it works for batch design.

I don’t document because someone tells me to.  I document because it helps my thinking process to write it down in a non-code way and work through the issues.  And that makes me wonder about programmers in general.

One of Richard’s points is that the development model of prototyping increases employee happiness.  I’d believe that too.  But why?  I think it may have to do with instant gratification (see my comments regarding children and oreo cookies).  I suspect that most people are not like me – ok, I’ve known that for a long time.  The thought process that I like to go through is very important to my satisfaction.  When I was a developer I was happiest when I could hold my design in my hands and be satisfied that I had thought the issues through and knew what I was facing when it came to the utterly boring task of actually coding it.

By far, I avoid coding when I have to develop a function to map some set of inputs to another set of outputs.  It is inevitably either a giant if..then..else statement or a case statement and I hate them.  Not because the constructs are ineffective, but because they are dull to put together.  Oh the drudgery!

As for most everyone else, the one Oreo cookie now is better than two later.  People like stuff now and that includes the code.  Code now is better than code later, right?  I guess I can’t fault people for wanting to get down to work if it makes them happy.  It’s probably why Agile makes people happy and looks like nothing special to me.  Even if you didn’t make me or even support me while I write a tech spec, I’d probably still jot down a few pages of design before coding anything.  It’s just the way I am.

So is it a matter of what managers have always known: select the right people, put them in the right job and get obstacles out of their way.  Was the problem with waterfall that we put instant gratification people into the roles that required patient deliberation?  Did corporate efficiency, which merged analysts and programmers into a single role because surely that’s cheaper, make waterfall-like approaches doomed to failure?  Maybe we never knew what “right people” meant.  While people decry waterfall as outmoded and inefficient, could it be that it just doesn’t work for most people?  But then again, could it work for most organizations if we restored the critical thinking roles to those who should be doing that work and delgated the coding to those built to do that job?

It reminds me of building a home (a common analogy for me, I know).  The architect (the analyst or designer, in programming) is the patient thinker who works out the details of the design in his mind and on paper.  The instant gratification people are the builders (the programmers), who don’t have the skills to be an architect because they aren’t built that way.  But they are effective at turning the architects ideas into reality.

Is Agile like putting home design in the hands of carpenters?  Can they put up a home without building plans?  Sure, and it’ll stay up and be an ok home.  But why do we still have architects and more importantly why have we gone so far as to consider their work so important that we license them for their jobs?  The builders know how to put a home together.  They do it every day.  You don’t get a license to be a builder, hell you don’t even have to be legally employed.  (In no way should you take that as a stance on immigration.  I am all for anyone who wants to come to the US to be here and don’t understand why we put up so many barriers.)  The point is, there’s an enormous gap between the education, training and requirements that an architect must meet that a carpenter does not.  It’s a very successful model for home building, so why should it not work in software development?  To me, it’s a brilliance of evolution – the ratio of planners and thinkers to the doers is right for one individual to design or direct the work of many.

*Yeah, yeah, I know it sometimes takes forever to put a home up.  Blame the project manager, not the architect.  The division of labor is right, the coordination of it is wrong.

Don’t give me that “I’ve got a new way to develop software” garbage

I decided today that every micro-level development process is at its core the same as every other.  From waterfall, to spiral, to agile, they are all the same!  There’s a movement afoot at my office to shift to agile development.  I’m pretty sure it’s the “perceived silver bullet” effect at work.  (By the way, my favorite blog rant on agile can be found at http://steve-yegge.blogspot.com/2006/09/good-agile-bad-agile_27.html).  And it was no different this morning when my peers were trying to convince me that my rant about Non-Value Add (NVA) activities in our development process I was making was an endorsement for agile, which it was NOT.

Let me give you some background on my conversation that led to the comments by my peers.  According to at least one Six Sigma source, the average process is 85%+ NVA.  A world-class, best-of-the-best process might be only 50% NVA.  Not exactly what you’d call miraculously efficient, but alas you can’t get rid of all NVA work.  For the moment, let’s accept that this is true.

I had extracted actual hours spent on 250 projects completed at my organization over the last few years.  Because of the way we do time billing, I could separate the hours down to project task level.  For example, for Project X, I knew there were 100 hours of Analysis, 200 hours of Design, 650 hours of Coding and 300 hours of testing.  All those numbers are made up, of course, but you get the idea.  I wanted to look at what percentage of the total time each task took up.  For example, Analysis is 100 hours of a total 1250 project hours or about 8% of the total project cost.

What I did with the data was dump it into a really nice boxplot (http://en.wikipedia.org/wiki/Box_plot).  I’m not going to show the data because I’d be worried I was giving away corporate secrets.  Frankly, by itself the data was boring anyway.  At a glance, it looked to me like project effort was evenly distributed among analysis, design, code and test.  Nothing really stood out as a huge percentage of the effort.

Out of curiosity, I extracted the industry average percentages of effort used by COCOMO II (http://csse.usc.edu/csse/) to distribute a total project cost into a project plan.  I happened to get my data from Costxpert, an implementation of COCOMO II.  I then put the industry averages for each task next to the boxplots I had and suddenly I had interesting data!  Turns out that the industry average spent on coding is about 50% of total project cost.  Does 50% sound familiar?  Ding!  That’s the same percentage as considered world-class for NVA activity.

Consider for a moment what the customer is paying you to do when they buy software services… they’re paying you to code and maybe to write user documentation and that’s it.  All those fun analysis and design documents you write that you then throw away are NVA.  So, if the industry averages spending 50% of the project effort on coding, whatever process they’re using is pretty efficient.  Sure, some will be better, some will be worse than that, but on average…

Ok, so why’s that interesting to me?  Well, the data from the 250 projects I had indicated the company I work for spent between 10-25% of total project effort coding on most projects (the 2nd and 3rd quartiles of the boxplot).  Wow!  The industry spends 50% on average on (potentially) value-add work and we max out around 25%.

And that’s what inspired my rant.  How was it that we spent so little time doing the work we were really being paid to do!?!?  It’s also where my coworkers claimed my rant was an endorsement for Agile, since Agile is very focused on getting down to coding and not so much on documentation, or as someone put it to me the “right” documentation.  But clearly every company in the industry averages is not doing Agile.  The company I’m at is the first I’ve even heard mention it, and Agile is not a new idea.  We can reasonably assume that the COCOMO data includes waterfall, spiral, RAD, and probably every other conceivable model for development.

Could it be that a properly designed waterfall or waterfall-like model might produce 50% value add activity?  I’d be willing to believe that.  Sure, by going Agile you increase the effort going into coding, but refactoring/rework coding because the requirements change is NOT value-add.  Getting it right the first time is; nobody’s paying you to make corrections because the design you have doesn’t work for the new requirement you’ve been given.  So, I’d even argue that Agile is probably nearing 50% NVA as well.

Which got me to thinking… what’s the real difference?  What’s the core process someone follows when we code something?  I’d argue that it is:

1. Figure out what you have today
2. Conceive what you want to build to meet the need
3. Build it

Regardless if you write anything down or not, the micro-level process you follow in your mind has to be that.  If someone asks you to code something into an existing system, you have to at least consider where you might add it to the system.  This might be a 5 second mental exercise or a 5 day documentation process, but it gets done.  Then, you conceive what you want to build.  Do I need lists?  Arrays?  What’s the algorithm?  You can’t write a line of code unless you have at least an inkling of what needs to be written.  Again, you might just think about it for a moment or you might write a comprehensive document.  Finally, you code your idea.

And that’s where I contend that nobody has a new idea in software development.  At the heart of it all is the same process with different levels of documentation going into each step.  For those processes which encourage relatively heavy documentation, you probably spend less time recoding since you instead spend time working it out on paper.  For those with lighter documentation you spend more time refactoring to adjust for new ideas coming in.

Now, that said, there are extremes of on both end of the spectrum.  Cowboy coding is the extreme of no documentation.  The government (and apparently the company I work for) is the extreme of too much documentation.  Near either end of the spectrum you get into an unhappy place where you are either paralyzed by documentation or unnecessary rework.  Either leads to massive amounts of NVA activity.  Anywhere in the middle is a workable solution, with some leaning towards documentation and others towards less.  And frankly, either is ok with me. 

But the point is this – they AREN’T different* or unique because at their core is the basis of human problem-solving – an incarnation of the scientific method and nothing else.

*  Note, I ignored the concept in Agile development that a usable deliverable is delivered more often than in a waterfall model.  Mostly I ignored it because multi-stream waterfall and other non-agile approaches compensate for this adequately in my mind.  Oftentimes we hear of phased project deliverables in which we deliver a series of releases each with successively more functionality yet follow a waterfall-like methodology within the phase.  Hence, were I willing to chop my releases into sufficiently small enough parts, this advantage of Agile methods could be created by a waterfall-like methodology as well.

It turns out ignorance is bliss

Go figure.  Actually, I think we’ve long known this.  Studies have shown (someone please provide me a link; I know I’ve read it) that stupid people are generally happier in life than smart people.  Why?  Well, it’s hard to worry about something if you don’t have the capacity to think about it.

But, here’s one better.  Research done at Cornell by Dr. Dunning indicates that not only are people be incompetent, but chances are when they are incompetent that they are incapable of detecting it.  You can read the brief NY Times article at: http://www.nytimes.com/library/national/science/health/011800hth-behavior-incompetents.html.  Or, if you’re interested, the super-awesome actual research with uber-cool statistics: http://www.apa.org/journals/features/psp7761121.pdf.

So why is this interesting to me?  Everyone knows this truth and has experienced it – at least those of us who have to give out yearly performance reviews have.  You have that employee who no matter how you try and tell them that they aren’t doing a good job argues and disagrees with you.  It’s always something else, someone else, certainly not them.  And on the flip side someone pointed out to me it’s the best employees who can never accept the glowing praise you heap upon them.  If you don’t have employees to do review for think about American Idol (I can’t believe I’m mentioning that awful, trivial show).  Those truly horrific people who go on the show honestly believe they are good.  It’s the same effect.

Why can we not do something with this knowledge!?  I have worked for more than one very large company and not once have I seen anyone test my level of competence (or as Dr. Dunning fears as well, my incompetence).  I have, however, been given programming tests and other job skills tests.  I’ve even administered tests at a prior company, but I’ve been forced to stick to job skill testing.  Job skill testing is great and all, but there’s a lot more to a good employee than whether they can program or not.  It’s the inferential testing that I believe could truly separate a great employee from a not so great one.

So here’s what I was pondering.  Could we design a test, probably a general logic test, and administer it to everyone we think we might hire.  Then, at the end of the test we ask the person taking to test to estimate how well they did – both in terms of how many questions he/she got correct and what percentile they fell in against their peers.  First off, it’d be easy to pick out the incompetent individuals.  They’d score poorly and think they did well.  However, we could also pick out the best.  They’d score well and they would rate themselves lowly – a victim of the false consensus.  This is the big win.  Is this a good indicator of the difference between someone who scores well out of chance and someone who truly has the awareness of how well they did and believes because it was easy for them that it must be easy for everyone else?

Now, my immediate reaction to the research was: who cares if they can’t tell how they did on the test, we just see they scored poorly and don’t have to hire that person.  It probably was your reaction as well.  Here’s why I rethought that reaction.  Maybe I’m making too big of an intellectual leap, but I’m wondering if better employees simply have a better self-awareness about almost everything.  Even if they don’t understand the domain well and therefore answer questions incorrectly, are they able to assess that they don’t understand the domain well.  In other words, could we extrapolate that a candidate who scores poorly but also understands that they did poorly would be a better candidate to train than someone who was blissfully unaware that they didn’t do well?

On a totally intellectual an impractical level, is ignorance a good thing when learning?  Let’s say for example I put a guitar in your hands and try to teach you to play (I don’t play guitar, but let’s say that I do).  At first, you really suck at it, but because you are unaware you suck at it you continue to play.  You may know that you’re not great, but you don’t think your god-awful.  Does this ignorance of one’s own ability lead you to continue to learn.  If you knew up front that you were awful, would you even continue to try?  Would it be too disheartening? 

Could this ignorance be an evolutionary trait people have?  Wouldn’t it make sense if you were a primitive tool user that your low-level of success would be interpreted by oneself as pretty good and therefore encourage you to continue trying whatever you were doing?  Thus, rather than get frustrated, we’d be willing to bang our heads against the wall for longer than someone who was keenly aware that it wasn’t working.  Sure, a lot of the time you’d fail anyway, but because you kept trying, sometimes you would succeed.

Now that I think about it, I believe I may be right that a level of self-awareness exists in better employees.  I, for example, wanted to learn to play guitar.  Note above that I said I don’t play.  My brother and my wife both play guitar and so they tried to teach me how to play.  I suck at playing guitar.  I don’t have the capability for it and I recognized this very quickly and gave up.  Now, if I had kept at it I probably would have ended up being ok but never great.  Was it my self-awareness of my own inability to play guitar that caused me to stop?  I think it may have been.

Yea though I walk through the valley of death…

Or the valley of despair, perhaps.  Have you heard of this concept before?  It’s a pretty simple idea about change management.  Let’s say you have a process that is operating at some level.  When you make change, people react badly at first.  There’s the hair-pulling, fretting, naysaying and general gnashing of teeth which occurs every time you make someone get out of their comfort zone.  And frankly, pretty much every change you make will make someone unhappy.  So the valley of despair represents this reality.  As you move from the old process to the new, the theory suggests that before you arrive at a new (hopefully better) steady state that the people doing the process must go through change themselves to get there.  And that change results in unhappy people who do not good things – like try to use the old process or use the new process poorly.

Anyway, you’re moving along at some rate and then you introduce a process improvement.  Suddenly, your data shows that productivity drops off and defects shoot up.  This isn’t what’s supposed to happen!  You just implemented a new process.  Things are supposed to be better, not worse.  Never has someone said it to me, but why shouldn’t we expect that the human reaction to new process wouldn’t cause things to get worse before they got better?  I’m guessing people assume all this garbage happens before you go live with the new process, but that doesn’t make any sense.  How could a person complete their coping process (you know, denial, bargaining, anger and eventually acceptance) if they haven’t experienced the change?  I full well expect that pretty much every new process is going to go south on you before it recovers.  If the valley of despair exists, then its effect should be felt in your data.

Not only that, but we should be able to prepare for the effect and either allow it to happen and predict it when we roll out the change so nobody freaks out, or we should design a new process that prevents the likely negative behavior that’s a result of resisting change.  I’d love to have some data to show this is true, but alas I haven’t got any good project data yet. 

Right now I’m supporting a project where what appears to be just this has happened.  There was a major redesign done to our workflow system to get code into production.  We used to ask for tons of data from the developers just to do an install and we’d get really bad data as a result.  Developers didn’t feel that giving the data was value add to them, so they did a shoddy job providing it.  I can hardly fault them.  In fact, it has no value add for them to deliver the data.  Sure, it might benefit someone else, but going back to the “people are selfish” rule that I laid out earlier, so what.

Anyway, a huge change was implemented that should have made life better for everyone.  We cut the number of fields that developers had to fill in about in half.  I, myself, was so excited that I believe I used the words “tears of joy” to describe how pleased I was with the change personally.  At the time I had to fill these forms out, so it hurt significantly whenever I had to enter one.  Of the few I did in my career, I’m pretty sure I got every one of them wrong in some way.

Well, several months into the new process and I’ve changed jobs.  Now I’m on the other side of the fence collecting the data to show how the process is doing. Lo and behold, there’s a distinct valley of despair in the data.  At first, the results were poor and they continued the downward trend for a couple months.  Then, during the last three months we’ve seen a steadily improving trend of first pass yield (the percent of workflow items that make it through without any issues).  I think I’m looking at that very valley that everyone talks about but never plans for.  We’ll know after the team does some more research.  For the moment, all we know is that the first pass yield is improving but not anywhere near where we want it to be.

Noah’s Ark with bad timing

So, what if it’d already been raining for 5 days and suddenly the booming voice of God came down and said “Noah, I want you to build me an Ark.”  I’m sure Noah would have been thinking (if not saying), “What the heck!?  Day 1, I was up to my ankles.  By day two,  I was waist deep.  It’s now day 5 and I finally can’t touch bottom anymore!  I’m treading water here and now you tell me to build a boat!?”

It’s a likely reaction in the software development world too.  At least one organization that I came into was in exactly this predicament.  Years of poor software development practices and a sudden massive turnover (100% of the team in less than 6 months) had left the team with their heads barely above water.

Each morning, calls would come in the from the help desk and 4 or more of us would be downstairs on calls with various people trying to get their systems back up and running.  It was a never ending cycle of put out fires, build more poor quality software which caused more bugs which caused more calls to come in.  We were drowning.  And the reaction of most of the members of the team was to simply try treading water faster.

It was in this case that despite Noah being almost over his head in water that God would have been right.  Regardless of how he got there, Noah wasn’t going to survive if he continued to tread water.  And neither was our team.

That’s when it became clear to me that building a boat (metaphorically speaking) was the only thing to do.  Frankly, it wasn’t worth me running down to the help desk to take phone calls anymore because we weren’t getting anywhere.  So I stopped – or at least I moved more slowly.  That’s right – I decided it was high time to ignore the phone calls.  Sure, the critical “I can’t do business at my location” had to be dealt with, but no longer was I dealing with the “half my systems aren’t working” calls.

Step 1, build a frame.  We had a help desk for a reason – to answer calls and resolve issues.  If we were down there answering the issues, what did we need the help desk for?  The foundation of a good software development organization is software that can be supported by someone else.  If you’re supporting your software – then you aren’t developing new stuff.  So, I started by building better checklists and strategies for the help desk.  Now, understand this – people who are not trained to understand software are not going to debug why your client/server application is crashing.  However, if you can identify common fixes that they can perform safely, you should.  My personal philosophy on this one was to use a sledgehammer to crack a nut.   Sure, it’s overkill, but it works.  And the same applies for supporting software.  If I gave the techs tools that might work to fix the problem, but they’d have to try several options then it would waste their time (and consequently mine).  So instead, if rebuilding the boot images would work for 60% of all the issues experienced in the field, then so be it.  That became standard behavior.  I will say, that this would come back to bite me a bit in the future.  Turns out that techs will attempt to apply any technique they know to even situations where it’s pointless.  That said, because rebuilding the boot images was safe, I didn’t really care.  They weren’t doing harm, but it did take some retraining to get them out of the habit of trying the same trick for everything.  By this point, it isn’t a boat but at least it frees up some of my time to make it more like one.

Step 2, siding.  Once we had the techs doing their own jobs on more tasks than they used to be able to we had a teensy bit of breathing room.  Imagine the team now clinging to our wood frame floating in the water.  We’re still cold and wet, but at least I can stop treading and just hang on.  So, where to go next with my software development team?  Well, there were all the bugs in the software, so we could clean up the code, but I didn’t think this was a good idea.  Every time someone created new software for the field they were creating bugs.  So, while I could clear the queue of defects out with some concerted effort the bugs would just be back.  Siding is the real purpose of the boat after all - to keep the water out. 

I created a development process for my team.  No need for anything fancy.  We had nothing, remember.  It was a basic analysis->design->code->test model.  I knew I’d have a revolt if we went too documentation heavy, so for starters I combined analysis and design into a single document.  You had to be able to convince one of your peers that you knew what you were talking about when you did the analysis and design.  Design included test case creation so that hopefully the developers would test their own stuff and so that when we did integration testing we had tests to execute.

I introduced proper source control.  We used to save the software out on the network drives; people overwrote each other’s stuff.  It sucked.  If there was one thing that could have been done for any team it would be introduce source control if they don’t have it.

That’s it.  A basic analysis & design template, peer review of the A&D, peer review of the code and a test plan.  It wasn’t long before the releases going into QA were better.  We started getting releases through QA in 30 days instead of 60 days.  There were still tons of bugs in the code but the queue wasn’t growing anymore.

Step 3, oars and a place to sit.  We are not talking about a fancy boat at this point.  It doesn’t move at all – no power system and it was leaking water.  Frankly, we had to get moving.  Prioritize and fix bugs was the next step.  Now, you’re probably thinking that we should fix bugs based on business criticality.  I disagree.  I prioritized bugs based on how annoying they were to our team.  If I constantly got called by the help desk because there was an issue that came up over and over, that’s what I wanted fixed.  It might have been important to the business as well, but I never asked.  If we were going to leave our old life behind then we needed to get away from those things that drag us down.  Having to recreate upload files from the raw data, for example, is annoying to us but doesn’t bother the business any.  They had no clue we were doing it.  So that’s where I went next with the team.  Now that the help desk could handle the bulk of the calls and the defect backlog wasn’t still growing we could focus on getting us to the place where we should be.  I wanted to put our life with the help desk far behind us, and the only way I was going to get there would be to put them (and myself) out of a job.

Step 4, stop bailing water.  When I finally looked around I noticed that despite us rowing slowly in the right direction that we were taking on a lot of water.  It took just a little bit of poking around to see why.  There were people who worked for me who “liked” treading water and wanted desperately to go back to that way of life.  I discovered that one of my team was drilling holes in the siding while the rest of us were trying to row. 

The great thing about being the manager is that change management is “easy” for your team.  You just tell them to do something new and they do it, right?  Well, I wish.  Most of my team embraced our new found mobility, but some people don’t change easily.  The familiarity of the cold, cold water draws people back for some reason.  And so it did with one of my own employees.  I tried coaxing him, pleading, yelling (bad managerial style, but after a while I was pissed) and finally resorted to working on terminating.  Yup, I was leading him to a makeshift gang plank I had on my boat.  I would be damned if one employee was going to destroy all the change I made.  Happily, about halfway down the gang plank he realized he’d rather be on the boat than back in the water and got in line.  You hate to have to push someone that far, but sometimes you have to do it.  In a bigger organization the use of reviews / raises would be a decent tool.  Too bad more organizations don’t look at process compliance as important.

Step 5, sails and a dagger board.  In sailing, a dagger board is designed to keep the boat from being pushed sideways in the wind.  It helps keep the boat on a straight path despite being pushed by a crosswind.  In the software development world, it was a bit like an autopilot.  I couldn’t steer the boat but at least it didn’t go as the wind blows.  I researched, recommended and purchased a workflow management tool.  I don’t usually do shameless pitches – but MKS’s Software Integrity and Integrity Manager are great products!  We reviewed IBM’s Clearcase and Clearquest, Serena’s TeamTrack, and MKS’s Integrity Manager and Software Integrity.  At the time, MKS’s software was far superior to the rest of the competition in almost every aspect.  Serena’s was prettier looking, but that’s hardly much of selling point.  For some reason Mercury’s ITG suite didn’t make our list, though now that I’ve used it in a later life, I’m glad.

At any rate, the point of a workflow management system was to get us on autopilot with our process.  Yes, prior to all this we did it entirely on paper.  It’s really annoying as a manger to have to review the contents of every release and all the paper checklists to see if they were all done.  But that’s how I discovered that I had a person drilling holes in my ship.  I’m not convinced that automation would have been a good thing before I understood our development process, but now that I had it down, I needed to free myself up to steer the ship.  Frankly, the developers were happy to have the automation too.  They hated printing things out as much as I hated reviewing them.

Step 6, a steering wheel.  Finally we had stability and the life we once led was a fading memory.  We heard from the help desk less and less every day.  By the end of my tenure, I talked to the help desk once every two months or so instead of several times a day.  QA was on record that my team produced the best software of any team in the entire department.  What was there left to do?  Be better, that’s what.  Now, at the time I didn’t have any Six Sigma training.  I’m pretty positive you don’t need to be successful, but a data driven approach doesn’t hurt.  My decision was to implement post-mortems.  At the end of every release, I reviewed every defect QA found, looked at common causes and made careful decisions to change one thing at a time.  Regardless of what happened in a single release, if I didn’t detect a pattern of behavior, I didn’t change anything.  It would take quite a few releases before I’d make a tweak.  Changes were usually minor, like adding items to our review checklists. 

The biggest change I made was to separate analysis and design into two documents.  Why?  Because although people were now figuring out what the right solutions were most of the time, they never considered alternative approaches.  By separating the two documents I could wrap up analysis with just design proposals – very high level suggestions on how to deal with key issues.  It gave some foresight into the design and allowed me to redirect people before they had spent too much time on a design that I didn’t like. 

I should note that a lot of people never do proper analysis.  They’ll often do a “high level design” and then a “detailed design”.  Don’t do it!  Understand that analysis should solely be for the purpose of understanding the situation you are in, not to decide on solutions.  You have to get people to think about the state of the world before you go and change it.  Yes, I insisted on proposals at the end of analysis (because you wanted to make sure they were going to go down the right path based on their conclusions), but the “solution” was 3-4 sentences about what the developer was going to do next.

Step 7, land-ho!  Sadly, by the time we’d gotten to where we were it was time to get off the boat.  I moved on to a new company, but I came back to check out my ship not that long ago.  2 years after I left the organization they still do software development almost exactly as I left it.  I were a few cracks showing, but they could be patched up with a good steward of the process.  It is true that even a good process will go bad after time; processes need adapting to the changing world.

But how I built up my ship is not nearly as important as how I got started.  More important than anything else was the decision to do the hard thing and build a ship instead of treading water.  We could have easily stayed on the path that we were on long ago and the team would still be there today.  So, when it seems like you’re drowning, despite being counter intuitive, you now know what the right thing to do is.

Communication is bad

Is my title a bit harsh?  Yes, probably.  I heard something at work today, something that has bothered me for a very long time and now that I have a place to write my thoughts I realized that there was no need for me to just dismiss it.  How often have you heard “we need to communicate better.”

It is an oft repeated mantra of why the organization fails to live up to its potential.  “If only we knew that in advance we could have reacted.”  Well, no kidding!  If I were omnipotent I could anticipate everything as well.  Communication sucks.  For every person you add to the conversation, the result gets worse and worse.  Take the example of 2 people talking.  Each one can talk to the other so we have 2 paths of communication.  If you add a third person the number of communication paths increases to 6.  Add a fourth person and it increases to 12.  The general formula is N(N-1).  5 people, 20 paths of communication.  10 people, 90 paths.  And it has a nasty growth curve.

I cut it off at 45 people just so you could see a nice curve, but you do the math.  100 people and you have 9,900 possible paths of communication.  How can it be that we reasonably think that even a small sized team is going to talk to each other?  It doesn’t make sense.

Worse yet, consensus isn’t a great thing!  Think large government for a good example.  Compromise doesn’t yield the best results.  In fact, by design it yields sub-optimal results for a given goal in order to provide a benefit to the other party.  In the case of work, compromise usually involves combining everyone’s ideas into a massive heap of garbage because you don’t want to leave anyone’s ideas out.  Worse yet, we replace competent individual thinking with group-think.  Bad business analysts are covered for with JAR sessions; bad designers with JAD and Agile (flame away oh agilistas, I know you want to).

For once I have an answer to my rant.  I believe it lies in interfaces.  Provided that any given programmer is reasonably competent (can produce a small program that does the right thing), interface based design may be the solution.  Forget communication, this is a strictly walled approach to development.  It allows for massive parallelism because it eliminates the need for any individual worker to know what the other is doing.  Rather, we set up a central designer.  S/he is responsible for breaking down the system design into isolated components – let’s call them objects.  S/he creates the interface spec for each component – what inputs it expects and what outputs it will provide in return.  It is this person alone who is responsible for the integration of components.  Within the walls of the work any individual is given, that individual can do as he wishes as long as the interface is faithfully implemented AND no side effects are used to enable communication across the interface.

There are shortcomings to this model.  Off the top of my head, it sub-optimizes.  It may be ideal to take advantage of the work of one object to speed the processing of another, but I’m willing to make this trade-off.  As Steve McConnell said in Code Complete of code optimization, you’d be surprised as to what needs optimization and what doesn’t.  Plus the advent of more powerful and continually cheaper hardware allows us to be somewhat wasteful of this resource.  It is plentiful, after all.

I have no idea if such a model is unique.  I certainly doubt it, but I can’t recall a formal methodology name for it at the moment (like waterfall, spiral or agile).  The idea of a surgical team approach to software development was presented by Fredrick Brooks in The Mythical Man Month and Microsoft’s early years were shaped by the concept of a Master Designer (I’ve read, but can’t recall the source at the moment).  Both of which emphasize the idea that a single individual control the overall design and delegate work out to subordinates.  Fredrick Brooks sees the subordinates as specialists (a tools person, for example) where the Microsoft approach is structurally similar to my (probably unoriginal) proposal but I can’t say explicitly calls out the idea of interfaces as the boundaries of units of work.

This model of development is certainly supported by Service Oriented Architecture, which creates clearly defined boundaries via web services, allowing developers to assemble greater value from smaller components (at the core of Web 2.0) plus encourages reuse (which I could care less about for the purposes of this entry).  So while the architecture of systems seems to be moving this direction, I haven’t seen an example of organizing a team around the same concept – that each individual will produce a smaller piece of stand-alone work that can be integrated by the central designer to build up a complete system.

I think this model comprises some of the best of all worlds.  Waterfall methodologies suffer from a lack of parallelism and rely on heavy intra-team communication to divide and conquer the work.  However, they benefit from the fact that there is no need to keep a business person engaged throughout the entire effort to make decisions.  They can be off doing other activities that are value added and trust (hopefully) that an adequate specification will result in an acceptable system.  On the flip side, Agile relies excessively on communication to make all decisions.  By forcing people into the room side-by-side poor individual decision making is compensated by assuring no individual ever need to think for himself – he can defer to a decision maker.  Of course, the downside is that your decision maker can’t do anything else while they make constant decisions about the project.  And what happens if he goes on vacation?

It is impractical to think that everyone can talk to everyone to compensate for a lack of individual foresight.  Organizations running on this model move sluggishly while they attempt to get the right people in the room just to make a decision.  But a model of development which mimics the hierarchical structure of businesses makes a lot of sense to me.  Rather than a manager being at the top of the structure, it’s a lead developer who drops developers into project roles to build small pieces of functionality with well known interfaces.  Developers gain a great deal of freedom to do as they wish in their space, achieve efficiency by allowing for massive parallelism and independent thought.

So there you have it, a rant with a solution attached.  Again, I make no promises that my idea is original or all that good, but it does seem to deal with the impracticalities of communication issue within a development team.  One can reasonably extend the concept to believe that “throwing it over the wall” might be a reasonable behavior if the acceptable interface (think entry criteria) were well defined.

Is this post NVA?

And what is NVA anyway?  Non Value-Add.  Anything you do that doesn’t change something towards your customer’s goal is NVA.  Generally speaking, if your customer wouldn’t pay you to do it then it is NVA.  So, if you are mowing a customer’s lawn, you driving there is not value added to the customer.  Sure, this seems crazy.  How are you going to mow the customer’s lawn if you don’t drive over to their house?  Regardless of the fact that you don’t know how else you’d do it, the customer isn’t willing to pay you for that.  Think about it, if you said “and one of the great features of our lawn mowing service is that we drive to your house” your customer would think you were crazy.  Fly for all they care – they just want their lawn mowed.

Life is full of NVA activities.  You really can’t avoid them unfortunately.  You can, however, minimize them.  And that’s where my rant begins.  There’s a distinct difference between minimizing an NVA activity and optimizing it.  For example, if I can’t figure out how not to drive to the customer’s house to mow his lawn then perhaps I can optimize the process.  For example, I could plan my mowing route so that I retraced my path as few times as possible and that I only made right turns because left turns mean I have to cross over traffic and that wastes time.  By the way, some package delivery services who shall remain nameless (primarily because I can’t remember which one it is) plan their routes using the latter philosophy.  I’m not kidding.  I’d call this optimizing.  You didn’t figure out how not to drive to the customer’s house in the first place, but at least you made it as efficient as possible.

In theory, I’m ok with this.  Sometimes NVA activity can’t be avoided and this is not a bad decision.  What is a bad decision (and one that I’ve seen done) is to optimize NVA activity rather than minimize it.  Bug fixing comes to mind.  You have to accept this truth first – bug fixing is NON VALUE ADD.  Yes, you have to fix bugs in code, but can we all agree that it’d be better if you just didn’t write bugs in the first place?  There’s lots of research on the cost of poor quality.  Numbers quoted usually indicate a cost of 100-1000 times what it would have cost to find it and fix it earlier.  That is, if I’d found and fixed the defect in the design phase it might have cost me $100.  If I don’t catch it and it gets to prod, it could be $10,000 – $100,000 to fix it.  Yes, you’re reading that right – one hundred thousand dollars.

But I’ve observed the following and if you’re not at least slightly disturbed how we got to where we are by the end, you’re not really grasping the issue.  Let’s say Bob (he’s fictional and does not represent anyone I actually know by the name of Bob, so don’t go blaming him) gets tasked with reducing maintenance labor costs at your company.  Maintenance labor includes a bunch of things – like upgrading hardware and applying security patches to the operating system and most significantly bug fixing.  Sure, projects create bugs but eventually those projects and their teams go away, so the projects (and their respective budgets) are no longer around to absorb the cost of fixing bugs.  The maintenance budget pays to keep the lights on, and that includes fixing bugs that no longer have a project to call home.

The first step is easy – Bob already figured out that bug fixing is the vast majority of the maintenance budget.  It makes sense to address bug fixes because that’s the best opportunity to save the company money.  As in all good Six Sigma, Bob starts in the Define phase which includes a lovely project charter.  Part of his charter is to define the goal and potential opportunity.  Let’s say for whatever reason, the business decides they need at least a $100,000 savings from Bob.  The bug fixing is costing the company $1,000,000 a year, so that’s 10%.  10% seems easy, right?

So Bob does some interviews about the bug fixing process since that’s where he was tasked to fix.  He does a value stream map as well.  Lo-and-behold, about 20% of the duration of the bug fixing process is waste.  It includes things like transporting the ticket around and waiting for someone else to look at it.  Bob does some quick math.  The total cost of bug fixing is $1,000,000.  If he eliminated this 20% waste, that’d be $200,000 the company could save!  It’s two times what Bob needs.  All he needs is a 50% reduction in the NVA activities of bug fixing.  50% less NVA means Bob gets his $100,000 payoff for the company.  And wouldn’t you know it, Six Sigma has a toolset for this.  They call it LEAN or Lean Six Sigma.

Let’s go back to the lawn mowing operation.  If I told you that you needed to remove 50% of the NVA costs associated with driving to the customer you’d probably think that was a tall order.  Chances are you weren’t that stupid to begin with when you laid out your mowing route.  You probably don’t drive back and forth from one end of the town to the other mowing lawns.  You probably have it at least regionally organized so you’re doing all the houses in a given development on one day.  You drive a big truck.  You have to just to pull the trailer, the lawn mower it carries, the spin trimmers, rakes and not least of all your crew.  I don’t think you’re going to optimize your NVA activity to make it 50% less costly.  Ok, maybe I’m wrong.  Maybe you could find 50% savings in your NVA activity.  I just heard the other day that something like 80%-90% of all work in the average process is NVA.  However, I should temper that by saying that the same individual quoted “world-class” processes as being only 50% NVA.

Bob had a much better option.  He was tasked with reducing the maintenance costs to the company.  Nobody told him to fix the bug fixing process – they told him to save the company money from maintenance.  Indeed, he properly noted that bug fixing was where the money was and he should go after it.  But if Bob had taken the time to consider it he would have realized that although he could optimize bug fixing and probably get his needed savings with a mammoth effort there was a better option.  Did he ever stop to think – why do we need maintenance at all?  Everything you do must be looked at with this lens.  “Do I need to do this to deliver value to the customer?”  Guess what, your customers don’t like bugs.  They sure as heck aren’t paying you to fix what you screwed up in the first place.  Try asking that question honestly about everything you do.  Yes, it sounds absolutely insane some times, but you have to ask if you want people to think about it.

“Oh my god,” you exclaim, “bug fixing is in itself NVA!  Don’t do it Bob!”  Cast your net widely and you see the world from the customer’s point of view.  They don’t see bug fixing as a process – they see software delivery from the time they send you requirements to the time the warranty expires.  What if Bob had done that?  Well, first off, his job would be a lot easier.  If he needs $100,000 savings, all he has to do is reduce the number of bugs being created by 10%.  Everything and anything else he saves the company is icing on the cake.  Software development is a big process and most likely there are lots and lots of opportunities to improve.  Not only that, his total opportunity now maxes out at $1,000,000.  If he 100% eliminated all bugs, Bob would save the company the entire amount.  If he just focuses on optimizing the NVA activities of an entirely NVA process his absolute maximum opportunity is $200,000.  All the bugs that were being created still get created but, by golly, at least we fix them quicker.  Best of all, Bob has lots more wiggle room for success with the wider view of the problem.  He only needs 10% improvement in the broader area as opposed to a 50% improvement if he looks more narrowly.  Small percentage improvements on big dollar figures still add up to big dollars.  Alternatively, he needs big percentage improvements to make much out of smaller dollar figures.

So why did Bob choose to optimize the NVA activities of an NVA process?  Well, I could envision some reasons.  Bob may not have thought bug-fixing was NVA.  This is probably likely.  People accept that bad things happen and that we should compensate for them.  I do not accept this, and I didn’t accept it before I got Six Sigma training.  You have to strive for better than that.  I don’t like it when my brand new car has defects, why should I expect anyone likes or accepts it when their brand new software has defects. 

Another alternative was that by keeping the scope small, Bob found the project to be more manageable.  If all you have to do is deal with the people who fix bugs, there are less people to talk to, less people to convince and most importantly a lot less possible root causes to look at.  I think along the lines of my rant about people being selfish that lazy probably fits into that.  If you can get what you want by exerting a minimum of effort your rewards are maximized.  Bob probably wouldn’t see an extra penny if he saved the company the full million; he was told to save them $100,000 and so he did as little as possible to reach that target.  It seemed easier that way.  However, since he was focused on one change to one very specific area he put all his eggs in one basket.  Had he gone after the larger defect issue, a dozen smaller changes that each yielded a 1% improvement would have met his goal and hedged his bets that one of his fixes wouldn’t pan out.

Bob’s probably the kind of guy who likes to load up a revolver and play Russian Roulette with his foot as well…  By the way, Bob’s gamble didn’t pay off in real life.  Though he minimized the NVA activities he’d identified, they turned out not to have any correlation to the cost of bug fixing.  See, if you do a time study (as I mentioned way above) then you’re looking at the duration of an activity, not the effort that goes into it.  Effort (the cost of fixing bugs) and duration (the time it takes to fix them) are only loosely related to each other.  If one person puts in an hour of effort, then the duration is at least an hour long.  But it could be much longer, I could put in an hour of effort over a work week at 12 minutes per day.  And in fact this is what was happening.

Now, rather than sitting in the wrong work queue being ignored and waiting to be reassigned, the tickets sat in the right queue being ignored…

How software development isn’t like a hamburger factory

There are always these articles where someone attempts to create a drawn out metaphor for something based on similarities to something unrelated… how software dev is like motorcycle maintenance (Zen and the Art of Motorcycle Maintenance) or like a restaurant (an article someone forwarded me on how Gordon Ramsay’s Kitchen Nightmares was likened to software development).  Here’s a post about how software ISN’T like something.  Software development isn’t like a hamburger factory (by which I mean McDonald’s or Burger King).

Imagine it, there you are a fresh-faced kid just out to make a few bucks so you can fill the gas tank of your cruddy Gremlin (thanks to Wayne’s World for immortalizing the epitome of the I-still-live-in-my-parent’s-basement vehicle) and take your girlfriend to the movies.  Anyway, I digress… while you’re standing there blissfully day-dreaming about the aforementioned girl an order comes in for you to make a hamburger.  You’ve gotten about 63 seconds of training on how to assemble a burger the Burger Factory way.  In fact, it’s on an 8 1/2 x 11 inch laminated job aid with little pictures and circles and arrows describing what each picture is.  You dutifully place down the wax paper wrapper, orient the bottom bun in the prescribed circle on the wrapper, slap on a wafer-thin patty, three pickles, a squirt of ketchup and mustard and plop on the top bun half.  It’s a masterpiece of understatement!  You glance up at the proper wrapping technique on your job aid and fold that 500 calorie delight up nice and tight.  Job well done.  Frankly, the 63 seconds of training doesn’t matter – the job aids, error proofing (note the nice little circle on the wax paper so you know exactly where to put the bun) and general familiarity assures us that you are now the world’s most complicated robot. 

It does make me wonder why all those employees haven’t been replaced by robots, by the way.  It’d probably be the coolest thing to walk into an almost totally unmanned fast-food joint and other than maybe interacting with a cashier, there isn’t a soul in there.  We’ve seen via the supermarket that we can mostly get rid of the cashiers too and it has the bonus of not worrying that the robot might spit on your burger…

Alas, that is not my point.  So, you’ve managed to make one burger.  It’s a nice repeatable process right?  Building on that last scenario, another order comes in for a burger.  This time, you plop down the wax paper wrapper and drop on the bottom bun.  But things have changed!  The bun is bigger and it’s square!  The patty is still round, and you’re relieved that ketchup and mustard still come out of their respective containers.  You drop on the square top and do your best to wrap up the ill-fitting bun in the standard wrapper.

Next order:  The wax paper wrapper is suddenly made of metal – and I don’t mean flimsy aluminum foil.  It’s a hefty 1/16th of an inch thick.  Buns are still square, but they’re significantly bigger yet.  You have to place about four patties on the bun just to get reasonable coverage and the ketchup jar now spews mayonnaise.

What the heck is going on here?  Your job aids are useless.  They don’t have a step for dealing with hefty metal plates instead of wax wrappers and where the heck is the ketchup?  Still, you put on the bun top and at least if someone took a quick glance at it they might conclude that this is indeed still a burger.

Thank god this doesn’t happen to you in real life.  Working at Burger Factory would be a challenging experience every day.  How could you expect a minimally trained teenager to deal with constantly changing inputs?  And yet, this is the life of software development.  Every “burger” you make developing software affects your ability to make the next “burger.”  The system you are trying to work on changes.  What once was a truth about the way the software was designed is suddenly just a fading memory of the core architecture that some well intentioned but relatively incompetent software developer violated.

It is an enormous frustration to anyone attempting to improve software development processes.  There’s no repeatability because the act of working on the system is destructive.  While I do mean that many poor-to-mediocre software developers do a lot of harm to the system even good software developers destroy the former truths about the system simply by enhancing it.  Documents about the system become outdated or document half-truths.  You’re lucky if you have documents in the first place.  The only thing you can trust about the system is what it does right now and you can only truly know by looking at the code.

And this is what really frustrates me.  When I watch people approach a Six Sigma project in the software world, it’s like watching a full-grown dog try and run under the chair legs like when he was a puppy.  It is just bad for all involved – upturned chairs and a dog with a headache and a confused look on his face.  I’m certain this is where significant distrust of Six Sigma becomes well justified.  How frustrating is it for all involved that you can’t find a nice correlation between pretty much any factor and the resulting output – good or bad software.  It still appears to be entirely an art.

 It cannot be reasonably called software engineering if it isn’t a science.  I think at this point two realities have to be accepted.  One, laymen don’t understand software and so treat it as if it was as pliable as silly putty.  You wouldn’t reasonably drive a bulldozer through the front door of your house and expect it to still be standing afterwards and yet metaphorically this is what people do with software every day.  “Hey, I know I told you that I’d only be running about 1000 records through your tool each week, but I tried 10,000,000,000 and it’s too slow!”  Two, process improvement people think software development is a science and not an art.  I envision this as being on the two worst ends of a possible spectrum.  First off we have people who think that anything done in software is free, that all software is easy to change and “what’s the big deal that I broke your core assumption about all the data coming into the system would have a unique identifier.  So what if I reused some IDs?”  On the other end we have people trying to help a company meet those crazy expectations of the customer saying “every time you change software, just follow this handy job aid and… what?!?  What do you mean the buns are now square instead of round?  My job aid shows round buns!”  And software is really at neither end of the spectrum.  It doesn’t just do as a I wish but yet isn’t created rigidly and predictably.

Let’s reset some expectations that created the above realities.  Software development (for the moment) is at best a methodology, not a process.  The difference is that a methodology is the hand-wavy this-is-what-we-kinda-sorta-do-except-when-we-don’t where process is we do steps a, b, c and d all the time, every time.  It’s a spectrum, of course, but you get the idea.  There is some point I think people could reasonably pick out where you’d say this is more descriptive (methodology) of what you do rather than prescriptive (process). 

That said, I asked a master black belt at one point what the best tools were for dealing with this lack of repeatability.   Needless to say, the answer was not satisfactory.  Six Sigma doesn’t have good tools or techniques for this space and solutions needs to be made.

Secondly, customers need to suffer to understand real costs of software development.  I’m thinking that every time a customer (or your manager who wants to get promoted and knows that the best way to get promoted is to fulfill every crazy customer request) asks you for a new feature you should be entitled to extract the cost somehow.  If you were building a house and you had all the walls up and the roof on and then you thought “hey, I think I’d like a third story on my house”, I’m certain without anything telling you that it was going to hurt you where it counts.  You probably should have thought of that earlier. 

The thing is, we don’t build software like we build houses.  Sometimes something that would seem very difficult under most circumstances works itself out quite nicely because of your particular design.  Other times, something really innocent sounding shatters the foundation of your work.  Anyway, it doesn’t hurt the customer in the same way it would if said customer was building his or her own house.  I think software developers need either a small mallet or perhaps a taser or something to impart the appropriate level of understanding. 

I mean really, why is it that people just accept that software requirements change?  (NOTE: I had to come back and edit this rant after I let my boss read my blog site.  Thank god she didn’t think I was crazy).  Personally I think just because software requirements is a methodology doesn’t mean you couldn’t get requirements right the first time.  Being loosely defined doesn’t mean poorly done.  That said, the cost of such up-front quality would probably be enormous and not worth it.  But what is missing is that people don’t have a sense of scale when it comes to requirements changes in software.   Frankly, after the walls and roof of the house are up, some things will be easy to change and some things are going to really suck.  All changes are not equal and the same is true with software dev.  The problem is, people don’t recognize it in the same way as we do with homes.  Certainly you can guess that if the walls haven’t been painted asking for a different paint color is not a horror.  But software has none of those common features, so it is hard to say when a change will be easy or difficult.  A lack of perspective is an unfortunate reality.  Further complicating this situation is your customer had experiences with other developers who, because of their design choices, were able to make some changes easily that are difficult for you to make.  On the flip side, every house you customer had built in their life was constrained by the same laws of physics.  It makes things a lot easier to explain if the house will fall down as a result of a requirement change.

“Managing expectations”, “under-promise and over-deliver”.  These are all reasonable ways of dealing with the reality of customers lack of understanding.  And yet, I think software developers and their management do the most disservice to themselves by not having their own sense about how difficult a change would be.

It’s probably not the most coherent rant, but the point is this:  from a process improvement methodology we cannot approach the problem like we are dealing with repeatable widgets.  Software dev pretty much never makes the same thing twice.  So let’s get over it and find a new path which needs new tools to predict success.  And frankly, I think if we can deal with our own realities then the customer lack of reality problem (though annoying) will fall into place.  Alas, I have no answers… I’m going to get some though, I hope.  My real job kind of depends on it.

Me! Me! Me!

Well, after three posts about process, I decided to spew my (probably not terribly original) idea on people and change management.  My observations with getting someone who doesn’t work for you to do something you want goes something like this:

You: “Hey, Bob, I’d like to change the way you <insert activity here>.  It has all these benefits if we do it this new way.”

Bob: “Sure, Fred, send me the new process documentation and templates and I’ll get right on that.”  (Assuming your name is Fred, of course, otherwise replace Fred with your own name.)  This conversation will also include a lot of smiling and nodding and general agreement from Bob.  He can’t say no to you directly probably, because he won’t look like a team player and that’s bad for Bob.

… several months pass …

You:  “Hey Bob, I was looking over the project documents and I noticed you didn’t follow the new process we had talked about six months ago even though you said you would.”

Bob:  “Yeah, well… <insert excuses or perhaps just a trailing off to mumbling as Bob wanders away >”

So what happened?  Why did Bob not change?  He said he would change, you detailed all the good reasons why he should change (hopefully) and still he didn’t do it.  The issue is probably somewhere grounded in what is commonly expressed as WIIFM (What’s In It For Me).  Problem is, oftentimes there’s nothing in it for Bob to change.  Oddly enough, even if it might be better for the company as a whole if Bob did whatever it was differently, Bob doesn’t see it.

So, that leads into my theory of human behavior, in three words.  People are selfish.  Genius, no?  Yes, WIIFM scratches the surface of this truth, but not necessarily in the right way.  Why does anyone do anything?  In economics terms – utility.  Whatever it is, be it a product or behavior, it has value (utility) to the person buying/doing it.  This holds true even for actions that appear very selfless – like charity.  How often do you see a person doing something charitable and being miserable?  I bet not that often.  Charity makes people feel good, and feeling good has a lot of utility to people.  Why does someone bring you expensive Starb*cks coffee “for no good reason?”  You can bet they either want something or that doing something nice makes them feel good and that’s worth more to them than the $4 they put out for that cup of caffeine goodness.  Yes, I’m sure there are some of you who are horrified to think that our society could function with everyone behaving this way.  Actually, it works out quite nicely because being selfish is not necessarily an I-win-you-lose arrangement.  Sure, that happens when Bob grabs the last donut and stuffs the entire thing into his gaping maw before you can blurt out “Hey!”  But look at the coffee example – both of you got something as a result of being selfish.  You got a nice steamy cup of liquid-energy and the giver got to feel good. 

And this is model which you can leverage to get what you want when it comes to change management.  You want process change because if you can save the company a million dollars you know a nice fat bonus check is on its way into your sweaty little hands.  Unfortunately, Bob isn’t going to see a cent of that, so why should he do anything different?  Maybe you have a very clear benefit to Bob – “if you make this change, we’ll be able to eliminate a huge group of users who calls you constantly to fix the code you screwed up in the first place.”  But, for the most part, the employees who need to change their behavior see little direct effect from the change.  Plus you have a time issue – the benefit they get is very distant from the expendeture they must make.  They have to learn to do something new, but it’s going to be months before enough of the bugs are out of the system that the users stop bugging them.  And frankly, researchers have observed in children that most kids will choose one Oreo cookie now vs. two cookies in a little bit.  You need a “now” benefit for the person changing, and I have one that everyone likes – their egos!

Yes, it’s the slimy salesman to the rescue.  Like it or no, if you’ve got nothing else to offer ego stroking has miraculous powers.  Now, you can’t go in saying “you’re a really great guy Bob, and my are you handsome.”  It’s too thinly veiled and chances are Bob is going to think you’re coming on to him.  That just creates a whole host of awkward conversations with HR.  Let’s go more subtle. Try “Bob, I need help.  Your team does this stuff really well, but some of the other teams are struggling…” 

Yes, it is a lie!  Lie, lie, lie!  There I’ve said it!  You are lying and if done right it is going to work.  It’s not a horrible lie, but it’s the kind that appeals to something Bob can have now – a bigger ego.  You get your process change and Bob gets one of the most valuable pieces of collateral – to feel good about himself.  He can feel good because he thinks he’s helping you out – which is true.  Really you’re probably getting more from it than him.  He can feel good because you’ve validated that there’s nothing wrong with him or his team (which is a lie), but Bob changes his behavior since he probably will be playing connect to dots in his head.  It goes something like:  “If I change to the new process, then Fred will give his monthly progress report, and on that nice little chart he’s got it’ll show my team doing better than all the other teams, and everyone is going to see that my team is superior to all the other teams, and my boss will notice that too and she’ll give me a raise or a promotion because I’m better than the other managers.” 

Selfishness!  Bob, wants to get promoted or get a nice raise so he can buy himself more nice things or amass more power!  And even though you can do nothing to directly influence that happens and because you are selfish you don’t really care, Bob will take advantage of the opportunity to differentiate himself because he (and his ego) think they could get something bigger AND he gets a “now” benefit for helping poor little you out.

Elephant carving is just like process creation

I mentioned in my last post about carving an elephant.  There’s a (bad) joke about it.

Q: “How do you carve an elephant?”  A:  “First, get a block of marble and then remove everything that doesn’t look like an elephant.”

Ok, I know it’s a stupid joke, but that’s not the point.  It reminded me of process design.  I am in training for DFSS (Design for Six Sigma).  One of the criticisms that I’ve heard leveled against Six Sigma (and don’t get my wrong, I _so_ have not drank the Six Sigma Kool Aid) is that it assumes that the process is repairable.  And true, the premise of DMAIC is that you can Improve (that’s the I in DMAIC, in case you didn’t guess) an existing process once you’ve Defined what the boundaries of said process and the potential opportunity, Measured the existing system’s capability to produce the desired result and Analyzed the data.  The C, by the way is Control.  Once you’ve fixed something, you want it to stay fixed.  This is NOT rocket science, people.

At any rate, DFSS assumes (and I’ll probably get this wrong, so flame away) that the process is so broken it isn’t worth repairing or it works quite well but just can’t reach the target you want.  For example, the current gasoline engine technology we have just isn’t going to make a car that goes 400 miles per gallon of gas.  Pretty much no matter what you do with the existing technology/process, you aren’t getting there from here.  There is a place for innovation in Six Sigma, and DFSS is that answer.  Again, please don’t take that as an endorsement of Six Sigma – the Kool Aid is still on my desk and I’m eyeing it suspiciously at the moment.

Anyway, my brief rant is this.  When you start to look for a new “innovative” solution, we already know (presumably) that the answer does not exist within the existing box.  If it did, you probably wouldn’t be throwing away what you had and starting all over.  Right?  Well, I’m not so sure.  First, lets take the case of the fantastically broken process.  I can bet you have one or more of these in your job today.  In fact, I’ll hazard a guess that the process by which you interact with support teams is a mess.  Why?  Because you have to submit a stupid trouble ticket and it disappears into some magical queue where it floats around and nothing seems to be happening on your problem until – Voila!  The ticket reappears in your email with a little notice that it has been cancelled because the developer (or technician) assigned to your problem can’t seem to reproduce the issue!  Sound familiar?  On a side note, this is pretty much why tech support asks you to reboot so often – they really don’t know much of anything and it is impressive how many issues can be resolved by a good ol’ reboot.

Now, I ask you.  Is the process by which your issue gets resolved difficult?  Has the process reached entitlement and simply can’t get any better?  Is it so broken that it must entirely be thrown away and built anew?  Given the opportunity, I think most people would say “yes” and clear the slate and design it all over again.  The answer (at least from my perspective) is no, no and no.  Do you honestly believe that a process such as this, no matter how horridly it works needs DFSS?  Again, I believe the answer is no.

So, what does this have to do with carving an elephant?  I’m going to propose that you a) can’t know if you need to think outside the box until you know what the box is and that b) can’t think outside of the box until you know where the boundaries of the box are.  And in this way it is similar to carving an elephant.  If you begin with the block of marble which you think could contain your elephant (your process), it is only once you carve away everything else that you’ll know if your block of marble in fact couldn’t contain the elephant.  How often do we go for “out of the box” thinking when a perfectly good answer exists within the box?  More importantly, the box is NOT defined by what you know.  It is defined by what the entire industry in which you are working in knows.  Chances are, someone else has carved your elephant from this greater block of stone.

In the case of your trouble ticket process – god knows there’s someone out there with a company who competently deals with trouble tickets.  As far as I’m concerned, that good process is probably not all that innovative and exists within the boundaries of everything known today, but you have to go looking for it in the boundaries of what is known today.

The other case I’d consider here is when you are designing a net new process.  Sure, you’ve never done it before, so DFSS makes sense.  Again, just like your fantastically broken process, have you gone and looked at what exists in the boundaries of what we collectively know today for an answer?  I bet you’d find answers for many of your “new” processes.  “Not Invented Here” runs amok (at least it does in the IT field).

So, I guess the short story is, just because you haven’t figured out how to carve an elephant from your block of marble, doesn’t mean that there isn’t a way and that maybe you should just copy the guy next door who has figured it out.  Save your creative energy for when you really need it.