Partner or Customer?

This is often a point of contention in the systems world.  Is the business a customer who we serve or a partner who we work together with?  One can see how we’ve made the bad shift to calling them business “partners.”  Partnerships are a peer relationships.  We are equal, we have a say in what happens to the system because we know best.  We have goals and needs that matter as much as the businesses’ goals.

This is so horrifically wrong I don’t even know where to begin!  First off, what’s the difference?

Partners work out their differences together.  Think of a husband and wife team working out their financial decisions.  They’re equals and have to agree on what matters to them collectively.  “Partners” also suggests we are stuck with each other, so we better figure out how to work together.

Customers, on the other hand, are always right.  We serve customers needs.  They tell us what they want and we do it.  By comparison, when we are customers we are not tied to one supplier.  We can choose our supplier which forces all of them to be competitive for our business or risk going out of business.

Who exactly wants a partnership?  I believe it is Systems who convinces the business this is in their best interest.  Why?  Systems people are generally relatively smart and they consider what they do to be highly specialized and so want to be listened to and have their advice taken.  In some ways they’re a lot like doctors.  We hold knowledge that other people don’t want to acquire or can’t acquire.  Just like it frustrates the hell out of your doctor when you come in and tell him/her what’s wrong with you, it makes Systems people really mad when you tell them how to build their systems.  After all, we have grand visions of beautiful works of engineering.  We are arrogant. 

Arrogant people do not make good partners.

Why does business agree to a partnership?!?  On the surface, it seems to make sense.  The thinking goes something like “if we have a partnership then we’ll get to see behind the curtain and finally understand why it costs a bazillion dollars to do anything.”  Again, like going to your doctor, if it only took you like 2 minutes to diagnose your own ailment so why does a doctor’s visit cost so much?  If the business can imagine the solution as a web page or two, why does it cost $5 million?  When a business has been unhappy with their systems costs for a long time, they enter into the “partnership” in hopes that once they find out what’s going on they’ll finally be able to put an end to it.  Stupid move.  It’s like giving an alcoholic a drink!  Once we’ve had that sweet, sweet taste of telling you how we can’t do your project because we need to re-architect this batch program or that web service it’s all over. 

This is why a business should always refer to systems (even internal systems groups) as a supplier.  And Systems, we should get used to calling our business a customer.  Like it or no, we are here to serve.  There is nothing wrong with serving our business!  Our business is the reason we exist.  Our business pays our salaries.  If you want to have the right to recommend something to your customer, you must first serve them faithfully so they know they can trust you. 

I don’t go to a new mechanic and immediately do all the services they recommend during an oil change!  If you do, seek help.  I don’t trust them yet to tell me what I need because I know they’re trying to sell me something.  After going to the same place a long time and having them talk me out of an expensive service I propose that I don’t need, then I’ll know that they are trustworthy and start taking their advice.

Most importantly, calling your business a customer will remind you of an important fact.  Systems is expendable.  As I’ve said in a previous post, we are largely cost centers and therefore it is in the best interest of the business to minimize the expense.  If you start thinking you are a partner, your costs will go up, you will get comfortable and settle in for a long, cozy relationship.  And then, one day, when you don’t expect it, the business will hire an outsourcing firm to do the work instead and FIRE YOU ALL!  Just because you are an internal systems group doesn’t mean you don’t have to be competitive with the real world.  If you can diagnose your own ailment, you can bet folks in the business are smart enough to figure out that their own systems group has become a waste of money.

A mechanism to break the company downfall cycle

Mark Buchanan has written an incredibly interesting work called “The Social Atom.”  I’d say it’s on par with the quality of Freakonomics, which was another very worthwhile read.  I won’t get into many details about his work, but to say that he provides reference to many scientific studies where simplified models, like those used to model physical systems, can demonstrate very complicated behaviors.

In particular I want to focus on a model regarding the rise and fall of companies (see pages 181-184 of his book).  In this model, each individual produces some output.  Also, the only other difference in individuals is their ambition; some people are lazy, others are more motivated.  As in the real world, you can work alone or you can work together and also as in the real world, if two motivated people work together then the whole is greater than the sum of their parts.  It pays to cooperate.  What happened?  Well, at first the people in this virtual world worked alone.  The more motivated people figured out it paid off to work with other motivated people because it increased the output above what they could do themselves and they benefited from it.  In fact, teams of motivated people got together and formed “companies.”  As the teams grew, the company output would grow as well, but each individual’s contribution to the output become proportionately smaller.  The less ambitious people in the company began to cheat.  Even though they were cheating the output of the company was about the same.  But when the cheating caught on among all the less ambitious people, the hard workers noticed and jumped ship.  And just like that, the “company” collapsed.

It’s a great model of how real companies rise and fall.  You eventually grow to the point where you simply pick up enough losers that the losers cause the people who were really making the difference to quit.  So what do we do?  I think Bell Labs stumbled upon it years and years ago without even realizing it.  First off, let me say that Bell Labs (the remainder of what happened when Ma Bell was split up) is not the company it used to be.  It may be exactly because the behavior I’m about to describe stopped happening.

Every so often, not yearly, but relatively often, the entire population of the company was force ranked.  The bottom X percent were then fired.  It sounds horrific, I know, but I think they were on to something. 

If losers do more damage to a company than just sucking down resources there’s a lot of motivation to get rid of them.  Not only do losers contribute nothing and take a share of the profits, but they also cause the people who make the money to quit.  They hurt the company twice.

Now, in the simulation it isn’t clear if joining a “company” was a mutual agreement or if one person just had to say “I’m working with that group,” but I’m not sure it matters.  In the real world, there is interviewing and screening processes that should allow you to hire just the best.  As we all know, the system of hiring good talent is far from flawless and so some losers are going to get in the door (by the way, if you’ve ever read a blog entry or other media about how fabulous Google is to work for, you can rest assured that every so often some loser is sneaking in the door…).  Eventually enough of these under performers are going to get in and do some serious damage by not contributing AND causing contributors to quit.

So, returning to Bell Labs’ employee removal system, a regular flushing of the lower ranks indeed makes sense.  First off, those who perform well are rewarded with knowing those who waste their time will sooner or later be gone.  And while you can be assured that some percentage of the losers who were just fired will be replaced with equivalent losers, some winners are going to get in the door as well.  Those winners will take up their place somewhere in the overall performance hierarchy and probably force some of the people who escaped the last cleaning down into the ranks who will be swept away next time. 

The key is to figuring out how often to clean house.  If you do it too often, eventually (in theory) the entire company would be filled with decent performers.  Eliminating the bottom X percent would then be firing perfectly good employees who simply aren’t the best.  At that point, it probably becomes bad for morale.  If you do it too infrequently too many good people will jump ship before the house cleaning occurs.

Me, I’d love to see the model re-done with one rule added:  if a sufficient # of the employees vote against a less ambitious employee they can be kicked out of the company.  How about that for a democratic process to keep the house clean?

How factories ruined efficient software development

The vision of a factory of workers churning out code has ruined efficient software development.  At first one might argue that it’s because thought workers don’t work that way, but I think it’s something much more simple.  Consider what a real factory does.  It takes some input and manipulates it in some way in a series of steps to produce an output.  Whether it’s a machine doing the work or a line of people, each step has someone/something processing an input and spitting out an output.  Each person or machine is highly specialized to do just that task and do it efficiently.

 The factory system is a pipeline where each person has a stream of inputs coming in and manipulates each into a stream of outputs going out.  If one person in the stream were to stop working on their inputs everything down the line from them would simply run out of work to do.  The resources would be idled, and those expensive resources (be it people or machines) would still have to be paid for but wouldn’t be producing anything. 

Factories are highly tuned to never allow those bottlenecks to happen so that nothing goes idle for long, if at all.  It’s easy to watch the widgets move through the system and it’s easy to see where they back up so it is easy to fix.  And here I bet you’re thinking I’m going to say something cliche’ like “you can’t see software widgets” or “because each part needs custom manipulation there are always backups in the system.”  The answer is no, I’m not.

The reason the factory model doesn’t run efficiently in software development is because there is too little going through the queue, not too much.  And Parkinson’s Law (“work expands to fill the time allotted”) takes care of the rest…

I think software does produce the equivalent of widgets.  Sure, they’re custom widgets, but they get similar types of work done at each step:

• A worker writes a requirement and passes it along…
• … to an analyst who writes up the change to the system and sends it to…
• … a developer who codes the changes and it goes to…
• … a tester who validates the change and finally…
• … it gets dropped into prod, or batched up and shipped off to production.

What doesn’t happen, however, is work doesn’t come in a nice flow.  The users don’t specify one requirement to be worked on, they specify hundreds or thousands which all get batched up and given to an analyst who manipulates them all until they’re ready for the next step and so on.  And what’s happening downstream?  Well, projects are like mini factories that pop up within the company.  At least, in a matrix staffed organization like I work in they do.  A team of people is selected to become the factory for the duration of the project and then they’re disbanded and reformed into new factories and so on.  So when you put your factory together, what happens if you have to batch all your processing all the way through?  It wastes resources.  While the analyst is working, the developer and tester are doing pretty much nothing.  And the requirements guy is just sitting around, his job pretty much being over.

Now honestly, there’s no need for it to be so inefficient.  It only gets that way because of turf wars.  If I want to run a project I have to put my hands on all the resources and scream “mine! mine! no touch!” or else someone else will swipe one of the resources I’ll need eventually and then what will I do when I get to that point?  So instead, I allocate an analyst and a developer and a tester at the beginning of the project and they sit around and do pretty much nothing until the requirements guy finally gets through his work.  Where I work, we’ll allow resources to be split across projects, but the rule of thumb is 2 or 3 projects max, so each project has to pay for 1/3 to 1/2 of the resource’s time regardless of what the resource does or doesn’t do.  Thumb twiddling, surfing the Internet and making trades in your virtual baseball league? 20 hours billed to the project.

And here you’re thinking that I’d say “Agile is the answer!”  And no, I’m not going to say that.  What I will say is that specialization is the mistake.  Highly specialized people make sense when there’s a pipeline of work to be done.  In a factory, there will always be another widget coming down the line for you to manipulate.  In a small company people take on many jobs because it’d be inefficient to have some guy whose job it is to just package the releases if you only put out one or two releases a year.  And that’s what these dynamically generated factories are: itty-bitty little companies inside bigger companies.

What does make sense is to build up the capability to queue work in software development.  Requirements people should churn out small requirements documents that are easily absorbed by a programmer/analyst (just one person) who picks it up and works on it until s/he’s done and it goes off to testing to be queued up there and dumped into the nearest available release.  Yes, Agile is a bit like this, but clearly it’s achievable without Agile.  I might even argue that what Agile found was that by ridding itself of highly specialized roles (like Architects and Analysts) that you could achieve efficiency.  The rest of the “people are smart, motivated and well intentioned” philosophy can be skipped.  Mediocre, mostly lazy people can still produce results if work isn’t allowed to expand forever.  And believe me, if you’re the only guy in the factory and things aren’t coming out the other end, it’s pretty clear who’s at fault.

UPDATE:  I was looking at some manufacturing blogs re: LEAN Six Sigma recently and discovered (no suprise here) that manufacturing already uses this model.  It’s called “one piece flow.”  The inefficient version is called “mass production.”

How not to improve a process

One of the things that Six Sigma is all about is data driven decision making.  This seems like a good thing to do intuitively, but I think people skip it anyway because getting the data is difficult a lot of time.  Now, your experience probably won’t be as disastrous as the one I’m about to describe to illustrate my point, but it could get there…

Fortunately, my example has nothing to do with a real process I was working on.  I was in black belt training and we were using the Statapult to learn about various statistical techniques.  I am, by nature, super competitive so I wanted to win the competition that was set up between the various teams.  That meant putting the most balls on target and then presenting out your findings (your learning experience) to the teams at the end.

For those of you not familiar with the Statapult, I’ll provide some background.  Essentially, it’s a very small catapult that fires wiffle and pingpong balls.  It has a bunch of adjustments on it.  Here’s a picture of it:

You have many adjustments you can make to it.  There’s the tension pin, the stop pin, the pullback angle, the rubber band connection point on the arm and the cup that holds the ball on the arm.  Two things need to be tackled with the statapult experiment.

1. You have to figure out how the various adjustments affect the distance the ball flies.  After all, the goal is to launch the ball onto some target a random distance away.
2. You have to figure out how to get the variation out of the system so that you can do #1.  This is critical.  There is a lot of noise in an uncontrolled statapult.  The rubber band is strong enough to move the entire statapult when released even if a couple hefty folks are holding it down.  And then there’s other things, like the tension pin which rotates in it’s slot as you pull back the rubber band.

In fact, it’s that very tension pin that taught me more about the statapult than anything else.  Here’s the important message that I want to demonstrate to you: do not improve a process based on your assumptions because you can do bad, bad things without realizing it.

After getting a day to play with the statapult, we decided (again, being very competitive) that we needed some serious improvements to control variation.  I went to the hardware store on the way home and bought:

• 4 2 1/2 inch lag bolts
• 16 washers
• 4 wing nuts
• A roll of foam insulating tape
• 2 metal plates
• A piece of 1/4″ plywood for a base (I had this in my garage)

When we came back in, I built a base to control our statapult’s movement.  We drilled holes in the plywood and pushed the bolts (with washers to prevent rip-out) through the board.  We then placed the statapult between the bolts and used the metal plates, more washers and wing nuts to hold the statapult to the base.  I attached the insulating foam to the underside of the metal plates to a) protect the statapult from damage (they’re insanely expensive) and b) to provide shock absorption. 

Finally we looked at what else we could do to take noise out of the system.  Well surely that spinning tension pin was a bad thing.  Every time we pulled back the arm it would rotate and the rubber band was clearly slipping over it.  This had to be a bad thing, so we clamped down the tension pin so it wouldn’t rotate.

By the time we were done with this, we were learning about Design of Experiments (DOEs).  We were doing a 2k DOE, which is pretty straightforward and we expected that when we were done that we’d have the best results out of the entire class.  After all, we had all the noise under control. 

The reality was that we had one of the worst results and we didn’t understand why.  The Master Black Belt teaching the class took a look at our data and concluded it was because of the firing angles we had chosen.  For a DOE to be effective it has to have at least one continuous variable to adjust.  All the other adjustments on the statapult are discrete, so it is important to use a wide range of firing angles to get a good calculation.  [EDIT:  As an astute reader points out (thanks Kent!), you CAN do an effective DOE without having a continuous adjustment.  For clarity, the advantage of having the firing angle be a wide range of continuous adjustment is that it is the only adjustment you have to fine tune your shot.  Since we narrowed the range too much, it appeared that we wouldn't be able to cover the entire range] Because of where we were in the classroom, if we used a wide range of options on the firing angle we’d launch the ball right into the wall.  This meant we couldn’t get a measurement on the shot and therefore wouldn’t be able to compute the results.  To compensate we used a firing angle of between 135 and 140 degrees for our experiments and it seemed like that might have not been enough variety in ranges.

Our MBB was kind enough to stay with us and do the experiment over with a wider range of firing angles (and a bit more room to fire) and see what we got.  The results still sucked.  It wasn’t the firing angle, but we discovered something very interesting.  When you do a DOE, you randomize the order in which you adjust all the parameters.  You do this to avoid introducing bias into the experiment.  You also take two measurements at each set of parameters.  So, if you were using settings A, B and C in the first shot, you might do 15 more shots with various settings before coming back around and doing settings A, B and C again.  Because it was getting late our MBB said to skip that randomizing because each time you do that you have to reset the whole device and it takes a while.  So, we were doing shot 1 and shot 2 at settings A, B, and C.  And we found out something crazy, the two shots weren’t producing anywhere near the same result each time.  If you do the same settings on the statapult and you’ve gotten rid of all the noise in the system it should produce the same result (or at least darn close to it).

That’s when we had an epiphany.  Locking down the tension pin rotation was a bad thing!  Yes, indeed the rubber band did move over the rotating pin but it was a good thing, not a bad thing.  By allowing the pin to rotate it equalized the tension on the rubber band on both sides of the pin.  Without it, sometimes the rubber band would be really tight and sometimes really loose and result in unpredictable shots.

Alas, it was now quite late and our MBB was unable to stay for round number 3.  One of my teammates and I did the entire experiment again with just the two of us (it’s much easier with 4 or more people).  And finally, we got a good DOE and a regression equation with a 95.6% R-Sqr (adjusted).

In the end, we didn’t win the competition.  It turns out there’s one more noise factor in the system we didn’t count on – having everyone else in class watch you during competition makes you really nervous.  We did come in second place, primarily because all our failures let us tell one of the best learning experiences about the statapult.

Getting back to my message for you of “do not improve a process based on your assumptions because you can do bad, bad things without realizing it” you can see how making bold assumptions about the statapult process resulted in us making the system work worse than it did if we just hadn’t played with it at all.  This is what you should take away about real-life process improvement.  It is not enough to think you know what’s right for the process, if you don’t understand how the process works, then don’t play with it.  Or do play with it, but have the fortitude to admit you screwed it up and now know enough to not do that again.

Getting the noise out of measurement

I discovered that we spend too much time some days trying to figure out how to slice up the data we want to measure.  The conversations about it make my head hurt!  I dread people coming to my desk to talk about the MBF format or data collection plans.  Conversations go something like this: 

Someone else: “We’d like to measure the old requirements process versus the new requirements process.” 

Me: “Ok, well, we’ll get the cost data from our timekeeping system and … “

Someone else: “No wait, we can’t do that, the data is too noisy and we’ll never be able to tell.  I want you to get the data for only this list of projects and only if they have these exact billing codes under them and only if they were started after July and only if the moon is full and …”

Me: [douses self with gasoline a'la the poor passenger stranded next to Ted Striker in Airplane]

Why do we do this?!?  First off, who says the data is too noisy to be measured?  Yes, there is noise in the data, there likely always will be.  The noise in this example is introduced by people who don’t know how to bill their time properly, people who know how to bill their time properly but are too lazy to switch to a new billing code, people who are deliberately not billing because they’re trying to fudge the budget and the list goes on.  But all that said, across the vast multitude of projects we run, some amount of this is always going on.

Of course, and then there’s potential measurement system noise.  Were I measuring the size of pieces of paper created by a cutting machine with a ruler, I might misread the ruler or be lazy and approximate.  So in addition to the paper cutting machine not producing consistently sized pieces of paper (part variation), I might be creating noise just by examining the paper.  Fortunately, when extracting data from a time keeping system, this isn’t happening.  The data is digital and the measurement system very reproducible.  No matter how many times I pull the same data set from the timekeeping system, I’m going to get the same result.  It’s also highly repeatable.  No matter who runs the job to pull the data you’ll still get the same result.

Secondly, who says your selection criteria is making things better?  In our efforts to get the perfect slice of data out of the system we forget the purpose of measuring in the first place.  We want to know if one population is different from another and that’s it.  Now certainly, if there’s enough noise in the data you will never be able to tell if the two populations are different.  But you haven’t really figured that out have we?  We just assume the data is bad and begin to over specify the sample population.  This is just as bad if not worse.  Random sampling is key to making assumptions about the larger population.  By adding in all these conditions on how we sample the data we are taking away randomness.  All the candidates are being cleaned out of the pool by us trying to find just the right candidates to prove our point.

And going forward, when you measure your new process, you will be unable to do this kind of careful selection.  All the new process data points will continue to have noise created by normal people behavior, like being lazy about timekeeping.  And then what exactly are you comparing?  A highly sanitized sample that isn’t representative of the population vs. a representative sample.  Even if you found a statistically significant difference in the two samples it’d be suspect at best.

In software development we often measure after the fact because our “widget creation process” turns out relatively few new units each year.  In order to get the required 20-30 minimum sample size we have to go back to old projects.  Since we didn’t plan on measuring these old projects, chances are there’s noise in the data from that project that we’re measuring.  Chances also are that the noise continues in the new projects we are going to measure after the change.  Rather than arbitrarily filter the old data to get something you think is noise-free, simply compare the new process to the old process, noise and all.  As long as the noise isn’t greater than the signal, you will still be able to detect a difference if you made one.  If after looking you can’t detect it, then start talking about noise in the data.  Regardless, it’s not like you are going to be able to go back and sanitize the data anyway, so if you can’t prove a change with the noise included, you probably can’t prove a change at all.

Unintentionally fixed price software development

I’ve been working on a project at work to improve our client pricing process.  Our business partners have long had complaints about how well (or not well, as it were) that Systems does these estimates. Some time a few months back we held a 2 day offsite to get our systems folks together with our business folks and decide on a new way to do estimates to improve the results. I think the results were less than stellar and it wasn’t until late last week that I understood why. We do three pricing estimates during any project: a high-level estimate done based on “back of the napkin” requirements, a more detailed estimate once we have requirements and a final estimate once we’ve completed the design.

Everyone got together in this offsite and decided it was all about communication and that we should set up a central team to coordinate estimates and get the right people in the room so that we got a good estimate.  At the same time, I was assigned to head up the tools team work – a group of folks who would focus entirely on what the right tool (excel spreadsheet, ms project, something else…) would be the way we would do and present estimates.  While doing my research for the tools, I met with all kinds of business and systems folks to understand what outputs they needed from an estimate.  I also learned some very interesting things along the way.

For one, the business was happy with the project actuals compared to the final estimate we did.  In fact, I pulled data for some 200+ projects and found that most of our projects fell in the +/- 10% range.  That is, our actuals were almost always within 10% of our estimates.  I think I’ve written about it before, but I should note that being off by more than 10% was cause to consider a project’s status “red” – a seriously out of control project.  It was more than a bit suspicious that so few projects fell outside that range.

Of course, if our estimates were so close to our actuals all the time, what exactly was the business unhappy about with our estimates?  It turns out they are unhappy with the quality of our early estimates.  It seems that the business wants to make major yearly decisions about which projects to run and if they ask us for an early estimate then they don’t expect subsequent estimates to double, triple or quadruple the cost.  In fact, more than one person I talked to said early estimates had to be 80% accurate.  One of these very respectable business folks used to work for a fixed price company so he knew that kind of estimate quality was possible.

We continued our research with this new information in hand and decided that we needed to work specifically on getting a high-quality early price.  In the absence of requirements, we believed that a parametric estimating model would be the best way to get a good early estimate.  Little did we know how easy it was going to be…

What we did was we built up a set of about 45 questions, mostly yes/no questions, that a business person could answer about a project.  Imagine a checklist that has items like “is product X being modified?”,  ”how many new web pages are being created?” and so on.

Then we dug up a sample of old projects and filled out the questionnaires for each project. Finally, we added the final budget to each project’s data.  From that we were going to build a regression equation.  A funny thing happened while building the regression equation.  I use Minitab to do my statistical work and I was having trouble getting it to build a regression equation using all 45 predictors.  In a complete fluke, I selected I small subset of predictors and tried to get an equation just to figure out why Minitab was spitting out an error.  The craziest thing happened!

Using just these 6 predictors I was about to predict the final cost of a project with > 95% accuracy! What was even crazier were the predictors!  They weren’t something complicated like “how many web pages are being modified” or “what is the approximate transaction volume.” Rather, I could get this super accuracy by answering these 6 questions:

1. Is Product A being modified?
2. Is Product B being modified?
3. Is Product C being modified?
4. Is Product D being modified?
5. Is Product E being modified?
6. Is Product F being modified?

Where I work, we offer a suite of products to our clients, so the products have inter-relationships.  Any given project/enhancement we do may affect one or more of the products we offer.  It’s quite common for an enhancement to affect a few of the products at once. Despite all the variety of enhancements we do, all the lines of code (at least one product has 750,000+ lines of code), all the different people and features pretty much every project is one price!

All this effort was going into getting the right teams of folks together to do an estimate and the reality is that I can tell you what the end result is going to be without asking a single person. How did we end up here?  How was it that no matter how complex or how easy a change was it cost the same thing?  When did we become a fixed price shop?

I have no proof, but I believe years of trying to improve our estimate quality got us here. Each team in their own way attempted to figure out how to make estimates quick, easy and accurate.  One of the teams I used to work for came up with staffing models.  The idea was that they estimated just the development effort and then extrapolated the rest of the project cost based upon that number.  So, for example, if you had between 1-2.4 person months (PM) of development, you chose staffing model #1.  Each model told you how much systems analyst, how much project manager, how much tech lead and so on that you needed for a coding effort of that size.  Simply take the base model, tack on the development effort, tweak slightly and ta-da… you have an estimate.  It turns out that most of the projects they do all fall into the same range (2.5 – 5PM) and so the same staffing model gets used over and over and over again.  Essentially, work is expanding to fill the time allotted.  The reason we’re a fixed price shop is that in order to never run over budget we’ve simply grown our estimates to the point where even if everything goes horribly wrong we can still get the project done.  Does it seem right to you that 1PM of coding would require the same percentage of effort from an analyst as 2.5PM of coding?  Is that even a reasonable differentiator?  Sometimes it’s just a lot of coding and requires almost no analysis (like repetitive code changes) and sometimes it’s not much coding at all but needs a lot of research.

What is it the business is really saying when they say they don’t like our estimates?  It isn’t that the estimate isn’t reliable, it is that we’re simply outrageously expensive.  At least we now know that there’s lots of room for improvement.