We’ve decided to do it the wasteful way

There are seven (or eight depending) forms of waste that LEAN recognizes:

1. Inventory
2. Overproduction
3. Over processing
4. Motion
5. Transportation
6. Defects
7. Waiting
8. (optional) Intellectual

Today, I learned from a friend of mine that we were going to deliberately introduce #6 as a valid way to behave.  Test Driven Development (TDD) is a development philosophy where you write a test case to fail, then write code until it works, and then execute the test case again to show that it now works.

Practically speaking, I get that we will probably never be able to eliminate all appraisal activities.  After all, unlike a faulty bolt which might cost us a nickle to replace, a faulty line of code could in theory cost us millions.  We appraise because it’s the only way we know how to check for correctness.  However, that doesn’t mean we have to accept appraisal as being necessary.  Far from it!

We know from the great research of Capers Jones and Barry Boehm that appraisal activities, especially testing are only about 40% effective per test type.  By comparison, visual inspection (aka peer review) can be 60-80% effective.  So why would you build a process around an ineffective practice?  And why, dear lord, why, would you run a test case you KNOW is going to fail.  You wrote it because it would fail.  If you’re writing test cases you don’t know will fail, are you paying any attention to what your system can and cannot do?!?!?

Now, one might argue that TDD is simply a manifestation of really solid use cases.  If we wrote adequately detailed use cases, they could in theory serve as the tests to appraise the system.  Fair enough, but that assumes that we are willing to exchange some level of intrinsic knowledge in our developers for complete and perfect specifications from our business.  The business will now tell us exactly how it has to work.  We can also reasonably assume that for TDD to be effective, you can’t just write the positive test case and show that it works.  You must also write all the negative cases, the exception cases where the system should deal with user error.  After all, those things must be specified to fail as well so that you can write code so they won’t fail.

Alas, this methodology is faulty to begin with.  There are known better ways than to appraise than through testing.  Why choose a particularly wasteful way to get your work done?  And if it’s just good use cases, then just call it good use cases not “test driven development.”

We were just waiting for you to quit…

Talk about doing an undervalued job.  A fellow that I worked with for a year or two just left the company for greener pastures.  Well, I assume that’s why he left, I never really got into it with him.  And I’m not particularly sure what “greener” really means.  I’ve come to the conclusion that all companies are dysfunctional in some ways, and that you just have to choose the dysfunction that you like the most (or hate the least, I guess).

Anyway, his office chair wasn’t even cold when an email crossed my inbox.  “I understand you are involved in some new metric work.  Dave [the former employee] used to produce a bunch of metrics.  Can we stop doing that?”

Ah, the truth comes out.  The manager in question had a full time employee.  He liked his empire to be a certain size (which is to say, not smaller than it currently was), so the manager justifies the employee’s existance as being critical to (in this case) our organizational metrics.  But the minute that employee leaves the company… “do we need to do this anymore?”

What does this say about us as people?  Are we afraid to confront a question as basic as “is this worth doing?”  Are you saying that in all the time that Dave was employed that we didn’t think he could do anything else, so as long as we had the same thing for him to do that would be ok?

If the only time you consider whether a task you are doing is still worth doing is when someone quits (or gets laid off or shuffles off this mortal coil)  then you’ve got some serious problems.  If we aren’t taking an approach that is constantly questioning whether or not what we are doing is bringing value to the customer, then WHAT ARE WE DOING!?!?

We put our blinders on, and we march.  Like good little lemmings off whatever cliff is coming up.  Insanity?  Didn’t anyone bother to ask the customers along the way if Dave’s metrics work was adding value to the company?  Apparently not.  Did we not want to hurt Dave’s feelings?  (Not that I think people shouldn’t be respected)  But if what Dave was doing wasn’t worth it, and yet you found Dave himself to be a good employee, I am certain we could have found something that Dave could do that would be worthwhile.

Don’t wait until someone quits to evaluate what you are or aren’t doing.  This time, it was just Dave (no offense Dave, if you read this), but sometimes it’ll be your biggest customer who up and quits on you instead.

Now is not the time to be sentimental

I was reading an article on CNN about some guy who received a postcard in the mail.  What was interesting about the postcard was that it was quite old.  The postal service had lost it somewhere and it was found.  They felt obligated to deliver to the address it was destined for and did so.

These stories are always fun for CNN because it’s odd that a postcard would show up that many years later.  And isn’t it a neat piece of history that we just discovered?  We like to look into the past and imagine what the sender and receiver of that postcard might have been like.  There’s an entire site (Found Magazine) dedicated to this ‘art’ of discovering stuff.

But the part of the article that inspired me was the short intro:

“When an Ohio man recently received a postcard sent 47 years earlier, he not only made a connection to Ohio’s history, but put a fresh spotlight on what a U.S. Postal Service spokesman calls “a lost art” that has fallen victim to e-mails and tweets”

‘…fallen victim to e-mails and tweets’?!?!?  OK, it’s a lost art, I’m willing to grant you that.  But a victim?

A victim of change, that is!  And what’s so wrong with that?  Things change all the time.  What is so wrong with this is the idea that clinging to an old thing when a new thing has been discovered that works better.  Email is faster, cheaper, very reliable (it doesn’t show up 40 years later) and conveys the same information that a letter did.  Even better, instead of a generic postcard in the mail you get a picture of the place a person is, right now, taken by their cell phone!  You can share in the experience to some degree.

And that’s the thing about change – change happens because people discover better ways of doing things.  Yet, rather than celebrate the change we mourn the loss of the old way?!?!  How stupid is that?

Change is going to happen.  Fighting it doesn’t make sense.  Regardless of how you loved the old way, were invested in the old way, are used to the old way.  Those that do not adapt die.

A control chart for non-normal untransformable data

Why in God’s name would I propose such a thing as the title?  Well, I invented it, sort of…

See, we had a very specific problem at work.  We wanted to track the quality of new code going through our test process and the existing types of control charts all failed to meet our needs.

Issue 1, how do we measure opportunity?  From a prior post, if you’re a regular reader, you’d know this story… we chose to measure opportunity as test cases.  Our score for a project is weighted defects per test case.  Data is relatively sparse, so we probably have to go with something like an individual observations chart.

Our measurement creates problems for all the control charts:

Option 1, a P or NP chart.  Won’t work because they assume that no more than 100% of the units can be defective.  With weighted defects per test case, if you have 1 high severity defect and 1 test case you’d end up with 9 defective units ((1 high defect * 9) + (0 medium defects * 5) + (0 low defects) / test cases.) per opportunity.

Option 2, a U or C chart.  C charts don’t make sense because we don’t have equal unit sizes per sample.  U charts don’t work because they assume a Poisson distribution, which we don’t have.  As a matter of fact, scores are distributed more like an exponential curve.  We have many cases where there are 0 defects with a long, long right tail.

Option 3, an I Chart.  They assume normally distributed data, but you can’t log transform data when it contains 0’s.  Other calculations using Box Cox failed to induce normality.  Ultimately, we did settle on adding a lambda of .5 (equivalent of a square root) transform because it helped with some of our product lines but not all of them.  All that said, I’m aware that empirically I charts actually still function well even in situations of non-normality, though they generate more false positives than a normally distributed population would.

And as you can see, we’re basically out of luck.  I went looking for control charts for non-normal data.  I found a proposal for Median charts, which was ok if you had subgroups, which I don’t.  There were some others, like Bootstrap, but any information I could find had to be purchased, so that wasn’t helpful to me.

Anyway, as I was sitting there thinking about the issue, John Tukey came to mind.  It was actually an unrelated conversation I was having with my father about box plots.  I had noticed in Wikipedia that the inventor of box plots was John Tukey and that Mr. Tukey happened to work for Bell Labs.  I recalled a story my father once told me about going to visit this great oracle (I’m exaggerating) regarding a statistical question, so I phone up my Dad to find out of the person he visited was the mythical John Tukey.  Turns out it was, which was cool.

Anyway, I was relating my whole “what’s going on at work?” story to my father, and talking about our control charts, etc. when suddenly it hit me.  Tukey… box plots … control charts … control charts based on box plot rules!!!

Box plots show the distribution of a sample including outliers.  Tukey’s rules for box plots were as follows:

1. Find the median of the data.
2. Find the first and third quartiles of the data.
3. Calculate the interquartile range (Q3 – Q1).
4. The whiskers of the box plot extend as follows:
   1. Q3 + (1.5 x IQ Range)
   2. Q1 – (1.5 X IQ Range)
   3. If the length of the whisker is longer than any observed data, clip the whisker to the observed data point.
5. All data points that exceed the whiskers are considered outliers.

Outliers!  That’s what we really want to know about, right?  Outliers are interesting.  Outliers deserve further study.

We can translate these into rules for constructing a control chart:

1. The center line is the median.
2. The UCL becomes Q3 + (1.5 X IQ Range).  We don’t clip it to the sampled data because we’re concerned about the theoretical limit, not the limit observed in the data we have.
3. The LCL becomes Q1 – (1.5 X IQ Range).  Again, we don’t clip it to the sampled data, but do clip it to zero if it would fall below zero.  This assumes that values less than zero aren’t possible in your data.  If they are, don’t do this.
4. In all other ways, it works just like an Individuals control chart.  Plot your observations along the X axis and away you go…

So, how do we test this?  One way would be to generate a series of random curves and see how it performs regarding false positives.  Since all our data would be known to come from one distribution, we wouldn’t be looking at true outliers that were the result of special cause variation.  That would give us some clue if the rules were overly sensitive.

I created 10,000 data point distributions for Lognormal, Exponential, Gamma, Least Extreme Value, Chi Square, F, and Poisson.  Admittedly, Poisson is probably silly since we have C and U charts for that kind of distribution.

Then, I tested each distribution against 3 possible tests:

1. Violating an upper or lower control limit
2. Having 8 points all greater or less than the median
3. Having 6 points in ascending or descending order

The results were quite positive.  For test one, the range of false positive was from 7.83% at worst (Lognormal distributions) to 1.31% at best (Poisson distributions).  Test 2 rates of false positives hovered around 3% while test 3 was between .2% and .4%.  All in all, not to shabby for something I dreamed up.  Anyone want to do the real math?  I’m not capable and have to stick to Monte Carlo simulations to test my work.

Anyway, there it is, a proposal for control charts for continuous data that cannot be adequately transformed by Box-Cox.  Note:  If you do use this, I’d LOVE (but of course can’t obligate you to) reference my blog as where you learned of the idea!  Thanks for reading!

Today is the day I boil the ocean

Boiling the ocean.  A platitude about not, um, biting off more than you can chew… one good metaphor deserves another, right?

The oft repeated phrase around the office “don’t boil the ocean” is a warning to all those who would take on more than feasible in their improvement projects.  It’s dumb.  I mean, it’s a silly idea, after all, what would I do with a trillion tons of boiled fish?  I hardly have a fridge big enough for it all.

But more importantly, it is silly because of what people think the converse of not boiling the ocean means.  For some reason if you are not going to boil the ocean, instead you barely boil a pot of water.  After all, we wouldn’t want to get out of hand, and a large pot of salted water could easily be mistaken for the ocean right?

Ridiculousness aside, we constantly make the mistake of over focusing our improvements on such a narrow area they they hardly achieve any benefit at all.  While it might be true that boiling the ocean is a daunting if not impossible task, boiling a pot of water is so simple it has hardly any benefit to the company at all.

And so it goes with our process improvements.  If I analyze a small problem in a focused area, it takes me a couple weeks and I achieve a modest, perhaps unmeasurable benefit.  The problem is, we’re salaried employees, and almost all of our costs are labor costs, so if I don’t make an improvement that is at least as large as the value of one person’s salary (ie, thus we can eliminate said resources) then it isn’t an improvement we can realize.  I can’t cut back hours; we still pay everyone the same.  We’d have to wait around until enough of these little things add up.  And that just doesn’t make sense.

I know, I know, nobody wants to be the guy whose work is responsible for laying people off.  I admire companies that have no-layoff policies to process improvement.  It’d be great if we made cars or televisions or something where we could achieve savings in material costs.  But we have few material costs, so there is little else to eliminate but people.

My own torment about people losing their jobs aside, any project we take on that is smaller than the cost of an employee isn’t going to lead to much benefit.  Indeed, my work must additionally pay for myself as well.  At least the equivalent benefit of two salaries must be saved or else it’s a wash.

So, if you don’t want to boil the ocean, that’s fine, but at least make an attempt at boiling the contents of a large swimming pool, or even a modestly sized lake.  It just isn’t going to do you any good to boil just enough water for a cup of tea for that employee you still have to, well, employ.

It’s a silly metaphor, I realize.

Another screwed up queue

Well, I’ve talked about Quizno’s and I’ve talked about Boston Market’s queuing failures.  Today on the block is Five Guys Burgers.  It’s a new place that just opened up, but I have to say I like their idea.  Their entire menu consists basically of hamburgers, hot dogs, french fries and soda.  There are various toppings for the burgers and dogs, but their menu is pretty basic.   Expensive, but basic.  I think my lunch cost me $9 and change.

Based on the crowd size I observed, I’d say they were doing pretty darn well too.  I entered the store about 11:45, just before the lunch rush.  It was busy, but not packed and the queue at the registers was relatively small.

They’ve got a single queue at the registers with two registers to serve people in a FIFO style.  I think that works pretty well.  Once you’ve ordered, a little slip prints out which is put down on the burger assembly line and follows your burger along with any instructions.  They’ve got about 3 guys working that line.

Right behind them is the burger/dog cooking folks.  There’s 3 people there as well, busily preparing burgers.  What is interesting to me is that the cashiers, who face the entry are responsible for a form of kanban.  They count the groups of people coming in the door and call out the potential demand before anyone orders anything.  In that way, there are enough burgers on the grill to meet the demand without having excess inventory of cooked burgers.  I was impressed with this.  Of course, in a place like McDonalds where the menu selection is much more vast, that probably wouldn’t work.

I do ponder how they know that all the folks coming in the door will order a burger, but it’s probably a reasonable assumption.  Though they offer hot dogs, I didn’t actually see any being consumed except for the one kid with his family.

Just to the right of the burger cooking folks you have a guy who’s job it appears to be to load the deep fryer with fries.  And next to him a guy who unloads the cooked fries, moves them to a holding area and waits for orders to come down the line that need fries.  This part of their queue is totally wrong.

When I ordered I was told that I was number 19.  Moments later, when I heard the guy at the end of the assembly line bellow out “10!”  I knew I was going to get some good information on how the line was flowing.  If it was working perfectly, I’d expect it to be a FIFO (first in first out) queue.  That’s how they’re set up, anyway, with a line of assemblers each doing their job, passing the work down the line, etc.

The thing was, after sitting and waiting a few minutes I started hearing numbers I didn’t expect.  “25!”  “28!”  “27!”  “32!”  Where was my order!?!?  I’m number 19, and they’re calling out numbers in the upper twenties/lower thirties.  My order was simple too – one burger with ketchup and pickles, a regular fries and a bottle of water (which was provided by the cashier when I ordered).  Eventually my number was called.

As I sat there eating my lunch (it was pretty good, actually), I listened to the conversation of the guys sitting next to me.  It turns out that they had each ordered in turn and been assigned numbers 38, 39, and 40.  But, like my experience, 38’s order didn’t come out first.  40’s did.  And that’s when one of them said “oh, you didn’t get fries, did you?”

I looked back over at the assembly line.  It’s a great place to visit for a look at a process because they do everything out in the open.  There’s no back room at all.  Remember the fry guy?  One thing I noticed about him was that there was one of him and 3, maybe 4, burger assemblers.  While burger assembly is somewhat more complicated than packaging fries, the workload distribution may have been wrong.

On the burger line, there was a guy who put down a slip of foil, put down the open bun, added ketchup and mustard as necessary.  The next guy, it appears to me added the “more complicated” condiments like bacon, lettuce and tomato.  I guess you need special training for that…

The third guy added the burger, which was pretty much provided as requested by the burger cookers.  He then wrapped the burger and pushed it to a fourth guy.  A guy, who’s job, so far as I can tell, was to affix yet another order slip to the outside of the paper bag and bag the burger.  The bagged burger was then transferred to another table, in a fairly chaotic manner, to await either delivery to the customer or the addition of fries.

And that’s where all hell broke loose.  See, the fry guy really does have a pretty simple job.  Take fries from the hopper and load them into a container and then put them into the bag.

Problem 1, the fry guy is stuffed into a corner.  He doesn’t have adequate space to organize his work.  Compared to the nice, pull style work going on the assembly line, fry guy had completed bags awaiting fries pushed on him.

Problem 2, because of #1, the fry guy can’t take a bag, process it (add fries if necessary) and move it down the line.  He’s just got a pile of bags, so he’s likely to work on them out of order.  There really wasn’t a logical way to organize the bags so that fry guy could fill them and move then along.  Instead, once filled they piled up waiting for another guy to take them from him.  And because fry guy was stuffed into a corner, he was constantly crossing over with the guy who took the completed orders.  They got in each other’s way.

Problem 3, the containers they have for fries are styrofoam cups!  This poses two issues.  One, super un-eco-friendly!  Not really an issue for the assembly line, but I thought I’d point it out.  Two, they’re too small for the typical order of fries.  When I finally got my bag, other than holding the grease in so that it didn’t soak through my paper bag, the styrofoam cup did nothing.  The fries overflowed the cup, were in the bottom of the bag, were generally everywhere but in the cup.

That problem stems from the cup being too small.  Since the fry guy can’t pre-load the cup and then drop it into the bag, he has to reach into the bag, make space for his cup to be placed down in there and then pour the fries into the cup while in the bag to minimize spillage.  It’s plain silly.  He’s got all this NVA unnecessary motion.

Problem 4, also because the cup is too small, he can’t prepare fries into cups in anticipations of a lull.  Now, I will say that they have very fresh fries so he isn’t going to have much inventory anyway, but if he could get ahead by even one or two cups of fries, I think he’d look a lot less harried.

Once fry guy eventually does his job, there’s YET another person whose job it is to take the paper bag from fry guy, yell out the order number and give it to the customer who walks up.  If fry guy were part of the assembly line, he probably could do that job.

Fry guy is like an afterthought for this place.  They have a really nice assembly line from the register through burger completion, but it has to take a detour if fries are involved.  Guess what, lots of people want fries.  Why isn’t fry guy in the assembly line – just the next guy to pass the bag as it moves towards the goal?

For now, the simple operation allows it work, but it clearly could work better.  My suggestion, if you have a Five Guys around you is to take a visit and see how it works.  Because of the open environment you can see the entire process really easily and learn a lot about how to make common sense improvements.

Tension Metric

I’m coining a new phrase today - ”Tension Metric.”  You heard it here first.  Why would I do such a thing, you ask?  Because I think it represents a concept that we overlook as a tool in making process improvements.

Now, my from my research (by which I mean I typed “tension metric” with the quotes into Google and got nothing back), we don’t have this concept in our lexicon.  I’ve heard “balanced scorecard”, which suggests you should have metrics for all aspects of your customers needs, but mine has a little twist.

A tension metric is a measurement taken in the same units as another measure to minimize the gaming of your measurement system.  For example, let’s say I want to measure defect containment rate (DCR).

Typically, this is measured as the number of defects found in test divided by the number of defects found in test and production combined.  DCR is a great measurement and one you want to drive towards.  However, in order to achieve a high defect containment rate, one possible side effect is that your quality assurance folks will over-test.  After all, though you don’t want defects to make it to the field, you also don’t want to pay a king’s ransom to have that happen.  If you’re going to have testing, and mind you, I believe testing is an NVA activity, then you want it to be effective and low cost.

So, what’s the “tension metric” for this?  I believe it’d be something like effort per contained defect.  In this manner, while trying to drive up DCR, you must also be mindful of how much effort you are exerting to contain a defect.  Notice I didn’t have “effort per test case” because that means you could write many more small test cases and drive that number down independent of driving up DCR.  Because both include the same unit measure (defect contained), we create a tension between the two. 

In theory, you’d have to solve the paradox – become more efficient and more effective – to move both metrics in a positive direction.  Otherwise, if the metrics were independent of one another, they could appear to both move in a positive direction without actually getting the result you want.

Anyway, it’s a thought, so I’m throwing it out there and laying claim to the idea – “tension metric.”

Simpler can be better

We over complicate.  Almost everything.  We have advanced strategies even when we haven’t been particularly good at the basics.

Take testing for example.  We’re trying to move from being black box testers to doing risk based testing.  We’re not good at the basics of black box testing.  Far too much gets by us when we attempt to cover all the functionality.  What is going to happen when we deliberately skip some testing because we don’t think it is “risky”?  We haven’t got a clue what risky means; we don’t even know what parts of the system are error prone.

But I didn’t learn an important lesson about simpler being better from work today.  After all, it’s Sunday and I’m not at work.  But more importantly, I learned simpler is better from playing a video game.

Our friends have a Nintendo Wii.  We also have one, though we don’t own many games for it.  Our daughter never plays the Wii at home.  And furthermore, she’s just under 3 years old.  Her skill with a Wii controller is, from what we can tell, fairly limited.  But when she sees it being played at our friends’ house, she really wants to try it.

And she asks very politely to play.  In this case our friends’ son was playing Wii Playground.  It’s a series of mini-games like dodgeball, tetherball, paper airplanes, and slot cars.  My daughter has no capabilities to do anything advanced with a Wii Remote.  She can wave it about and press a button and hold it.  She has no idea about pressing a button several times, or pressing the B button.  All she can do is press the A button, and wave it about.

So our friends’ son set our daughter up with slot cars.  There are all kinds of cool things you can do with slot cars.  You can press “A” twice to make the car turbo boost, you can move the remote left and right to switch lanes, you can press “B” to activate special powers.  And doing all this, you can hop between lanes, catching speed boosts on the track and blasting in front of your opponents.

Now my little one doesn’t know about winning or losing, she just likes to try.  I’m happy for her to do that.  After all, she’s not even 3.  Anyway, once she had the remote in hand, the race started and she held down the “A” button to make the car go.

Sure enough, the car took off down the track.  Our little one didn’t move the car between lanes, she didn’t use the turbo boost, she didn’t activate any special powers.  She didn’t ram her opponents off the track.  Despite all the capabilities that this game had, she used none of them.  She just put the pedal to the metal (so to speak) by holding onto the “A” button.

She won!  She won 3 races in a row… against computer opponents, who have no idea that it’s a 2 year old playing against them!  Computer opponents don’t take it easy on you.  I’ve played the slot cars game – I have LOST at the game.  I tried the advanced strategies.  I tried to jockey for the best slot to be in, use the speed boosts, knock my opponents off the track.

Simpler was better.  My daughter, doing nothing but holding “A” beat the computer consistently.  We tend to over think, to try and use advanced techniques to get a huge edge.  We don’t need those things!  Sure, my daughter didn’t beat the opponents by a hundred car lengths, or even ten.  She just beat them.

Think simple.  Try basic.  Try boring.  Try consistent, but unexciting.  The results might surprise you.  They sure surprised the heck out of me.

Single data-point testimonials

The justification for using a Knowledge Management solution went something like this:  there was a oil company who had a knowledge management community for their oil rigs.  One day an oil rig went down, and when an oil rig isn’t running it is very expensive.  The foreman on rig went out to the community looking for advice on how to get the rig back up, and a problem that would have taken weeks for him to figure out had been experienced by someone else.  They got the rig back up in just a few days instead!  KM is a huge success and everyone should be doing it!

::cough, cough:: bull#$%^!  The testimonial, the worst marketing garbage ever used to justify a business decision.  Now granted, perhaps the several weeks savings from that one oil rig paid for the price of having a KM solution many times over.  The stakes were very high and the cost of KM very low.  But one chance event does not a solution make.

You’d have to be able to show a pattern of improved results with and without knowledge management solutions.  And we cannot explore the alternate reality where this company didn’t have a KM solution in place and an oil rig went down with the same issue.  For all we know, the foreman would have called the most knowledgeable guy around and gotten the same answer regardless of whether there was a formal knowledge management system in place.

A single-data point testamonial is worthless.  One cannot attribute a chance event to something else without being able to show a pattern of causation.  And yet, companies come in all the time to sell us stuff and they always have that great story about how their product saved the life of 101 cute little dalmations, or what ever other compelling story might make you buy their product.

Now, I’d be OK if they had a testimonial which showed a pattern of changed behavior, but that story is boring.  The statistics are boring.  The 2 sample T test that goes along with it is boring.  The intervention analysis that goes along with it is boring.  Boring, boring, boring.

If you’re not a geek like me, you wouldn’t find any of that analysis interesting at all.  The treatment group, the control group… dull, dull, dull.  Despite how critically important all that analysis really is to making a decision, you will probably decide to buy a product, or at least look more closely, based on a great story that someone tells you.

What’s the latest diet fad … Hoodia Rush (it contains Acai… I don’t know what that is, I just type “diet pill” into google and clicked the first link).  “Hoodia Rush worked for me!”  OK, you took Hoodia Rush and you also dieted and exercised… um, what leads you to believe it was the pill?  Two other possible causes were introduced as well – diet AND exercise.  And yet, here’s a smiling handsome man or hot woman on TV telling you, with a smile, that it works.  It must work, right? 

By the way, in case you didn’t know, a friend of mine (I’m sorry to admit this) actually signed up for one of some company’s before and after picture campaign.  You know, the fat, flabby frowny picture on the left, the trim, muscled smiling picture on the right… anyway, the way they do that is they get people who are fit, take the picture for the “after” shot, then the person eats like a pig and lets themselves go to hell and THEN they take the “before” shot.  It’s much easier to do it backwards.  He’s still out of shape, by the way.  In the long run, while he got paid for the gig, I think it might have done longer term damage.

Resist the urge!  It isn’t about a single great story even if the story is true.  Which is to say – they did X and then Y happened.  Notice I didn’t say they did X which CAUSED Y to happen.  It isn’t about false causation.  It’s about patterns and changes in those patterns as the result of things we do.  No matter how many puppies lives were saved in this tear jerking story, it does not make a valid case to purchase new technology or implement a new process.

Don’t be a sucker!  If the testimonial is a single-data point story, you can almost bet that the company doesn’t have a solid, yet boring, story to tell based on sound research.  They have marketing.