Laugh when you can,
Apologize when you should,
And let go of what you can't change.

January 03, 2008

The Reality of Statistics

As an industrial engineer, and somewhat of a statistician, I have to comment on these statistics I continue to hear during the presidential debates. I am not going to speak politics here, other than to name a statistic mentioned. On Meet the Press, Huckabee stated that there has been a lower percentage of deaths of our troops since the surge and this indicates a move towards victory. I don't recall the exact numbers, but for the purposes here, it doesn't matter. Let's play with numbers for a moment; I'm going to use small numbers for the sake of simplicity.

If there are 10 troops and one is killed, that means 10% of troops are killed.
If there are now 100 troops (surge) and 5 are killed, that means 5% of troops are killed.

In this example, a surge causes a lower percentage of troops to be killed and yet more of our troops are dying. I take issue with this being considered a metric for victory. It does not seem to be an indicator of much of anything and may in fact skew the data to look like a victory when in fact it is an increased failure.

In addition, delving a little deeper into this metric reveals that the metric itself cannot measure victory. The implication of Huckabee's statement is that a lower percentage of troops killed equals victory. Under this idea, by removing all troops and thereby gaining a 0% kill rate, complete victory would be achieved. By this metric, a surge should never have been considered. So again, the metric appears to measure little, but instead just be another number to be thrown at the public so they will believe something positive is being done.

What I would give to work in the government and establish metrics! I have found in my short career in the metrics field that the majority of people have no clue what they're doing when it comes to metrics. The thing about statistics is that you can interpret them to support almost any argument (with the same data). To have good metrics that measure real issues and make real improvements, you need an expert in the field; you also need someone with integrity that will report the data and metrics for what they really are.

My point? Beware this election season (and everywhere else in your life) as people throw statistics around like they are the selling points to their argument. Things are not always as they appear.

1 comment:

Anonymous said...

This is why I love you. Not the only reason, of course, but it's a big one. :)