Tag: analysis

Abraham Wald

As Allied Planes in World War II were being shot down at a devastating rate,  the US Air Force wanted to reinforce the planes with armor. However, every pound of extra armor took away the attack capacity of the planes. The person tasked with solving the problem was Abraham Wald, a Romanian Jew who’d fled the Nazis to become a Professor at Columbia University. He was recruited in the US Army efforts in World War II as part of the Statistical Research Group.

The Air Force supplied Wald with the data available – number of bullet holes grouped by their location on all the planes that returned to base. Most bullets were around the tail gunner and the wings. So, the army suggested reinforcing these regions with armor.

In what has become a legendary piece of analysis, Wald disagreed. He noted that the fact that the planes had survived despite these injuries suggested these areas needed no reinforcement. All the other planes had likely been struck elsewhere – the engine and the cockpit for example. Those were the areas that needed to be armored.

Wald gave us survivorship bias. In simpler words, he reminded us to look for insight by being very mindful of the data that isn’t in the room. And, as I think of a recent error that rose from jumping to a conclusion with available data, I realize I must spend more time channeling Abraham Wald.

Selection bias and winners – The 200 words project

Here’s this week’s 200 word idea thanks to TimHarford.com, Lifehacker.com and Professors David Dranove and Brett Saraniti at Kellogg.

In 1943, the American statistician Abraham Wald was asked to advise the US air force on how to reinforce their planes. Only a limited weight of armor plating was feasible, and the proposal on the table was to reinforce the wings, the center of the fuselage, and the tail. Why? Because bombers were returning from missions riddled with bullet holes in those areas.

Wald explained that this would be a mistake. What the air force had discovered was that when planes were hit in the wings, tail or central fuselage, they made it home. Where, asked Wald, were the planes that had been hit in other areas?They never returned. Wald suggested reinforcing the planes wherever the surviving planes had been unscathed instead.

As blogger Tim Harford points out, this makes for a classic example of selection bias and also a great life lesson. It is natural to look at successes. But, if we don’t examine our failures, we may end up putting our time, money, attention or even armor plating in entirely the wrong place.

Abraham Wald planesSource and thanks to: Digitalroam.typedpad.com

‘The data that isn’t present may tell as important a story as the data that is.’

The anatomy of a mistake

Last weekend, I decided to add 75 new friends I’d gotten to know over a week to the 200 words project with an option to opt-out.

In hindsight, that was a mistake. I judge it as one because it has given me an uneasy feeling in the pit of my stomach ever since. What I should have done is made it an opt-in. That it seems so obvious now makes it even more exasperating.

Rather than kick myself for the result of the decision, it makes for an interesting case for analysis. What led to the mistake? I can think of 2 reasons –

1. Sleep deprivation and illness. Low sleep, not feeling great => limited willpower. Beware of decisions you make when you are low on willpower.
2. No clear decision process. Generally, behind every bad decision is the gap left by the absence of a process. In this case, I have always added folk to the 200 words project via a mix of opt-ins and opt-outs depending on how well I know the person. It is my way of staying in touch with people and it turns out to be very nice to be in touch with folk from 6 years ago thanks to a consistent weekly email. Additionally, there is a lot of effort that goes into making it useful – so I don’t see it as a spammy share (talk about bias…). And, therein lies the problem with an undefined decision making process that, in reality, is very intricate. I’m better off simplifying it by just saying opt-in only. And that’s what I’ll do from here on in.

A few other learnings I have taken away –
1. Behind every mistake is a bad decision. Don’t kick yourself for the mistake, fix the decision. Every mistake, thus, is just a learning opportunity. And yes, this is hard to implement. I seem to get there thanks to this blog but it only happens after kicking myself a few times.
2. You amplify the bad reactions and forget the good.  This is hard to correct as it is human nature to focus on a negative reaction. It is a good reminder, however.
3. No one cares about you and your mistakes as much as you do. Again, a reminder.
4. Bad feelings in the stomach are a sign that some insecurity has been pricked. This one is an insecurity of feeling unfairly judged.
5. Keep flexing your “rejected” muscle. Try, fail and get rejected, try again. Trying again matters a lot. Your “rejected” muscles need to be working well so you don’t get too upset with yourself after you make a mistake.

A friend wondered aloud yesterday as to how I managed to think of a learning every day. I pointed to the more-than-average number of mistakes I seem to make. I hope he’s reading this one. :-)

Finally, to add a dash of perspective, I did remind myself that these aren’t mistakes my grandchildren will ever even know of. That’s a good reminder, too (since we are in reminder zone).

We live and we learn.