Tag: data

Directional answers and precise answers

I use simple a rule of thumb for the difference between attempting to convert a directional answer into a precise answer – 10x+ time investment.

Attempting to find a reasonably good pair of headphones might just take you 10 minutes on Amazon. However, if you’re looking for the best headphones you can find for your budget, you could easily spend 100+ minutes searching.

Similarly, if you want to predict the effect of your strategy on key metrics, you might be able to get to a directional range in 2 hours. But, if you want to isolate and understand every variable and its interactions, you’ll want to budget at least 20+ hours of work.

This is analogous to being a satisficer versus a maximizer. Attempting to maximize everything you do by getting precise answers is very expensive. Thus, the implication in conducting analysis as in living life is similar – assume you need directional answers unless it is clear that precise answers are the way to go.

Every once a while, you’ll realize that you’d have been better off finding that precise answer. But, thanks to this strategy, you’ll have enough buffer time to course correct. :-)

Abraham Wald

As Allied Planes in World War II were being shot down at a devastating rate,  the US Air Force wanted to reinforce the planes with armor. However, every pound of extra armor took away the attack capacity of the planes. The person tasked with solving the problem was Abraham Wald, a Romanian Jew who’d fled the Nazis to become a Professor at Columbia University. He was recruited in the US Army efforts in World War II as part of the Statistical Research Group.

The Air Force supplied Wald with the data available – number of bullet holes grouped by their location on all the planes that returned to base. Most bullets were around the tail gunner and the wings. So, the army suggested reinforcing these regions with armor.

In what has become a legendary piece of analysis, Wald disagreed. He noted that the fact that the planes had survived despite these injuries suggested these areas needed no reinforcement. All the other planes had likely been struck elsewhere – the engine and the cockpit for example. Those were the areas that needed to be armored.

Wald gave us survivorship bias. In simpler words, he reminded us to look for insight by being very mindful of the data that isn’t in the room. And, as I think of a recent error that rose from jumping to a conclusion with available data, I realize I must spend more time channeling Abraham Wald.

In data we (don’t) trust

There’s a growing legion of companies and product teams that aspire to call themselves “data driven.” When they make decisions, they tell tales of how Google tested 40 shades of blue and eliminated the need for intuition and gut-based decision making.

But, as data might suggest, extreme beliefs in any approach are problematic and a belief in data driven decisions is no exception.

For the data to point the way, we need suitable problems, the right inputs and tracking based on good questions and thoughtful hypotheses, reliable data pipelines, good analytical judgment in overlooking outliers and picking a robust methodology, and versatility in the tools to analyze and interpret the outputs. Every once a while, all of these align and it all just works.

But, for the most part, we’re better off marrying a desire for data with a healthy skepticism for what it is telling us. It is that skepticism that will ensure we keep pushing for the right questions and iterate our way into insights that get us closer to the truth.

Better to be data informed than data driven.

The world sucks – vote for me

“The world sucks – vote for me.” That’s the vote-getting narrative that has worked wonderfully well for our politicians. And, the media has complied as well. That’s not because they want to, but because their business model requires them to sell negativity and fear. As humans, we’re drawn to that stuff.

The folks at “The World in Data” from the University of Oxford have created a public good to help counter this. They are a team of 3 who’ve gathered evidence on data around indicators of progress such as extreme poverty, education, and child mortality. And, to make it easy to consume, they’ve helped us visualize this in “The World as 100 People.”

the world in data

Of course, the world doesn’t suck. In fact, things are better than they’ve ever been. Does this mean things are better for everyone? No, it doesn’t. There is and always will be plenty to do. But, it also means a lot of the current narrative around the state of the world is nonsense.

What is the probability of “The World in Data” actually making the difference they seek? Very low. They don’t have the megaphones that our politicians have. And, there aren’t that many folk who are interested in looking at empirical data if it doesn’t show up as a sensational article on their Facebook feed.

But, maybe we could help improve the odds? Perhaps we could share their findings, one person at a time? Maybe we could better educate ourselves, our families and our communities? And, maybe, just maybe, we could use all this data to make better decisions.

Thank you, “The World in Data” team, for giving us the opportunity to make a difference. Now, it is on us to spread the word.

PS: Their blog has only 58 followers on Feedly. That’s a travesty. Let’s fix that.

Selection bias and winners – The 200 words project

Here’s this week’s 200 word idea thanks to TimHarford.com, Lifehacker.com and Professors David Dranove and Brett Saraniti at Kellogg.

In 1943, the American statistician Abraham Wald was asked to advise the US air force on how to reinforce their planes. Only a limited weight of armor plating was feasible, and the proposal on the table was to reinforce the wings, the center of the fuselage, and the tail. Why? Because bombers were returning from missions riddled with bullet holes in those areas.

Wald explained that this would be a mistake. What the air force had discovered was that when planes were hit in the wings, tail or central fuselage, they made it home. Where, asked Wald, were the planes that had been hit in other areas?They never returned. Wald suggested reinforcing the planes wherever the surviving planes had been unscathed instead.

As blogger Tim Harford points out, this makes for a classic example of selection bias and also a great life lesson. It is natural to look at successes. But, if we don’t examine our failures, we may end up putting our time, money, attention or even armor plating in entirely the wrong place.

Abraham Wald planesSource and thanks to: Digitalroam.typedpad.com

‘The data that isn’t present may tell as important a story as the data that is.’