The Weekly Buzz: Forecasting
“A recession is coming.” “This salesperson will be a good hire.” Our clients this week have been confronting a number of problems that ask them to try to predict the future. (Sadly, the marketing wizards can’t help.) As it turns out, people are pretty bad at prediction.
Hiring involves decisions about people, and people are complicated, multi-faceted things. You can’t just evaluate someone on a couple scales of one to ten (“years experience,” “certifications,” and “college GPA,” let’s say) and know how they are going to work out in your organization. People often rely on how much they like a candidate or go with their instinct. This can lead them to ignore attributes that are actually essential to performing the job successfully. Good managers find ways to reduce the role of instinct in hiring by developing methodical approaches, ranging from checklists to role plays to careful reference checks—and then use them every time.
Michael Lewis, the author of Moneyball, looks at problems with “gut instinct” approaches to decision-making and data in his book The Undoing Project. The book looks at how Nobel prize-winners Amos Tversky and Daniel Kahneman disrupted much of what we know about decision-making. Philip Tetlock has built on their ideas and founded the Good Judgment Project to try to apply those insights to improve forecasting in the real world of business and international policy. Keys to success include understanding statistics, avoiding bias, and taking in divergent viewpoints before settling on a prediction.
When Big Data Doesn’t Help
When it comes to applying statistics to data to make decisions, analysis is being transformed by big data and machine learning. But some things are just too complex or specific to predict with any degree of confidence: after analyzing all sorts of data multiple different ways, AI predicted the 2018 World Cup would be won by Spain, with a likelihood of 17.8%. Unsurprisingly, this not-particularly-confident prediction didn’t pan out. At least it was a more scientific approach than using animals! Paul the Octopus may have started the trend, but his very successful Japanese successor had a tough year. Mmm, sashimi!