During the first week of the NBA regular season, I declared that all writers covering the league in the early going should be required to read Daniel Kahneman's new book, Thinking, Fast and Slow. One of my Twitter followers had a better point, replying that everyone should read the book.
I've read a number of books that have helped shape the way I think. How We Know What Isn't So by Tom Gilovich, a collaborator with Kahneman, was useful in college. So too was Influence: The Pyschology of Persuasion, by Robert Cialdini. Still, nothing has done as complete a job of explaining the way humans reason--and the errors these systems cause us to make--as Kahneman's opus.
With the exception of a brief digression on the Hot Hand (the influential original paper studying the topic was co-authored by Gilovich and Kahneman's long-time research partner, the late Amos Tversky, as well as Robert Vallone), Thinking, Fast and Slow has nothing to do with basketball. Yet I couldn't help but consider how various conclusions applied to hoops, both in terms of scouting and statistical analysis. For each key chapter, we'll take a look at what Kahneman writes and what it means in basketball terms.
Chapter 6: Norms, Surprises and Causes
Key quote: "[A] large event is supposed to have consequences, and consequences need causes to explain them. [...] System 1 is adept at finding a coherent causal story that links the fragments of knowledge at its disposal."
As Thinking, Fast and Slow makes clear, the human mind is designed to tell "stories," even when there isn't enough evidence to put together a coherent plot or doing so requires ignoring contradictory information. Dean Oliver has said that the role of statistical analyst is to use the numbers to tell stories, but we must be careful not to be overconfident in doing so. For an example from my own work, I'll point to Nick Collison. My argument that Collison's true value is reflected by plus-minus rater than his individual statistics ignores that Collison's plus-minus ratings were pedestrian before 2009-10. I still suspect the plus-minus trend is more than a fluke--it's continuing again this season--but I might need to temper my belief in it.
The desire for good stories certainly influences award voting. Many years ago, John Hollinger mused in Pro Basketball Forecast 2005-06 about the "New Guy" effect. When a team substantially improves (or declines), this change is often credited entirely to a new star player or coach at the expense of smaller additions that are also important.
Chapter 7: A Machine for Jumping to Conclusions
Key quote: "The tendency to like (or dislike) everything about a person--including things you have not observed--is known as the halo effect. ... [T]he halo effect is a good name for a common bias that plays a large role in shaping our view of people and situations."
An example of the halo effect that stands out to me is from baseball, where I recall reading well over a decade ago that Detroit Tigers scouts were so enamored of Gabe Kapler's unexpected development that they grew to believe he would be a good base stealer. Kapler did steal 23 bags one year, but finished his 12-year career with 77 steals.
Key quote: "Information that is not retrieved (even unconsciously) from memory might as well not exist."
Kahneman calls this concept "what you see is all there is," and I think it's especially meaningful in terms of evaluating players in terms of character. My concern isn't that character is unimportant; rather, it's naïve to think we can really understand what players are like from a few brief glimpses. These judgments often need to taken for what they really are--guesses.
Chapter 9: The Law of Small Numbers
Key quote: "The results were straightforward: ... It was evident that even the experts paid insufficient attention to sample size."
Of everything in the book, this might be the most meaningful takeaway from a basketball perspective. If trained statisticians tend to be overly confident in small samples, how much more does this problem affect the rest of us? As a result, I find myself more cautious than ever before in drawing conclusions early in the season. We need many games to determine whether a new level of performance represents a real change and not merely randomness. Jumping to early conclusions is the easiest way to look foolish as a sports analyst.
Chapter 13: Availability, Emotion and Risk
Key quote: "The lesson is clear: estimates of causes of death are warped by media coverage. The coverage is itself biased toward novelty and poignancy."
Alas, we're not interested in physical death here, but figurative death--losing a game--is of great relevance. I think the availability bias explains some of the conservative decisions we see from coaches in late-game situations. A coach is more likely to easily retrieve the memory of the time his team lost by intentionally fouling a player in the act of shooting a three than the time the opposition merely came down and hit a three-pointer, to name one example. Crazy finishes are more likely to end up on SportsCenter, convincing coaches they are more common than they really are. That's what a coach remembers when making the decision on whether to foul or not, as opposed to the numbers that capture all possible situations.
(For the record, I'm still something of a situational pragmatist when it comes to fouling intentionally, but I do think this explains coaches that are completely (and irrationally) opposed to the move.)
M. Haubs of The Painted Area made an excellent point that this also applies to clutch situations. As problematic as they already are because of small sample sizes, last-second shots are also subject to availability bias. Remember that famous Michael Jordan commercial about all the game-winning shots he'd missed? Well, how many of those do you remember? And how much harder did you have to think to recall any of them rather than his famous game-winners, like The Shot over Craig Ehlo and his championship-winning shot against Utah in the 1998 NBA Finals?
Chapter 17: Regression to the Mean
Key quote: "Causal explanations will be evoked when regression is detected, but they will be wrong because the truth is that regression to the mean has an explanation but does not have a cause."
In this chapter, Kahneman explains the perverse feedback loop: Success in a given trial is often largely due to randomness (or luck), so the player who is praised for a great shooting night is likely to perform worse the following game while the player who gets chewed out for bad decision making will probably do better the next night. This leads coaches, in this example, to erroneously believe that yelling at a poor-performing player caused them to improve when it was nothing more than the law of regression.
Another NBA example that comes to mind is the persistent belief in "the scouting report." The story goes that a reserve promoted to the starting lineup might play well for a few games, but will struggle as soon as opponents scout him better. I've always found this odd because it implies both that scouting is highly effective (in that teams figure out how to shut down these players) and also ineffective (in that it takes an extended period of time for the new scouting report to get out). It now seems probable that players fade after a fast start simply because they're returning to their true talent level, which gets attributed to scouting.
Chapter 18: Taming Intuitive Predictions
Key quote: "Intuitive predictions need to be corrected because they are not regressive and therefore are biased."
This applies most obviously to preseason projections; the best ones tend to be regressed heavily to the mean, which is why SCHOENE will rarely predict that any team will win or lose 60 games. While we know some team will reach this mark, the chances of any individual team getting there are usually less than 50 percent. Another intriguing application is the draft. Our best guesses at how players will perform should be fairly closely grouped because before they enter the NBA we just don't know what they might become. Anyone guaranteeing a rookie will become a star or bust is peddling overconfidence in an uncertain world.
Chapter 19: The Illusion of Understanding
Key quote: "Hindsight bias has pernicious effects on the evaluations of decision makers. It leads observers to assess the quality of a decision not by whether the process was sound but by whether its outcome was good or bad."
This is pretty self-explanatory. I would say what unites analytical types even more than belief in the use of statistics is the paradigm that process is more important than results. If I was in charge of this site, I'd consider putting that in the Basketball Prospectus masthead.
Key quote: "Because luck plays a large role, the quality of leadership and management practices cannot be inferred reliably from observations of success."
Here, Kahnemann evaluates CEOs, but the same logic can be applied to GMs and coaches. So much of what they do is out of their control, and so much determined by randomness--one draft pick can make or break an NBA franchise--that we tend to attribute too much of their success or failure to their own efforts. Or, as some writer known as The Big Ten Wonk once put it pithily, regress your view of your coach toward the mean.
Chapter 20: The Illusion of Validity
Key quote: "In other words, people who spend their time, and earn their living, studying a particular topic produce poorer predictions than dart-throwing monkeys who would have distributed their choices evenly over the options."
The evidence for the ability of experts to make correct predictions is largely nonexistent. More importantly, experts tend to struggle to admit they were wrong because of their confidence in their own opinions. Apply that the next time you read any basketball predictions, including those on this site. I also wonder whether this might be relevant to the draft process. For all the time, money, scouting and analysis teams do to pick the right player, is it possible they (we) do no better than the wisdom of the crowd? I'm not sure how to evaluate that question, but it's a sobering possibility.
Chapter 31: Risk Policies
Key quote: "Decision makers who are prone to narrow framing construct a preference every time they face a risky choice. They would do better by having a risk policy that they routinely apply whenever a relevant problem arises."
In the fourth section of the book, Kahnemann discusses how humans understand risk. The narrow framing example above certainly seems to describe many coaches. It's easy for them to dismiss what statistics say over the long term because the only relevant outcome to them is the game at hand. Sometimes, this might be appropriate. In the one-and-done format of the NFL Playoffs or the NCAA tournament, for example, the decision that is better over the long run may be too risky if a missed opportunity means the end of a season.
Still, there are examples of coaches who see the bigger picture. Going back to the question of fouling up three, I think of DePauw coach Bill Fenlon (best known as the guy who coached Butler's Brad Stevens), who read the relevant literature and resolved to foul every time, no matter the results in any specific situation.
Chapter 32: Keeping score
Key quote: "Decision makers know that they are prone to regret, and the anticipation of that painful emotion plays a part in many decisions."
Here, to me, is the most relevant discussion at all to how coaches approach the decision-making process. We all seek to avoid regret, and one way to do so is by avoiding risks. When a football coach plays for a field goal his kicker misses, it's easy to explain away the outcome as a failure by the player rather than the decision-making process. If the coach aggressively throws for the end zone, by contrast, the possibility of a turnover or a sack is largely on the coach and is likely to produce regret.
When coaches choose the safer option, they are not being obtuse, or ignorant. They're being human. As Thinking, Fast and Slow continually reinforces, that's often synonymous with being fallible.
This free article is an example of the kind of content available to Basketball Prospectus Premium subscribers. See our Premium page for more details and to subscribe.
Kevin Pelton is an author of Basketball Prospectus.
You can contact Kevin by clicking here or click here to see Kevin's other articles.