From Completely Useless to Moderately Useful
In 1955, a
twenty-one-year-old Daniel Kahneman was assigned the formidable task of
creating an interview procedure to assess the fitness of recruits for the
Israeli army. Kahneman’s only qualification was his bachelor’s degree in
psychology, but the state of Israel had only been around for seven years at the
time so the Defense Forces were forced to satisfice. In the course of his undergraduate
studies, Kahneman had discovered the writings of a psychoanalyst named Paul
Meehl, whose essays he would go on to “almost memorize” as a graduate student.
Meehl’s work gave Kahneman a clear sense of how he should go about developing
his interview technique.
If you polled psychologists today
to get their predictions for how successful a young lieutenant inspired by a
book written by a psychoanalyst would be in designing a personality assessment
protocol—assuming you left out the names—you would probably get some dire
forecasts. But
Paul Meehl wasn’t just any psychoanalyst, and
Daniel Kahneman
has gone on to become one of the most influential psychologists in the world.
The book whose findings Kahneman applied to his interview procedure was
Clinical vs. Statistical Prediction: A
Theoretical Analysis and a Review of the Evidence, which Meehl lovingly
referred to as “my disturbing little book.” Kahneman explains,
Meehl reviewed the results of 20
studies that had analyzed whether clinical
predictions based on the subjective impressions of trained professionals
were more accurate than statistical
predictions made by combining a few scores or ratings according to a rule. In a
typical study, trained counselors predicted the grades of freshmen at the end
of the school year. The counselors interviewed each student for forty-five
minutes. They also had access to high school grades, several aptitude tests,
and a four-page personal statement. The statistical algorithm used only a
fraction of this information: high school grades and one aptitude test. (222)
 |
| Daniel Kahneman |
The findings for this prototypical study are consistent with
those arrived at by researchers over the decades since Meehl released his book:
The number of studies reporting
comparisons of clinical and statistical predictions has increased to roughly
two hundred, but the score in the contest between algorithms and humans has not
changed. About 60% of the studies have shown significantly better accuracy for the
algorithms. The other comparisons scored a draw in accuracy, but a tie is
tantamount to a win for the statistical rules, which are normally much less
expensive to use than expert judgment. No exception has been convincingly
documented. (223)
Kahneman
designed the interview process by coming up with six traits he thought would
have direct bearing on a soldier’s success or failure, and he instructed the
interviewers to assess the recruits on each dimension in sequence. His goal was
to make the process as systematic as possible, thus reducing the role of
intuition. The response of the recruitment team will come as no surprise to
anyone: “The interviewers came close to mutiny” (231). They complained that
their knowledge and experience were being given short shrift, that they were
being turned into robots. Eventually, Kahneman was forced to compromise,
creating a final dimension that was holistic and subjective. The scores on this
additional scale, however, seemed to be highly influenced by scores on the previous
scales.
When commanding officers evaluated
the new recruits a few months later, the team compared the evaluations with
their predictions based on Kahneman’s six scales. “As Meehl’s book had
suggested,” he writes, “the new interview procedure was a substantial
improvement over the old one… We had progressed from ‘completely useless’ to
‘moderately useful’” (231).
Kahneman
recalls this story at about the midpoint of his magnificent, encyclopedic book
Thinking, Fast and Slow. This is just
one in a long series of run-ins with people who don’t understand or can’t
accept the research findings he presents to them, and it is neatly woven into
his discussions of those findings. Each topic and each chapter feature a short
test that allows you to see where you fall in relation to the experimental
subjects. The remaining thread in the tapestry is the one most readers familiar
with Kahneman’s work most anxiously anticipated—his friendship with
AmosTversky, with whom he shared the Nobel prize in economics in 2002.
Most of the ideas that led to
experiments that led to theories which made the two famous and contributed to
the founding of an entire new field,
behavioral economics, were borne of casual
but thrilling conversations both found intrinsically rewarding in their own
right. Reading this book, as intimidating as it appears at a glance, you get
glimmers of Kahneman’s wonder at the bizarre intricacies of his own and others’
minds, flashes of frustration at how obstinately or casually people avoid the
implications of psychology and statistics, and intimations of the deep fondness
and admiration he felt toward Tversky, who died in 1996 at the age of 59.
Pointless Punishments and Invisible
Statistics
When
Kahneman begins a chapter by saying, “I had one of the most satisfying eureka
experiences of my career while teaching flight instructors in the Israeli Air
Force about the psychology of effective training” (175), it’s hard to avoid
imagining how he might have relayed the incident to Amos years later. It’s also
hard to avoid speculating about what the book might’ve looked like, or if it ever
would have been written, if he were still alive. The eureka experience Kahneman
had in this chapter came about, as many of them apparently did, when one of the
instructors objected to his assertion, in this case that “rewards for improved
performance work better than punishment of mistakes.” The instructor insisted
that over the long course of his career he’d routinely witnessed pilots perform
worse after praise and better after being screamed at. “So please,” the
instructor said with evident contempt, “don’t tell us that reward works and
punishment does not, because the opposite is the case.” Kahneman, characteristically
charming and disarming, calls this “a joyous moment of insight” (175).
The
epiphany came from connecting a familiar statistical observation with the
perceptions of an observer, in this case the flight instructor. The problem is
that we all have a tendency to discount the role of chance in success or
failure. Kahneman explains that the instructor’s observations were correct, but
his interpretation couldn’t have been more wrong.
What he observed is known as regression to the mean, which in that
case was due to random fluctuations in the quality of performance. Naturally,
he only praised a cadet whose performance was far better than average. But the
cadet was probably just lucky on that particular attempt and therefore likely
to deteriorate regardless of whether or not he was praised. Similarly, the
instructor would shout into the cadet’s earphones only when the cadet’s
performance was unusually bad and therefore likely to improve regardless of
what the instructor did. The instructor had attached a causal interpretation to
the inevitable fluctuations of a random process. (175-6)
The roster of domains in which we fail to account for
regression to the mean is disturbingly deep. Even after you’ve learned about
the phenomenon it’s still difficult to recognize the situations you should apply
your understanding of it to. Kahneman quotes statistician David Freedman to the
effect that whenever regression becomes pertinent in a civil or criminal trial
the side that has to explain it will pretty much always lose the case. Not
understanding regression, however, and not appreciating how it distorts our
impressions has implications for even the minutest details of our daily
experiences. “Because we tend to be nice to other people when they please us,”
Kahneman writes, “and nasty when they do not, we are statistically punished for
being nice and rewarded for being nasty” (176). Probability is a bitch.
The Illusion of Skill in Stock-Picking
Probability
can be expensive too. Kahneman recalls being invited to give a lecture to
advisers at an investment firm. To prepare for the lecture, he asked for some
data on the advisers’ performances and was given a spreadsheet for investment
outcomes over eight years. When he compared the numbers statistically, he found
that none of the investors was consistently more successful than the others.
The correlation between the outcomes from year to year was nil. When he
attended a dinner the night before the lecture “with some of the top executives
of the firm, the people who decide on the size of bonuses,” he knew from
experience how tough a time he was going to have convincing them that “at least
when it came to building portfolios, the firm was rewarding luck as if it were
a skill.” Still, he was amazed by the execs’ lack of shock:
We all went on calmly with our
dinner, and I have no doubt that both our findings and their implications were
quickly swept under the rug and that life in the firm went on just as before.
The illusion of skill is not only an individual aberration; it is deeply
ingrained in the culture of the industry. Facts that challenge such basic
assumptions—and thereby threaten people’s livelihood and self-esteem—are simply
not absorbed. (216)
The scene that follows echoes the
first chapter of Carl Sagan’s classic paean to skepticism Demon-Haunted World, where Sagan recounts being bombarded with
questions about science by a driver who was taking him from the airport to an
auditorium where he was giving a lecture. He found himself explaining to the
driver again and again that what he thought was science—Atlantis, aliens,
crystals—was, in fact, not. "As we drove through the rain," Sagan writes, "I could see him getting glummer and glummer. I was dismissing not just some errant doctrine, but a precious facet of his inner life" (4). In Kahneman’s recollection of his drive back to
the airport after his lecture, he writes of a conversation he had with his own driver,
one of the execs he’d dined with the night before.
He told me, with a trace of defensiveness,
“I have done very well for the firm and no one can take that away from me.” I
smiled and said nothing. But I thought, “Well, I took it away from you this
morning. If your success was due mostly to chance, how much credit are you
entitled to take for it? (216)
Blinking at the Power of Intuitive Thinking
 |
| Malcolm Gladwell |
It wouldn’t
surprise Kahneman at all to discover how much stories like these resonate.
Indeed, he must’ve considered it a daunting challenge to conceive of a sensible,
cognitively easy way to get all of his vast knowledge of biases and heuristics
and unconscious, automatic thinking into a book worthy of the science—and
worthy too of his own reputation—while at the same time tying it all together
with some intuitive overarching theme, something that would make it read more
like a novel than an encyclopedia.
Malcolm Gladwell faced a similar challenge
in writing
Blink: the Power of Thinking without Thinking, but he had the advantages of a less scholarly readership,
no obligation to be comprehensive, and the freedom afforded to someone writing
about a field he isn’t one of the acknowledged leaders and creators of.
Ultimately, Gladwell’s book painted a pleasing if somewhat incoherent picture
of intuitive thinking. The power he refers to in the title is over the thoughts
and actions of the thinker, not, as many must have presumed, to arrive at
accurate conclusions.
It’s entirely possible that
Gladwell’s misleading title came about deliberately, since there’s a
considerable market for the message that intuition reigns supreme over science
and critical thinking. But there are points in his book where it seems like
Gladwell himself is confused.
Robert Cialdini, Steve Marin, and Noah Goldstein
cover some of the same research Kahneman and Gladwell do, but their book
Yes!: 50 Scientifically Proven Ways to be Persuasive is arranged in a list format, with each chapter serving as its
own independent mini-essay.
 |
| Robert Cialdini |
Early in Thinking, Fast and Slow, Kahneman introduces us to two characters,
System 1 and System 2, who pass the controls of our minds back and forth
between themselves according the expertise and competency demanded by current
exigency or enterprise. System 1 is the more intuitive, easygoing guy, the one
who does what Gladwell refers to as “thin-slicing,” the fast thinking of the
title. System 2 works deliberately and takes effort on the part of the thinker.
Most people find having to engage their System 2—multiply 17 by 24—unpleasant
to one degree or another.
The middle part of the book
introduces readers to two other characters, ones whose very names serve as a
challenge to the field of economics. Econs are the beings market models and
forecasts are based on. They are rational, selfish, and difficult to trick.
Humans, the other category, show inconsistent preferences, changing their minds
depending on how choices are worded or presented, are much more sensitive to
the threat of loss than the promise of gain, are sometimes selfless, and not
only can be tricked with ease but routinely trick themselves. Finally, Kahneman
introduces us to our “Two Selves,” the two ways we have of thinking about our
lives, either moment-to-moment—experiences he, along with
Mihaly Csikzentmihhalyi (author of
Flow)
pioneered the study of—or in abstract hindsight. It’s not surprising at this
point that there are important ways in which the two selves tend to disagree.
Intuition and Cerebration
The Econs versus Humans distinction, with
its rhetorical purpose embedded in the terms, is plenty intuitive. The two
selves i
dea, despite being a little too redolent of
psychoanalysis, also works well. But the discussions about System 1 and System
2 are never anything but ethereal and abstruse. Kahneman’s stated goal was to
discuss each of the systems as if they were characters in a plot, but he’s far
too concerned with scientifically precise definitions to run with the metaphor.
The term system is too bloodless and too suggestive of computer components;
it’s too much of the realm of System 2 to be at all satisfying to System 1. The
collection of characteristics
Thinking
links to the first system (see a list below) is lengthy and fascinating and not easily summed up
or captured in any neat metaphor. But we all know what Kahneman is talking
about. We could use mythological figures, perhaps Achilles or Orpheus for System
1 and Odysseus or Hephaestus for System 2, but each of those characters comes
with his own narrative baggage. Not everyone’s System 1 is full of rage like
Achilles, or musical like Orpheus. Maybe we could assign our System 1s idiosyncratic
totem animals.
But I think the most familiar and
the most versatile term we have for System 1 is intuition. It is a hairy and
unpredictable beast, but we all recognize it. System 2 is actually the harder
to name because people so often mistake their intuitions for logical thought.
Kahneman explains why this is the case—because our cognitive resources are
limited our intuition often offers up simple questions as substitutes from more
complicated ones—but we must still have a term that doesn’t suggest complete
independence from intuition and that doesn’t imply deliberate thinking operates
flawlessly, like a calculator. I propose cerebration. The cerebral cortex rests
on a substrate of other complex neurological structures. It’s more developed in
humans than in any other animal. And the way it rolls trippingly off the tongue
is as eminently appropriate as the swish of intuition. Both terms work well as
verbs too. You can intuit, or you can cerebrate. And when your intuition is
working in integrated harmony with your cerebration you are likely in the
state of flow Csikzentmihalyi pioneered the study of.
While Kahneman’s division of
thought into two systems never really resolves into an intuitively manageable
dynamic, something he does throughout the book, which I initially thought was
silly, seems now a quite clever stroke of brilliance. Kahneman has no faith in
our ability to clean up our thinking. He’s an expert on all the ways thinking
goes awry, and even he catches himself making all the common mistakes time and
again. In the introduction, he proposes a way around the impenetrable wall of
cognitive illusion and self-justification. If all the people gossiping around
the water cooler are well-versed in the language describing biases and heuristics and errors of
intuition, we may all benefit because anticipating gossip can have a
profound effect on behavior. No one wants to be spoken of as the fool.
Kahneman writes, “it is much
easier, as well as far more enjoyable, to identify and label the mistakes of
others than to recognize our own.” It’s not easy to tell from his
straightforward prose, but I imagine him writing lines like that with a wry
grin on his face. He goes on,
Questioning what we believe and
want is difficult at the best of times, and especially difficult when we most
need to do it, but we can benefit from the informed opinions of others. Many of
us spontaneously anticipate how friends and colleagues will evaluate our
choices; the quality and content of these anticipated judgments therefore
matters. The expectation of intelligent gossip is a powerful motive for serious
self-criticism, more powerful than New Year resolutions to improve one’s
decision making at work and at home. (3)
So we encourage the education of
others to trick ourselves into trying to be smarter in their eyes. Toward that
end, Kahneman ends each chapter with a list of sentences in quotation
marks—lines you might overhear passing that water cooler if everyone where you
work read his book. I think he’s
overly ambitious. At some point in the future, you may hear lines like “They’re
counting on denominator neglect” (333) in a boardroom—where people are trying
to impress colleagues and superiors—but I seriously doubt you’ll hear it in the
break room. Really, what he’s hoping is that people will start talking more
like behavioral economists. Though some undoubtedly will, Thinking, Fast and Slow probably won’t ever be as widely read as,
say, Freud’s lurid pseudoscientific On
the Interpretation of Dreams.
That’s a tragedy.
Still, it’s pleasant to think about
a group of friends and colleagues talking about something other than football
and American Idol.
Characteristics of System 1 (105): Try to come up with a good metaphor.
·
generates
impressions, feelings, and inclinations; when endorsed by System 2 these become
beliefs, attitudes, and intentions
·
operates
automatically and quickly, with little or no effort, and no sense of voluntary
control
·
can
be programmed by System 2 to mobilize attention when particular patterns are
detected (search)
·
executes
skilled responses and generates skilled intuitions, after adequate training
·
creates
a coherent pattern of activated ideas in associative memory
·
links
a sense of cognitive ease to illusions of truth, pleasant feelings, and reduced
vigilance
·
distinguishes
the surprising from the normal
·
infers
and invents causes and intentions
·
neglects
ambiguity and suppresses doubt
·
is
biased to believe and confirm
·
exaggerates
emotional consistency (halo effect)
·
focuses
on existing evidence and ignores absent evidence (WYSIATI)
·
generates
a limited set of basic assessments
·
represents
sets by norms and prototypes, does not integrate
·
matches
intensities across scales (e.g., size and loudness)
·
computes
more than intended (mental shotgun)
·
sometimes
substitutes an easier question for a difficult one (heuristics)
·
is
more sensitive to changes than to states (prospect theory)
·
overweights
low probabilities.
·
shows
diminishing sensitivity to quantity (psychophysics)
·
responds
more strongly to losses than to gains (loss aversion)
·
frames
decision problems narrowly, in isolation from one another