Why do I care so much about the Malcolm Gladwell issue? First is the matter of scientific integrity in journalism. Science-oriented folks care about it, and most everyone else doesn’t. For instance, here is John Horgan from Slate:
Almost four years ago, an esteemed science journalist — OK, it was me — suggested that the days of truly momentous scientific discovery might be over. One symptom of science’s plight, I predicted, would be that my fellow science writers would become increasingly desperate for and willing to invent “revolutionary” theories. To my delight, Malcolm Gladwell has provided the most spectacular confirmation of my hypothesis to date.
Compare this to the Columbia Journalism Review:
The answer to this charge is: Of course Gladwell lacks rigor – he’s a feature writer, not a brain scientist. Why some people – including the corporate titans who pay Gladwell’s speaking fees – seem confused about this I haven’t a clue. I can’t also help but wonder what would prompt the Times to haul out the heavy gun that is Pinker to shoot down a collection of magazine miscellany.
The reason why is that the way in which we think about probability and statistics determines the way we model the world, and the way we model the world profoundly effects the way we think, behave, and solve problems. A faulty map, like the kind that Gladwell spreads to millions of powerful and wealthy people, causes us to collectively trip and fall in the territory. The only problem is that the more of us use the faulty map, the easier it is to write off our faulty navigation based on uncontrolled external factors. This is groupthink on a stupendous scale. We have trouble identifying our mistakes if we all make them in the same way.
I am not a scientist. I didn’t even attend college for more than a few classes. I don’t pretend to be a scientist, but I do form science-based opinions based on the results reported by real scientists, whom I admire. But the real people with power in this world are journalists and politicians, not scientists. As an online journalist/intellectual type with more than 1,500 short popsci articles under my belt, I see my task as spreading scientific literacy to as many people as possible, and by extension the people in power, so they can make decisions based on empirical evidence and not folk theories. I see a responsibility to scientists and researchers to absorb as much of their material as I can and translate it into non-specialist language that any educated person can digest.
Malcolm Gladwell breaks that responsibility. Instead of trying to be interesting while being factual, he arbitrarily makes up counterintuitive ideas and then cherry-picks anecdotes and evidence to support them. This makes a mockery of science. I was shocked to see unscientific language being used in a review of What the Dog Saw (Gladwell’s latest book) for my city’s newspaper, the San Francisco Chronicle:
The book – divided into three sections on minor geniuses, intriguing theories and personality analysis – is grounded on a bedrock of strong character portraits.
What the hell? A bedrock of… character portraits? Character portraits form a bedrock? This is a very low intellectual standard.
What is the alternative to character portraits? Well, studies that record the intelligence testing results of many tens of thousands of people and follow up on additional traits such as job performance, trainability, delinquency rates, vocabulary understanding, ability to deal with unexpected situations, identification of problems, dealing with orders, and a huge library of other g-loaded tasks, to give one example. Where does this wealth of information come from? To quote Gottfredson 1997:
Civil rights law and regulation have led many employers in recent decades to scrutinize more carefully the validity of their selection procedures (Sharf, 1988). They have also prompted a sometimes desperate search for less g-loaded selection procedures (procedures less highly correlated with intelligence) in order to reduce disparate impact of selection devices on minority hiring and thus employers’ vulnerability to employment discrimination lawsuits (Gottfredson & Sharf, 1988). As a result, there now exists a very large body of evidence concerning the predictive validity of various mental aptitudes, personality traits, and physical capabilities (e.g., see Gottfredson, 1986b; J. Hogan, 1991; R. Hogan, 1991; Landy, Shankster, & Kohler, 1994; Lubinski & Dawis, 1992; Schmidt, Ones, & Hunter, 1992; Stokes, Mumford, & Owens, 1994). Many of these data have been metaanalyzed.
This data is all there, yet Gladwell writes that it is impossible to determine how good of a teacher someone will be from their intelligence tests, or how well a starting quarterback will perform based on their draft position. Actually, you can use these metrics — though the estimation will not be perfect, it’s almost always better than guessing without information. Here’s a couple more quotes on Gladwell from New York magazine’s book review:
Leon Wieseltier, the literary editor of The New Republic, has said, “What Gladwell is marketing is nothing but marketing—the marketer’s view of the world. But that view of the world is, I’m afraid, idiotic.” The judge and legal scholar Richard Posner, in a scathing review of Blink for TNR, complained that it was “written like a book intended for people who do not read books.”
In a marketer’s view of the world, science doesn’t really matter. If it helps you sell something, great, otherwise, who cares?
There will probably always be marketers writing books on marketing. What is scary is when these marketing books acquire a vague scientific veneer that sends them screaming to the top of bestsellers lists. Most marketing books are complete, utter fluff — the reason that Gladwell does better than his competitors is that non-scientists can understand his work and consider it scientifically informed on some level. Back to Janet Maslin’s “Mr. Gladwell has a great penchant for quantifiable data.”
The first issue, which I’ve just described, is a conflict between Gladwell and established science on intelligence. But what concerns me even more is deeper. It’s a conflict between Gladwell and Bayesian reasoning itself. Instead of thinking about something and carefully considering all sides of an issue, Gladwell advocates making decisions in the time it takes to blink. This strategy can work alright for tasks like facial recognition, but in complex situations, it becomes worse than useless. Piles upon piles of scientific studies of human decision-making have determined that going with our “gut feeling” often leads straight down the rabbit hole to Fail-Land.
Reading Pinker’s article, I figured that his main qualm with Gladwell — also mine — is that Gladwell urinates all over statistical analysis just because it’s not perfect. (Implementing a true Bayesian rationalist would require infinite computing power.) Pinker’s most important points are at the top of the second page of his review. The page that most clearly elucidates Pinker’s motivation — and sheds light on the entire article and what the conflict is really about, which I’d wager 90% of the commenters on the issue haven’t realized yet because they think the issue is more about pretending you’re an expert than using flawed statistical reasoning that rots the core of our society — is a short piece published by the editors of the newspaper on how the idea for the review began:
Malcolm Gladwell recently said that if he were trying to break into journalism today, he would start by getting a master’s degree in statistics. The Harvard psychologist Steven Pinker, who reviews Gladwell’s “What the Dog Saw” on this week’s cover, might second this advice. Asked via e-mail what is the most important scientific concept that lay people fail to understand, he responded: “Statistical reasoning. A difficulty in grasping probability underlies fallacies from medical quackery and stock-market scams to misinterpreting sex differences and the theory of evolution.”
Difficulty in grasping probability is another way of saying difficulty in following the axioms of probability theory, which is another way of saying difficulty in using Bayesian reasoning. The axioms of probability theory are here to stay. They seem even more fundamental than the laws of physics. Try to fight probability theory, and you will eventually lose.
There’s a problem with probability theory: it’s not sexy. Humans do not follow it at the conscious level very frequently because evolution is a lazy designer that follows a “good enough” philosophy of organism-making. So, there are two choices — attempt to twist our minds into a configuration that follows probability theory more faithfully, or accept nonsense that makes us feel good. Most people who have this choice choose the latter. Twisting is difficult, but ultimately necessary, and with brain-computer interfacing it will eventually become much easier.
One of the fundamental ideas in Bayesian probability theory is that you assign prior probabilities to different possibilities based on prior knowledge. Gladwell is arguing that we throw away explicit priors and trust our gut, or assign all mutually exclusive future possibilities an equal probability weighting. Yet, that is just another type of prior — one conveniently supplied at the subconscious level by our Bayesian brains. Now, it may be, in some cases, that the “hidden prior” that exists in our brains might be superior to a hastily assembled explicit prior, especially for tasks for which there was a strong selection pressure and evolution has a strong incentive for not messing up — like facial recognition. For evolutionarily novel decision problems, such as judging the predictive value of IQ tests, forget it. We have to follow the data and see what it says, because our personal opinions are untrustworthy. Gladwell’s habit of throwing away predictive indicators altogether will do us absolutely no good.
As we head into a dangerous period of technological development, it is more important than ever to be educated about statistical reasoning. Cognitive biases like scope neglect — behaving the same way whether 1,000 or 1,000,000 lives are at sake — will be our downfall if we aren’t careful. Our “downfall” could be our literal extinction, from molecular nanotechnology, AI, or synthetic biology, as Bill Joy pointed out in his famous article.
The statistical ignorance that Pinker rails against ties in to why some people think that AI is straight-up impossible or implausibly difficult — they view their own intelligence as a magical engine (the holistic view) rather than a large number of individually uninteresting but collectively powerful prediction and control algorithms (a reductionist view). Statistical analysis and decision theory still has a ways to go before creating AGI (in my view), but part of the reason why some people think that AGI is centuries off is that the achievements that these fields have already produced have gone under-recognized and unregarded by some of the best-selling authors of our time. Some of these authors will continue ignoring the power of statistical reasoning right up until the day a Bayesian AGI walks right up to them and shakes their hand.
3 Responses »
