Freakonomics is a well known book that got mostly very positive reviews. It is written with intelligence and wit. Its subject matter is of interest to me, and its approach is a kind of critical thinking that I could be expected to appreciate. All-in-all, one would think I would rate this book very favorably. Certainly I had high hopes for it when I picked it up.
Instead I found it to be just pretty good.
The subject matter falls under the category of applied economics. More specifically, the authors look at various phenomena of social behavior, from important to quirky, and seek to quantify and explain them. It’s kind of like what Nate Silver does with politics.
I prefer Silver. For one thing, he has more humility about the limits of what he does. He’s always conceding that he can only measure what is measurable, that the world often provides only sample sizes that are worrisomely small or otherwise flawed, that it’s best to incorporate some intuition and common sense to account for non-quantifiable factors, etc.
Levitt and Dunbar write in a more cocky, breezy style of “We crunched the numbers and this is how it turns out things are” without a lot of qualifications or acknowledgement of the fallibility of their approach.
But the limitations are still there, even if they choose not to dwell on them. I’m skeptical their methodology fits all problems they use it on equally well. I found my confidence level in their claims varying from chapter to chapter.
An example–one of their quirky ones–where I think their approach is convincing is their examination of sumo wrestlers throwing matches.
In sumo wrestling tournaments, each wrestler engages in 15 matches. If he finishes 8-7 or better, he moves up in ranking. Ranking is hugely important in terms of the money and other perks of sumo.
When a wrestler enters his final match on the bubble with a 7-7 record, everything rests on that last match. If he enters with an 8-6 record, he has very little incentive to win. (There’s almost no difference between finishing 9-6 versus 8-7. Whereas if he enters his last match at, say, 12-2, there can be a big difference in prize money between finishing 13-2 and 12-3.)
What the numbers show is that although you might think that a wrestler with an 8-6 record and a wrestler with a 7-7 record are pretty equal, with a slight edge to the 8-6 wrestler, in fact the 7-7 wrestler wins about 80% of the time. Furthermore, the next time those two wrestlers meet, the one that had been 8-6 wins disproportionately (though not quite as disproportionately) often–about 60% of the time. If they meet again later, the percentages revert back to about 50%, which is what one would have expected all along.
The data points strongly to the conclusion that some of those 8-6 wrestlers threw their match, and that at least part of the incentive was an arrangement that they were promised a win in their next bout with the same opponent, when they might well need it more. This is further corroborated by other evidence, such as the fact that the numbers stray even farther from expectations when there is some connection between the wrestlers, like that they fight for stables that have a relationship with each other.
That kind of analysis reminds me of an article I read some years ago that established the high likelihood of cheating on a certain ESP test that had shown positive results.
I don’t remember it exactly, so the details are surely off, but roughly what happened is that the tests given to subjects were multiple choice where they had to write 1, 2, 3 or 4, and the conclusion was that the researcher had gone back afterwards and surreptitiously changed some 1s to 4s where 4 was the right answer (because that’s the easiest one to change without making it obvious you’re writing over a previous answer).
If a researcher were to cheat like that, you’d expect the answers to contain fewer 1s and more 4s than chance, and for 1s and 4s to be right more often than 2s and 3s.
For example, if there were 400 questions, with the correct answer being a 1, 2, 3, or 4 100 times each, and a subject with no ESP ability chose at random, you’d expect him to pick each number about the same number of times, and to go about 25-75 on each of them. If that happened, and a cheater was grading the tests and changed, say, 20 wrong 1s to right 4s, then his record would be more like 25-55 on 1s, 25-75 on 2s, 25-75 on 3s, and 45-75 on 4s. Overall that would make him 120-280 rather than the chance performance of 100-300. And that’s just the kind of distribution skeptics found when they closely examined the records of subjects alleged to have done better than chance.
Another issue the authors look at is that of whether real estate agents will really get the best price for your house that they can. Certainly there’s an incentive for them to do so: their cut will be bigger the bigger your sale price. But the data shows that this incentive is not enough, compared to other incentives like wanting to get your deal done quickly so they can work on other deals. The proof is that when real estate agents sell their own houses, they do so for a higher average price, even controlling for all other plausibly relevant factors.
The best known phrase from the book is one of the chapter titles: “Why do drug dealers still live with their moms?” The answer, perhaps surprising, perhaps not, is that drug dealing is like most capitalist enterprises in that the overwhelming majority of people in that line of work make extremely little money at it. They do it in spite of the terrible pay (and in spite of the even more terrible working conditions), because they hope to work their way up to be one of the less than 1% of drug dealers who hit it big and get rich (though even most of them are rich quite briefly and then are dead or in prison).
Yes, the percentages are daunting, but most ghetto 15 year olds either don’t know that and think success in that field is more likely than it is, or they realize that getting rich by being a brain surgeon or day trader or whatever is even less likely for someone raised in their environment. So they desperately fight each other, and often kill each other, in pursuit of that one small shot they might have of making it big. And in the meantime they live with their moms and have a second job at McDonalds when they’re not on a street corner selling crack, because they make almost no money unless and until they get to the highest rungs in their drug dealer organization.
Like I say, at times the analysis feels shakier than other times to me. Sometimes it’s a matter of highlighting measurable things, not because they’re necessarily the most important, but precisely because they are measurable.
For example, I’m always uncomfortable with analyses of what works and doesn’t work in education that are based only on the most quantifiable outcomes, such as scores on the SAT or other standardized tests. The authors are guilty of this in their discussion of parenting. Their conclusion is that nothing parents do as far as their child raising matters much if at all (though what parents are–rich, intelligent, etc.–matters a great deal). But they base this on quantifiable things like test scores.
To me, the goals of good parenting should involve things like how happy the child is (both during childhood and then as an adult), how well they treat people, how capable they are of thinking rationally and coming to true rather than false beliefs, how willing they are to think for themselves and challenge authority, etc. But how do you measure stuff like that?
So they fall back on test scores and claim it really doesn’t matter how you parent. OK, maybe it doesn’t matter, much, for that, but that leaves open the possibility that it matters a great deal for the more important things.
They also occasionally get away from their data-driven analysis and toss in some anecdotes instead. An example of where that leads them to a dubious conclusion is in their brief discussion of race and politics. They claim there is a gap between what people tell pollsters and what they then actually do. Specifically, they say people claim to support an African American candidate or oppose a racist candidate more than is reflected in how they then proceed to vote. In support of this, they cite the cases of David Dinkins underperforming his pre-election polls, and David Duke overperforming his pre-election polls.
The thing is, from what I understand this is a well-researched phenomenon–called the Bradley Effect, after Tom Bradley–and that it has been largely debunked. Evidently, studies have shown either that the Bradley Effect doesn’t exist, or that it’s much, much weaker than most people would expect.
(That doesn’t mean, by the way, that racism in elections is a myth. It just means that there are not a significant number of people who are influenced by race in how they vote but who lie to pollsters and claim they’ll be voting the other way.)
In their discussion of crime, and why it has gone down substantially in recent years, the authors claim that almost all the causes people from all along the political spectrum point to turn out to have little if any explanatory power. They say there are two main factors supported by the data.
One is the massive shift toward punishment–fewer limits on policing, longer sentences, highest incarceration rate in the world, limitations on the rights of the accused, etc. The other is the higher number of abortions. Many of the people who would have been most likely to commit crimes were never born, because the potential mothers of those people–living in poverty, poorly educated, etc.–disproportionately chose abortion.
I’m neither fully convinced by their discussion of the crime rate, nor highly skeptical. I’m more on the fence–a good chance it’s roughly true, but I wouldn’t be surprised if a more thorough look at the evidence contradicted at least part of what they claim.
The more simple the issue–such as looking for suspicious correlations in the outcome of sumo matches–the more confident I am in their conclusions. The more complex the issue–such as crime or parenting, with their myriad of quantifiable and unquantifiable factors–the less confident I am.
The book is a fun read, and an educational one, as long as you don’t assume that all their claims have been somehow proven in a hard science sense.
Numbers tell us some things, and they don’t tell us others. They probably tell us most, but not all, of what Levitt and Dunbar think they tell us.