I can’t actually believe I’m going to write a “book review” on my blog, but I am. Or, rather, I’m going to write a review of two particular subsections of a book. I just finished reading, Superfreakonomics, the followup to the immensely popular and successful Freakonomics, both Dr. Steven Levitt and Stephen Dubner. I was a huge fan of the first book, and while I’d say I’m a fan of the second book, there were two particular instances I found troubling. In the first book, there was a pervasive attitude of, “I don’t care what you tell me, I care what the data tell me.” This lead to both revolutionary ideas – the crime drop of the 90s was a byproduct, in large part, of Roe v. Wade – and obvious ones that it was nice to have proof of – your real estate agent has his/her own best interest at heart, not yours. In the second book, this trend continued for the most part, with two notable exceptions. In both of these cases, Levitt and Dubner took some data basically at face value, and, in both cases, I don’t think the data underlying the arguments are strong enough to warrant that.
The first instance is in Chapter 2, when they reference the work of Anders Ericsson, father of the so-called “10,000hr Rule.” Ericsson’s work was the foundation for a lot of other books – Outliers, Talent is Overrated, and The Talent Code. Generally speaking, Andersson is of the belief that, to borrow from the title of Geoff Colvin’s book, “talent is overrated.” Or that what we think of as talent is often the result of other things like the “birthday bulge,” whereby a nine year old boy born on Jan 2 and a nine year old boy born on Dec 31 are considered the same age, as of Jan 1, with regards to sports teams, despite the fact that one is basically a full year older, or, to give some better perspective, the one boy is more than 10% older (at least at that young age). 10% is a lot. There is no denying the influence of that “extra year” on development. But Ericsson takes his conclusions further when he starts talking about deliberate practice – the source of the 10,000 “rule.” And this is where the data get a little thin. The Science of Sport bloggers, Dr. Ross Tucker and Dr. Jonathan Dugas of South Africa, pretty thoroughly dismantle Ericsson’s data, mostly by pointing out that his rule doesn’t present any standard deviation. The most effective example, I think, is Michael Phelps, who was 5th in the Olympic final for the 200m butterfly at the age of 15, roughly four years (i.e., well shy of 10,000hrs) after he started swimming competitively. Ultimately, by looking at the data – the same data that Levitt and Dubner should have considered but didn’t, largely because they gave the topic relatively short shrift due to it’s “extensive” coverage in the aforementioned books – Tucker and Dugas show that, unsurprisingly, the 10,000 hour rule is pretty much bunk. Some people need a lot more practice to “make it,” some a lot less, and some never make it regardless of how much they practice. It’s a very insightful two part series – very “freakonomics-esque” – that is covered in Part 1: Genes vs. Training and Part 2: Genes and performances. The basic summary is this: the data show that the standard deviation from 10,000 hours (which is indeed roughly the average amount of deliberate practice it takes to become a “master”) is so large as to basically totally discredit the idea that it is practice that makes champions. They conclude the second part with a nice quote, “To become an Olympic champion, the very best of the best, you need to tick the boxes. Genes is without a doubt one of those boxes. But so too are opportunities. And so is success genetics or training? It’s both. In fact, it’s 100% genetic, and 100% training.” As always, trust the data.
The second area is not so much an area where I think Levitt and Dubner didn’t do a good job with the data as much as they presented something where the source of said data maybe ought to be questioned. In particular, I’m talking about the obviously hot-button (pun intended) issue of Global Warming (or Climate Change), which is covered in Chapter 5. One of the major focuses of the chapter is on solutions to the problem of climate change that are proposed by Nathan Myhrvold’s Intellectual Ventures, a think-tank located in Bellevue that aims to solve the world’s problems. All of them. But on the topic of climate change, there is member of Myhrvold’s cadre of scientists who gets given a bit more a free pass than I think he ought to. Lowell Wood is a protege of Edward Teller, he of atomic bomb fame. Sort of. Wood and Teller, most recently, were the “brains” behind the “Star Wars” missile defense program. This program is trotted out in the book as evidence of Wood’s “credentials.” He started work at IV after leaving Lawrence Livermore National Laboratory. Anyone who is even vaguely familiar with the “Star Wars” program would label it an unmitigated disaster. Those who are even more familiar will tell you that it was doomed from the start. Robert L. Park was the director of the National Physical Society and is author of several books, including, Voodoo Science, which takes a very hard look at both Teller and Wood and their projects. Obviously, Park is just one man, and his opinion is certainly subject to bias. But what isn’t subject to bias is the data, because good data generally are (you can say “is” depending on how picky you want to be) immune from bias, at least if the data are collected correctly. Park is, like Levitt and Dubner, a “what’s the data say?” kind of guy. And, by more accounts that just Park’s, Star Wars was a failure. Park goes into detail quite thoroughly – skewering Teller and Wood pretty convincingly – in Voodoo Science. It’s a great read for anyone who is interested; I’d recommend it to Levitt and Dubner. But putting aside the obvious potential for bias involved in any general assessment of someone’s character (meaning Parks’ opinion of Wood and Teller), the most obvious thing to consider is that you have a guy – Wood – with basically the unlimited resources of the federal government behind him, and he couldn’t even demonstrate basic proof of concept of his signature project (Star Wars), and yet somehow this is a guy who we should look to solve climate change? Granted, being totally wrong about one idea doesn’t mean you’ll be totally wrong about another (though Wood is apparently trying to make Star Wars for mosquitoes, endeavoring to shoot malaria carrying mosquitoes with lasers instead of Soviet ICBMs), and there are skeptics – covered in the book – who have run similar models to Wood and gotten the same results. But when it comes time to execute a plan to implement said model, I’m not sure I’d take advice from a guy who has a relatively massive black mark on his record. What I found surprising is that both Levitt and Dubner seem like born skeptics. They seem totally unlikely to believe anyone. And yet I think they both, in a state of semi-reverence, showed an undue amount of faith in someone who, by all accounts, has wasted an extraordinary amount of tax payer dollars. At least in this case, he might only waste private money. But I would have hoped for the same sort of skepticism that was showed towards police chiefs and others who have made bold claims that the data do not support. I guess, as often seems to be the case with folks like these, the *actual* (as opposed to theoretical) data are always coming.
Ultimately, I was a bit disappointed in the book, because it seemed to be a case of, “what will be popular,” as opposed to the original book which is, “what do we think is interesting.” The climate change chapter seems to me to the most obvious example of this, mostly because the data are so unreliable on that topic. As Myhrvold himself says, paraphrasing, it’ll take us as long to develop the computers that can model climate change as it will to simply see what happens. And, given that, I think it should be obvious that the data in such a case cannot be trusted. But, I guess, Levitt and Dubner were simply demonstrating that, as always, people respond to incentives. And, in this case, the incentive was just to sell more books.
[Addendum 2011.08.19] Riding my bike after writing this post, I realized that I should not have been surprised to see Lowell Wood’s name associated with projects of the sort that IV is working on. Wood, like Teller before him, built his career around the “boogeyman” of the era – the prospect of global nuclear war with the USSR. Once that threat passed, Wood was left without a “bad guy” to fight. I do not believe, from what I have read, that Wood is a fraud. Rather, I think that he is a true believer; he genuinely believes that he has the solutions to the potential and real woes that could and are plaguing mankind. So once the USSR threat ceased to exist, he moved on to the latest and greatest boogeyman – global warming & climate change, as well as, it appears, various diseases. I think that iconoclasts and entrepreneurs and inventors and tinkerers are wonderful. I just wish that more respect was given to those that actually have a track record of solving problems and that more skepticism was shown to those who have a track record of not solving them.
Jordan Rapp 
Do you mean that data is unreliable for the climate in general, or data as to pumping sulfur into the air to reverse warming is unreliable?
Love the two sports scientist articles, very enlightening.
LikeLike
Jordan,
Pardon the ignorance but how has Roe v. Wade reduced crime? I get infanticide but what else would it reduce?
LikeLike
Brian,
“This lead to both revolutionary ideas – the crime drop of the 90s was a byproduct, in large part, of Roe v. Wade – and obvious ones that it was nice to have proof of – your real estate agent has his/her own best interest at heart, not yours.”
These two ideas are in the Freakonomics book. I would recommend reading it. Unless Jordan wants to paraphrase or even write out the chapters.
LikeLike
Jack – rather than answering your question directly, which I'm not sure is the most honest way to go about it, I'll present instead four statements, which I believe would receive 95% agreement from the educated folks in the community:
1) Since at least the dawn of the industrial revolution, and perhaps since the end of the little ice age, the earth, in general, has been getting warmer.
2) Human civilization is very likely a (but not the only) contributing factor to this warming.
3) We have no real idea what this warming trend means for the future either the planet or for human civilization.
4) Pursuant to #3, we have no real idea what we should do about – if anything – about said warming trend.
LikeLike
In re: the failure of the “Star Wars” project, I think you give the idea short shrift from a political economy perspective.
As an actual, functional and operational project, you're absolutely correct that it was an unmitigated failure. Missile shield programs have evolved pathetically, and it truly is a wonder why development continued in the Bush administration (well, maybe not, but that's another post that would involve vitriol and disgust, and nobody to see that…).
There is, however, an argument to be made that the Cold War was won because the United States was winning the battle to upset the notion of M.A.D. The Soviet Union saw this, and when it realized it couldn't keep up, perestroika and glasnost were implemented, the world saw how much of a wreck it actually was, and the falling of the Wall was the inevitable end.
I don't heartily agree that the Cold War was won based on superior defense spending and manufacturing (that idea makes me shudder); totalitarian governments of that size always topple. Doesn't mean there isn't an argument to be made that Star Wars was a success.
Thanks for the Sports Scientist articles, and the interesting book review, and good luck with the season!
LikeLike
Chris – I'm aware of the “Star Wars was effective because it bankrupted the Soviets” logic. But that logic, in my opinion, certainly was not Wood's and/or Teller's logic. It may have been the logic of the people who funded them, but I don't think it was either of theirs (either collectively or individually). And certainly that same logic is not, I don't think, what we want to employ with regards to climate change. There's no “opponent” to bankrupt through escalation.
Brian – the gist of the Roe vs. Wade leading to reduced crime is covered in Freakonomics as well as in a standalone research paper by Dr. Levitt. The gist of it is this. Statistically speaking, criminals are most likely to be males aged ~20 +/- ~4 years. There is also a very high statistical correlation between “unwantedness” and becoming a criminal; basically, children born to a parent or parents that do not want them are more likely, statistically speaking, to become criminals. So there is an especially high probability that a male aged 20+/-4 who was an unwanted child is going to be a criminal. What's one possible way to eliminate the possibility that those individuals become criminals? Well, one way is to prevent them from ever “existing” in the first place. If they are never born, then they can't become criminals. Levitt is quick to point out – and I will as well – that this is NOT an argument for (or against) abortion. It's merely stating that unwantedness and becoming a criminal have a high statistical correlation, and legal and easily accessible abortion is one clearly very definitive and effective way to reduce the number of unwanted children. That is not to say it's a “good” way or the “best” way or anything like that. That's a moral argument, and I believe people on both sides are absolutely entitled to their opinion. That is why I am pro choice; I am not pro abortion; I just think people should have the right to choose. But the data shows that because Roe v. Wade made abortion legal in all states and therefore more readily accessible, it reduced the number of unwanted children. And when those unwanted children would have been of the age when they would, typically, have started to engage in criminal activity, they couldn't, because they hadn't been born.
LikeLike