[Addendum 2011.08.19] Riding my bike after writing this post, I realized that I should not have been surprised to see Lowell Wood’s name associated with projects of the sort that IV is working on. Wood, like Teller before him, built his career around the “boogeyman” of the era – the prospect of global nuclear war with the USSR. Once that threat passed, Wood was left without a “bad guy” to fight. I do not believe, from what I have read, that Wood is a fraud. Rather, I think that he is a true believer; he genuinely believes that he has the solutions to the potential and real woes that could and are plaguing mankind. So once the USSR threat ceased to exist, he moved on to the latest and greatest boogeyman – global warming & climate change, as well as, it appears, various diseases. I think that iconoclasts and entrepreneurs and inventors and tinkerers are wonderful. I just wish that more respect was given to those that actually have a track record of solving problems and that more skepticism was shown to those who have a track record of not solving them.
I can’t actually believe I’m going to write a “book review” on my blog, but I am. Or, rather, I’m going to write a review of two particular subsections of a book. I just finished reading, Superfreakonomics, the followup to the immensely popular and successful Freakonomics, both Dr. Steven Levitt and Stephen Dubner. I was a huge fan of the first book, and while I’d say I’m a fan of the second book, there were two particular instances I found troubling. In the first book, there was a pervasive attitude of, “I don’t care what you tell me, I care what the data tell me.” This lead to both revolutionary ideas – the crime drop of the 90s was a byproduct, in large part, of Roe v. Wade – and obvious ones that it was nice to have proof of – your real estate agent has his/her own best interest at heart, not yours. In the second book, this trend continued for the most part, with two notable exceptions. In both of these cases, Levitt and Dubner took some data basically at face value, and, in both cases, I don’t think the data underlying the arguments are strong enough to warrant that.
The first instance is in Chapter 2, when they reference the work of Anders Ericsson, father of the so-called “10,000hr Rule.” Ericsson’s work was the foundation for a lot of other books – Outliers, Talent is Overrated, and The Talent Code. Generally speaking, Andersson is of the belief that, to borrow from the title of Geoff Colvin’s book, “talent is overrated.” Or that what we think of as talent is often the result of other things like the “birthday bulge,” whereby a nine year old boy born on Jan 2 and a nine year old boy born on Dec 31 are considered the same age, as of Jan 1, with regards to sports teams, despite the fact that one is basically a full year older, or, to give some better perspective, the one boy is more than 10% older (at least at that young age). 10% is a lot. There is no denying the influence of that “extra year” on development. But Ericsson takes his conclusions further when he starts talking about deliberate practice – the source of the 10,000 “rule.” And this is where the data get a little thin. The Science of Sport bloggers, Dr. Ross Tucker and Dr. Jonathan Dugas of South Africa, pretty thoroughly dismantle Ericsson’s data, mostly by pointing out that his rule doesn’t present any standard deviation. The most effective example, I think, is Michael Phelps, who was 5th in the Olympic final for the 200m butterfly at the age of 15, roughly four years (i.e., well shy of 10,000hrs) after he started swimming competitively. Ultimately, by looking at the data – the same data that Levitt and Dubner should have considered but didn’t, largely because they gave the topic relatively short shrift due to it’s “extensive” coverage in the aforementioned books – Tucker and Dugas show that, unsurprisingly, the 10,000 hour rule is pretty much bunk. Some people need a lot more practice to “make it,” some a lot less, and some never make it regardless of how much they practice. It’s a very insightful two part series – very “freakonomics-esque” – that is covered in Part 1: Genes vs. Training and Part 2: Genes and performances. The basic summary is this: the data show that the standard deviation from 10,000 hours (which is indeed roughly the average amount of deliberate practice it takes to become a “master”) is so large as to basically totally discredit the idea that it is practice that makes champions. They conclude the second part with a nice quote, “To become an Olympic champion, the very best of the best, you need to tick the boxes. Genes is without a doubt one of those boxes. But so too are opportunities. And so is success genetics or training? It’s both. In fact, it’s 100% genetic, and 100% training.” As always, trust the data.
The second area is not so much an area where I think Levitt and Dubner didn’t do a good job with the data as much as they presented something where the source of said data maybe ought to be questioned. In particular, I’m talking about the obviously hot-button (pun intended) issue of Global Warming (or Climate Change), which is covered in Chapter 5. One of the major focuses of the chapter is on solutions to the problem of climate change that are proposed by Nathan Myhrvold’s Intellectual Ventures, a think-tank located in Bellevue that aims to solve the world’s problems. All of them. But on the topic of climate change, there is member of Myhrvold’s cadre of scientists who gets given a bit more a free pass than I think he ought to. Lowell Wood is a protege of Edward Teller, he of atomic bomb fame. Sort of. Wood and Teller, most recently, were the “brains” behind the “Star Wars” missile defense program. This program is trotted out in the book as evidence of Wood’s “credentials.” He started work at IV after leaving Lawrence Livermore National Laboratory. Anyone who is even vaguely familiar with the “Star Wars” program would label it an unmitigated disaster. Those who are even more familiar will tell you that it was doomed from the start. Robert L. Park was the director of the National Physical Society and is author of several books, including, Voodoo Science, which takes a very hard look at both Teller and Wood and their projects. Obviously, Park is just one man, and his opinion is certainly subject to bias. But what isn’t subject to bias is the data, because good data generally are (you can say “is” depending on how picky you want to be) immune from bias, at least if the data are collected correctly. Park is, like Levitt and Dubner, a “what’s the data say?” kind of guy. And, by more accounts that just Park’s, Star Wars was a failure. Park goes into detail quite thoroughly – skewering Teller and Wood pretty convincingly – in Voodoo Science. It’s a great read for anyone who is interested; I’d recommend it to Levitt and Dubner. But putting aside the obvious potential for bias involved in any general assessment of someone’s character (meaning Parks’ opinion of Wood and Teller), the most obvious thing to consider is that you have a guy – Wood – with basically the unlimited resources of the federal government behind him, and he couldn’t even demonstrate basic proof of concept of his signature project (Star Wars), and yet somehow this is a guy who we should look to solve climate change? Granted, being totally wrong about one idea doesn’t mean you’ll be totally wrong about another (though Wood is apparently trying to make Star Wars for mosquitoes, endeavoring to shoot malaria carrying mosquitoes with lasers instead of Soviet ICBMs), and there are skeptics – covered in the book – who have run similar models to Wood and gotten the same results. But when it comes time to execute a plan to implement said model, I’m not sure I’d take advice from a guy who has a relatively massive black mark on his record. What I found surprising is that both Levitt and Dubner seem like born skeptics. They seem totally unlikely to believe anyone. And yet I think they both, in a state of semi-reverence, showed an undue amount of faith in someone who, by all accounts, has wasted an extraordinary amount of tax payer dollars. At least in this case, he might only waste private money. But I would have hoped for the same sort of skepticism that was showed towards police chiefs and others who have made bold claims that the data do not support. I guess, as often seems to be the case with folks like these, the *actual* (as opposed to theoretical) data are always coming.
Ultimately, I was a bit disappointed in the book, because it seemed to be a case of, “what will be popular,” as opposed to the original book which is, “what do we think is interesting.” The climate change chapter seems to me to the most obvious example of this, mostly because the data are so unreliable on that topic. As Myhrvold himself says, paraphrasing, it’ll take us as long to develop the computers that can model climate change as it will to simply see what happens. And, given that, I think it should be obvious that the data in such a case cannot be trusted. But, I guess, Levitt and Dubner were simply demonstrating that, as always, people respond to incentives. And, in this case, the incentive was just to sell more books.