Respectfully, Nate Silver Is Still Wrong: The Ticker
Nate Silver had a long post on his FiveThirtyEight blog yesterday responding to my column about the impact that campaigns, rather than extrinsic factors such as the economy or an opponent's ideology, have on election results.
Two thoughts jump out immediately when I read his piece. First, I was unquestionably correct when I referred to Silver as “brilliant.” Second, we agree far more than we disagree (as he notes in his post). The latter will surely come as a disappointment to a number of political scientists who were looking for a smackdown, if my Twitter traffic of the past 48 hours is any indication.
A few points merit some elaboration:
It goes without saying that underlying factors play a big part in election results: Looking at the three metrics that Silver cited in his piece, it is obviously easier to get an incumbent president re-elected if people approve of the job he has done, the economy is forecast to grow, and his opponent is seen as being on one of the far poles of the ideological spectrum. Barry Goldwater had no strategy to beat Lyndon Johnson, for certain.
But if that was all Silver was arguing, then I doubt the New York Times would have given him several thousand words and a Sunday Magazine cover story to say it. What my column was reacting to was his extensive efforts to apply this wisdom with mathematical precision. One example was the conclusion that if gross domestic product growth in 2012 were 0.0 percent, and the president’s approval rating was 43 percent, then Texas Governor Rick Perry, as the Republican nominee, would have a 59 percent chance of winning. (More on that specific forecast later.) Starting with the headline, “Is Obama Toast: Handicapping the 2012 Election,” the crux of the piece, as I read it, was to suggest that models relying on “objective” data could forecast the election outcome, even before we know how the campaign is conducted, or how the candidates perform.
The gravamen of my disagreement with this contention isn't -- as some loyal Silver-ites have tweeted -- a rejection of, or ignorance about, social science. Rather, it's just the opposite: My critique flows from the fact that social science needs extensive data, assembled under reasonably controlled or constant conditions, to be truly scientific. Presidential campaigns, held on an extremely episodic basis (once every four years), under vastly varying conditions, simply don’t produce enough reliable data in a given time frame to make the exercise of forecasting based on that input meaningful.
Silver acknowledges this in his post. He even draws on baseball as a metaphor to provide a better illustration of the problem than I did. Silver notes that baseball lends itself toward Moneyball-style analysis more readily because players participate in 162 games a year; by contrast, presidential elections offer data for would-be “political Moneyballers” just once every four years. In terms of the specific question at hand (i.e., the prospects that an incumbent will be re-elected), a typical Major League player has more at-bats in a single week (more than 20) than the number of times incumbent presidents in either party have sought reelection in the past 100 years (14).
And even this point understates how much easier it is to model baseball than presidential campaigns. How reliable would baseball statistics be in forecasting future performance if players used wildly different equipment from game to game, played some games in hurricanes and others in ice storms, and the mound was a different distance from homeplate in every stadium? Yet this is a perfect way to describe how presidential campaigns are conducted, with incredible variation in the fundamental rules of the game, technologies and conditions. Virtually nothing about the $1 billion, super high-tech, super-targeted, super-organized, effort to re-elect President Barack Obama will resemble Gerald Ford's campaign of 1976 -- let alone Harry Truman’s in 1948. In baseball, in 2012, they will still be using wooden bats, for goodness' sake!
All of this is to say that the people who want to Moneyball-ize presidential campaign forecasting should show a great deal of modesty about the applicability of their tools because they just don’t have enough data collected under comparable conditions to make the exercise reliable. Now, in fairness to the “wonk” types, it should be pointed out that the “hacks” engage in a similar fallacy with their endless commentary about how a given election is a reiteration of a past race. Look no further than the current conventional wisdom that 2012 will be 2004 in reverse, with Obama in the role of President George W. Bush, and the would-be Republican candidate, Massachusetts Governor Mitt Romney, playing the part of the Democratic challenger, Senator John Kerry of Massachusetts. It's a tempting form of analysis -- and one that I have succumbed to from time to time -- but highly dubious, just the same.
What I would say to both the data jocks and the pundit class is this: Presidential campaigns are sui generis, happening too rarely, with too much variation in circumstances and rules, and always subject to unpredictable events, to place much stock in historical analogies or statistical models. Underlying facts and conditions matter, of course. But in reasonably close elections -- and the vast majority of our presidential contests come down to a shift of less than 5 percent of the electorate, one way or the other -- the performance of candidates and their campaigns matter a great deal, and can be decisive. That was my single most important contention, and Silver’s post seemed to acknowledge as much. I will take that as a happy point of agreement and end the debate there.
I have to go back to Silver’s forecast that under conditions of zero GDP growth and an Obama approval rating of 43 percent, Perry would have a 59 percent chance of winning. Sadly, because Perry’s current odds of being the Republican nominee are approaching zero, and happily, because both the GDP number and presidential approval rating will probably outperform Silver's predicates, there is no way of testing his forecast. But I do wish we could employ something like the “Holodeck” from Star Trek to hold such an election, in which case I would readily give Silver and any of his adherents even money on the outcome (even though his model suggests that I should get odds).
Candidates and campaigns can't outperform overwhelming circumstances against them, no doubt. But anyone who has watched Perry’s performance in the campaign to date surely wouldn't want to put their money on the model, I suspect. The candidates and their campaigns matter. Period.