August 14, 2006

Bizarre polling results

Over at TAPPED, there is a brief post about recent baffling polling results:

Via Billmon by way of The Arabist, I see that apparently 30 percent of Americans, according to a new survey, can't recall what year 9-11 happened. And five percent don't remember the day and month of 9-11. As Billmon asks, "I wonder how many of them know who's buried in Grant's tomb?"
Often, people quote results like these to lament the intelligence of the American public, but I think that assessment is far from the whole story. The chief problem with these polls is that they assume it is worth my time to stay on the phone with a pollster and think about the questions I am asked. Experimental economics calls this a flat-payoff problem. There is a cognitive cost associated with me thinking about what month and day 9/11 occurred on and as a result if I am a rational utility maximizer that doesn't care what the pollster on the phone thinks about me, then it is in my interest to just say whatever the hell I want to the pollster. Now I am not saying that this is how everyone should act, just that if 5% of subjects polled don't know what day and month 9/11 occurred on, it might not be a problem with them, but with the mechanism by which the information was elicited. This complaint is completely independent of a number of other problems that plagues polling, especially the sensitivity their results have to simple wording of the question.