The aggravating nature of survey data

11/16/2014 10:31:00 PM
Danielle Kurtzleben has a post about how Americans--and everyone else--massively overestimate the unemployment rate. The actual rate is 5.9 percent, yet the average survey response was 32 percent. All the other countries surveyed also massively overestimated their unemployment rates:
Survey respondents massively over estimate the unemployment rate in every country.
While it's fun to sit back and say "haha people are stupid," I think there is a perfectly reasonable explanation for why the respondents answered the way they did:
Currently, 40.8 percent of americans do not have a job.
In fact, while the American survey respondents estimated that the unemployment rate was 32 percent, more than 40 percent of Americans do not currently have a job.

This gets at the most frustrating thing about survey data. The public doesn't really define unemployment the same way economists do. Economists want a useful labor market metric, so we want exclude everyone who doesn't want a job--children, full-time students, people with serious disabilities, the retired, the fabulously wealthy--from our measure of unemployment. We do this by asking two questions in the Current Population Survey: "Do you have a job?" and "Are you looking for a job?" You are only counted as unemployed if the answer to the first is "no" and the second is "yes."

However, most of the public is not trained in how economists define unemployment, which is non-obvious and not necessarily intuitive. For the most part, the public defines the term based on intuition when they hear it, and "unemployment rate" sounds like it is the percent of the whole population that doesn't have a job, not a percent of the subset of the population that wants a job as economists define it. It's true that the survey Kurtzleben cited did explicitly define the term for respondents, but that's the nature of surveys: respondents tend not to read all of the question before answering. This is extremely frustrating to anyone who has ever attempted to design a survey for research, but extraordinarily common and to be expected.

So overall, I'd say there's nothing to see here. The estimate of 32 percent is quite reasonable--actually low-baling it a bit--of what the respondents thought they were being asked. That is all.