with most of it, hey, talk to Mick after the talk … The Web has really caught on, and it looks like an incredibly low cost means of survey data collection. So, the average face-to-face interview now costs, what, maybe $750 or $1,000, on average, to do a face-to-face interview in an area probability sample. It’s literally true—you can get, for $2 or $3 dollars a case, an interview on the Web. So lots of firms, particularly in the market research community, have rushed to adopt this new technology. Yet, I think we all think—those of us who have looked at it carefully— for government applications and high-end academic applications there’s a lot of research that’s going to be needed before federal agencies accept the Web.

And why is that? Well, there are several key issues involving sampling and coverage, non-response, and then there’s this host of measurement issues.

What are some of the sampling issues? For sampling purposes, there are two key obstacles for Web surveys. For most survey populations likely to be of interest to the federal government or academic survey researchers, there’s no good frame. There’s no frame of Internet users. And even if there were a good frame of Internet issues, it’s not clear … there are significant coverage problems; only a fraction of the population—a growing fraction—has Internet access. So for most surveys there would be huge coverage problems.

Because of these problems—the absence of a frame and the presence of these significant coverage problems—there have been four methods of sampling that have been adopted, mostly by the market research community. There are what might be called general invitation samples, volunteer panels, probability samples, and then—as a special case of probability samples—intercept samples, and I’m going to talk briefly about those four.

The general invitation samples are simply Web sites where you can go and fill out a survey, if that’s your cup of tea. I love this; there’s, for example, a Web site [called] MrPoll.com, and you can go there and register your opinion on various issues, at no charge to you. [laughter] These are the moral equivalent of those 900 polls on TV, except that they’re even cheaper to do, right? When you do a 900 poll, you actually pay a few bucks to participate. Here, it’s free. It’s a completely self-selected sample, so it’s a sample of convenience; it has unknown sampling bias and it’s obviously restricted to the Internet population. So, not a really good sample.

There are also volunteer panels. The largest of the volunteer panels— at least the largest of the volunteer panels that I’m aware of—is the Harris Interactive panel. I think they’re up to several hundred thousand members of the panel. And you can get a probability sub-sample of this

The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement