Survey Automation Report and Workshop Proceedings (2003) / Chapter Skim
Currently Skimming:
Panel Discussion: How Can Computer Science and Survey Methodology Best Interact in the Future
Panel Discussion: How Can Computer Science and Survey Methodology Best Interact in the Future
Pages 226-246
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 226... ...
Each of the panelists will have just a few minutes to give their particular perspective about five to seven minutes give their perspective on material both of yesterday and today in the broad theme of, "Can we all get along? " The broad theme is: what can we as survey researchers and all of the panelists here are really survey researchers learn from computer science in the broad sense, partic
|
From page 227... ...
Tony Manners from the Office for National Statistics in the U.IC. And Susan Schechter from the Office of Management and Budget.
|
From page 228... ...
And so we need to think about reducing complexity. A good friend of mine who's a consultant used to say that the thing about survey researchers I've noticed in working with them he was not a survey researcher by trade is that they seem to derive great pride from managing complexity when they should be deriving pride from eliminating complexity, from simplification.
|
From page 229... ...
Informating systems are more designed for professionals really have more to do with empowering people, using technology to give people information so that they can do a better job at whatever it is we've chosen to do. It seems pretty clear that these systems have gone down the route of automating of turning face-toface interviewers into what we've done to CATI interviewers, which is the automaton route.
|
From page 230... ...
And I think that's not really the point of extreme programming, any more than it was the point of rapid prototyping. It is, granted, it's a way to come to grips with the fact that there's pressure to produce enormous amounts of software, with functionality that mostly no one wants or needs, as rapidly as possible.
|
From page 231... ...
In area sampling, which is used in most major household surveys in the U.S., once one gets down to the final stage, the traditional approach has been in a lot of these surveys to identify a relatively local area below a block group, typically and then within that area to train and send into the field field workers whose job is to basically in a very systematic way go through the area that's so designated and construct a list or a sampling frame of housing units for purposes of sampling and selecting the final sample of households. I'm from North Carolina and, in our state, what we're attempting to do is to mount an effort to produce an annual ongoing longitudinal survey of households in our state.
|
From page 232... ...
This is for a community, Carrboro, which is adjacent to Chapel Hill. Now the thing I wanted you to see in all of this is, first, the overlay of the property tax parcels to the TIGER maps, and then to also note the link to this of some data that was available from the property tax offices that provided information, in essence, on what they had things lilce number of buildings on the property, what the property value was, the type of strllctllre on the property, and so forth.
|
From page 233... ...
The other essential question we thought we needed to answer was: given that you have a sample of these parcels, and you send people to the field, will they be able to locate the correct person? That was probably the lacy question for us to answer and, again, our data from this limited field test suggests that the answer is yes.
|
From page 234... ...
. Our Labour Force Survey, which is lily your Current Population Survey, is a 40-minute interview.
|
From page 235... ...
We had, on our Expenditure and Food Survey which is a little lily your Consumer Expenditure Survey, but it's got nutrition as well that we produced basically 99 tables out of the instrument. It was literally 99, and it was a painful process.
|
From page 236... ...
Next, we'll hear from Susan Schechter from the Office of Management and Budget. SCHECHTER: I'm going to spend just a few minutes talking about the OMB process, from an official perspective, and then I'm going to give you a couple of my opinions on how that integrates with survey automation.
|
From page 237... ...
In fact, 80 percent of the entire burden that OMB approves during the year are IRS forms. SO, we're not talking just about a process that fits just the survey world; we're really talking about a process that fits to a tax report form, a birth form, any form of administrative reporting, a federal loan application, etc.
|
From page 238... ...
The agencies certainly have a responsibility to document their methods to OMB but a lot of that depends on the familiarity that the desk officer has with survey research. Some of the desk officers work much more with forms, not with surveys, with administrative lcinds of data collection, and they are trained in policy areas, in economics, etc.
|
From page 239... ...
I'm not sure, for example, what the current status of the National Crime Victimization Survey is, but the last time we reviewed it it was still paper-and-pencil because they didn't have the money to pay for the conversion to CAPI. So if OMB is not going to provide that money, and yet there's a legislative requirement to offer an automated reporting option, I guess at some point there's going to have to be a meeting of those minds.
|
From page 240... ...
I think that archiving will end 11p becoming more of an issue, because people are going to be able to go back and say, "Well, you based some rule on a study that was done that was unpublished I'd lilac to see some of that data. I'd lilac to be able to understand how you came to the conclusions that you came to." So I do think that, between this Government Paperwork Elimination Act and the Data Quality Initiative, it will impact the OMB processI'm just not sure how it will.
|
From page 241... ...
So what I think we should do is, many of the problems and many of solutions I heard yesterday are really on the order of one smart graduate student working over the summer to demonstrate how these lcinds of tools could be applied to a module of the survey enough to get people excited about the idea. The other final comment I would malce, and then I promise I will turn it over to the discussion, is that one of the things I learned yesterday is that despite being extremely complex, as the big surveys are they're not as complex in the sense that we can analyze them.
|
From page 242... ...
I would say that the statistical agencies themselves are probably going to come up with a common framework for how they're going to define and look at data quality. The essence of this particular statutory initiative was that rules that are passed that are very costly to implement are often based on studies that EPA may do or other agencies, that HHS may do, and then based on the results of rules they implement a legislative initiative that might require industry to spend a fair amount of money to do something.
|
From page 243... ...
I think one of the worst problems we've always had working with surveys, or any old data set, is that the documentation not only is incomplete but it hasn't been structured. And I thinly Tony Manners said that we need to think in terms of developing surveys within a structured system that captures those decision points, that captures the conceptual points as well as the survey development.
|
From page 244... ...
And it may be, and there's plenty of evidence to suggest, that if I'm that respondent I won't be fully forthright if you ask me the question in a conventional simplistic way. And I may have to find some more complex alternative such as preserving confidentiality through use of technology to remove identifying information some states have legal implications in that re
|
From page 245... ...
In the academic world of software engineering and computer science, there's very little emphasis on things lilac maintenance, support, and documentation. And everybody who has gone from teaching computer science to being VP of engineering or something which is my route you discover this and you lcind of look back and say, "Gee, I wish I had had a course that was teaching about documentation." And I don't lcnow how to get that back into play; it's a lcind of general problem that we seem to have a mismatch on that, and we see it there, too.
|
From page 246... ...
In the academic world of software engineering and computer science, there's very little emphasis on things lice maintenance, support, and documentation. And everybody who has gone from teaching computer science to being VP of engineering or something which is my route you discover this and you lcind of look back and say, "Gee, I wish I had had a course that was teaching about documentation." And I don't lcnow how to get that back into play; it's a land of general problem that we seem to have a mismatch on that, and we see it there, too.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.