National Academies Press: OpenBook

Survey Automation: Report and Workshop Proceedings (2003)

Chapter: Prospects for Survey Data Collection Using Pen-Based Computers

« Previous: Interface of Survey Methods with Geographic Information
Suggested Citation:"Prospects for Survey Data Collection Using Pen-Based Computers." National Research Council. 2003. Survey Automation: Report and Workshop Proceedings. Washington, DC: The National Academies Press. doi: 10.17226/10695.
×

COHEN: Sorry; basically, urban versus rural.

NUSSER: Well, in the urban areas you’re more likely to be using maps. And in rural areas you’re more likely to need to rely on photographs. But I still maintain that there’s still a lot of overlap; one of the areas that we’re proposing to focus on is new developments. You have— TIGER isn’t going to have lines in there for the new roads, necessarily. And your urban datasets on the streets may or may not have those; you may have a recent photograph, however, and you might want the enumerator or some staff member to record what those lines are, either by GPS or by annotation. Harry?

ROBINSON: Partly a comment or a perspective … your dichotomy of route-based versus map-based maps very well onto the different ways that people use and test applications. When people aren’t familiar with an application, we give them a tutorial and they work step-by-step through it. The same way when they test: they record step-by-step. And as they get more cognizant they get more of a map-like view.

NUSSER: Yes; I think actually if you give interviewers instruments, it’s the same thing as well. Their thinking, over time and with experience, becomes more of a navigation view in mind. Yes, Miron?

STRAF: If you can track across time, have you looked at any effects of climate change?

NUSSER: No, we haven’t looked at that at all; that’s a big area.

CORK: Thank you, Sarah.

PROSPECTS FOR SURVEY DATA COLLECTION USING PEN-BASED COMPUTERS

Jay Levinsohn and Martin Meyer

CORK: Continuing and building on some of the themes addressed in the last talk, we invited Marty Meyer and Jay Levinsohn to talk about prospects for data collection using handheld portable pen-based computers. Jay is the manager of technology issues and Marty is a research programmer and analyst at RTI, and—apparently, in the tag-team situation here—Jay will be doing the presenting today.

LEVINSOHN: Good morning. We’re going to talk about … we’ve been talking mostly, the past couple of days, about things to put on devices. We’re at the other end of this: after you’ve done all this development and after you’ve designed, what are the tools you can take in the field and what equipment can you use to display and to show the items you’ve developed? And, in sort of a global sense, what role, if any, do these newer and smaller devices play in survey data collection? Can handhelds be used effectively in the field for survey data?

Suggested Citation:"Prospects for Survey Data Collection Using Pen-Based Computers." National Research Council. 2003. Survey Automation: Report and Workshop Proceedings. Washington, DC: The National Academies Press. doi: 10.17226/10695.
×

And what’s out there. Below that, what are the software design and data transmission issues that need to be considered if you’re thinking about this equipment?

It’s relatively early. You know, handheld devices have been out for a while; you’ve seen some of the antiques being used. The Newton devices have held, I think, their ninth birthday this year. So there have been devices out there for that long, but their appearance in the market, their volume and their pricing, have begun to change very, very rapidly in the past 12 months. The Newton was a $1,000 device when it came out, and there is pretty much nothing in that price range for these small handhelds now.

So, we see the use of the handheld whenever the full laptop is not an option. There are certainly differences in capability and form factor for these devices, and I think you’d want to make the decision to use a handheld where it’s most effective. Clearly, for doorway screening, where you need a light, portable device with a good screen; hospitals and medical clinics, where you’re moving around and very mobile, going from room to room collecting data. Another example is diary applications, where people are recording information through the day. There have been studies, EPA studies, where you want to report your environment all day to assess your exposure to chemicals. Or, if you’re working in a plant, how many hours did you spend in this room versus that room? How much were you exposed? These are easy devices to carry around to do that. In market research, people tracking you down in the mall and getting to you. Fundamentally, any situation where the data collector is on the move, needs to be standing during the interview, and needs something small and light.

So far, it looks like you would need to restrict your attention to shorter and simpler questionnaires. Some of the—Moore’s Law will probably have some impact. These machines are expanding in capability fairly rapidly, certainly in data storage capacity, and the ability to plug things into them to expand their capability is improving very rapidly. I haven’t seen so much change in the speed of these devices as you see in other machines, and that may be related to the effort to keep battery consumption down and weight down. And maybe because the market is still relatively small. But I think that may change and you may start to see this doubling of capacity every 12 to 18 months.

Based on our experience at RTI, we’ve been using as a handheld the Apple Newton which—in many ways—is a very innovative and very clever device. It’s small; it’s light, about a pound and a half. It has a large screen, by today’s handheld screen size standards. And it’s been around for a long time. We’re using this in the National Survey on Drug Use and Health, and we’ve had it in the field for three and a half

Suggested Citation:"Prospects for Survey Data Collection Using Pen-Based Computers." National Research Council. 2003. Survey Automation: Report and Workshop Proceedings. Washington, DC: The National Academies Press. doi: 10.17226/10695.
×

years. So we have collected a large amount of in-field experience with this device—over two million hours of use—and it’s been positive. The device has held up; the interviewers have learned how to use it; we can maintain it. Even though it’s out of production, and the software is frozen, it’s still a working device. Apple is still maintaining the device. So our experience with it has been good. We’re doing fairly complicated screening with it; sample selection is built into the screening algorithm. And we’re using it nationally, in every state, so we have used it in every climatic condition you can think of: freezing in Alaska in the winter, summers in DC and New Orleans, and we have people in Hawaii. So it’s done very well. Probably, its utility and repair rate is half that of a laptop. It has no moving parts; it seems to take a beating fairly well. We bought 1,700 of these devices, and over a four-year span have lost about 100 due to getting run over by baggage trucks, being left on top of the car. But, in general, they’re doing very well.

So we’d say that the answer to “for this class of device, is it useful in the field?” is yes. We’ve been happy with those results.

We’re also beginning the search for a replacement for this product. The Newton, and the Newton developers, have moved on to Palm. They’ve moved from Apple to start the Palm [platform], they’ve moved to Mindspring. This is sort of the seed device in this market, and the products today are still vaguely reminiscent—there’s a Newton flavor in all of them. More so in the Palm class, but certainly in the others. So we’re looking for a more modern device that’s faster, that has more capacity; the memory capability on the Newton is somewhat limited, and the speed is slow. What we’ve found is that we’ve filled the Newton in terms of software capability, and speed has been an issue for some of our interviewers who have a large workload. Some percentage of our interviewers, when they get up to a large number of cases, it begins to bog the Newton down more than we would like. Plus, it’s not in production [anymore], so we need to look at new devices.

The questions are, do we want one … Currently, on the drug survey, we’re using two devices. We do doorway screening with the Newton; we go in the house and do interviews with a regular production laptop. Do we want to look at that model, or do we want to move toward a single device?

So, we’ve set up some target requirements for evaluating these devices: what kind of device do you want for doorway screening or a very mobile activity? What kind of device do you want to do an in-person interview with a pretty broad range of respondents? Our age range is from young to old, good vision to bad vision, good reading skills to bad reading skills. So the population for the interview is pretty broad, and it can be a pretty lengthy interview—anywhere from 20 to 70 minutes. So

Suggested Citation:"Prospects for Survey Data Collection Using Pen-Based Computers." National Research Council. 2003. Survey Automation: Report and Workshop Proceedings. Washington, DC: The National Academies Press. doi: 10.17226/10695.
×

do you want one device to do that, or do you look at two?

These are sort of a list of things we looked at for this application; perhaps the column on the left is a more general thing to look at. These are things you need to be concerned about in evaluating a device. Battery life, screen readability, can you hold it in one hand. Durability—will it hold up? What happens if you drop it? What happens if it gets wet? What happens when it’s 105°? Most of these devices have problems above 100° and below 20°. And as you go further away from those extremes they begin to stop. At 110°, 115° the LCD screens tend to … you can’t see them. They turn black. Or, when it gets to 5° and the wind’s blowing in Alaska, the screens again lose their ability to display. You’re not going to be doing much doorway screening in those conditions, though. [laughter] They’ re working at it …

PARTICIPANT: Do the interviewers tend to get frozen?

LEVINSOHN: The interviewers give out first, that’s true. You know, I think there’s a human effect: if the people can stand it outside, the devices will typically work.

There are also things you need to evaluate, in how it looks under indoor and outdoor light, and how it deals with very strong direct light. If you’re in a very sunny environment, some of the LCD screens don’t do that very well. And you want to look at what communications tools are there, how much memory can you get in it. We feel like there should be permanent storage; need permanent storage in these devices. Some of the Palms, some of the other devices optionally have a permanent storage. On the simple Palm device, if the power goes off, you’ve lost all the things in it, unless you have a permanent storage device. But most of them now have a slot you can plug something in to, and the things that go in there are getting bigger every week. And cheaper.

Vendor support is an issue. We’ve been in the field with a discontinued item since we started. It’s worked, but it’s a little scary. Nothing can change, so if there had been sort of a sea change in transmission protocols or maintenance procedures, it would have left us at risk at some point. It’s a little hard to guess; even big players can drop an item. So the market is pretty volatile in these devices, and I think it’s hard to guess. So it may not be a given vendor—it may not be Panasonic you’re worried about—but [are] there at least going to be a couple of vendors in this market providing alternate equipment and supporting protocols, software, replacement parts, and hardware?

Operating systems—you know, you may not have a lot of choices with a handheld device. They tend to be more esoteric operating systems, and you’re going to end up programming in that environment. In the laptop environment, you might be looking at a Windows environment—and if you’re looking at Blaise or standard questionnaire devel-

Suggested Citation:"Prospects for Survey Data Collection Using Pen-Based Computers." National Research Council. 2003. Survey Automation: Report and Workshop Proceedings. Washington, DC: The National Academies Press. doi: 10.17226/10695.
×

opment package—you might have to have a Windows 98/2000/XP-type operating system.

So what’s the landscape for these devices? There are Palm-class devices which—now—there’s a number of brands. They’re this size form factor, 3 inch screen. They come in black-and-white or color, and the displays are pretty good and pretty readable. They are small; extremely light. Depending on the models, very, very good battery life; some of these will, with a couple of AA batteries, last days or weeks. The newer models are rechargeable; they’re certainly good for a day’s work. They have good capability. They tend to be relatively slow and have small amounts of memory, but you can plug in cards with the newer Palms. And there’s a good mix of manufacturers, several people; Palm makes these, Mindspring makes these. There’s a new vendor that pops up all the time.

The next class is something called a Pocket PC. This is, in some ways … there are emerging standards for these devices. They’re bigger in terms of screen size; they’re bigger in terms of processor speed. There is a significant difference in the processor speed. The 200 Mhz are the top end of these devices; some of the earlier Pocket PCs are slower. They have more memory; they have a different operating system, and the screen is significantly bigger. And they also come in color or black-and-white.

We talked about clamshell or hinged handheld PCs. We don’t have one of those to display, but by clamshell we mean a device that folds in half and the screen would be the top half and the keyboard the bottom half. Sort of like a PC, this format, but much smaller; maybe a half or a third of a PC. A lot of those run on CE; you can also find them for Windows.

Then there are Windows CE tablets, and this is an example of a Fujitsu tablet. This is an older-model Fujitsu. It has a bigger screen, so now you get to see a significantly bigger screen size. It would have the same computing capabilities as some of these devices, but it has a nice large screen.

Then there are what we call Windows tablets, which are bigger machines. This is an example of a Fujitsu tablet that has a somewhat smaller screen size. And this is the big Fujitsu that has the biggest screen. All these differ in weight; we have a dramatic change in weight and size and portability and battery life between these two formats. But you have a huge screen difference. This one is a Windows 2000 capable machine; they’re reasonably fast computers, 400 to 800 MHz machines so that they’re beginning to rival what’s on a desktop; an 800 MHz machine is a pretty current desktop PC. They have large amounts of internal memory, RAM, and they have a hard disk. So a big change … the CE class

Suggested Citation:"Prospects for Survey Data Collection Using Pen-Based Computers." National Research Council. 2003. Survey Automation: Report and Workshop Proceedings. Washington, DC: The National Academies Press. doi: 10.17226/10695.
×

machines generally do not have a hard disk, they’re using some form of volatile or non-volatile memory internally. So that might have some impact on your ability.

Then there are sub-notebooks and miniature laptops, which are just small laptops. They tend to be, oh, they vary in size, but they’re typically a laptop configuration and they’re hinged.

Finally, this last class is relatively new, and still the form is emerging. The standards, it’s not clear what they will be. These are smart cell phones—cell phones built on top of a [personal digital assistant (PDA)]. So you might have a cell phone sitting on top, or inside, a PDA; it might be a Palm operating system. Not only do you have a phone built-in, you could also have a Palm OS built in. There would be messaging capabilities and abilities to send and receive built into the phone, and capability to program and connect those two devices together. There’s a class of wireless e-mail devices, like the Blackberry and some others, that might be appropriate for small data collection formats. I’m not sure that any of the federal-size surveys we’ve talked about here could use these smaller devices, but they’re interesting and they’re changing quite quickly.

Of course, wait ’til next month [laughter] and this list will be longer. The handheld of this is now the volatile part of the marketplace. A few years ago, you saw this kind of development in laptops, where every week someone was coming out with a bigger screen size or larger hard disk. They’d finally get high-end Pentium processors into the laptop and, every month, the manufacturers were beating themselves to the market with another device. That’s exactly what’s happening now. So, all these devices that we bought—I don’t think you can buy any of them now, and we just bought these six or seven months ago. All new models; I guess the Fujitsu is still the same. But the Pancentra is really not on their main market list. This is their newest device, the sort of mid-size screen; I think this is a Pentium 600, so it’s a pretty high-end computer, and a small—very good—screen. The Compaq and all the Palms we bought, the Pocket PC models we bought, have all been leapfrogged by other offerings, both from the same vendors and from other vendors.

It’s very rapid, and—if you’re going to pick a device—you’re going to have to take a guess. I think you can pick an area, like the Pocket PC, and assume that that environment and those standards are going to be there, but the device you do your development on is not the one you’re going to be going into the field with. And it may not even be close, if you’ve got a long development cycle.

There are a class of design and performance trade-offs that you have to evaluate based on the application you’re looking at. The different screen types … the screens come in different types. There are some

Suggested Citation:"Prospects for Survey Data Collection Using Pen-Based Computers." National Research Council. 2003. Survey Automation: Report and Workshop Proceedings. Washington, DC: The National Academies Press. doi: 10.17226/10695.
×

that are optimized for indoor use, that are backlit, that will work in a dark room, look real good inside—and disappear when you go outside. There are outdoor screens that are just the opposite; they work very well under bright light that’s reflected off of the screen and are very good, but go into a dim hallway and they’re not very good. And then there are screens that are built with some backlighting or edge-lighting that try to do both. So you have to evaluate that in terms of the application going on. If you do a lot of outside work, that’s more important; if you’re all inside, you may want the best indoor screen you can get. Display size and resolution is really a form factor issue; there are surprising standards within the devices. No one has really come up with a Pocket PC that has a really super 1280×1024 resolution; they’re sticking within that framework. Permanent storage capability, capacity, are trade-offs between these devices. Battery life, clearly, is a big difference; the battery life on these devices with a hard drive, with a big screen, with backlighting, are going to be nothing compared to smaller devices like the Palm.

So you may need multiple batteries, you may need battery management. The manufacturers … that may be today’s big lie, how long the batteries last. After “the check’s in the mail,” the “four-hour battery life” may be the next big lie. You can get four hours, maybe, but it depends very much on what you’re doing, and if you’re using the device continuously. On the health survey that we’re running, we have .WAV files and a lot of disk access that may go for an hour and 10, 15 minutes. And that’s a pretty consumptive activity; we’re spinning everything in the machine, and in the laptops we have, the battery lives don’t last very long. So it’s a question you want to evaluate and if it’s not going to meet your needs, you have the issue of managing multiple batteries— buying batteries, which are expensive, and providing some tool for your interviewers to charge them. You might have to buy one of these little “toasters” for them to put the batteries in and charge them up at night, or they could get car adapters and charge them in the field if they can do that, or they can go to lunch and find some place where they can plug up.

Durability is an issue. My suspicion is that these devices will be like the Newton; there [are] no moving parts, they’re relatively light, and they’ll probably hold up pretty well. And there is a lot of experience in the field with these devices, you know, as a commercial application. As you move up in screen size, get a little more parts on it and a few more cards in it, it may get less durable. With the bigger devices, like this Fujitsu, if we drop it, I don’t know. [laughter] And I’m not going to do it. We couldn’t get the Fujitsu engineers who came to do this test for us, either. They’re reasonably well-constructed, but they have a lot of parts

Suggested Citation:"Prospects for Survey Data Collection Using Pen-Based Computers." National Research Council. 2003. Survey Automation: Report and Workshop Proceedings. Washington, DC: The National Academies Press. doi: 10.17226/10695.
×

in them and the screens are fragile. The screens are big and if you drop it on a corner, I think it’s history. They’re relatively expensive devices; this is well over $2,000.

Size and weight are an issue; transmission capabilities—what devices are on there and what speeds will they run at. Software and questionnaire development tools are all things that need to be balanced in picking a device.

This table sort of let us put these things together in one place, and one of the questions we had in evaluating a new device for the National Survey on Drug Use and Health was: should we stick with a handheld device for screening and a laptop for indoors, or should we try to do all of it with this device [atablet]? Go to the doorway, screen, go indoors, set this up on a stand with a keyboard and turn it into the equivalent of a laptop. There would be a lot of advantage going back to one device; go to a stronger operating system, go to one device to cut down inventory. So we’re trying to evaluate that. This table sort of got at our sense, or initial thoughts, about whether there was one device that could do both things. Basically, we said that it’s only the tablet. We certainly couldn’t do the interview … we have 2 gigabytes of .WAV files that we play for the sound parts of the questionnaire, and it eliminates most things. If we only do one device, it would have to be a tablet. But the Pocket PC, we felt, would be viable for our screening operation; the Palm was a little too slow and a little small and—at the price differential—wasn’t enough to get too worked up over. If cost is a very big issue, then you’re looking at Palm devices. To get below $200 per unit, you’re looking at Palm.

So we felt that the Pocket PC would be fine for the screening operation and a likely replacement for the Newton—better in many ways. And we felt that the tablet would be a good device for doing both, if we needed to, with compromises. This is a little heavy to be at the doorway but, still, it would work. And we’re about to start focus group testing with our interview staff and with some elderly people to see how this screen works for visibility for doing an interview. And that’s going to be done by the end of the month.

Well, once you’ve made some decisions about the physical parameters of the machines, what are you going to do for development? The development tools are pretty good for these devices, and dramatically different than when the Newton came out. The Newton had a very limited set of development tools and a small developer community, and all of those guys are doing something else now. There are pretty good development tools for the Palm OS environment; the basic units are where a lot of development work is being done. There’s CodeWarrior, which is a C-based tool. There are some free tools; there are a lot of other tools for the Palm environment. There are rapid application development tools;

Suggested Citation:"Prospects for Survey Data Collection Using Pen-Based Computers." National Research Council. 2003. Survey Automation: Report and Workshop Proceedings. Washington, DC: The National Academies Press. doi: 10.17226/10695.
×

there is a package called AppForge, which plugs into Microsoft Visual Studio. It’s very usable. There’s a Canadian company which makes something called MSBasic that’s easy for development. And there are lots of third-party tools. EntryWare is something I stumbled across recently, which is a questionnaire development tool which will allow you to author questionnaires and run them. All of these tools are, you know, emerging fairly rapidly—seem to be adequate for the task you want to do.

I’d say that there’s more software, more development tools, in the Palm environment than in the Pocket PC platform right now, but that will probably change. There are tools from Microsoft; there is a Microsoft standard for the Pocket PC, Microsoft is backing this device and moving in this area, and providing good support tools for it. So there are synchronization software and development tools, so that the Microsoft embedded tools … There are currently embedded C and embedded Visual Basic tools, and they will be built into the emerging Microsoft .net software. AppForge also makes a tool for the Pocket PC, and there’s a third-party Visual Basic compiler and other people doing it. There are some relational database packages out there and tools to get at, for example, SQL server or other tools on a network. And there are some Java and Sun tools available, as well. We’ve been doing prototyping using embedded Visual Basic, and some of those tools.

The C++ environment might be the heavier-duty tools that you’re really concerned about performance or doing real-time applications, or heavy number-crunching, where you’ve got to squeeze things out, squeeze performance. You’ll have somewhat more options and more control. The Visual Basic environment has less control and is less integrated into the system APIs and the other applications. That may change over time; people are beginning to do a lot of development for these devices, and I think it’s a big commercial market for software as these devices spread around.

This is a second page of different applications that could be done through these two tools.

Tablet PC development tools are the same tools you’re familiar with in desktop applications. Tablet PCs are running [Windows] NT, 2000, or XP—all run and do pretty well on those. If you move to the Tablet PC you have fewer limitations and less learning curve; developing for the Pocket PC or the Palm, you need emulators or you need the devices, you need to be able to move the software to the devices. You need to worry about—typically, for development on the Palm or the Pocket PC, you do your development on a PC, you compile your code, select a target device, and move it over. Or you have an emulator device that runs on your PC where you can emulate, do quick debugging, flash it

Suggested Citation:"Prospects for Survey Data Collection Using Pen-Based Computers." National Research Council. 2003. Survey Automation: Report and Workshop Proceedings. Washington, DC: The National Academies Press. doi: 10.17226/10695.
×

up to see what it’s going to look like—you still need to port it over to the device. The environments to do that are OK, but it’s a little more complicated than working directly on a PC. And you need to target … most of these devices have different chips in them, so there’s a different target set if you’re working for the Cassiopeia or the Compaq, or this one has a different RISC chip in it, and the compiler needs to be told, “Where am I going? How do you want me to compile this software?”

Desktop software ports easily to Tablet PCs, and it’s easy to move it. You can provide … pop-up keyboard stuff is easy to do, and you have the availability of these add-on wireless keyboards. One of the things that comes with these new devices is a set of new capabilities that has not been in the traditional laptop environment: they all have touch screens. That’s sort of one of the requirements in what we’ve collected, that they all have touch screens. They have handwriting recognition that’s not bad; may not be ready for formal data collection if you care … if small error rates are a problem, handwriting recognition still has a way to go. It might be great for signatures, for annotations, for things where the context will allow you to get the right answer. Single error mistakes in an address or a ZIP code or a Social Security number are a killer. But it allows you to get signatures and other graphic information that’s pretty good. You can add cameras; most of these devices plug in for video stuff. The new ViewSonic tablet that’s just been released has a built-in video camera, so you can do that. Global positioning hardware we’ve seen connected, and software is being developed for this that allows that activity. And they’re all either coming with built-in wireless or are easily adaptable for wireless, and the varieties of wireless are expanding pretty rapidly. So these capabilities are either new because they’re just getting out there or because of the form factor.

We wanted to talk briefly about options, now, for how you put these things together and how you connect your equipment in the field. This is also an evaluation that you need to do. Most of these devices are thought of as assistants to a desktop PC; for the commercial market, most of them are talking about applications where they’re going to be synchronized with a desktop PC. You sync your mail, you sync your schedule. They expect you to be attached to a PC and doing updates periodically. That’s a model you can use, so that you can transmit—and they provide good software that’s pretty good for those activities and also has outlets if you want to do your own file movement or data transmission. You can talk to a laptop and have your laptop talk—through traditional channels—back to your corporate LAN or other computer networks you’re working with. You can use the host PC to communicate with FTP or SQL server databases. That implies that you have a host PC around. Some applications, the handheld might be the only device

Suggested Citation:"Prospects for Survey Data Collection Using Pen-Based Computers." National Research Council. 2003. Survey Automation: Report and Workshop Proceedings. Washington, DC: The National Academies Press. doi: 10.17226/10695.
×

they have, so this model may or may not work. Using the canned sync functions, and the ActiveSync or HotSync software that comes with the packages, puts you in a tighter framework for development; you might want to develop your own software for that kind of communication.

Another scenario is that you upload data to the desktop and that you’re still doing your communication with dial-up. This communication can be done by wireless; there are infrared ports on these, and I think that all of these have infrared ports. This works for syncing, and it’s not a bad method. It’s built-in. You can use BlueTooth or 802.11.b Internet for high-speed links between these machines; you can do dial-up back to the LAN.

This third scenario is sort of a direct connection between handhelds, and these could be dial-up; this could be a network, so that if you were in a clinic and had wireless Ethernet in the building, you could be doing this. And you can have different communications, either to communications gateways or the corporate network. Pocket PCs support dial-up networking; like the desktop PC, [they have] RAS connections that allow you to dial-in and make connections. You can do e-mail, you can do data transfers. All of these are sort of another level of effort than doing it on a desktop; the desktop is older, it’s been all worked out, and there are lots of models. This stuff is all very new, and there are new utilities to support it. There’s clearly a learning curve and another level of effort to get this level of communication software in place. I wouldn’t say it’s canned; we’re beginning to look at that to see which model … if we went to a single device, we would be doing this from a Windows PC where we know how to do it. If we go to two devices, our decision is [whether] we’re going to have the handheld communicate to a laptop and let the laptop do the communications back to the company. Or do we want these indirectly. Right now, we’re using dial-up communications with the Newton, and our field force calls in practically every night from this machine, using 57 KB modems. It’s been very reliable, I’d say, for dial-up communications, which [still] means that you have lots of recalls and the line gets dropped depending on where you are—Iowa, it seems, is not a good place to do dial-up, must be the town our interviewer lives in. Over the national network, dial-up is not the same. And certainly things like wireless, cable modems, and DSL don’t have that kind of availability. But that model has worked very well with that device, and that model is available with the newer devices; they all are able to have a modem connected to them and do dial-up activities.

We’re looking at things like doing more complicated access, where they would be doing SQL server transactions, getting access to and from the database. We’re not thinking about doing this in real-time with wireless; I think in time that will be an option, and depending on the geogra-

Suggested Citation:"Prospects for Survey Data Collection Using Pen-Based Computers." National Research Council. 2003. Survey Automation: Report and Workshop Proceedings. Washington, DC: The National Academies Press. doi: 10.17226/10695.
×

phy it’s an option now; you can do cell-phone-like connections in some cities, you can do wireless Ethernet in other cities. But you can’t nationally; there’s no national wireless network that would be available to a national field force. It will come, but I think that the rate of expansion there will be slower; ability to get cable modem and DSL—two products that are pretty well understood, people know how to do them—is still very erratic, even within counties. It’s all market-driven; if there aren’t enough people there, you aren’t going to get it. Marty lives 6 miles from me; he can’t get DSL, he can’t get cable modem, he’s got a satellite. I can get either one because I’m inside the city limits, and the more rural you are the worse it gets. So I think those issues will cloud the wireless area for a while. I’m not confident about predicting where that will turn out; the rollout has been slower than I thought it would be. I thought it would be easier to get cable modem or DSL because I think it would be a good way to make a fortune for the phone companies. But they don’t think that way yet.

That brings us to this screen on future trends. OS vendors are driving the technology trends; they are developing standards for these, and that’s a good thing. There are hardware and software standards, and ergonomic standards, that are beginning to get there and become well-defined. Microsoft has taken the lead on these, where they’ve defined a standard for the Tablet PC and the Pocket PC. They’re talking about a smart phone standard. The Palm is well-developed, fairly mature, stable platform that’s been around, but the Pocket PC is giving them a lot of competition. If I were making projections, they’re beginning to eat the market they had. So that there’s going to be a lot of competition, and I see the Pocket PC as a very formidable competitor to the Palms, which could even move down to a niche market because of price competition. Pocket PCs are in the $300 to $600 range; Palms are in the $150 to $500 range. There’s a lot of overlap in the price, and the Pocket PC is a much more capable device. That’s a good and a bad thing; it’s a much more complicated device, and if you’re looking for a calendaring, simple e-mail, expense report machine the Palm may be it, because it’s simple. The Pocket PC has capability to do a bunch more things. Fujitsu has been the dominant player in the tablet market for a long time; that’s about to change, I believe. Maybe not their dominance, but the fact that there was no one else. Many vendors are threatening to release a machine; ViewSonic has already released one, and several other players have machines, and ViewSonic’s machine is significantly cheaper. So I expect the competition in that area to change pretty rapidly, and we’ll see some price reductions in those areas.

Convergence of technology seems to be on its way; the capabilities of these machines are all being put together. There are things that you can

Suggested Citation:"Prospects for Survey Data Collection Using Pen-Based Computers." National Research Council. 2003. Survey Automation: Report and Workshop Proceedings. Washington, DC: The National Academies Press. doi: 10.17226/10695.
×

do on all of them. The Buck Rogers idea might be that your cell phone will be more capable than these other devices in a few years. That’s likely to come, and the capability to run a large operating system like Windows on very small devices seems to be coming. I think that if battery consumption and speed weren’t such big issues that it could fit on these smaller devices. The data storage is a big thing; you need probably a gigabyte of storage to run one of these big operating systems, but that’s not far away from these devices. You can buy big plug-in non-volatile memory for these machines, so it’s something that you can look at or guess that this capability will continue to grow, and it will be more capable within the same machine. It may be that you can even expect a Windows environment or a Linux environment to be available, that you can run on these devices; I think I’ve recently seen a Linux-based PDA on the market.

Finally, this is sort of a price review on the machines we brought with us. The Tablet PCs are pretty expensive. And these are ballpark prices; these are not GSA prices or what you could negotiate if you were buying a large number of them. But it’s suggestive. Some of these handhelds— these purchase prices were five or six months old. This Casio is an industrial model of a Pocket PC; it’s got a rubber case, you can drop it, it’s got plugs so it can be used in the rain and won’t get wet. But it’s very expensive. These Palm PCs are down in the $150 to $350 range. As a sort of checkpoint, the Apple MessagePad was $1,000 to $1,200 when it came out—and it’s not as capable as any of these devices except that, as we’re seeming, screen real estate is pretty important, and this has a nice-sized screen. It’s only black-and-white, so it’s not as visible and you can’t code things in color. But that’s a basic review of the handheld price market and the tablet market at this point. I guess we could put the ViewSonic out here, at $2,000, and that’s just—we saw that vendor a month ago. Maybe not even a month ago, a few weeks ago. So I think there will be pressure on these prices.

Questions?

PARTICIPANT: Have you done any usability testing with your interviewers? I’m just curious; is there any point at which small is too small?

LEVINSOHN: We have scheduled focus groups in about 10 days, and we have two concerns: one, the question is, will you go in the field with this device or this one? And we’re asking: can you use it? Will you be comfortable doing that? Do you want to carry a big device? We’re going to ask whether they would do doorway screening with this; is it too heavy, would you use it. So we’re going to ask our interview staff. There is also concern that these screens are not as good or as big as a laptop—you can [get] a laptop with a 15- or 16-inch screen now. We’re

Suggested Citation:"Prospects for Survey Data Collection Using Pen-Based Computers." National Research Council. 2003. Survey Automation: Report and Workshop Proceedings. Washington, DC: The National Academies Press. doi: 10.17226/10695.
×

in the field now with a Gateway laptop with a 14-inch screen. And, at the time, that was a big, capable screen. These are 10-inch screens, so they’re smaller; so they’re going to old age homes, to collect elderly people together, have them do the interview on this and see if this is OK. Or how does this compare with doing the interview on the Gateway laptop with the bigger, brighter screen. I think there are two ergonomic issues that we know we are concerned about; the focus groups may tell us that there are some more. There’s certainly the durability issue, too, that we haven’t gotten a feel for. Yes?

PARTICIPANT: [inaudible; essentially, speculating that it seems like the best option now would appear to be the tablet PC, with the hopes that the units would become more cost-effective in time]

LEVINSOHN: I think that’s depending on your time frame; for this survey, we need a new device on January 1, 2004. So I’m not sure I’d be willing to do that, because we’d need to buy that device six months— depending on how bold you are, you’d back it up four to twelve months. In that time frame, I don’t know if I’d do that; in longer time frames, I really would. And porting—it depends. There is no Blaise for a Pocket PC; there, you could certainly write your own questionnaire administration tool. The tools, I believe, are strong enough to do that, but that’s a big development activity. So, I think that you might decide that, well, I’m going to develop for Windows and, if worse comes to worse, I’ll be working with something that’s bigger than I want it to be. You could take that path. For a lot of doorway screening, you might have to convince your interviewers to carry that extra weight. But, we’re carrying both devices now.

PARTICIPANT: [Unintelligible]

LEVINSOHN: The health study is a self-administered questionnaire, so that these devices would be turned around and given to the respondent to enter data on. I think these devices, certainly, would work with some training. I don’t think using the pen for entering a lot of information with a first time user is a great idea; there’s some training involved. It’s sort of like using a mouse if you’ve never done one, or using this little pointing stick in the middle of an IBM keyboard. First half-hour you use it, it goes all over the block … So it depends, some. If you’re going to approach somebody for the first time and hand them one of these devices and say, “Please answer these questions. It’s very sensitive information, so I don’t want to see you do it,” they may need some training on how to use the pen. Or you may want to develop your interface where it’s very simple and they just touch the odd thing. But I think that these devices are fine for collecting small amounts of information where, at this time, you don’t need a questionnaire development tool like CASES or Blaise or something else. There is this EntryWare

Suggested Citation:"Prospects for Survey Data Collection Using Pen-Based Computers." National Research Council. 2003. Survey Automation: Report and Workshop Proceedings. Washington, DC: The National Academies Press. doi: 10.17226/10695.
×

package; there are probably a couple of others you could do devices on.

You could put bar code scanners on this. So, these are beginning to show up in industry for sort of vertical applications, inventory applications. There, they do wireless transmission back to the database. So I think that there are … we’ll see tools for that. Yes?

PARTICIPANT: For PDA-class devices [unintelligible]

LEVINSOHN: Come again?

PARTICIPANT: For PDA-class devices, how important do you think color is?

LEVINSOHN: I think it’s important; it gives you another dimension. You don’t have much screen real estate, and I think one of the changes between designing for these 14-inch screens, 10-inch screens, and the 3-inch screens is that you give up things that you were used to doing. Even the width of your slider bar becomes an issue. Color gives you another encoding dimension, so you can use color to say, “red things mean this, blue things mean this.” And I wouldn’t like to give that up. I think it also improves visibility. The Compaq screens and the new HP handheld screens are very sharp and crisp with their color displays.

GROVES: I have sort of another question, I guess. You’ve had the Newtons for nine years …

LEVINSOHN: Five … they’ve been out for nine …

GROVES: Oh, OK. But, do you have a sense at this point in time that you’re getting better at predicting what ought to be on the next platform—what the winner is, what the loser is, and how far behind the cutting edge we should be to be wise in this?

LEVINSOHN: Well, I think it may be a little easier now than it was five years … there wasn’t much else. So, if you wanted a handheld five years ago, you were going to pick a Newton. If you want one now, I think you [should] say: do I want a Pocket PC? Do I want a Palm OS? Do I want a Windows-class machine? So I wouldn’t pick a vendor; I wouldn’t recommend that. But I think you can pick a Pocket PC standard, program to that, and then choose from whatever vendors have survived. HP bought Compaq or vice versa—one of those two devices may disappear next year. And they’re both probably—in my mind, at this point—at the top of the stack in handheld devices. But I can’t imagine that they’re going to continue on producing two and compete against each other.

But moving the software is pretty easy; anything we’ve developed so far, in our prototypes, moves from any one of these devices to another with a recompile, at worst. Sometimes you can just move the code if they’re the same chip. So I wouldn’t pick Cassiopeia, or Compaq—I’d say we’re developing for Pocket PC or that we’re developing for Windows and hope to get small enough—we’re going to rely on the ingenuity of the industry to give us a device. I think you need to … if you’re making that

Suggested Citation:"Prospects for Survey Data Collection Using Pen-Based Computers." National Research Council. 2003. Survey Automation: Report and Workshop Proceedings. Washington, DC: The National Academies Press. doi: 10.17226/10695.
×
Page 211
Suggested Citation:"Prospects for Survey Data Collection Using Pen-Based Computers." National Research Council. 2003. Survey Automation: Report and Workshop Proceedings. Washington, DC: The National Academies Press. doi: 10.17226/10695.
×
Page 212
Suggested Citation:"Prospects for Survey Data Collection Using Pen-Based Computers." National Research Council. 2003. Survey Automation: Report and Workshop Proceedings. Washington, DC: The National Academies Press. doi: 10.17226/10695.
×
Page 213
Suggested Citation:"Prospects for Survey Data Collection Using Pen-Based Computers." National Research Council. 2003. Survey Automation: Report and Workshop Proceedings. Washington, DC: The National Academies Press. doi: 10.17226/10695.
×
Page 214
Suggested Citation:"Prospects for Survey Data Collection Using Pen-Based Computers." National Research Council. 2003. Survey Automation: Report and Workshop Proceedings. Washington, DC: The National Academies Press. doi: 10.17226/10695.
×
Page 215
Suggested Citation:"Prospects for Survey Data Collection Using Pen-Based Computers." National Research Council. 2003. Survey Automation: Report and Workshop Proceedings. Washington, DC: The National Academies Press. doi: 10.17226/10695.
×
Page 216
Suggested Citation:"Prospects for Survey Data Collection Using Pen-Based Computers." National Research Council. 2003. Survey Automation: Report and Workshop Proceedings. Washington, DC: The National Academies Press. doi: 10.17226/10695.
×
Page 217
Suggested Citation:"Prospects for Survey Data Collection Using Pen-Based Computers." National Research Council. 2003. Survey Automation: Report and Workshop Proceedings. Washington, DC: The National Academies Press. doi: 10.17226/10695.
×
Page 218
Suggested Citation:"Prospects for Survey Data Collection Using Pen-Based Computers." National Research Council. 2003. Survey Automation: Report and Workshop Proceedings. Washington, DC: The National Academies Press. doi: 10.17226/10695.
×
Page 219
Suggested Citation:"Prospects for Survey Data Collection Using Pen-Based Computers." National Research Council. 2003. Survey Automation: Report and Workshop Proceedings. Washington, DC: The National Academies Press. doi: 10.17226/10695.
×
Page 220
Suggested Citation:"Prospects for Survey Data Collection Using Pen-Based Computers." National Research Council. 2003. Survey Automation: Report and Workshop Proceedings. Washington, DC: The National Academies Press. doi: 10.17226/10695.
×
Page 221
Suggested Citation:"Prospects for Survey Data Collection Using Pen-Based Computers." National Research Council. 2003. Survey Automation: Report and Workshop Proceedings. Washington, DC: The National Academies Press. doi: 10.17226/10695.
×
Page 222
Suggested Citation:"Prospects for Survey Data Collection Using Pen-Based Computers." National Research Council. 2003. Survey Automation: Report and Workshop Proceedings. Washington, DC: The National Academies Press. doi: 10.17226/10695.
×
Page 223
Suggested Citation:"Prospects for Survey Data Collection Using Pen-Based Computers." National Research Council. 2003. Survey Automation: Report and Workshop Proceedings. Washington, DC: The National Academies Press. doi: 10.17226/10695.
×
Page 224
Suggested Citation:"Prospects for Survey Data Collection Using Pen-Based Computers." National Research Council. 2003. Survey Automation: Report and Workshop Proceedings. Washington, DC: The National Academies Press. doi: 10.17226/10695.
×
Page 225
Next: Panel Discussion: How Can Computer Science and Survey Methodology Best Interact in the Future »
Survey Automation: Report and Workshop Proceedings Get This Book
×
Buy Paperback | $80.00 Buy Ebook | $64.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

For over 100 years, the evolution of modern survey methodology—using the theory of representative sampling to make interferences from a part of the population to the whole—has been paralleled by a drive toward automation, harnessing technology and computerization to make parts of the survey process easier, faster, and better. The availability of portable computers in the late 1980s ushered in computer-assisted personal interviewing (CAPI), in which interviewers administer a survey instrument to respondents using a computerized version of the questionnaire on a portable laptop computer. Computer assisted interviewing (CAI) methods have proven to be extremely useful and beneficial in survey administration. However, the practical problems encountered in documentation and testing CAI instruments suggest that this is an opportune time to reexamine not only the process of developing CAI instruments but also the future directions of survey automation writ large.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!