5
Life-Cycle and Training Issues

5.1 THE LIFE CYCLE FOR INFORMATION TECHNOLOGY SYSTEMS

The initial decision to procure an information technology (IT) system is only one dimension of the life cycle of that system. In the lexicon of information technology, the “life cycle” of a system begins with its initial purchase or acquisition—that is, when the system is first delivered. Concurrently, people must be trained to use, operate, and maintain the system. Problems in operation are inevitably discovered, ranging from small software bugs to major design flaws—and many of these problems must be fixed. Fixing a problem involves development of a putative fix and then testing the fix to determine that the problem is resolved and also that no other problems are introduced. In principle, the fix—or, more properly, the complete fixed system—must be legally certified under state law before and in time for an election. Then the problem fix must be deployed to the entire installed base of systems. In addition, new capabilities are often desired by the user, and a vendor may develop upgrades to accommodate those needs; upgrades must go through the same process of development, testing, and deployment as do problem fixes.

One of the most important dimensions is the cost and effort associated with continuing operation of the system over its expected lifetime. A second is that the expected lifetime of an information technology system may well be much shorter than budget-constrained state and local governments would either expect or prefer—and the Help America Vote Act



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 96
Asking the Right Questions About Electronic Voting 5 Life-Cycle and Training Issues 5.1 THE LIFE CYCLE FOR INFORMATION TECHNOLOGY SYSTEMS The initial decision to procure an information technology (IT) system is only one dimension of the life cycle of that system. In the lexicon of information technology, the “life cycle” of a system begins with its initial purchase or acquisition—that is, when the system is first delivered. Concurrently, people must be trained to use, operate, and maintain the system. Problems in operation are inevitably discovered, ranging from small software bugs to major design flaws—and many of these problems must be fixed. Fixing a problem involves development of a putative fix and then testing the fix to determine that the problem is resolved and also that no other problems are introduced. In principle, the fix—or, more properly, the complete fixed system—must be legally certified under state law before and in time for an election. Then the problem fix must be deployed to the entire installed base of systems. In addition, new capabilities are often desired by the user, and a vendor may develop upgrades to accommodate those needs; upgrades must go through the same process of development, testing, and deployment as do problem fixes. One of the most important dimensions is the cost and effort associated with continuing operation of the system over its expected lifetime. A second is that the expected lifetime of an information technology system may well be much shorter than budget-constrained state and local governments would either expect or prefer—and the Help America Vote Act

OCR for page 96
Asking the Right Questions About Electronic Voting of 2002 (HAVA) does not provide funding for continuing operations or system replacement. It is thus helpful to consider some of these other dimensions explicitly. 5-1. What is the life-cycle cost of any particular electronic voting system? The initial procurement cost of any information technology system is generally only a fraction of its total life-cycle cost, which also includes costs associated with operations, maintenance, upgrades, and training.1 (Put differently, within a few years of initial purchase, many voting jurisdictions have found that other nonprocurement expenditures exceed the initial purchase cost.) Moreover, there is generally considerable uncertainty about estimating or even identifying collateral costs. For example: Extra work by employees with high-tech skills may be required to support elections staff or poll workers in the field. Necessary security measures and security audits may well increase costs. Skilled program and contract managers with IT experience are generally needed, but may not already be on staff in the purchasing jurisdiction. County or municipality employees already drawing salaries may be used instead of other poll workers, thus rendering their costs invisible. Training and education expenses for in-house IT staff to develop an understanding of a system in enough detail to make authoritative statements about a system’s operational properties may increase costs. In addition, costs beyond initial procurement can increase dramatically in later years if vendor support for the purchased configuration is not available. Over some period of time, it is virtually inevitable that this will be the case, either because the vendor will have made available upgrades to the initially deployed system and no longer supports that system, or in less common instances because the vendor has simply gone out of business. Note also that upgrades are not necessarily a positive thing, 1   For example, various studies of the total cost of ownership of personal computers in a work environment suggest that acquisition costs are less than 20 percent of the total cost of ownership per year. Assuming a useful lifetime of 3 years, acquisition costs are well under 10 percent of the total cost of ownership over the entire lifetime of the system. See John Taylor Bailey and Stephen R. Heidt, “Why Is Total Cost of Ownership (TCO) Important?” Darwin Magazine Online, November 2003, available at http://www.darwinmag.com/read/110103/question74.html.

OCR for page 96
Asking the Right Questions About Electronic Voting or more precisely, they generally come with both costs and benefits. Upgrades often fix problems but have also been known to introduce new (and unanticipated) problems. 5-2. What assurances can a vendor offer with respect to long-term support? Given that elections happen relatively infrequently, continuity of the election process is an important requirement. Purchasers of electronic voting systems (that is, states or local election jurisdictions) must have assurances that a vendor will be able to support those systems for an extended period of time. (Historically, voting machines have had life-times measured in decades, but it is likely that any information technology system will be obsolescent and thus hard to support and maintain in much shorter time frames. Consider, for example, that the World Wide Web is now only 10 years old, and even automatic teller machines are programmed for replacement on a 15-year life cycle.) Purchasers might thus be concerned about issues such as the following: The presence of a sustainable business model and adequate capitalization that will allow a vendor to stay in business over the expected lifetime of the equipment. A vendor that goes out of business is not just a problem for its investors or owners; it is also a problem for the jurisdiction, because such a vendor will no longer be able to provide equipment and software support. The presence of a proven quality assurance infrastructure that can support the systems being sold over their entire life cycle. The cost of switching vendors in the event that a vendor goes out of business or proves unsatisfactory in its contract performance. A well-known strategy of vendors seeking business is to capture a purchaser with low initial costs and technology that makes it difficult for the purchaser to switch vendors later on. Such a strategy makes a great deal of sense from the vendor’s perspective, but it leaves purchasers more or less at the mercy of the vendor in the middle of the system’s life cycle. Contract provisions that ease the transition to another vendor should such a transition become necessary. For example, performance bonds are a common practice in the voting systems industry,2 but performance bonds are usually large enough to place significant barriers to entry for both existing companies and new entrants in the field. (A typical performance bond is a substantial fraction of the total 2   A performance bond is issued to one party of a contract (in this case, the purchaser) as a guarantee against the failure of the other party (in this case, the vendor) to meet obligations specified in the purchase contract.

OCR for page 96
Asking the Right Questions About Electronic Voting value of a contract—it may even be 100 percent of the contract—and it serves to incentivize the vendor to fulfill the terms of the contract and to enable the purchaser to find another vendor should the original vendor default.) Another practice is source code escrow, which calls for the deposit of source code files and appropriate documentation with a mutually trusted third party during and after the completion of the contract. In the event that the vendor becomes unable or unwilling to continue to provide service to the purchaser, the source code is released to the purchaser so that it can seek another party to assume the first vendor’s responsibilities. Source code escrow is a common, even routine, commercial practice. However, this practice alone does not provide complete protection against the risk of vendor difficulties. 5-3. What are alternatives to purchasing complete integrated voting systems? Outright purchase of an integrated electronic voting system is only one procurement model. Two other models are leasing rather than buying the system and purchasing election services rather than owning and operating voting machines. The first model may entail greater cost over the long run, while the second raises many questions about the appropriateness and legality of privatizing essential government services. A third approach is based on the procurement of individual components of a system from separate vendors (or lessors) that are subsequently integrated into a functional system. Such an approach requires the development and promulgation of standards for data and program interfaces that allow different functional modules to interoperate. 5-4. How difficult will it be to change vendors if the original vendor becomes unresponsive or too expensive? Vendors have many incentives to capture the loyalty of a purchaser, because a loyal customer represents a steady and predictable income stream. But there are many methods for building loyalty. Some are incentives—by offering the purchaser various services and perks not readily found elsewhere, a vendor can build a more enduring relationship with the purchaser. Others are disincentives—by forcing the purchaser to pay certain costs if it wishes to change vendors, a vendor may be able to lock in the loyalty of the purchaser even if it would be in the interest of the purchaser to change vendors in the absence of such disincentives. For example, if the vendor is the only source of expertise on the operation and maintenance of a system, the purchaser is necessarily dependent on the vendor for support. But purchasers who are highly dependent on a vendor for support tend to pay much more than if they have access to independent sources of expertise. In principle, a purchaser can develop such expertise in-house or can contract for it with third parties

OCR for page 96
Asking the Right Questions About Electronic Voting other than the vendor. In either case, the utility of this expertise is greatly enhanced by having access to the source code of the software running on the system in question. As a general rule, systems that are designed in accordance with widely accepted standards, in a modular fashion, and with clearly defined interfaces are easier to support and maintain in the long run, because other vendors can also design system components in the same way and thus increase the likelihood that those components will interoperate with system components already in place. 5-5. What logistical and administrative issues arise regarding the physical management of a voting system? Election officials are responsible for the physical handling of machines before, during, and after an election, and different kinds of equipment have different handling requirements. For example, some systems require storage in climate-controlled environments. For subsequent auditing purposes, electronic records (e.g., flash memory cards) may require storage. Paper records must be stored in fire-resistant containers; how should flash memory cards be stored? Electronic equipment is typically more delicate and fragile than nonelectronic equipment—what procedures need to be followed in moving units between their storage locations and polling sites? 5.2 POLL WORKER TRAINING Perhaps the most significant training issue that arises with electronic voting systems is the one associated with poll workers. Poll workers play an essential role in the electoral process today. Poll workers are individuals who assist with the polling process essentially on a volunteer basis (they are usually paid a token amount for their work on Election Day and for being trained, but in no sense can the job of poll worker be regarded as a significant income-producing job). Poll workers have many responsibilities on Election Day, including the following: Physically setting up the voting site on Election Day (placement of tables and the like), Turning on the individual voting stations being used before the polls open, Resolving problems with voting stations if such problems arise during polling hours, Checking voter registration and acting as gatekeeper for voter access to individual voting stations, Answering any questions that voters may have about the mechanics of voting, and

OCR for page 96
Asking the Right Questions About Electronic Voting Ensuring that ballots (or totals) are delivered properly to the tabulation authorities. Box 5.1 describes the experience of an actual poll worker in the 2004 election to provide some on-the-ground context for the realities of poll working. As obvious as the point is, it is sometimes forgotten that it is this myriad of on-the-ground experiences that shape the perceptions of ordinary citizens regarding an election. In the context of electronic voting systems, the range of things a poll worker might be responsible for doing in each of these categories is arguably even larger than when nonelectronic systems are used. This is not to say that every poll worker will necessarily experience a wider range—only that he or she must be trained to handle a larger number of contingencies. In general, poll workers must know how to use the systems at least as well as any voter would need to know, and they must know still more than that, because they will be the first line of assistance for voters who are confused about how the system works. Poll workers must know enough about the system in use to be able to recognize a problem that arises at a voting station, and then to take action to correct the problem. Against this backdrop, some important training questions arise: 5-6. What is the nature and extent of the training required to make poll workers sufficiently knowledgeable about an electronic voting system? The nature and extent of needed training depend significantly on the design and capabilities of the voting system in question. As a general rule, a system with greater functionality and that customizes its interactions to a voter’s needs will require more training.3 Poll workers may also need some knowledge about how to set up a voting station. For example, a voting station may need to be rebooted after it suffers a system crash, and a poll worker may be the only one available to do so promptly. A useful benchmark might be a comparison with the training required for poll workers prior to the introduction of electronic voting. If a significant amount of additional material must be covered in the same training time, training problems might be reasonably expected, especially if there are changes from year to year in operating procedures and interfaces (as is often the case with IT-based systems). New media for training, such as DVDs, videocassettes, and online Web-based education, may provide more complete poll worker training than has previously been possible. 3   Examples of customization include presentation in different languages, use by persons with disabilities, and ticket versus individual choice voting.

OCR for page 96
Asking the Right Questions About Electronic Voting Box 5.1 An On-the-Ground, First-Person Report of a Poll Worker I [Leslie Sussan, Montgomery County, Maryland] served twice as an election judge in Maryland, first as an assistant chief judge in the primary and then as a check-in judge in the General Election. My experience was that goodwill and good intentions were plentiful. Still, problems with the quality of the training, the conditions we had to work under, and the unintended effects of the voting systems themselves were very evident. Weeks before each election, election judges attended a mandatory 4-hour training session and were given a binder with instructions. The training sessions tended to highlight the many changes from the preceding election rather than proceeding through exactly what to do at each step. The effect was like the old joke of a native instructing a tourist to turn left where the red barn used to be. The first training I went to was almost all lecture, with only about 30 minutes’ practice with an actual voting machine we saw for the first time. The second training devoted more time to role plays, but the presentation of what to do was fast and cursory, and most people had not read the manuals beforehand, resulting in lots of confusion. Little effort was made to explain the purpose of particular documents or requirements; the rush to get through “what to do” left no time for “why.” The inadequacy of the training was evident when election judges tried to use their common sense to fill in the gaps in their memory and understanding. Check-in judges would ask to see identification from all voters, for example, because doing so seemed self-evidently reasonable to them. Yet, the law clearly required identification only from certain first-time voters. The judges at my precinct argued over when to give provisional ballots, because the guidance from the Board of Elections had changed between the two elections. Most of the election judges were at or well past retirement age. We were basically amateurs, rather than trained professionals, and were paid only a token amount. We met at the poll the night before in order to set up some materials that could be prepared in advance and in order to meet the “team” for the first time. On Election Day, we had to arrive by 6:00 a.m. and were not permitted to leave the premises at all until the election was completed. The polls closed when the last person in line at the end of the voting period finished voting, and the procedures to shut down the polling place and secure everything took until 10:00 p.m. It does not take much imagination to understand how easily errors could occur with people in their sixties, seventies, and eighties working for 14 to 16 hours straight (after working on set-up the night before as well) with few breaks and little food. During the day, we were supposed to be allowed occasional breaks to eat food we had brought in, while the chief judges substituted so as to maintain one election judge from each party at each station. This worked fine during the primary, but the high volume at the general election made it very difficult to keep the stations fully manned when the chiefs had to perform their other responsibilities too, such as handling provisional voters. Breaks became brief and rare, and some

OCR for page 96
Asking the Right Questions About Electronic Voting times single individuals tried (improperly) to keep lines moving while substitutes were juggled. Also, I was the only Spanish-speaking judge in a heavily immigrant precinct, and in the general election I often was called away from my post to translate (particularly because, although ballots were provided in Spanish, the audio versions available for blind or illiterate voters were not in Spanish and I had to read the ballot through for two voters). We had an amazing number of forms, checklists, tallies, and documents to maintain and double-check. So, the check-in judges were supposed to find a voter’s name in a voter registry listbook; find a matching voter authorization card (known as a VAC, a term unfortunately also applied to another document in the process) in a box; get signatures on both and initial them; make a mark on a running tally of the number of voters checked in by party registration; prepare an electronic plastic voting card; and send the person to the voting unit judge. The number of signatures in the log, VAC cards, and marks ought to all correlate. Then, the voting unit judge takes the voter to a machine; notes the number of the machine on the VAC, marks a running tally sheet on the machine; verifies that the electronic card pulls up a ballot properly; and returns the used VACs to the check-in judges. The number of marks on the tally sheets should match the electronic record in the machine of how many votes were cast on that machine. But every redundant record offers more opportunity for inconsistencies. A check-in judge who did not see the point of the tally by party in a general election focused instead on getting the line moving. A voting unit judge walked a voter to a machine and began to record the number of the machine on the VAC, when the voter asked a question. After answering the question, the judge showed the voter how to put in the plastic card and get a ballot and then tried hopelessly to remember whether or not she already marked the tally sheet for the machine. The biggest complaint by voters was the interminable waits, and most of all the apparent inequity because some segments of the alphabet seemed to have much shorter lines than others. The segments of the alphabet were set up in advance to provide for the number of pairs of check-in judges assigned to the precinct. Preprinted signs were set up for each line (say, A-G, or L-R), and the bound registry list books were divided into the same groups as were the boxes of VAC cards. It was impossible to readjust on the spot when it became clear that the voters turning out did not fall evenly into the assigned divisions, because the list books could not be disassembled. Binding the books reduces the risk of tampering by adding or removing pages, but the binding could be done by individual letters to allow some flexibility. The check-in judges were the ones whom the voters blamed. I would serve again, and there was something heartwarming about so many very ordinary people working so hard to make an election happen, but there was also quite a bit that was disturbing about seeing the sausage-making close up. SOURCE: Leslie Sussan, Montgomery County, Maryland.

OCR for page 96
Asking the Right Questions About Electronic Voting 5-7. How will election officials know that a poll worker has been adequately trained? As procedures become more complex and the possible contingencies more varied, an assessment of a poll worker’s knowledge may become necessary before he or she is selected to work at the polls (recognizing that requiring potential poll workers to undergo an assessment may well intimidate or discourage some of them from volunteering at all). A related issue is how the poll worker himself or herself will know about the adequacy of training. Poll workers who are conscientious may well be uncertain about various aspects of problem resolution and will want to know how to remedy those gaps in knowledge. 5-8. How will poll workers get help when unanticipated questions or issues arise? Almost independently of the training that a poll worker receives, it is virtually assured that some poll workers will encounter unanticipated problems during Election Day. Thus, some mechanism must be available to provide poll workers with assistance on a timescale that does not significantly interfere with the voting process. When these problems involve the operation of voting stations, the help mechanism will most likely be the responsibility of the vendor. 5-9. What is the nature of the help mechanism(s) provided by the vendor? Help mechanisms can take a variety of forms, and all may be relevant to a given situation. Vendors may provide documentation (e.g., sets of frequently asked questions) to help facilitate problem resolution, provide answers over a help line, or provide in-person support at the polling place. However, consider the following: For complex systems, documentation cannot be both comprehensive and easy to use. Furthermore, users must generally have some familiarity with the system in order to use documentation effectively. Though help lines can be quite effective in resolving simple problems, it is often difficult for a help line specialist to diagnose and provide advice on a more complex problem, especially when the specialist cannot see the station with the problem and the poll worker must describe the problem in words. In general, in-person assistance cannot be provided as rapidly as when help lines are used (assuming that help lines can handle peak call volumes). Also, though in-person assistance is usually the most efficacious method for problem resolution, it is also the most expensive and generally the least timely (because an individual must be dispatched to the appropriate location).4 4   In at least one locality in the 2004 election, a vendor put a very large number of field technicians on call to provide prompt service. If this was part of its contract with the locality in question, the question arises as to whether the contract provides for such staffing for the

OCR for page 96
Asking the Right Questions About Electronic Voting New technologies, such as chat rooms or instant messages, may provide new channels for responsive assistance. 5-10. What consequences flow from any vendor inability to provide adequate problem resolution on Election Day? Given that the vendor provides various assurances that its systems will be usable on Election Day for voting purposes, it is reasonable to ask about the strength of those assurances. For example, the vendor might be required to post a performance bond that is forfeited if a certain level of problem resolution is not attained (e.g., forfeited if more than 5 percent of help requests cannot be resolved in 30 minutes). 5-11. How can local election officials attract and ensure an adequate base of volunteers who can cope with the challenges of new electronic voting systems? Problems of poll worker training may be exacerbated by the demographics of poll workers extant in many jurisdictions, where they are often individuals without much experience with technology.     lifetime of the systems in use or whether such staffing was intended as a “loss leader” to provide reassurances for other prospective buyers. If it was not part of the contract, the question arises as to the vendor’s inclination to provide similar levels of support in the future.