2
An Experimental Innovation Inducement Prize Program at NSF

OVERVIEW OF AN EXPERIMENTAL PRIZE PROGRAM

In view of the NSF’s lack of experience and the federal government’s limited experience in the design and administration of prize programs intended to induce innovation of national significance, the committee recommends that the agency take an experimental approach to implementing the congressional directive to award such prizes, especially during the program’s formative period.1 By an “experimental program” we do not mean that prizewinning would necessarily entail use of experimental methods or that the program would be set up as a rigorous scientific experiment with appropriate controls. We mean that the program should be conducted with careful attention to evaluation, measurement, and use of feedback about the program itself. There is much to learn about how to determine the appropriate goals and objectives for inducement prizes,

1

There is precedent for an experimental approach to innovation policy. For example, in the early 1970s interest emerged in using federal procurement to induce technological innovations thought to be of national importance. One prominent response was the Experimental Technology Incentives Program, or ETIP, implemented by the National Bureau of Standards (now National Institute of Standards and Technology, or NIST). ETIP sought to learn how to redesign procurement processes so as to strengthen the incentives for potential suppliers to develop new and improved technologies that were not available in the commercial or governmental marketplaces. See Lewis, Jordan D. “Incentives for Technological Change, A Progress Report.” Experimental Technology Incentives Program. March 26, 1975.



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 18
Innovation Inducement Prizes: At the National Science Foundation 2 An Experimental Innovation Inducement Prize Program at NSF OVERVIEW OF AN EXPERIMENTAL PRIZE PROGRAM In view of the NSF’s lack of experience and the federal government’s limited experience in the design and administration of prize programs intended to induce innovation of national significance, the committee recommends that the agency take an experimental approach to implementing the congressional directive to award such prizes, especially during the program’s formative period.1 By an “experimental program” we do not mean that prizewinning would necessarily entail use of experimental methods or that the program would be set up as a rigorous scientific experiment with appropriate controls. We mean that the program should be conducted with careful attention to evaluation, measurement, and use of feedback about the program itself. There is much to learn about how to determine the appropriate goals and objectives for inducement prizes, 1 There is precedent for an experimental approach to innovation policy. For example, in the early 1970s interest emerged in using federal procurement to induce technological innovations thought to be of national importance. One prominent response was the Experimental Technology Incentives Program, or ETIP, implemented by the National Bureau of Standards (now National Institute of Standards and Technology, or NIST). ETIP sought to learn how to redesign procurement processes so as to strengthen the incentives for potential suppliers to develop new and improved technologies that were not available in the commercial or governmental marketplaces. See Lewis, Jordan D. “Incentives for Technological Change, A Progress Report.” Experimental Technology Incentives Program. March 26, 1975.

OCR for page 18
Innovation Inducement Prizes: At the National Science Foundation how to set the terms and conditions under which prize contestants are recognized, establishing an effective time frame for award of a prize, whether and how to partner with or outsource prize administration to nongovernmental entities, and a number of other issues discussed in this report. To reduce these uncertainties as well as to contribute to innovation, NSF should proceed with care in implementing an inducement prize program, beginning relatively modestly and learning as experience in designing and implementing prizes is analyzed and accumulated. Proceeding in this way need not delay program implementation. In fact, we recommend that NSF initiate a few somewhat specialized prize contests relatively quickly while preparing for more ambitious prizes to follow. The notion of a prize program as a learning opportunity carries with it the obvious corollary suggestion that NSF offer several prizes—on the order of 5 to 10—under its initial program. Multiple prizes offer an opportunity to try various contest designs and administrative approaches.2 They also afford an opportunity to build public interest in the program by making relatively modest awards for successful shorter-term efforts while allowing enough time for more significant, longer-term contest objectives to be reached. Within an experimental program early progress can be made on relatively straightforward approaches to contest administration, leaving time for later development of more complex contests and mechanisms. In the following sections the committee outlines an approach to the design of an inducement prize program that emphasizes learning and program improvement in the formative years, leading to more ambitious contest goals with greater visibility and higher stakes for participants and NSF. The program would fund prizes in several fields of endeavor that differ with respect to the scale and scope of the prize, the timing of its 2 The committee suggests a program of this scope based largely on three considerations. First, the number of prizes should be large enough to allow for some variation in the parameters of the various contests to facilitate learning from the program’s experience. Second, we expect that mounting each prize contest will be a substantial intellectual and administrative challenge that will draw heavily on NSF staff and advisory resources, which suggests keeping the number of prizes manageably small. Third, we have some concern that a large number of prizes might detract from the prize program’s goal of focusing on high-priority national need. To the extent that financing activities to compete for a prize will depend on corporate sponsorships, philanthropy, and volunteer efforts, it will be important to understand whether the pool of such resources for supporting contestants is limited. The interest of the general public in such contests may also be limited. Thus, during program evaluation, NSF should assess the degree to which each subsequent contest attracts contestants and sponsors.

OCR for page 18
Innovation Inducement Prizes: At the National Science Foundation announcement, the duration of the prize contest, how winners are determined, the engagement of expert and interested groups, and other factors. In the remainder of this chapter a number of issues in the design of such a program are discussed and preferred alternatives suggested. The next chapter discusses a process and criteria for selecting some initial prize topics. It discusses several potential prize topics to illustrate selection processes and criteria. ADMINISTRATION AND DESIGN ISSUES We begin with a general description of an innovation inducement prize program and then discuss a number of specific issues regarding such a program. The committee recommends that NSF establish a multiyear, continuing innovation inducement prize program. NSF should assume primary responsibility for the design and implementation of the program. NSF should also take full advantage of the interests and capabilities of other federal agencies as well as of diverse private parties, when appropriate enlisting them as partners in the program. NSF should consult widely with professional and technical societies, industry associations, entrepreneurs and investors, industrial R&D managers, public interest organizations and others in designing and implementing the prize contests it sponsors. NSF should experiment with a variety of types of prizes, as discussed below, as well as with the mechanisms used to induce a wide range of contestants to pursue the prize awards. The committee envisions that, at least initially, the funds used to pay for the prize awards (i.e., the prize purse), as well as those needed to administer the prize program contests, will be appropriated for these purposes by Congress. Such funding should be in addition to rather than a substitute for funds appropriated to NSF in support of research and education in science, mathematics, and engineering. As indicated above, the committee anticipates that in addition, contestants will undertake private investments in the research and development activities entailed in competing for the prizes. The magnitude of these investments will depend on the nature of the prize objective and the opportunities for publicity and other benefits, among other factors. The committee does not believe that the primary reason for government sponsorship of a prize contest is to save taxpayers’ money relative to the cost of accomplishing the objective through a grant program or procurement. Rather, the primary reason for offering a prize is to attract different parties to contribute to a recognized societal or scientific objective. The NSF program should offer prizes in a range of fields to address a diverse set of national challenges. Prizes would differ substantially in

OCR for page 18
Innovation Inducement Prizes: At the National Science Foundation size and be administered in several ways. All of the prize contests would follow a common set of general administrative rules. Each contest would have a set of rules tailored to the specific circumstances, as discussed below. “First-past-the-post” and “Best-in-class” Contests There are two principal types of innovation inducement prize contests. The more familiar type is the “first-past-the-post” contest, in which a prize is awarded to the first team3 that accomplishes a stated objective in accord with the contest rules.4 Such a contest may be held open for as long as required for a winner to emerge or it may end at a date certain. In either case, we suggest that NSF award prizes using a “first-to-achieve” rather than a “first-to-report” rule. In the latter case, if no team reaches the objective by the contest termination date, no award is given. The committee believes that contests of finite duration are generally preferable, although the lead time can be quite long depending on how ambitious the scientific or engineering challenge is. A second major type of contest is the “best-in-class” contest, in which the winning team is the one that comes closest to achieving a stated objective within a specified time period. For a best-in-class contest, one option is for the rules to specify a minimum level of accomplishment; and if no team reaches that minimum, the prize will not be awarded. In both types of contest, entries may be evaluated on the basis of objectively measurable outcomes or the judgments of expert panels. Contests designed around objectively measured outcomes are less subject to influence and controversy about outcomes and in the committee’s judgment are generally preferable, although we recognize that this may not always be feasible if the range of possible solutions to a challenge is broad. A contest may be structured to award a prize only for accomplishing the final contest objective. In an especially challenging contest, progress may be enhanced by awarding a series of prizes to recognize accomplishment of intermediate objectives. In such contests, award of intermediate prizes may also be conditioned on meeting preset minimums of performance. Other important features of inducement prize contests include the 3 We use the word “team” throughout to denote an individual, an organization, a group, a partnership or any other entity that participates in an inducement prize contest. 4 To minimize priority disputes, contest rules may have to specify how progress is to be documented.

OCR for page 18
Innovation Inducement Prizes: At the National Science Foundation breadth and significance of the societal need that informs the contest goal and objective, the anticipated degree of innovative advance needed to accomplish that goal, and, of course, the size of the prize purse. The form of the prize award can also vary and can range, for example, from a cash prize to a procurement contract. The committee recommends that NSF consider a variety of contests that would represent different kinds of challenges, feature different award levels, and test different contest designs. Some contests would reward accomplishment of simple and clear-cut, albeit challenging scientific or technical objectives, which if realized, would open the door to a number of important innovations in the future. The objectives of such contests could be stated in objective, quantitative terms, and it would likely make sense to set them up as first-past-the-post contests. Prizes could range from $200,000 to perhaps $2 million each. NSF could initiate its prize program by announcing three or four such contests each in a different field, during the first two years of the program. Depending on experience with these contests, NSF could then offer one or two additional prizes of this type each year thereafter. Exclusive of administrative and support costs,5 this part of the recommended program could require an initial prize award budget of $800,000 to $8 million, and perhaps $400,000 to $4 million per year on an ongoing basis thereafter. Other NSF contests should offer considerably larger awards to encourage innovations that address significant economic, social, or other challenges to the United States. They would likely elicit much more ambitious and competitive responses from contestants than would be expected from the prizes at lower levels. Prizewinning innovations would likely be highly complex and expensive, would address objectives well beyond the current state of the art, and be intended to become integral to major changes in complex sociotechnical systems such as those of energy supply, environmental protection, education, or health care. Setting objectives for such contests would likely involve the considered judgment of multidisciplinary groups of experts, just as reaching the objectives could 5 NSF funds personnel and related costs from a separate account not rolled into program budgets. Typically, these and other administrative costs are a small proportion of program costs, about 7 percent on average. In the case of the prize program, in-house personnel costs (for, say, a program director, two professional staff, and one administrative staff) would be modest; but the cost of planning and consultation to determine the topics of prize contests, advertising the contest to a wide range of potential participants, and ongoing evaluation could push administrative costs higher than the average for NSF programs, whether or not these functions are contracted out.

OCR for page 18
Innovation Inducement Prizes: At the National Science Foundation require the efforts of large teams with contributions from many disciplines and fields of expertise. The prizes in contests of this type could range from $3 million to perhaps $30 million each. Designing a contest at this level would probably require one to three years of preliminary planning, consultation, and analysis. NSF might offer one new contest of this magnitude every few years, and each contest would be planned to continue for 5 to as many as 10 years. Thus, this part of the program would entail a budget for prize purses of $5 million to $50 million annually at steady state, exclusive of administrative and supporting costs, depending on the number of contests offered and the prize for each. With either scale of prize contest the objectives should be stated in objective, measurable terms, and the committee generally prefers the first-past-the-post format. Nevertheless, depending on the nature of the objective and expected technological advance, either small or large contests could incorporate subsidiary best-in-class prizes for accomplishment of an intermediate objective. The small- and large-scale prize contests suggested here represent what the committee believes are reasonable bounds on the scope of prizes that it would make sense for NSF to offer. They were selected for purposes of exposition of the design choices and their implications. We can envision that other intermediate types of prize contests will evolve as NSF gains experience with an innovation inducement prize program. A few potential topics of both small- and large-scale prize topics are described in Chapter 3. We turn next to a discussion of a number of prize program design issues, including program administration and funding; the roles of NSF, other agencies, and the external communities; selection of prize topics and contest objectives; contest rules; and program evaluation. NSF’s Role in Program Administration and Funding NSF should take primary responsibility for developing and implementing the innovation inducement prize program. However, there are a variety of opportunities to make full use of the interests and capabilities of other federal agencies and a number of private sector entities in the profit and nonprofit worlds. Here we address the role of the NSF and how other institutions could be engaged in the NSF prize process. Because NSF will be accountable to Congress for the conduct of the prize program and its results, the committee is of the view that NSF must execute certain tasks itself. For each contest NSF should decide on the topic or focus, set the rules, and be responsible for deciding the winners. NSF will also determine whether and how other entities contribute to the program’s design and administration. NSF will seek annual appropria-

OCR for page 18
Innovation Inducement Prizes: At the National Science Foundation tions, request any needed changes in authorizing legislation, and report on the program’s status and progress to Congress and the general public. To perform these functions NSF should establish an innovation inducement prize program staff, which we will call here the “Office of Innovation Prizes” or OIP. OIP could be set up using existing program and administrative support staff, perhaps augmented by one or two new hires or individual consultants or contractors experienced in prize administration. OIP should be able to call on the program directorates and staff across the NSF for scientific and technical expertise to help with contest design and with evaluation of prize applications and submissions. OIP would plan the prize program, manage contractor selection and oversight, ensure that contractors carry out the prize program tasks in conformance with all relevant governmental rules and regulations, and administer a process of formative and summative program evaluation on behalf of the director and the National Science Board. OIP would also be the focal point for interagency participation in prize contests for which NSF determines that such participation is desirable. The decision to award a prize should be made by the director of NSF upon recommendation from the OIP. The decision of the director in any particular prize contest on the technical merits of applications should be final, but a mechanism for appeal on procedural grounds needs to be provided. The director, acting through OIP, should establish a formal external advisory committee on innovation prizes that would include leading scientists and engineers, experts in the innovation process, people with some experience in inducement prize implementation, and such others as the director views as appropriate. The advisory committee would do just that, advise; it would not have a role in the decisions to award specific prizes to specific winners. Ad hoc, contest-specific committees of expert advisors could be convened to participate in or verify award decisions. The committee recommends that NSF request and Congress provide separate “no-year” budget authority for funds to support the prize program awards. NSF would be barred from using these funds for purposes other than the prize program unless agreed to by Congress. No-year budget authority would help ensure that funds remain available to make occasional large prize awards when multiyear contests are completed. At present the legislative authority for the NSF innovation inducement prize program is limited to guidance from the House Appropriations Committee in the FY 2006 Science, State, Justice, and Commerce Appropriations bill and associated committee report language. In the committee’s view it will be important to the long-term success of the prize program, including its financial stability, for NSF to seek formal authori-

OCR for page 18
Innovation Inducement Prizes: At the National Science Foundation zation of the program from Congress in addition to the encouragement it has received from the Appropriations Committee. A broad reading of the NSF Act of 1950 suggests that operating a prize program is within the NSF’s general authority to spend funds in support of innovation, and the committee believes that the agency can proceed with the first set of prize contests on that basis. NSF was established as an agency whose principal mode of operation was to make grants and enter into contracts for the support of research and of education in scientific and engineering fields. Adding authority to make prize awards, and to do so in cooperation with private entities, would ensure that the NSF has the full support of Congress in carrying out such programs. Authorizing legislation could also be considered, if necessary, to modify Section 1870(f) of the NSF Act to allow NSF to accept donated funds that are intended to support the prize program or a particular prize.6 The committee cautions against seeking legislative authority for the prize program that is overly detailed and prescriptive as to the topics of prizes, size of prizes, types of contestants, rules for competing, and so on. NSF needs the flexibility within broad legislative grants of authority to experiment with the design and administration of a prize program if it is to be successful. Role of Non-NSF Entities in Program Administration and Funding We anticipate that teams from the private sector and nonprofit institutions will be the principal contestants for innovation inducement prizes. Entities other than NSF are expected to fund most if not all of the research and related activities that contestants conduct in pursuit of the prizes. In designing a prize program, it is important to take into account the likely motivations for nongovernmental entities to provide financial and other resources to contestants to ensure that the program as designed is responsive to the potential of those motivations. Several motivations were identified by the committee, including the following: Some for-profit firms may compete for the prizes or support the research of other contestants in the expectation that they will be able to profit from commercialization of the research results, regardless of whether the contestants they support actually win a prize. For such firms 6 Section 1870(f) of the NSF Act permits NSF to accept and use donated funds, but only “if such funds are donated without restriction other than that they be used in furtherance of one or more of the general purposes of the Foundation.” It is not clear whether this language would allow NSF to accept funds donated specifically for the prize program or a particular prize.

OCR for page 18
Innovation Inducement Prizes: At the National Science Foundation the monetary value of the prize may be of relatively limited consequence; the much greater reward would come from profits earned in the marketplace and from publicity associated with the contest and with winning teams. To enjoy such rewards these firms need to own or control the commercialization of any intellectual property, including trade secrets that are developed in pursuit of a prize. Another motivation for private support of prize-seeking R&D is that supporters may hope to benefit from being recognized as supporters of a research team. In a sense, supporting an R&D team that is pursuing an NSF prize would constitute a form of advertising not unlike that associated with financing of sports teams or athletic events. A supporter motivated in this way may have little or no direct interest in exploiting prizewinning innovations. Instead, it expects to benefit from the association with the celebrity of competing teams. For this kind of supporter, prize contests need to be organized in such a way that the public relations aspects of the contest are managed to their benefit. Yet another motivation for private support of contestant teams is the supporter’s commitment, personal or corporate, to the goal of a particular contest. Such a supporter may be more interested in seeing that the goal is achieved than that any particular team achieves it. This type of supporter may even be willing to augment the prize purse offered by NSF. Nonprofit foundations and wealthy individuals may assume this role; they may not be interested in bringing attention to themselves through a prize contest—some may even prefer to remain anonymous. For this kind of supporter, the structure of the contest objective is its most important aspect. We anticipate that colleges, universities, and other nonprofit organizations will be active supporters of teams competing for some NSF innovation inducement prizes, either using discretionary resources or teaming with for-profit entities. For example, some of the leading participants in the DARPA autonomous vehicle prize competition (including the 2005 grand challenge winner) were university-automotive firm partnerships. In addition to the possibility of using the publicity associated with the contest to raise private funds in support of their contestant teams or for other research purposes or simply to help recruit faculty or students, educational institutions may see involvement of student and faculty teams in innovation prize competitions as an exciting and creative educational experience for students, especially those enrolled in engineering, design, or problem-solving courses. Educational institutions may view team sponsorship as a useful means of marketing to potential students, research collaborators, and donors. For them, excitement, team participation, and photo opportunities may be the most salient benefits of supporting competing teams.

OCR for page 18
Innovation Inducement Prizes: At the National Science Foundation NSF should look to a variety of external entities to assist in aspects of prize program design and administration. Innovation inducement prize contests have typically focused on goals that may be the responsibility of other agencies, such as National Institute of Health (NIH), Department of Energy (DOE), Department of Housing and Urban Development (HUD), Department of Transportation (DOT), Department of Justice (DOJ), or Environmental Protection Agency (EPA). We recommend that, in cases where it is determined to be appropriate for NSF to award a prize related to another agency’s portfolio, NSF consult with the agency in developing the terms of the prize. In some cases it may be desirable for NSF to partner with another federal agency to offer a prize. Such partnerships may be especially productive when the collaborating agency does not have a strong R&D program of its own and could benefit from NSF’s access to scientific and engineering expertise. NSF should consult widely with professional and technical societies, industry associations, venture capitalists, industry research managers, public interest organizations, and others in identifying potential prize topics. These kinds of groups can help in identifying key unmet needs in their areas of expertise and in understanding how particular constituencies may respond to contest designs. In some cases associations have developed “road maps” of R&D priorities that may be useful guides to selecting prize topics. The same kinds of organizations may be useful conduits for disseminating information about new contests and may help NSF reach beyond its traditional academic constituencies. The committee recommends that NSF experiment broadly with new mechanisms for outreach to communities with ideas for prize topics, especially to communities beyond the ones NSF usually reaches. For example, Web sites, blogs, and chat rooms are tools that can be used to reach specialized constituencies. NSF could fund workshops organized by various kinds of organizations to identify important prize topics. It could even offer rewards for the best ideas for prize contests. There may be important benefits to NSF in contracting through the Office of Inducement Prizes with one or more external organizations to assist in designing and, even, administering some aspects of each prize contest, perhaps with different organizations under contract for each prize. These organizations would bring specialized expertise to bear in crafting the technical rules necessary to specify the objective of each of the prizes or in judging whether a contest objective has been met. To help design a prize NSF could contract with scientific and technical societies whose members have deep knowledge of the technical domain in which a prize is offered. NSF could also contract with organizations experienced in designing and offering innovation inducement or other kinds of prizes to help administer a contest.

OCR for page 18
Innovation Inducement Prizes: At the National Science Foundation Apart from providing technical expertise, contractors may be able to engage in fund raising in support of the awards, a role that may not be comfortable for NSF or even permitted under current law.7 The contractor model could also facilitate employing a range of professional expertise not generally associated with NSF, such as major prize administration, media management and outreach, marketing and branding, and outreach to potential contestants in sectors that NSF does not normally reach. The Intergovernmental Personnel Act could provide a useful vehicle for NSF to obtain on a limited-term basis the staff services of individuals who are experienced in prize design and administration. The committee recommends that NSF make use of external contractors in whatever ways will enable it to move most expeditiously and effectively to mount a vigorous innovation inducement prize program. Selection of Prize Topics, Goals, and Objectives NSF should decide on the prize topics, goals, and objectives for each contest. The Office of Inducement Prizes would manage the selection process, reaching out widely to both generalist and specialist expertise in doing so. NSF should select the first set of prize topics and goals within a few months, although detailed contest objectives and rules may take more time to develop. Following announcement of the initial set of prize contests, the committee envisions a more extended process for selecting the more ambitious prize topics, determining the prize goals within those topics, and designing the objectives of each contest to best match the circumstances that surround likely goal achievement. INDUCEMENT PRIZE CONTEST RULES Innovation inducement prize contests would be governed by a number of rules. Broadly, these can be divided into (1) administrative rules that would apply generally to all such contests sponsored by NSF and (2) rules that would be specific to each contest. Here we discuss and offer recommendations regarding some of the most important of these rules. We suggest two caveats at the outset. First, since one purpose of the innovation inducement prize program is to open federal support to entities that have not traditionally received NSF funding or even federal fund- 7 See footnote 6 for what the NSF Act says about NSF acceptance of donated funds.

OCR for page 18
Innovation Inducement Prizes: At the National Science Foundation ing more generally, it is important that the rules be kept as limited and as simple as possible. Second, we are aware that variations in these rules will undoubtedly be desirable in certain cases, and we counsel against adopting rigid rules that are difficult to tailor to circumstances. As a general principle, the committee recommends that NSF make the rules no more restrictive than for other NSF funding mechanisms, and perhaps less restrictive in cases where NSF believes that would be appropriate to the prize instrument. Administrative Rules Applicable to All Contests NSF should consider adopting a number of program rules or guidelines that would apply uniformly to all such contests. NSF should consult with the prize program advisory committee and other interested parties to ensure that these rules are consistent with the best understanding of what is needed to make prize programs effective; and once they are formulated, NSF should consider incorporating them in a prize policy manual analogous to the NSF Grants Policy Manual. General rules are needed to govern registration, eligibility, appeals of award decisions, contest termination decisions, and control of intellectual property. Contestant Registration After a prize contest is announced, teams would be given a finite but reasonably generous window of time in which to register with NSF or its designee their intention to compete for a prize. A period of six months to as many as three years after contest announcement is reasonable, depending on the details of the contest requirements. Registration should identify the principal participants in teams, including the team leaders and his or her affiliations. Registration would, at a minimum, acknowledge team understanding of and agreement to abide by contest rules, including acceptance of the decisions of NSF regarding prizewinners. We do not foresee a general need to prequalify contestant teams, although this may be required if competing would require access to specialized federal resources. A key advantage of registration is that it would provide NSF an early indication of the likely interest in pursuing a particular prize and possibilities for early publicity. Some inducement prize contests have required payment of an entry fee both to defray part of the cost of prize administration and to discourage frivolous applications. The committee recommends against charging such fees at initial registration, especially since one purpose of the prize program will be to encourage the widest possible participation, including

OCR for page 18
Innovation Inducement Prizes: At the National Science Foundation by teams that may have limited resources, such as independent inventors and students. Compliance with Applicable Regulations The registration procedure should include a statement of intent to comply with all applicable regulations in carrying out experiments or tests in pursuing a prize. NSF should beware of creating excessive red tape around such certifications, but some due diligence to ensure the legitimacy of contest winners would be prudent. On the other hand, with the exception of requiring documentation of progress toward a contest objective in a manner subject to verification, the kinds of cost-accounting and recordkeeping standards imposed on recipients of federal grants and contracts should not be applied to prize contestants, since there is no public need to account for the contestants’ use of private sponsor funds or for the winners’ use of the contest award funds. Conflicts of Interest The issue of conflict of interest may arise in connection with NSF’s (or any other government agency) offering inducement prizes for accomplishments in areas in which it also awards grants. It has been suggested that an agency could favor its own grantees or contractors in determining which team has actually won. In the committee’s view, however, if the rules for determining winners are well written and the criteria for winning are stated objectively, it should not be possible for the awarding organization to bias the results in favor of “its” teams. As NSF is the principal supporter of research in many fields, it would be contrary to the purpose of a prize program to exclude the best researchers because they happened to have received a grant award from NSF in a related area. Indemnification We believe it is not necessary to ask contestants to sign waivers indemnifying the NSF for any legal liabilities that arise from the research, testing, or commercialization of an innovation developed in pursuit of a prize. As a federal entity, NSF is protected from most liability actions. Many applicants of the type NSF would wish to attract would not likely possess the resources that would be required to make any such indemnification meaningful. The cost of insurance to contestants for protecting the government against liability actions may also be prohibitive for such entities.

OCR for page 18
Innovation Inducement Prizes: At the National Science Foundation Foreign Participation A key issue in designing the contest rules is whether a prize could be awarded to persons or organizations outside the United States or who are not citizens or permanent residents of the United States.8 We note that NSF has statutory authority to make grants and other awards to non-U.S. persons and entities, but that it can do so only with the advice and approval of the Department of State.9 On the other hand, NSF funds awarded to U.S. institutions are commonly used to support research carried out in those institutions by noncitizens, including temporary residents, such as foreign graduate students and visiting researchers. Under certain conditions NSF permits its grantees to make sub-awards to non-U.S. entities. Thus, the law governing NSF’s activities does not bar supporting foreign persons or entities. Although non-U.S. entities could be accommodated as prize contestants on the same terms as those that are applied to other NSF award programs, there are other considerations. Foremost is the fact that one purpose of the NSF innovation inducement prize program is to strengthen domestic U.S. competitiveness. Opening the prize competitions to non-U.S. entities could conflict or appear to conflict with this purpose. Awarding a prize for a technical advance that may ultimately be commercialized is not the same as supporting fundamental research with the intention that the results be available to all. We recommend that eligibility for NSF’s innovation inducement prizes be limited to U.S. entities, for which the appropriate test is whether the institution or organization with which the applicants are associated is incorporated or registered in the United States and whether the winning team, whether affiliated with such an institution or independent, is led by U.S. citizens or permanent residents. This policy would allow for the participation of the U.S. subsidiaries of foreign-owned companies as well as the participation on U.S.-led teams of immigrants with student or work visas. Federal Involvement Another important issue is whether applicants may include federal employees or use federal funds or facilities in competing for an innova- 8 This section does not take into account the possibility that as experience is gained with the inducement prize programs, it may prove worthwhile for NSF to consider partnering with one or more other countries to offer inducement prizes on an international basis. Consideration of the pros and cons of this approach was beyond the scope of this study. 9 42 USC 16, Sections 1870 and 1871 (portions of the NSF Act).

OCR for page 18
Innovation Inducement Prizes: At the National Science Foundation tion inducement prize. We recommend that federal employees not be eligible to lead or directly participate in teams competing for the prize.10 Organizations substantially funded by the federal government on a sustained basis, such as Federally Funded Research and Development Centers and Government-Owned, Contractor-Operated federal laboratories, should not be eligible to sponsor competing teams. Federal facilities could be used and federal laboratory expertise could be tapped by nonfederal entities if they are made available for use by all teams on an equitable basis.11 We recommend against banning the use of federal funds in doing research toward a prize, in part because segregating funds by source and purpose is a difficult practical matter in large organizations, and in part because we see no principled reason for barring use of such funds. In fact, NSF may chose to mount a mixed program that would offer both grants to participants chosen on a competitive basis and a prize for the most successful research result. Early Termination of Contests NSF should reserve the right to terminate a prize contest before a winning team appears if, in the sole judgment of NSF, an entity that did not register to participate in a contest had, in fact, developed an innovation that would win the prize had it registered. This could happen, for example, if a foreign entity that was not eligible for the U.S. competition were the first to succeed, or if a U.S. entity that chose not to enter the contest or that made its innovation after the registration period had closed was the first to succeed. While contest termination may be unsettling for NSF and the registered teams, it is certainly preferable to having to award a prize to a team demonstrably not best in class or first to achieve the contest objective. Contest rules should specify how NSF would acknowledge the accomplishments of the noncontestant entity and how it would notify contestants in the event of contest termination. Appealing Award Decisions As noted earlier, it is the committee’s view that because of the unusual nature and high visibility of the program, final determination of 10 This ban on participation should be absolute for employees of NSF. The rules may allow for participation by employees of other federal agencies and contractors on their own time if such activity would be otherwise consistent with their employment agreements. 11 “Equitable” does not necessarily mean “equal.” For example, a user fee for testing a prototype at a major federal laboratory could be charged to large corporations but be waived for small firms, academics, student groups, or independent inventors.

OCR for page 18
Innovation Inducement Prizes: At the National Science Foundation prizewinners should be made by the director of NSF and that the decision of the director on whether a particular innovation achieved the contest goal should be final. For the rare instance in which an award is challenged on procedural grounds, the NSF should provide an appeals mechanism.12 The general contest rules should spell out the procedures for filing and hearing such appeals. Intellectual Property We recommend that the federal government not seek to own or control the disposition of intellectual property developed by contestants in the course of seeking NSF innovation inducement prizes, with one exception. The exception would be that if the winner of a prize chooses not to put the winning innovation into commercial practice within a reasonable time period and if it declines to license it to another U.S.-based entity wishing to commercialize the invention, the winner would be required, as a condition of the award, to enter into good faith negotiations with the other party for a license to be awarded under terms and conditions typical for the industry or technology sector. A winner could avoid even that limitation on its exercise of its intellectual property by foregoing the prize purse. The committee counsels against any requirement that the winner make the intellectual property underlying the winning innovation available to the world at no cost or on concessional terms. At the extreme, such a requirement would likely discourage potential entrants from participating for fear that if they won, they would lose control over their innovations. If the winning innovation were developed in any part using federal funds, the Bayh-Dole Act would apply, allowing the contestant to take title to and license or sell the intellectual property developed in pursuit of the prize, subject to federal government rights to practice the invention for government purposes without royalty and ability to “march in” and take control of the invention in extraordinary circumstances. If no federal funds were used in making the winning innovation, the federal government should have no equity position from which to demand march-in rights to prizewinning intellectual property. It is possible that an innovation proposed as a prizewinner would 12 NSF’s formal reconsideration procedure, described in the Grant Policy Manual, for NSF decisions not to make a grant award is not appropriate for the prize program in the committee’s view because the ultimate authority to decide such a case is a level below the director.

OCR for page 18
Innovation Inducement Prizes: At the National Science Foundation make use of components or methods that had been patented or copyrighted by parties other than the contestant. To guard against possible infringement a team that is under consideration for the award of a prize should be required to make a good-faith effort to identify such intellectual property and should be required to acknowledge that it would need licenses from its owners to put the winning innovation into commercial or research use. Contest-specific Rules In addition to the general administrative rules applicable to all innovation inducement contests discussed in the preceding section, specific rules would be needed for each individual contest. These rules would define the form of each prize, the criteria for winning the prize, the method by which winning would be determined, and when appropriate the criteria that a potential contestant would need to demonstrate its competence to pursue a prize. In the committee’s view crafting these prize-specific rules is likely to be a challenging and different task for each contest. The experience of other organizations, including DARPA, NASA, and the X-Prize Foundation, has been that writing contest rules requires extensive consultation with experts, affected parties, and potential contestants to ensure that the stated prize goals are clear, understandable, and unambiguous; that the proposed prize goals represent important societal needs and opportunities; that the contest objectives embody a significant yet achievable advance over the current state of the art; and that it will be possible to determine in a relatively unambiguous way whether a particular innovation should qualify as the winner. Rules are also needed to specify the form of any notification or application required from contestants who believe they have won a contest, the nature of the information required to be submitted in this notice or application, and the requirements for witnessing or other validation of a team’s claim that it has achieved the contest objective. Defining the Contest Objectives It became apparent to the committee during its deliberations that defining the precise form of the objective of a contest is a challenging task. It is one thing to articulate the thought that one or more goals of national significance may make an interesting prize topic. It is another thing to express that thought in the form of the accomplishment of an objective that would help reach the national goal in a way that would be new, different, and a substantial advance over what can presently be accomplished

OCR for page 18
Innovation Inducement Prizes: At the National Science Foundation at about the same time in the absence of a contest. That is, the task of translating the broad societal goal of a contest into a concrete proximate objective that can be measured can be demanding.13 The task is somewhat different when the innovation being sought is relatively fundamental in character, reflecting an advance in science or basic technology, versus when the innovation represents the commercially successful application of a new idea, method, or technology. For more technically focused contests, achieving a defined technical objective may suffice as an indicator of success. The private marketplace (or, sometimes, government procurement) usually determines whether a more applied innovation is successful. Success of such innovations is measured by sales, profitability, market share, consumer acceptance, winning a competitive procurement, and the like. For NSF innovation inducement contests that seek to encourage innovations that address important societal problems, however, the committee believes that nonmarket objectives, rather than marketplace acceptance, should be used to judge success. Accomplishing a well-chosen, well-defined nonmarket objective should in some sense stand as a surrogate for market success; moreover, it can contribute to such success. The U.S. innovation system as a whole, at least in comparison with other national systems of innovation, is adept at picking up good technical ideas and carrying them on to the marketplace. An NSF inducement prize program can play a valuable role in this process not only by identifying sound technical solutions to problems but also by conferring a “halo effect” that may help winners attract further investment. In every case the challenge is to determine what should be set as the concrete objective of a prize contest. Suppose, for example, that NSF were to consider offering a prize with the broad goal of encouraging innovation in nanotechnology. There is widespread agreement that nanotech- 13 The Ansari X-Prize, for example, used in lieu of a market test the objective requirements that a craft could reach suborbital heights (100 km) with a human cargo twice in a two-week period and return safely, with a maximum of 10 percent of craft weight to be replaced during the two-week interval. These contest conditions were apparently intended by the prize organizers to be a reasonable surrogate for what could not yet be achieved or evaluated—development of a successful commercial space tourism business. In the government market arena, DARPA’s Grand Challenge prize for autonomous off-road vehicles established a series of contest objectives based on discussions with potential customers for such vehicles in the military services. Winning required completion of a 132-mile course through rugged desert countryside within 10 hours, as well as successful completion of a series of preliminary qualification tests. Devising an objective for each contest that can serve in place of actual market success appears to be the most challenging intellectual aspect of contest design.

OCR for page 18
Innovation Inducement Prizes: At the National Science Foundation nology offers great promise for innovative applications in many fields, including medicine, high-strength materials, electronic devices, telecommunications, quantum devices, coatings, cosmetics, and so on. In light of this great promise, for what specific achievement should an inducement prize be offered, that is, what should be the contest objective? The contest objective could be a new fundamental scientific capability that could be widely applied in many domains of nanotechnology, or the objective could be demonstration of a flagship application thought to offer great promise but also great technical challenge. The committee suggests that it would be essential to confer with scientists and engineers active in the field of nanotechnology and to elicit from them desirable objectives for innovations that, if realized, could substantially advance nanotechnology applications. In addition to discussions with scientists and engineers, NSF may find it useful to examine existing technology road maps in which expert communities have identified the most important technical problems to be solved to enable advances in their fields.14 For more ambitious contests more complex judgments will have to be made about whether an innovation is likely in the longer term to achieve market success, for example, in new sources of energy. There are many options for designing a contest in this field. Such a contest could be framed, for example, by offering a prize for a dramatic innovation in energy supply generally, for an innovation in carbon-neutral energy systems, or for an innovative catalyst for conversion of biomass to usable transportation fuels. Each of these operates at different levels within the sociotechnical energy system and each poses its own dramatic yet different challenges. Framing an energy supply contest around broader definitions of the societal goal may lead to more significant developments, yet may also make it challenging to define contest objectives that are sensible and for which winning can be objectively assessed. Conversely, framing an energy supply contest around a narrow goal may yield interesting innovations within the scope of the contest, yet fail to stimulate innovations that might be of greater overall importance but that would fall outside the contest objective. Furthermore, to help achieve 14 These road maps are exemplified by the International Technology Roadmap for Semiconductors, 2005 edition, International Roadmap Committee, online at http://www.itrs.net/Links/2005ITRS/Home2005.htm. A comprehensive history and review of technology road maps in general, including the various semiconductor road maps, can be found in. Schaller, Robert R. “Technological Innovation in the Semiconductor Industry: A Case Study of the International Technology Roadmap for Semiconductors.” Ph.D. Dissertation, Fairfax, Va., George Mason University, April 2004, full text on the web at http://www.xecu.net/schaller/schaller_dissertation_2004.pdf.

OCR for page 18
Innovation Inducement Prizes: At the National Science Foundation goals of such broad societal significance and great technical challenge, it may be useful to structure the contest to include a series of prizes for achieving intermediate objectives, as well as a grand prize to recognize a substantial capstone innovation that fulfills a more ambitious objective. Defining the contest rules for this kind of complex area will take a substantial commitment of talented staff and advisers, resources, and time by NSF. The committee recommends the broadest possible outreach by NSF to the technical, professional, public interest, and industrial communities in defining prize and contest goals and objectives. Form of the Prize Most innovation inducement prizes feature a single cash award to the winner. The winner may also enjoy substantial good will and publicity. NSF may wish to follow that practice. We can imagine other types of awards, such as a cash prize provided over a period of several years, using lottery payouts as a model. Another type of prize would be a guarantee that the government would procure a certain number or value of products or services based on the prizewinning innovation. Yet another form of prize could be a guaranteed research grant or contract to the winner or the winner’s institution in support of the winning team’s continuing research. An interesting suggestion is that NSF could offer as part of a prize package a tour of the U.S. facilities it manages in Antarctica, a rare opportunity that could stimulate a great deal of interest on the part of both contestants and the general public. Of course, any variation from a single cash award could result in a more complicated process for administering the prizes, which may not be worth its cost to NSF or to the winner. Demonstrating Qualifications to Compete For some contests NSF could require that potential participating teams demonstrate their competence through a preliminary contest or demonstration. This approach has been used by DARPA in its autonomous vehicle prize competitions. DARPA’s motivation was that because the comparison of different teams’ accomplishments entailed use of expensive military resources, it was appropriate to ensure that participants had a reasonable chance of performing competitively. In the NSF case this may be necessary only if part of the contest similarly involved access to and use of sophisticated federally owned measurement or testing equipment or facilities, or if there are aspects of the prize that would require administrative resources that increase significantly with the number of contestants. Such a procedure amounts to a preliminary contest and would re-

OCR for page 18
Innovation Inducement Prizes: At the National Science Foundation quire careful attention to criteria for being able to move forward, just as for the main contest event. Method for Determining Contest Winners For first-past-the-post contests structured around accomplishment of objectively and quantitatively measurable objectives, the method of determining winners would be relatively straightforward. Teams that believe they have met the contest objective would submit a claim to that effect to NSF or its implementing contractor along with supporting evidence. NSF would use its own staff to confirm the claim or would contract with a separate measurement organization to check the claims of the applicants. For best-in-class prizes to be awarded on the basis of performance criteria involving a degree of expert judgment, NSF or its contractor would convene expert panels to judge the representations of competing teams. Verification by a separate panel of judges may be desirable to ensure credibility of the decision. AWARDING INNOVATION INDUCEMENT PRIZES One of the purposes of an innovation inducement prize program is to raise public awareness of the importance of innovation to the economy and society and to generate excitement about the enterprise. Prizes should be awarded in a manner consistent with this purpose. This suggests that officials at NSF, the White House, and the Congress be engaged in public award ceremonies with a level of effort expended to make the award ceremony, as well as demonstration of the innovation in cases in which this would be effective, a significant newsworthy event. Larger awards will entail relatively large sums of money to be awarded to teams that develop innovations of great national significance. Public and official recognition of these prize awards should be on a par with that accorded the annual awards of the Nobel Prizes in the sciences or to the annual award by the President of the National Medals of Science and of Technology. In such cases the award ceremony could be a commercially sponsored event on national television, perhaps packaged as a national celebration of innovation. EVALUATING THE INNOVATION INDUCEMENT PRIZE PROGRAM To ensure that NSF garners the greatest possible information and wisdom from its experiences with the program, it should engage one or more

OCR for page 18
Innovation Inducement Prizes: At the National Science Foundation external evaluation teams in tracking the development and implementation of the program from its earliest stages. The program should include a vigorous formative evaluation, which should be carried out competitively by teams from academia or appropriate nonprofit or for-profit evaluation firms. The evaluation teams should be managed by the OIP and report directly to the director of the NSF and to the National Science Board. In the program’s early years it would be too soon to attempt a summative evaluation, that is, to evaluate whether the program accomplished its ultimate goals or to assess its contribution to the nation. Instead, a formative evaluation strategy would focus on measures that would provide insight into whether the program’s anticipated short-term impacts were being achieved on a continuing basis, and which features of the prize contests are more or less successful. Among the relevant indicators of positive near-term program impact could be whether the contests attract large numbers of contestants; whether those contestants are a more diverse group than the traditional NSF constituency; whether, to what extent, and from what sources private funds are forthcoming to support the research activities of contestants or even to augment the prize purses; whether prize rules and processes functioned well; whether the contestants make related innovations that are successfully spun off from the main prizeseeking activity; and whether the program significantly advances innovation in the area of the prize topics. Assessments could also be made of whether public awareness of and interest in innovation or the impacts of a specific prize are enhanced by the prize program, and whether NSF’s public image is affected by its sponsorship of the program. Among the relevant program design variables to be examined are the duration of the contest, the size of the prize purse, whether the contests are first-past-the post or best-in-class, whether a contest is administered by NSF or by a contractor, and whether a contest includes prizes for achievement of intermediate objectives. Evaluation teams could also examine whether NSF’s administration of the program is effective as measured by accomplishment of NSF’s own stated goals for the program.