National Academies Press: OpenBook

The Advanced Technology Program: Assessing Outcomes (2001)

Chapter: Findings and Recommendations

« Previous: Summary of Symposium Proceedings
Suggested Citation:"Findings and Recommendations." National Research Council. 2001. The Advanced Technology Program: Assessing Outcomes. Washington, DC: The National Academies Press. doi: 10.17226/10145.
×

Page 83

III

FINDINGS AND RECOMMENDATIONS

Suggested Citation:"Findings and Recommendations." National Research Council. 2001. The Advanced Technology Program: Assessing Outcomes. Washington, DC: The National Academies Press. doi: 10.17226/10145.
×

Page 84

Suggested Citation:"Findings and Recommendations." National Research Council. 2001. The Advanced Technology Program: Assessing Outcomes. Washington, DC: The National Academies Press. doi: 10.17226/10145.
×

Page 85

Review of The Advanced Technology Program

Summary of Findings

In response to a Senate mandate, the National Institute of Standards and Technology (NIST) asked the National Research Council (NRC) to review the performance of the Advanced Technology Program (ATP) in light of its legislated goals and to make recommendations for improvements, where appropriate, in the operations of the program. This task was addressed by a committee previously formed under the direction of the National Academies' Board on Science, Technology, and Economic Policy (STEP) to carry out an assessment of governmentindustry partnerships. The Committee responsible for this analysis was not charged with a review of questions of principle with regard to the desirability of government-industry cooperation. Recognizing that partnerships are an integral part of the U.S. innovation system, the Committee has taken a pragmatic approach focusing its work on the operation and assessment of government-industry partnerships. 1

Government-industry cooperation to achieve national goals has played a key role in U.S. economic development. 2 Continued U.S. leadership in technological progress is essential for the long-term growth of the domestic economy at a rising

1 The scope of the Committee's work is described in the Preface.

2 See the overview of the history of government-industry collaboration in the Preface of this volume. As discussed in the Introduction, the U.S. government has played a significant and supportive role in advancing technological progress in industries ranging from aircraft and biomedicine to information technology and the Internet. The ATP is a public-private partnership to develop new technologies with broad applications. The program makes competitive awards on a cost-share basis to individual companies and larger awards to joint ventures.

Suggested Citation:"Findings and Recommendations." National Research Council. 2001. The Advanced Technology Program: Assessing Outcomes. Washington, DC: The National Academies Press. doi: 10.17226/10145.
×

Page 86

standard of living. 3 Substantial domestic U.S. investment in research and development—both public and private—is the prerequisite for sustaining U.S. economic growth in a global economy. 4 A leading role for the United States in the development and commercialization of new technologies is essential to the continued competitive success of U.S. industry in global markets. Governments around the world have recognized the importance of new technologies to their economies and have encouraged public-private partnerships to develop and anchor them within their national economies. The long-term goal of these programs is to achieve greater productivity growth through the creation of knowledge that can be applied to industrial processes, products, and services. 5 The logic behind government funding of certain types of R&D activities is that government awards provide incentives to firms to undertake high-risk R&D projects with substantial potential benefits for the economy as a whole. 6 In the middle of the 1980s the United States began focusing more attention on cost-shared partnerships as a means of developing new technologies.

As noted in the Introduction, the Committee's assessment of the Advanced Technology Program is contributing to the Committee's review of government-industry partnerships programs in the United States and abroad. This assessment of the ATP should thus be understood as one element of the Committee's multi-year study of a wide variety of partnerships. Carrying out this analysis of the ATP




3 See Michael Borrus and Jay Stowsky, “Technology Policy and Economic Growth,” in Lewis Branscomb and James Keller, editors, Investing in Innovation: Creating a Research and Innovation Policy, Cambridge, MA: MIT Press, 1998. The contribution of technology to economic growth is now well recognized. See P. Romer, “Endogenous Technological Change,” Journal of Political Economy, 98(5):71-102, 1990. See also G. Grossman and E. Helpman, Innovation and Growth in the Global Economy, Cambridge, MA: MIT Press, 1993.

4 Romer, “Endogenous Technological Change,” op. cit.; Borrus and Stowsky, “Technology Policy and Economic Growth,” op. cit. See also National Research Council, Allocating Federal Funds for Science and Technology, Washington, D.C.: National Academy Press, 1995. The report notes that federal investments in R&D have produced enormous benefits for the nation's economy, national defense, health, and social well-being. Ibid., p. 3.

5 See the paper by Maryann P. Feldman and Maryellen R. Kelley, “
Leveraging Research and Development: The Impact of the Advanced Technology Program,” in this volume.

6 As noted by Feldman and Kelley in this volume, “The logic for public investment is that, in the long run, the economic benefits to consumers, other firms and the larger national economy will exceed the private returns realized by the firm that received the research award, and thus justify the public investment.” Ibid. The rationale for government funding of certain types of R&D activities, as articulated by Zvi Griliches, is that this funding encourages firms to undertake R&D projects in which the public rate of return exceeds the private rate of return. This includes, for example, the case in which an industry as a whole may benefit from the development of an enabling technology. Private firms typically use some predetermined benchmark rate of return known as a hurdle rate. The project will only be acceptable if the expected rate of return is above that benchmark. By reducing the cost of the project, government funding will increase the expected rate of return and may make private companies willing to pursue them. See Z. Griliches, “The Search for R&D Spillovers,” Scandinavian Journal of Economics, 94(Supplement):29-47.


Suggested Citation:"Findings and Recommendations." National Research Council. 2001. The Advanced Technology Program: Assessing Outcomes. Washington, DC: The National Academies Press. doi: 10.17226/10145.
×

Page 87

has informed the Committee's deliberations and allowed for comparative points of view on a range of partnership activities. As part of this assessment, the Committee organized two major symposia and a workshop to review the program's operation and also drew on the substantial body of independent analysis of the program. The initial symposium provided an overview of the program in terms of its goals, operations, assessment, achievements, and challenges while providing an opportunity for critics to voice their views. The symposium summarized in this volume focused on possible improvements to the program, findings from the ATP assessment effort, issues such as “crowding out” and the relationship of the ATP to venture capital, the roles and needs of large companies in such a program, and feedback from users, some of whom have received other types of awards. The collection of papers included in this volume provide insights into the operation and impact of the ATP and are illustrative of the substantial program of external and internal research it has under way. The meetings and research are of course complemented by the exceptional expertise of the Committee responsible for the NRC review of government-industry partnerships. 7 Keeping in mind the limitations and advantages of the Committee's analysis, the core findings and recommendations of the study are listed below.

Core Findings and Recommendations 8

  • 1. The Committee finds that the Advanced Technology Program is an effective federal partnership program. The selection criteria applied by the program enable it to meet broad national needs and help ensure that the benefits of successful awards extend across firms and industries. Its cost-shared, industry-driven approach to funding promising new technological opportunities has shown considerable success in advancing technologies that can contribute to important societal goals such as improved health diagnostics (e.g., breast cancer detection), developing tools to exploit the human genome (e.g., colon cancer protection), and improving the efficiency and competitiveness of U.S. manufacturing. 9
  • 2. The program's peer review of applicants for both technical feasibility and commercial potential supports its goal of helping advance promising new tech

7 The members of the Committee are listed in the front matter.

8 These summary findings and recommendations are elaborated and documented below. In addition to the papers and proceedings in this volume, the Committee issued National Research Council, The Advanced Technology Program: Challenges and Opportunities, Washington, D.C.: National Academy Press, 1999. The ATP assessment program also provides extensive documentation regarding the contributions of the program. See
Annex D in this volume. See also William F. Long, Advanced Technology Program: Performance of Completed Projects: Status Report Number 1, NIST Special Publication 950-1, March 1999.

9 See
Section I in this chapter. For a summary of the differentiating characteristics of the ATP, see Maryann Feldman's analysis in
Section C of the Introduction and the study by Feldman and Kelley, “
Leveraging Research and Development: The Impact of the Advanced Technology Program,” both in this volume.

Suggested Citation:"Findings and Recommendations." National Research Council. 2001. The Advanced Technology Program: Assessing Outcomes. Washington, DC: The National Academies Press. doi: 10.17226/10145.
×

Page 88

nologies that are unlikely to be funded through the normal operation of the capital markets. 10

  • 3. The program has set a high standard for assessment involving both internal and independent external review. The quality of this assessment effort lends credence to the program's evaluation of its accomplishments. 11
  • 4. The extensive assessments of the program show that it appears to have been successful in achieving its core objective, that is, enabling or facilitating private sector R&D projects of a type, or in an area, where social returns are likely to exceed private returns to private investors. 12, 13
  • 5. The Committee does recommend a series of operational improvements designed to make this program more effective and suggests several measures

10 With regard to the ATP selection process see the presentation by former ATP Director, Lura Powell, in the first volume of this study, National Research Council, The Advanced Technology Program: Challenges and Opportunities, op. cit., pp. 53-56; with regard to the role of venture capital finance and the need for a bridging mechanism, see the statement by Todd Spener of Charter Financial in the same volume, pp. 90-91, as well as the presentation by Joshua Lerner of the Harvard Business School, pp. 88-90. See also the presentation by venture capitalist David Morgenthaler in Panel I of the Proceedings of this volume and the summary of his statement in Section C of the Introduction to this volume. See also Lewis M. Branscomb and Philip E. Auerswald, Taking Technical Risks: How Innovators, Managers and Investors Manage Risk in High-Tech Innovation, Cambridge: MIT Press, 2001, Chapter 5 and passim.

11 See Section I in this chapter and the description of the program, its current results, and the ATP assessment effort by Rosalie Ruegg and the positive review of the assessment program by Irwin Feller of Pennsylvania State University in Panel II in this volume. See also the panel discussion led by Richard Nelson of Columbia University, including the description of the ATP assessment, its early beginnings, and its focus on tools for assessing technology spillovers in National Research Council, The Advanced Technology Program: Challenges and Opportunities, op. cit., pp. 71-80.

12 See, for example, the paper by Maryann Feldman and Maryellen Kelley, “ Leveraging Research and Development: The Impact of the Advanced Technology Program,” in this volume. The study by Albert N. Link, “ Enhanced R&D Efficiency in an ATP-funded Joint Venture,” documents the impact of an ATP joint venture designed to reduce the costs and timing required to develop a suite of new technologies for the U.S. printed wiring board industry. The study finds a dramatic effect on R&D efficiency, resulting in cost savings on the order of $35 million while reducing cycle times for new product and process development. The project resulted in productivity improvements for member companies, diffusion of new technology to other producers, and improved competitive positions for and retained employment at participating companies. The study by David Austin and Molly Macauley, “ Estimating Future Benefits from ATP Funding of Digital Data Storage,” estimates substantial consumer welfare gains from ATP-funded innovations in digital data storage although the final impact is dependent on the adoption of the technologies. Similarly, the paper by Tayler H. Bingham, “ Estimating Economic Benefits from ATP Funding of New Medical Technologies,” projects substantial social returns, much larger than the projects' private returns, primarily due to the projected positive spillovers to patients treated with new technologies. These technologies focus on the diagnosis and treatment of cancer; the treatment of diabetes, damaged ligaments and tendons; and the transplanting of xenogenic organs. The overview of the progress of ATP awards by Rosalie Ruegg, “ Taking a Step Back: An Early Results Overview of Fifty ATP Awards,” documents both commercialization progress and knowledge creation and dissemination. The latter is documented through outside recognition of the project's technical accomplishments, patents filed and granted, patent-tree citations, collaborative

Suggested Citation:"Findings and Recommendations." National Research Council. 2001. The Advanced Technology Program: Assessing Outcomes. Washington, DC: The National Academies Press. doi: 10.17226/10145.
×

Page 89

designed to bring the benefits of the ATP to other national initiatives and to state technology programs through enhanced cooperation. 14

I. Accomplishments of the Advanced Technology Program

  • A. Meeting Legislative Goals:

    The Advanced Technology Program is achieving the goals ascribed to the program in the Omnibus Trade and Competitiveness Act of 1988. As initially stated, its goals were “to assist U.S. business in creating and applying the generic technology and research results to (1) commercialize significant new scientific discoveries and technologies rapidly and (2) refine manufacturing technologies.” (P.L. 100-418). The ATP emphasizes economic growth and advances the competitiveness of U.S. firms by fostering technologies with potentially large net social value for the nation that might not otherwise emerge in time to maximize their competitive value. 15

  • B. Supporting Enabling Technologies:

    The ATP focuses its support on enabling technologies that face substantial technical barriers yet which also have the potential for broad-based economic benefits. Program goals and examples of technologies illustrating the ATP approach and meeting the program's current operational objectives are:

    • 1. Improved manufacturing efficiency and competitiveness. ATP contributions are illustrated by Extrude Hone's contribution to manufacturing efficiency and the environment and by the successful U.S. Printed

    relationships, and knowledge disseminated through new products and processes. Ruegg records substantial evidence that benefits are extending well beyond those captured by award recipients. The papers cited above are included in this volume.

    13 For an excellent review of the factors affecting the generation and impact of social returns or spillovers, see Adam B. Jaffe, Economic Analysis of Research Spillovers: Implications for the Advanced Technology Program, NIST GCR 97-708, December 1996. For additional ATP-supported research on social benefits, see Edwin Mansfield, Estimating Social and Private Returns from Innovations Based on the Advanced Technology Program: Problems and Opportunities, NIST GCR 99-780, January 1996; William F. Long, Performance of Completed Projects, Status Report Number 1, op. cit.; Wesley M. Cohen and John Walsh, R&D Spillovers, Appropriability, and R&D Intensity: A Survey-Based Approach, Gaithersburg, MD: National Institute of Standards and Technology, Forthcoming; and Michael S. Fogarty, Amit K. Sinha, and Adam B. Jaffe, ATP and the US Innovation System: A Methodology for Identifying Enabling R&D Spillover Networks with Application to Microelectro-mechanical Systems (MEMS) and Optical Recording, Gaithersburg, MD: National Institute of Standards and Technology, Forthcoming. 14 See Section II and Section III in this chapter.

Suggested Citation:"Findings and Recommendations." National Research Council. 2001. The Advanced Technology Program: Assessing Outcomes. Washington, DC: The National Academies Press. doi: 10.17226/10145.
×

Page 90

Wiring Board (PWB) consortium. 16 Other contributions include an “ion implantation” technology to reliably process larger, and hence more productive, 300mm wafers economically. The ATP helped fund the development of advance process control (APC) technology for semiconductor production, which increases process consistency and yield. 17 Working with large and small companies, the program also helped develop a novel insulating material to improve performance of computer chips. 18 These innovations should help maintain the exceptionally high historical annual growth in productivity, on the order of 25-30 percent, which characterizes the semiconductor industry.

  • 2. More rapid commercialization of technologies with positive spillovers, such as the mammography diagnostic instrument recently brought to market. 19 Work is also under way to develop miniaturized DNA analyzers designed to increase the speed of research and medical testing for diseases such as HIV, strep infections, or cancer. 20
  • 3. Contributing to the development of technologies embodying recent scientific discoveries, such as the award to PPL Therapeutics to develop a way to produce valuable stem cells from adult human cells, possibly creating a non-controversial alternative to the use of embryonic stem

15 The Introduction to this volume provides the policy context which led to the creation of the ATP and other cooperative programs and summarizes legislation designed to encourage cooperative technology programs. The legislation establishing the ATP is reproduced in Annex A.

16 For a discussion of the manufacturing and environmental efficiencies made possible by Extrude Hone's advanced manufacturing processes, see the presentation by Larry Rhoades in Panel III of this volume. For a summary of the accomplishments of the PWB consortium see the analysis by Albert Link, “ Enhanced R&D Efficiency in an ATP-funded Joint Venture” in this volume.
17 The APC technology was developed in cooperation with SEMATECH and leading U.S. firms, such as Honeywell, Inc., Advanced Micro Devices, and IBM, among others.
18 Developed by Texas Instruments and NanoPore, Inc., a small New Mexico-based company, the insulator is called Xerogel, which consists of a highly porous, glass material used as a low dielectric constant insulating layer in integrated circuits. The innovation has led to an estimated twenty patents and patent applications and represents a positive development for U.S. industry.
19 Recently approved for clinical use by the Food and Drug Administration, the new system represents a significant technological advance in breast cancer detection. It uses a unique amorphous silicon detector that provides high-quality imaging which can be digitally enhanced and rapidly verified. A 1995 ATP project awarded to General Electric and EG&G Reticon developed a new manufacturing process that significantly reduced the manufacturing cost of the amorphous-silicon panels used in the new detection system, making this superior detection system more affordable and available to a greater number of women. See http://www.nist.gov/public affairs/update/upd000410.htm#Health . See also footnote 40 in this chapter.
20 See National Research Council, The Advanced Technology Program: Challenges and Opportunities, op. cit., p. 55. Microtechnologies under development offer significant advances in the convenience and speed of DNA analysis. One such company, Affymetrix, has developed chip systems which can detect genetic variations related to HIV, cancer, and drug metabolism. The company has also received a grant from the Human Genome Research Institute.

Suggested Citation:"Findings and Recommendations." National Research Council. 2001. The Advanced Technology Program: Assessing Outcomes. Washington, DC: The National Academies Press. doi: 10.17226/10145.
×

Page 91

cells. Stem cells hold the promise of fighting diseases ranging from heart failure to Parkinson's. 21

  • 4. Catalyzing and supporting research partnerships between industry on the one hand and U.S. university researchers and federal laboratories, on the other. Through 2000, 176 universities have been involved in the program, participating in over half (56 percent) of the program's 522 projects, either as full participants or subcontractors. Some 50 projects have included federal laboratories. These partnerships help speed the transfer of publicly-funded basic research and expertise to industry. 22
  • C. An Exceptional Assessment Effort:

    The ATP assessment program has produced one of the most rigorous and intensive efforts of any U.S. technology program. This program has two elements: an in-house effort based at NIST Headquarters and an external effort contracted with the independent National Bureau of Economic Research. 23 The quality, quantity, and analytical range of these studies are impressive. Over 58 case studies and other assessments have been completed; substantial additional work is under way. 24 With regard to this assessment program, several points emerge:

    • 1. It is important to note that these studies, by their very nature, do not endorse every aspect of the program. They do provide valuable insights into the operation and impact of the program.
    • 2. The broad scope of the studies offers insights into the operations of the U.S. innovation system, for example, with respect to early-stage


    21 See Erika Jonietz, “Sourcing Stem Cells: Could New Research End the Embryo Debate?” Technology Review, January/February 2001, p. 32.

    22 For an earlier discussion of this point, see the presentation in
    Panel II by Rosalie Ruegg, Director of the ATP Economic Assessment Office at the time of the conference. The universities most involved in ATP projects include: Stanford University; the University of Michigan, Ann Arbor; the Massachusetts Institute of Technology; Cornell University; Johns Hopkins University; the University of Minnesota; Carnegie Mellon University; Pennsylvania State University; the University of California, Berkeley; and North Carolina State University.

    23 See the discussion of the ATP assessment program in
    Panel III of National Research Council, The Advanced Technology Program: Challenges and Opportunities, op. cit., pp. 70-82, especially p. 79. This section describes ATP's substantial involvement of outside experts, both in the selection process through peer review, and the valuable input provided from the outset of the program through consultations with leading economists from the National Bureau of Economic Research (NBER), which included important contributions by Professors Zvi Griliches and Edward Mansfield. This collaboration continues under the overall direction of NBER's Adam Jaffe.

    24 See
    Annex D of this volume for a list of studies commissioned by the ATP Economic Assessment Office.

Suggested Citation:"Findings and Recommendations." National Research Council. 2001. The Advanced Technology Program: Assessing Outcomes. Washington, DC: The National Academies Press. doi: 10.17226/10145.
×

Page 92

finance of promising technologies and the impact of the intellectual and economic spillovers derived from the program. 25

  • 3. These studies are also making a contribution to our understanding of the U.S. innovation system and to the development of methodologies to measure the impact of federal and state technology programs such as the ATP. 26
  • 4. Few other federal technology programs have embraced this level and intensity of assessment and sought to apply its results as diligently as the ATP. 27

II. Recommendations to Improve the Program

  • A. Extend the window for award applications, accelerate the decision-making process for awards, and extend substantially the period in which awards can be made: New, commercially-relevant technologies are often time sensitive. Fixed periods for firms to apply to the program and long delays in notification of awards may reduce the attractiveness of the pro-



    25 National Research Council, The Advanced Technology Program: Challenges and Opportunities, op. cit., pp. 79-80. See also Adam Jaffe, “The Importance of ‘Spillovers' in the Policy Mission of the Advanced Technology Program,” Journal of Technology Transfer, 23(2):11-19, 1997; E. Mansfield, Estimating Social and Private Returns from Innovations Based on the Advanced Technology Program: Problems and Opportunities, op. cit.; Wesley M. Cohen and John Walsh, R&D Spillovers, Appropriability, and R&D Intensity: A Survey-Based Approach, op. cit.; and D. Mowery, J. E. Oxley, and B. S. Silverman, Knowledge Spillovers and R&D Joint Ventures, Gaithersburg, MD: National Institute of Standards and Technology, Forthcoming.

    26 As noted above, an excellent example is the recent work by L. M. Branscomb and P. E. Auerswald, Taking Technical Risks, op. cit. Initially sponsored by the ATP, this volume reviews some of the factors affecting early-stage financing and notes a serious gap between the creation of an idea and its realization in a technology that meets market requirements for investors. With regard to interaction with state programs see Marsha R.B. Schachtel and Maryann P. Feldman, Reinforcing Interactions Between the Advanced Technology Program and State Technology Programs, Volume I: A Guide to State Business Assistance Programs for New Technology Creation and Commercialization, Washington, D.C.: U.S. Department of Commerce, April 2000.

    27 For example, the SBIR program, currently allocated over $1.2 billion annually, is six times larger than the ATP, yet it has been subject to almost no systemic external assessment, apart from a series of GAO reports and the recently completed National Research Council study, The Small Business Innovation Research Program: An Assessment of the Department of Defense Fast Track Initiative, Washington, D.C.: National Academy of Press, 2000. Similarly, widely used cooperative research and development agreements (CRADAs) normally have limited assessment mechanisms. These limitations are described by D. Mowery in “Using Cooperative Research and Development Agreements as S&T Indicators: What do We Have and What Would We Like?” a presentation before the National Science Foundation conference, Workshop on Strategic Research Partnerships, 13 October 2000, publication of proceedings pending. Some partnership programs have benefited from regular assess-

Suggested Citation:"Findings and Recommendations." National Research Council. 2001. The Advanced Technology Program: Assessing Outcomes. Washington, DC: The National Academies Press. doi: 10.17226/10145.
×

Page 93

gram, in particular to new, small firms. Faster decision-making also would enhance the value of the debriefing process for unsuccessful firms. 28 This revolving application process will provide greater opportunity for applicants. It will also give the ATP management earlier and more accurate information concerning the rate of awards than is available under an annual award process. 29

  • B. Retain the debriefing process for unsuccessful applicants: Unsuccessful awardees find the debriefing process after an unsuccessful application to the ATP to be valuable even though more than three-fifths of the non-winners do not proceed with any aspect of the R&D project that they proposed to ATP. 30
  • C. Concentrate a significant proportion of the awards in selected thematic areas: One of the key features of the ATP is its use of general competitions, which are open to proposals from all areas of technology. The goal of the program is to compensate for market imperfections that result in underinvestment in certain types of technologies, a goal that distinguishes it from mission-oriented (and mission-constrained) R&D programs. 31 These general competitions should be maintained. At the same time, they could be usefully supplemented by allocating a proportion of ATP funding in selected thematic areas where the current technological opportunities are particularly promising for broad economic or social benefits. Awards to thematic areas can also be a means of addressing elements of important national missions and of generating synergies between related projects, and among companies, laboratories, and universities in areas of current technological promise. In

  • ment such as the Program for Next Generation Vehicles and the Advanced Battery Consortium. See National Research Council, Review of the Research Program of the Partnership for a New Generation of Vehicles: Sixth Report, Washington, D.C.: National Academy Press, 2000, and National Research Council, Effectiveness of the United States Advanced Battery Consortium as a Government-Industry Partnership, Washington, D.C.: National Academy Press, 1998.

    28 For a description of program modifications undertaken by the ATP management in the course of this review, see Alan P. Balutis and Barbara Lambis, “
    The ATP Competition Structure,” in this volume.

    29 Faster decision making has also been a concern for the SBIR program. The Department of Defense launched a successful initiative known as the Fast Track for firms able to demonstrate the ability to attract third-party finance. See National Research Council, The Small Business Innovation Research Program: An Assessment of the Department of Defense Fast Track Initiative, op. cit., passim.


    30 See the paper by Maryann P. Feldman and Maryellen R. Kelley, “
    Leveraging Research and Development: The Impact of the Advanced Technology Program,” in this volume.

    31 J. Lerner and C. Kegler, “Evaluating the Small Business Innovation Research Program: A Literature Review,” in National Research Council, The Small Business Innovation Research Program: An Assessment of the Department of Defense Fast Track Initiative, op. cit., pp. 309-314. For an informed discussion of U.S. technology policy see L. M. Branscomb and R. Florida, “Challenges to Technology Policy in a Changing World Economy,” in Branscomb and Keller, Investing in Innovation, op. cit.

    Suggested Citation:"Findings and Recommendations." National Research Council. 2001. The Advanced Technology Program: Assessing Outcomes. Washington, DC: The National Academies Press. doi: 10.17226/10145.
    ×

    Page 94

    these cases, the program should attempt to reap the higher returns from realizing complementarities and synergies among projects and R&D-performing institutions. 32

    • D. Enhance current efforts to integrate assessment results into the decision process: As noted, the quality of the ATP assessment effort is a major attribute of the program. The integration of the results of the assessments must remain a major goal of the program. 33 The early release of outside assessments to the research community would facilitate the dissemination of the research results.
    • E. Increase the Nation's Return on the Operation of the Program:
      • 1. Maximizing Return: As noted above, the program is achieving its goals. 34 It has a deserved reputation as a program that is well managed and under which awards are fairly awarded. 35 In our view, based on this review of the program, the ATP could use more funding effectively and efficiently, consistent with the goals set for the program. A more predictable funding base would also ensure that the program continues to attract quality proposals, provide flexibility to address new opportunities and ensure the maximum return on existing investments.
      • 2. Stability for R&D Funding: In any case, every effort should be made to provide greater stability in the funding of the program. The current instability creates uncertainty for participants and potential applicants about the funding of multi-year program commitments and is particularly difficult for small firms. 36


    32 The generation of “social capital” made possible by these awards underscores the role of government finance for technological innovation.

    33 See Panel II, in this volume, where current efforts to integrate evaluation findings are described.

    34 This does not mean that all awards are crowned with success. As would be expected for a high-risk R&D program, a significant portion of the awards do not succeed. This experience strongly parallels the experience of venture-backed investment. For example, one study found that out of a sample of 794 venture capital investments made over three decades, only 22.5 percent ultimately succeeded in going public: see P.A. Gompers, “Optimal investment, monitoring, and the staging of venture capital,” Journal of Finance 50(5):1461-1489. Concerning program evaluation, see the discussion in Panel II, especially the remarks by Irwin Feller of Pennsylvania State University, in this volume and Panel IV, especially the presentation on “ Economic Returns to New Medical Technologies” by Taylor Bingham of the Research Triangle Institute, also in this volume. Concerning program accomplishments (and failures), see the overview provided by Rosalie Ruegg, “Taking a Step Back: An Early Results Overview of Fifty ATP Awards,” in this volume.

    35 Feldman and Kelley, “The Case for Government R&D Additionally,” op. cit. The authors provide evidence that the investment community attaches value to the ATP awards through the highly selective and competitive nature of the award process. A significant percentage of even the non-winners in the selection process found the debriefing process to be helpful suggesting useful guidance for improving the firm's technical and/or business planning was made available.


    Suggested Citation:"Findings and Recommendations." National Research Council. 2001. The Advanced Technology Program: Assessing Outcomes. Washington, DC: The National Academies Press. doi: 10.17226/10145.
    ×

    Page 95

    Image: jpg
    ~ enlarge ~




    • F. Continue Focus on Small Business: A significant portion of the program funds (i.e., more than 60 percent) are awarded to small business. This reflects small business's unique capabilities as a source of low-overhead innovation. 37 Notwithstanding this recognition of the innovative capabilities of small business, the diversity of the ATP awards, involving both large and small companies, is an important feature of the program, and should be retained (see G below). The substantial size of the ATP awards, their multi-year disbursement, and the opportunity to collaborate with other institutions (e.g. universities) and larger firms make ATP funding particularly attractive to small firms. The ATP can thus contribute to the development of new technologies that meet its criteria of broad social benefits and enhance returns on the U.S. investment in research.
    • G. Retain Joint Ventures and Large Company Involvement: The participation of large companies is a unique and valuable characteristic of the

    36 Roger Noll and Linda Cohen emphasize the need to avoid large swings in annual funding for R&D programs. See The Technology Pork Barrel, Washington, D.C.: The Brookings Institution, 1991, p. vii.

    37 David B. Audretsch and Roy Thurik, Innovation, Industry, Evolution, and Employment, New York: Cambridge University Press, 1999.

    Suggested Citation:"Findings and Recommendations." National Research Council. 2001. The Advanced Technology Program: Assessing Outcomes. Washington, DC: The National Academies Press. doi: 10.17226/10145.
    ×

    Page 96

    ATP. 38 Large companies bring unique resources and capabilities to the development of new technologies and can be valuable partners for technologically innovative companies new to the market. 39 The participation of larger companies can also ensure better access to downstream markets for the small firms with which they collaborate under this program. 40 Accordingly, awards to joint ventures involving large companies should be retained. The current 60 percent funding requirement for large companies should also be retained; it should not, however, be significantly increased.

    • H. Coordinate ATP with SBIR: The SBIR and the ATP programs are different in important ways. However, they can be understood as separate steps on a national innovation ladder. In cases where applicants to the ATP do not have sufficiently developed business plans, but do have sound technologies, they might well be remanded automatically to an appropriate SBIR program. To the extent their technology has met the requirements of the ATP, SBIR program managers could be assured of the potential of the proposed technology. 41






    38 Dr. Mary L. Good describes the leverage offered by an ATP award to win internal support for a promising technology at Allied Signal. As Dr. Good describes it, the award fit the conditions associated with ATP (i.e., early technology development, an enabling technology, and collaborative work with universities resulting in the creation of a new material). She adds that the capabilities of a large company (i.e., expensive equipment and experienced technologists) were crucial to the success of the award. L. M. Branscomb et al., Managing Technical Risk: Understanding Private Sector Decision Making on Early Stage, Technology-based Projects, NIST GCR 00787, prepared for the Advanced Technology Program, April 2000, p. 42

    39 For a further discussion of this point see C. Hill, “The Advanced Technology Program: Opportunities for Enhancement,” in Branscomb and Keller, Investing in Innovation, op. cit., pp. 159-160. Hill suggests that because R&D decisions are often decentralized, large firms may operate much like independent, small firms particularly for projects that have high ratios of social to private returns.

    40 The development and marketing of the digitally-enhanced mammography diagnostic instrument (referred to in footnote 19) illustrates the synergy between large and small firms. The substantial marketing advantage of an established firm such as GE means the benefits of this new technology are rapidly and widely distributed. The laboratory manager responsible for developing the mammography diagnostic technology, Dr. Bruce Griffing, states that this promising technological development might well not have occurred in the absence of a government R&D award from the ATP. As noted above, this diagnostic system produces substantially fewer false positives. The lower false positive diagnoses reduce the need for expensive “work-ups” with the associated health care costs and personal trauma. Over time, the technology has the potential to virtually eliminate costs associated with film storage, retrieval, and transmission. The social benefits or spillovers appear substantial. The development of this technology also illustrates the impact federal R&D awards can have on decision making in large companies where multiple options, established hurdle rates, and technological and market uncertainties mitigate against even promising technologies. As Dr. Griffing remarked in a recent seminar, “There is a valley of death for new technologies, even in the largest companies.” Between Invention and Innovation: Mapping the Funding for Early Stage Technologies, Carnegie Conference Center, 25 January 2001, Washington, D.C.



    41 To a limited extent, this process already occurs in reverse. Successful applicants to an SBIR program may subsequently apply to the ATP. However, firms that do not qualify in an early stage of their development for an ATP award may well meet the different criteria for an SBIR award. There are cases

    Suggested Citation:"Findings and Recommendations." National Research Council. 2001. The Advanced Technology Program: Assessing Outcomes. Washington, DC: The National Academies Press. doi: 10.17226/10145.
    ×

    Page 97

    III. New Initiatives for the Program

    • A. Increased Collaboration on National Initiatives:

      ATP's collaboration with agencies responsible for national initiatives such as the Human Genome should be substantially increased. The Advanced Technology Program has established a “core competency” in its ability to screen, select, monitor, and assess projects of technological and commercial promise. As such, the ATP would be a valuable partner to research agencies and SBIR programs by working with them to develop valuable enabling technologies based on their investments in health and other areas such as environmental remediation. 42

      The National Institutes of Health have shown unparalleled capability in the funding of basic health-related research and have made enormous progress in specific areas such as the sequencing of the human genome. However, NIH investments tend to be focused on the generation and demonstration of new research ideas. The comparative advantage of the ATP is its ability to provide R&D funds to stimulate specific sectors and companies with the potential to develop these new ideas as commercial products and therefore make them available to a much wider group of users. An example of this approach is the ATP support for DNA tools, which is converting research findings into methods, devices, and reagents that actually work. 43 This type of collaboration between the ATP and health researchers should continue and expand.

    • B. Matching Grants by States:
      • 1. In some states, firms that receive ATP awards are currently eligible for grants from the state government. The NIST management should establish a regular outreach program to coordinate awards after the review process (or in conjunction) with state development programs.
      • 2. Matching State Funds: Consideration should be given to providing matching state funds for ATP awardees.44 Expanding the ATP's interaction with state programs to support high-technology companies

        where firms have progressed from an early SBIR award to an ATP award. See Donna Fossum, et al, Discovery and Innovation: Federal Research and Development Activities in the Fifty States, District of Columbia, and Puerto Rico, Science and Technology Policy Institute, MR-1194-OSTP, 2000.

        42 See the statement delivered by Jeffrey Schloss on behalf of Francis Collins, the Director of the National Human Genome Research Institute at the National Institutes of Health, in the first volume of the Committee's review of the ATP. See National Research Council, The Advanced Technology Program: Challenges and Opportunities, op. cit., pp. 56-59.

        43Ibid., p. 58.

        44 The ATP currently refers potential recipients of its funds to state science and technology program offices for technical assistance. See C. Hill, “The Advanced Technology Program: Opportunities for Enhancement,” in Branscomb and Keller, Investing in Innovation, op. cit., p. 165. Positive interactions currently take place between state and federal programs such as the ATP. See Marsha R. B.

    Suggested Citation:"Findings and Recommendations." National Research Council. 2001. The Advanced Technology Program: Assessing Outcomes. Washington, DC: The National Academies Press. doi: 10.17226/10145.
    ×

    Page 98

    within their borders would have a number of advantages. Making awards in parallel with state governments would:

    • a) Increase Certification: First, parallel awards would increase the certification impact of the ATP award in the local community by raising the firm's profile at the state level. This certification effect can serve to attract private investors by reducing uncertainty concerning the quality and potential commercial applications of the firm's technology. 45
    • b) Leverage Program Funding: Second, parallel awards might enable the Advanced Technology Program to reduce the size of its base award to individual small business applicants, thereby significantly expanding the reach of the program at no additional cost. In cases where the award size remains constant, the leverage of the award would be significantly and immediately increased by the addition of state funds. Cooperation with state programs would have the additional benefit of aligning the ATP's resources with state efforts, particularly in existing or nascent technological clusters, thereby improving the opportunities for the program and the awardees to reach critical mass.
    • c) Expand “Best Practice” Selection: The ATP has exceptional expertise in the review of technically-sound, commercially-feasible proposals by small independent companies and joint ventures operating with the advantages of large companies (noted above). Care would be required to ensure that an alignment of awards does not compromise the ATP's rigorous selection process. At the same time, ATP cooperation with state agencies would have the advantage of leveraging the ATP's expertise in selection and assessment, contributing to the quality of the state selection process, and the reach of the NIST-based ATP while preserving the current quality of the ATP selection and assessment program.

    The Steering Committee *

    Schachtel and Maryann P. Feldman, Reinforcing Interactions Between the Advanced Technology Program and State Technology Programs, Volume 1: A Guide to State Business Assistance Programs for New Technology Creation and Commercialization, NIST GCR 00-78, April 2000.

    45 Feldman and Kelley, The Case for Government R&D Additionality, op. cit., conclude that “winning an ATP award significantly increases the firm's success in attracting additional funds from other sources for R&D activities.” Their findings “provide strong evidence that the ATP award confers a halo effect on winners that makes them more likely to attract other funding when compared to non-winners of the same size…with projects of similar business and technical quality.”

    * For the Committee membership, see the front matter.

    Suggested Citation:"Findings and Recommendations." National Research Council. 2001. The Advanced Technology Program: Assessing Outcomes. Washington, DC: The National Academies Press. doi: 10.17226/10145.
    ×
    Page 83
    Suggested Citation:"Findings and Recommendations." National Research Council. 2001. The Advanced Technology Program: Assessing Outcomes. Washington, DC: The National Academies Press. doi: 10.17226/10145.
    ×
    Page 84
    Suggested Citation:"Findings and Recommendations." National Research Council. 2001. The Advanced Technology Program: Assessing Outcomes. Washington, DC: The National Academies Press. doi: 10.17226/10145.
    ×
    Page 85
    Suggested Citation:"Findings and Recommendations." National Research Council. 2001. The Advanced Technology Program: Assessing Outcomes. Washington, DC: The National Academies Press. doi: 10.17226/10145.
    ×
    Page 86
    Suggested Citation:"Findings and Recommendations." National Research Council. 2001. The Advanced Technology Program: Assessing Outcomes. Washington, DC: The National Academies Press. doi: 10.17226/10145.
    ×
    Page 87
    Suggested Citation:"Findings and Recommendations." National Research Council. 2001. The Advanced Technology Program: Assessing Outcomes. Washington, DC: The National Academies Press. doi: 10.17226/10145.
    ×
    Page 88
    Suggested Citation:"Findings and Recommendations." National Research Council. 2001. The Advanced Technology Program: Assessing Outcomes. Washington, DC: The National Academies Press. doi: 10.17226/10145.
    ×
    Page 89
    Suggested Citation:"Findings and Recommendations." National Research Council. 2001. The Advanced Technology Program: Assessing Outcomes. Washington, DC: The National Academies Press. doi: 10.17226/10145.
    ×
    Page 90
    Suggested Citation:"Findings and Recommendations." National Research Council. 2001. The Advanced Technology Program: Assessing Outcomes. Washington, DC: The National Academies Press. doi: 10.17226/10145.
    ×
    Page 91
    Suggested Citation:"Findings and Recommendations." National Research Council. 2001. The Advanced Technology Program: Assessing Outcomes. Washington, DC: The National Academies Press. doi: 10.17226/10145.
    ×
    Page 92
    Suggested Citation:"Findings and Recommendations." National Research Council. 2001. The Advanced Technology Program: Assessing Outcomes. Washington, DC: The National Academies Press. doi: 10.17226/10145.
    ×
    Page 93
    Suggested Citation:"Findings and Recommendations." National Research Council. 2001. The Advanced Technology Program: Assessing Outcomes. Washington, DC: The National Academies Press. doi: 10.17226/10145.
    ×
    Page 94
    Suggested Citation:"Findings and Recommendations." National Research Council. 2001. The Advanced Technology Program: Assessing Outcomes. Washington, DC: The National Academies Press. doi: 10.17226/10145.
    ×
    Page 95
    Suggested Citation:"Findings and Recommendations." National Research Council. 2001. The Advanced Technology Program: Assessing Outcomes. Washington, DC: The National Academies Press. doi: 10.17226/10145.
    ×
    Page 96
    Suggested Citation:"Findings and Recommendations." National Research Council. 2001. The Advanced Technology Program: Assessing Outcomes. Washington, DC: The National Academies Press. doi: 10.17226/10145.
    ×
    Page 97
    Suggested Citation:"Findings and Recommendations." National Research Council. 2001. The Advanced Technology Program: Assessing Outcomes. Washington, DC: The National Academies Press. doi: 10.17226/10145.
    ×
    Page 98
    Next: Welcome »
    The Advanced Technology Program: Assessing Outcomes Get This Book
    ×
    Buy Paperback | $80.00 Buy Ebook | $64.99
    MyNAP members save 10% online.
    Login or Register to save!
    Download Free PDF

    This report examines the operations of the APT, reviews its extensive assessment program, and provides NRC Committee findings concerning the ATP’s operations and recommendations for potential improvements to the program. The report includes a summary of a major conference held in April 2000 as well as seven papers, including surveys of the industry participants or users of the ATP program, a summary of the results of fifty awards, detailed assessments of major joint ventures, and a description of the current selection process. It is the most comprehensive study to date of the program’s origins, operations, achievements, and assessment. Its conclusion: the program works.

    1. ×

      Welcome to OpenBook!

      You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

      Do you want to take a quick tour of the OpenBook's features?

      No Thanks Take a Tour »
    2. ×

      Show this book's table of contents, where you can jump to any chapter by name.

      « Back Next »
    3. ×

      ...or use these buttons to go back to the previous chapter or skip to the next one.

      « Back Next »
    4. ×

      Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

      « Back Next »
    5. ×

      Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

      « Back Next »
    6. ×

      To search the entire text of this book, type in your search term here and press Enter.

      « Back Next »
    7. ×

      Share a link to this book page on your preferred social network or via email.

      « Back Next »
    8. ×

      View our suggested citation for this chapter.

      « Back Next »
    9. ×

      Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

      « Back Next »
    Stay Connected!