National Academies Press: OpenBook

The Advanced Technology Program: Assessing Outcomes (2001)

Chapter: Panel IV: Assessing Progress: Case Study Cluster

« Previous: Panel III: Stimulating R&D Investment
Suggested Citation:"Panel IV: Assessing Progress: Case Study Cluster." National Research Council. 2001. The Advanced Technology Program: Assessing Outcomes. Washington, DC: The National Academies Press. doi: 10.17226/10145.
×

Page 145

Panel IV:

Assessing Progress: Case Study Cluster

INTRODUCTION

David Austin

Resources for the Future

The coming session, Dr. Austin said, would present a series of studies on a variety of topics. The first would be a presentation by David Ayares on a project designed to address the pressing national need for organ transplants. The second reviews case study methodologies for evaluations, and the third focuses on economic returns in medical technologies.

As background information, he called attention to three recent ATP studies assessing the ATP program:

  • Estimating Future Consumer Benefits from ATP-Funded Innovation: The Case of Digital Data Storage 18

  • Managing Technical Risk: Understanding Private Sector Decision Making on Early Stage Technology-based Projects 19

  • Performance of Completed Projects: Status Report Number 1 20

18 D. Austin and M. Macauley, Estimating Future Consumer Benefits from ATP-funded Innovation: The Case of Digital Data Storage, NIST GCR 00790, April 2000.

19 L. M. Branscomb, K. P. Morse, and M. J. Roberts, Managing Technical Risk: Understanding Private Sector Decision Making on Early Stage Technology-based Projects, NIST GCR 00787, April 2000.

20 W. F. Long, Advanced Technology Program, op. cit.

Suggested Citation:"Panel IV: Assessing Progress: Case Study Cluster." National Research Council. 2001. The Advanced Technology Program: Assessing Outcomes. Washington, DC: The National Academies Press. doi: 10.17226/10145.
×

Page 146

Each of these studies has value in its own right. Collectively, they underscore the quality and scope of the analysis referred to by Professor Feller and Dr. Wessner.

THE FIVE LITTLE PIGS:A TALE OF XENO-ORGAN TRANSPLANT

David Ayares

PPL Therapeutics, Inc.

PPL Therapeutics, said Mr. Ayares, is new to the ATP, having received its first grant only five months previously. The company—a U.S. subsidiary of a foreign company—is working to develop the technology of transgenic animals, based on earlier work at the Roslyn Institute in Scotland. At its Blacksburg, Virginia, facility it has two technology focuses:

  • transgenic production, that is, transplanting genes for various human proteins into farm animals, to produce pharmaceutically and nutritionally important proteins in their milk; and

  • xenotransplantation, that is, genetically modifying pigs so as to produce cells and organs suitable for transplantation into humans without spurring immune rejection of the grafts.

PPL's lead product is alpha-1-antitrypsin (AAT), a compound used in treating cystic fibrosis and other diseases. It is produced in the milk of sheep at very high rates, making the process much cheaper than the traditional recombinant methods. The company is also working on recombinant methods for producing a variety of other human proteins for therapeutic and nutritional purposes. It has several products in clinical trials. The company, PPL Therapeutics, became widely known a few years ago for producing Dolly, the cloned sheep. Cloning of animals is a necessary step in order to multiply quickly the amounts of the various pharmaceutical products that the original transgenic animals yield.

Xenotransplantation

This technology addresses a critical shortage of human organs for transplantation purposes. Altogether more than 62,000 patients are on waiting lists for lungs, hearts, or kidneys, and every day 11 of them die while waiting. The demand for transplants is growing, but the supply of human organs is not. The potential market is estimated at $10 billion per year.

Xenografts from pigs have several advantages:

  • the supply is potentially unlimited (through cloning);

Suggested Citation:"Panel IV: Assessing Progress: Case Study Cluster." National Research Council. 2001. The Advanced Technology Program: Assessing Outcomes. Washington, DC: The National Academies Press. doi: 10.17226/10145.
×

Page 147

  • pigs as sources of organs seem to be most nearly compatible to humans;

  • the organs, tissues, and cells can, in principle, be engineered to overcome rejection; and

  • it is possible to induce a recipient to tolerate a transplant by a variety of means.

The Big Bad Wolf and the Five Little Pigs

Echoing David Morgenthaler's observations, Dr. Ayares remarked that maintaining financial solvency can be difficult for a medium-sized biotechnology company, especially in today's volatile markets. From a financial perspective, most of the past decade has been hard on such companies. Some have gone out of business, and nearly all have been strapped for operating funds. To safeguard its future, PPL has built financial houses of straw, sticks, and brick for its xenograft program—just as the pigs in the old story did.

In this story the Big Bad Wolf represents these financial threats. There are five little pigs, not three, because that is the number of the first cloned litter of transgenic pigs.

PPL's xenograft program has the following specific areas of focus:

  • hyperacute rejection (which we address by deleting the pig gene that produces a particular sugar molecule that evokes this violent and catastrophic rejection mechanism);

  • delayed xenograft rejection;

  • T cell-mediated rejection; and

  • pig development.

To pursue these problems, the company has spent $8 million since 1996 on research in the United Kingdom and the United States.

“And He Huffed and He Puffed…”

PPL Therapeutics, the parent firm, raised $34 million near the fourth quarter of 1998. It was spending at the rate of about $20 million annually, with two products in clinical trials and no corporate partner for one of them. The company knew it would run out of money by the end of fiscal 2000. The board of directors was calling for the company to focus on its lead products (proteins in milk), and the xenograft program was given until September 1999 to find a partner or be forced to shut down.

To raise money, we aggressively pursued deals with several large pharmaceutical companies and with other xenograft companies. We wrote a business plan, which we showed to venture capitalists and private investors. We also pursued joint ventures. None of these efforts bore fruit. Risks and costs were too high, and payoffs too distant for potential investors.

Suggested Citation:"Panel IV: Assessing Progress: Case Study Cluster." National Research Council. 2001. The Advanced Technology Program: Assessing Outcomes. Washington, DC: The National Academies Press. doi: 10.17226/10145.
×

Page 148

Time was running out for us. But then we stumbled on the ATP. One of our senior scientists—Irina Polejaeva—met an ATP officer at an embryonic stem cell meeting in Madison, Wisconsin. After discussing the xenograft work, the officer suggested we should submit an ATP proposal.

Once we understood that the program was open to us, we submitted an ATP proposal, “Cloning Pigs: A Solution to Overcoming Rejection in Organs for Transplantation.” Its goals were fourfold:

  • 1. develop gene knockout technology in primary pig cells, using a knockout gene, the alpha 1,3-galactosl transferase gene;
  • 2. develop pig cloning technology;
  • 3. clone pigs using the genetically modified alpha-1.3-GT gene; and
  • 4. use success in the preceding to develop strategic alliances for commercialization.

By the second quarter of 1999 we were still burning $18 to $20 million per year. We had lost our lead partner for AAT—our main product. The big pharmaceutical companies decided that investments in xenografts were too long-term and high-risk for comfort. Our business plan was still in the hands of venture capitalists, who had not offered funding.

PPL made the first cut of the ATP selection, and was invited to give an oral defense in June 1999. On October 6, we were awarded an ATP grant of $2 million over three years.

But our partners in AAT were still awaiting the results of phase II clinical trials. We were still spending $20 million a year, and the board decided that spending needed to be cut substantially to give the company time to achieve some product revenues from AAT. PPL cut its U.K. and U.S. staff by 30 percent. PPL, Inc.—the American company—lost 12 of its 34 employees. Research on rabbits (to produce calcitonin) and cows (for a variety of useful proteins in milk) was dropped.

Life Inside a Brick House

But as news of the ATP award spread, the phone began to ring. Big pharmaceutical companies who previously thought our research was too risky, wanted to talk. Venture capitalists were no longer hemming and hawing when we talked to them; they wanted to talk about structuring a spin-out company. Xenograft companies are trying to license the patents and otherwise cooperate. The stock price has doubled, giving a market value $80 million. And the board of directors is willing to support xenografts as long as it takes to find a partner, whether it is a major pharmaceutical company or a venture capital firm.

On March 5, 2000, the world's first cloned pigs were born. We have teams of world-class scientists working toward clinical trials, commercialization, and great

Suggested Citation:"Panel IV: Assessing Progress: Case Study Cluster." National Research Council. 2001. The Advanced Technology Program: Assessing Outcomes. Washington, DC: The National Academies Press. doi: 10.17226/10145.
×

Page 149

medical and economic benefits. The future we envision is an infrastructure of high-technology pig farms and associated products and services that will make transplantation safer, more convenient, and easier to arrange. The backlogs will disappear. For thousands of people each year, life will be transformed from a steady decline to full health.

Without ATP there would be no cloned pigs. Success would have been delayed by years, and perhaps forever. The entire field of xenotransplantation would have withered or shrunk. The big bad wolf of inadequate financing would have won.

EXTENDING CASE STUDY METHODOLOGIES FOR TECHNOLOGY POLICY EVALUATION

Todd A. Watkins

Lehigh University

Inadequate Incentive for Innovation

The links between competitiveness and technology, and the questions those links raise for technology policy or, more broadly, industrial policy, have major implications for a nation's wealth, living standards, and national security. Policy-makers and economists have long understood technological innovation as important for economic growth. It is also well understood that under some circumstances competitive markets will fail to generate adequate incentives to innovate.

Market Failure and Spillovers

The market failure justifications for government support of R&D are well rehearsed and widely discussed in the innovation economics and technology policy literature. Three main market failures are ubiquitous for R&D: increasing returns, from both externalities and indivisibilities (such as fixed R&D costs), and uncertainty. All are important for thinking about science and technology policies, but the increasing returns due to non-rival-good externalities are of most relevance for the ATP's focus on spillovers.

Empirical evidence from a wide variety of studies, industries, and methodologies supports the view that the social benefits of R&D are consistently high and, importantly for policymaking, higher than the private benefits, by factors typically ranging from 30 to 80 percent and sometimes as much as 300 percent. The persistence of high social rates of return to R&D is an indication that not enough is being invested in R&D from a social maximizing point of view. Otherwise returns to R&D would return to more normal rates.

Recognizing this chronic underinvestment, when the ATP was created by the Omnibus Trade and Competitiveness Act of 1988, its goals were “to assist U.S. business in creating and applying the generic technology and research results to

Suggested Citation:"Panel IV: Assessing Progress: Case Study Cluster." National Research Council. 2001. The Advanced Technology Program: Assessing Outcomes. Washington, DC: The National Academies Press. doi: 10.17226/10145.
×

Page 150

(1) commercialize significant new scientific discoveries and technologies rapidly and (2) refine manufacturing technologies” (P.L. 100-418). This has since been reformulated to emphasize that the ATP will foster economic growth and the competitiveness of U.S. firms by advancing technologies that have potentially large net social value for the nation but would not otherwise emerge in time to maximize their competitive value.

By subsidizing specific commercially-relevant high-risk R&D projects, the ATP aims, among other things, to accelerate the benefits from emerging technologies, to widen potential applications areas, and to increase the likelihood of technical success. In other words, the economic rationale for the ATP is, at its core, an R&D market failure argument. Hence, maximizing net economic spillovers is a central policy goal. By extension, a central issue for policy debate, program design, and program evaluation is whether the ATP succeeds at fostering spillovers and how it might increase them.

Research Support: ATP Awards Do Generate Technical Advance and Positive Spillovers

From the ATP Economic Assessment Office, there is a growing body of research evidence that ATP projects do indeed foster technical advances, increase the competitiveness of participating firms, and generate positive spillovers. However, as mentioned above, a wide variety of empirical studies of industrial R&D, including privately funded R&D indicate that R&D on average does all three as well, whether funded by the government or not. It is insufficient for ATP evaluations to find only increases along these three dimensions. To be shown effective, the ATP must succeed in sponsoring technologies with above average net return to the nation and that would not have emerged without the ATP.

Key Questions

Does the ATP deliver as advertised? Four questions are central to previous ATP evaluation research.

  • 1. Is knowledge advanced? Have ATP projects advanced scientific and technological knowledge?
  • 2. Is company performance improved? Have ATP projects increased the economic and competitive performance of U.S. companies?
  • 3. Are there net spillover benefits? Have ATP projects generated additional net benefits that spillover to the broader economy of non-participants?
  • 4. Does ATP pick the right projects? Does it succeed in identifying high-spillover projects relative to what would happen without it?

Questions 1 and 2 are the most straightforward to investigate and have been the principal focus of most existing ATP evaluations. They also have the clearest

Suggested Citation:"Panel IV: Assessing Progress: Case Study Cluster." National Research Council. 2001. The Advanced Technology Program: Assessing Outcomes. Washington, DC: The National Academies Press. doi: 10.17226/10145.
×

Page 151

affirmative evidence already. Questions 3 and 4 remain significantly under-explored because the methodologies previously employed did not enable investigating them. For example, most existing ATP evaluation studies use only ATP participants in their research designs. So, even though questions 1 and 2 are the most straightforward questions, most of the studies acknowledge that they will miss advances in organizations that do not participate in ATP, such as competitors, suppliers, and those in largely unrelated markets.

Direct Evaluation and Counterfactuals

Existing evaluation methods begin by estimating the direct effects of ATP on its participants (gathering data through interviews, surveys, document review and/ or expert forecasts) using metrics such as revenues, productivity gains, resource savings, decreased product maintenance costs, improved product quality, or reduced time required to launch new products. Evaluating the incremental effect of the ATP award then is a counterfactual (or countertemporal) exercise of comparing the current (or estimated future) state to what would have been (or was before) without ATP. However, by not including non-participants, the studies either ignore or have to estimate indirectly any effects (positive or negative) on the competitiveness of other players such as suppliers and competitors in the market.

Multi-Flavored Spillovers

Existing studies' focus on ATP participants is even more problematic when it comes to evaluating Questions 3 and 4 on spillovers. Part of the methodological problem is that spillovers come in two flavors:

  • those to customers, competitors, and suppliers within the markets served by the innovators (so called market spillovers); and

  • those to others not in the same markets, perhaps even completely unrelated to the innovator (otherwise known as knowledge spillovers).

In particular, it has limited ATP studies almost exclusively to market spillovers and, indeed more narrowly, almost exclusively to market spillovers to customers/users. Explicit evaluation of knowledge spillovers (value created for those other than players in the target market) is conspicuously absent. Nor can such participant-user isolated methodologies be of significant help in understanding spillover pathways or factors that facilitate or inhibit them.

Capturing the Social Value of Market and Knowledge Spillovers

So, how can case study methodologies be improved to more explicitly account for both market spillovers and knowledge spillovers? We extend the tradi-

Suggested Citation:"Panel IV: Assessing Progress: Case Study Cluster." National Research Council. 2001. The Advanced Technology Program: Assessing Outcomes. Washington, DC: The National Academies Press. doi: 10.17226/10145.
×

Page 152

tional case study methodology of Mansfield et al. in two significant ways that we believe are unique. 21 Implicitly, case study methods to date exclusively have focused on market spillovers. Knowledge spillovers have generally been the implicit focus of statistical studies using aggregate industry statistics, patent analysis or macroeconomic modeling. I am unaware of any study, either within or outside of ATP, of the returns to innovation that explicitly attempts to track both market and knowledge spillovers. It is clear that both are relevant to measuring the social value of innovative activity. It is unclear whether existing methodologies significantly misestimate spillovers by not explicitly accounting for both.

Snowballs: Mapping Technology Diffusion

To explicitly investigate both market spillovers and knowledge spillovers through case studies, the first methodological extension we developed was a combination of a snowball interviewing process—to identify and then include in our interviewing key customers, suppliers and competitors for market spillovers— coupled with patent and publication citation analysis to identify organizations outside the target markets for knowledge spillovers. We are thereby mapping the technology diffusion process at least one layer into the broader network by including in the case non-participating customers, competitors, suppliers, and other organizations.

Finally, the second methodological extension concerns the means of accounting for the economic value of technologies displaced from the market by innovations emerging from the projects. In theory, the incremental economic value attributable to an innovation (and, by extension, policies that support them) nets out the economic value of what would have been if the innovation had not occurred. This is either (or both) a forecasting or historical counterfactual exercise, subject to large uncertainties and subjectivity.

Dynamic Displacement

In empirical practice this fact has required subtracting for the forecast value of the displaced defender technologies already in the market. Note, however, that this counterfactual, “compared-to-what” valuation of a new technology takes a static view of the “old” market. It essentially ignores the potential value of parallel research projects elsewhere, the results of which were not yet in the market. However, parallel research projects among competitors are likely within markets characterized by dynamic strategic interaction among competitors in rapidly moving high technology. When a new innovation enters a market, it might not only

21 See E. Mansfield, “Social and private rates of return from industrial innovations,” Quarterly Journal of Economics, 91(2):221-240, 1977. See also Z. Griliches, The Search for R&D Spillovers, Cambridge, MA: Harvard University Press, 1990.

Suggested Citation:"Panel IV: Assessing Progress: Case Study Cluster." National Research Council. 2001. The Advanced Technology Program: Assessing Outcomes. Washington, DC: The National Academies Press. doi: 10.17226/10145.
×

Page 153

displace the existing defender in the market but also preempt a second, or third or more almost-rans that would have otherwise emerged. We call this “dynamic displacement” and believe it is theoretically superior and in the context of counterfactual estimation no more subject to error empirically.

When both the R&D race winner and loser are U.S. firms, then using a static defender technology to evaluate the value of the winner's R&D will lead to overestimates of its net value to the nation. If government subsidies were responsible in part for the emergence of the first new technology, the appropriate subtraction for “compared to no government subsidy” would include the value of whatever second new technology (if any) would have eventually emerged instead. Subtraction only for the static old technology is appropriate only when no competitive parallel R&D project would have been eventually successful.

To take an extreme example to make the point, an evaluation of the economic value of government funding of CERN, the European particle physics research center, could add up to literally hundreds of billions, and perhaps trillions, of dollars, because CERN invented the World Wide Web (WWW). Clearly the WWW stimulated a very large share of the Internet economy. But we would only assign all those trillions as returns to government funding of CERN if we believe that no one else would have developed any alternative. Given how rapidly computer and semiconductor and software technologies have advanced, it is impossible to believe something else like it, would have escaped the world without CERN. CERN, though out front, was not alone in seeking relatively easier access to widely dispersed electronic information. In other words, organizations operating in similar technological areas tend to face similar innovations possibilities frontiers. Would we really all still only be using Veronica and Gopher through Unix instruction sets?

Static vs. Dynamic Models

The empirical attraction of the static defender is that it is known. Dynamic also-rans are counterfactual. Yet to assign value to a static defender in empirical practice, we also undertake counterfactual historical and forecasting analysis: what would the world have been like in 1990, 1995, and 2000 if it were still using only Veronica and Gopher? We see little reason to believe that estimates like this will on average be better than estimates to dynamic questions such as: who are the two organizations that you believe were closest to introducing similar technologies?; or how many more months or years longer might it have taken them to introduce it? In addition, by using a snowball interviewing process, such estimates can be verified or modified by then going to interview the two organizations thus identified.

So, where counterfactual estimation is necessary, we are investigating a dynamic interpretation of the defender technology for determining the counterfactual economic value displaced by innovations, comparing both a static and dynamic

Suggested Citation:"Panel IV: Assessing Progress: Case Study Cluster." National Research Council. 2001. The Advanced Technology Program: Assessing Outcomes. Washington, DC: The National Academies Press. doi: 10.17226/10145.
×

Page 154

defender interpretation in order to evaluate the difference between the two approaches in terms of their estimates of net social value as well as their empirical feasibility.

ECONOMIC RETURNS TO NEW MEDICAL TECHNOLOGIES

Tayler H. Bingham

Research Triangle Institute

Mr. Bingham recalled prehistory, a time when the inventor and the person with the need for an invention were probably the same person. Surely a farmer invented the plow. The industrial revolution, however, was a period of the division and specialization of labor. Different people in different places, embedded in different organizations performed different activities. This trend has not abated. We are continuing to grow more specialized. Markets were traditionally viewed as effectively coordinating the various productive activities quite well, at least until the early part of the twentieth century. Prices sent approximately the right signals about the values and costs of products and services.

The modern view has become more sophisticated. We now discuss the possibility of so-called market failures. Some of the benefits of certain economic activities (such as research and development) are not always appropriable by those who undertake them. The implication is that the returns to the inventor and the returns to society differ. Where there are market failures, private markets do not provide the socially desirable type and number of goods such as R&D, and the government has a role.

He and his colleagues at the Research Triangle Institute (RTI) assessed the economic impacts of NIST support for new medical technologies for the ATP, under contract to NIST. 22 Dr. Sheila Martin (who has since left the Center to become policy advisor to the Governor of the State of Washington) directed the study.

They found that there are a number of reasons to believe that the difference between the benefits to society and those to inventors is large in the medical field. The ATP is one response to this gap between the public and private returns to R&D.

Seven Projects Assessed

The RTI team worked with NIST on seven multiple-application tissue-engineering projects, including projects addressing the diagnosis and treatment

22 S. A. Martin, D. L., Winfield, A. E. Kenyon, J. R. Farris, M. V. Bala, and T. H. Bingham, A Framework for Estimating the National Economic Benefits of ATP Funding of Medical Technologies: Preliminary Applications to Tissue Engineering Projects Funded from 1990 to 1996, prepared by the Research Triangle Institute for the Advanced Technology Program, NIST GCR 97-737, April 1998.

Suggested Citation:"Panel IV: Assessing Progress: Case Study Cluster." National Research Council. 2001. The Advanced Technology Program: Assessing Outcomes. Washington, DC: The National Academies Press. doi: 10.17226/10145.
×

Page 155

of diseases such as cancer and diabetes, the treatment of damaged ligaments and tendons, and procedures for the transplantation of xenogenic organs.

Methods Used: Life Without Railroads

To evaluate the potential contribution of ATP for the tissue engineering projects requires the need for a counterfactual—what would have happened without NIST support? To address that question, they borrowed a technique used by Robert W. Fogel in 1964. 23 Fogel wondered what the economy would have looked like in 1890 if railroads had never been developed—that is, if the defender technologies, such as rivers, canals, and roads, had prevailed as the nation's dominant transportation networks. In modeling this counterfactual world, he found that the railroad had contributed only 5 percent to economic welfare.

They posed a parallel question. They sought to compare the benefits and costs of the new health technology including the effects of the investments made by the ATP, with those of the defender technologies (the “non-ATP” case). They modeled the entire process, from R&D to health outcomes. RTI assumed that the innovators are rational: they are motivated not by curiosity, but by the chance to earn income from their technologies.

Shakespeare's Metric

How do you measure the health consequences of these technologies? Shakespeare, in As You Like It, identified seven ages of man, in which we start out in dependency and eventually end up in dependency again. A person's health stock does decline over time. But there are ways to improve our health, and the impacts of these techniques can be measured. They used a measure called the “quality-adjusted life-year” as a gauge of the health quality impacts of the technologies. Figure 3 illustrates the concept schematically.

Briefly, a person who is in perfect health is assigned a value of 1; a person who is dead is assigned a value of 0; and those at other states in between the two are assigned values in between. Estimates of where one is located along the spectrum, given an illness or disease, are based on interviews by health workers with people in various states of health.

The team estimated the statistical impacts of the technologies on the health of a cohort of patients and then calculated the probabilities of improvements in their health as a result of the impacts of NIST support of those technologies.



23 Robert W. Fogel, Which Road to the Past?: Two Views of History, New Haven: Yale University Press, 1964.

Suggested Citation:"Panel IV: Assessing Progress: Case Study Cluster." National Research Council. 2001. The Advanced Technology Program: Assessing Outcomes. Washington, DC: The National Academies Press. doi: 10.17226/10145.
×

Page 156

Image: jpg
~ enlarge ~

FIGURE 3 Schematic illustration of quality-adjusted life-year concept




The final question they addressed is “How much does society benefit from the contribution of NIST to these technologies?” The technologies are linked to the national economy by

  • wage and salary incomes;

  • owners' profits;

  • tax payments;

  • health expenditures; and

  • health state, as represented by the value of a quality-adjusted life year.

The value of a life-year is traditionally estimated by economists according to people's willingness to pay to save or extend life. Statistically, this figure is about $7 million per life. The value of a life-year is valued something in the neighborhood of $0.25 million. Using this measurement, it is then possible to adjust for quality of life, in calculating society's benefit from medical technology.

Summary of Findings: Higher Social Returns

The study estimated the expected social return on the public (ATP) investment in these technologies at $34 billion, in net present value terms, or an IRR (internal rate of return) of 116 percent. The expected total social return on both public and private investments is estimated at $109 billion, net present value, or an IRR of 115 percent. The expected total private return on these investments is estimated at $1.6 billion ($914 million of it attributable to public funding), net present value, or an IRR of 12 percent.

Suggested Citation:"Panel IV: Assessing Progress: Case Study Cluster." National Research Council. 2001. The Advanced Technology Program: Assessing Outcomes. Washington, DC: The National Academies Press. doi: 10.17226/10145.
×

Page 157

These results illustrate two important points about ATP's role in funding these technologies:

  • the ATP significantly impacts the expected social and private returns on these projects; and

  • the social returns are far greater than the private returns. Private companies, therefore, tend to underinvest in these technologies from society's standpoint. For this reason public incentives to the private sector are important in pursuing these technologies.

Because the seven technologies studied have not yet been applied, the study is a prospective one, based on preliminary results and on the expectations of the innovators involved and other informed people. Whether these expectations will be met is uncertain. However, the methodology developed in this study will allow ATP to update the results of this study, as more data on actual applications become available.

DISCUSSANT

Henry Kelly

White House Office of Science and Technology Policy

High Public Returns on R&D

Henry Kelly of the Office of Science and Technology Policy in the White House, recalled Harry Truman's perhaps apocryphal dictum: “If you took all of the economists and laid them end to end, they'd point in all directions.” With respect to the ATP, there is one place they all point in the same direction, that is, their agreement that the social rate of return to R&D investments is much higher than the private rate of return to R&D investments. It is beyond debate at this point.

In fact, we are probably underestimating the social rate of return, because there are so many intangible factors that are not measurable. The complexity of these spillovers for example—with knowledge spreading in a network of interactions among researchers in different fields and with different goals—is difficult to capture in an economic model. If one speeded up the development of one technology, how would that result propagate through the rest of the R&D system? Externalities that are not captured in conventional markets, such as the value of an additional year of life—or of a better, healthier life—are perhaps even more difficult to assess.

Suggested Citation:"Panel IV: Assessing Progress: Case Study Cluster." National Research Council. 2001. The Advanced Technology Program: Assessing Outcomes. Washington, DC: The National Academies Press. doi: 10.17226/10145.
×

Page 158

ATP: A Place to Welcome New Ideas

The Council of Economic Advisors 24 did a literature survey a few years ago showing that the average social rate of return to R&D is more than twice the private rate. The debate should not be whether there is an opportunity for public investment. The issue is whether we are doing it efficiently and well. We have a reasonably good track record in agencies with well-defined missions, such as the Departments of Energy and Transportation, and in areas where the government is the sole buyer, such as the Department of Defense and NASA. The exception is in the great collection of interesting and important crucial areas that do not fit in the missions of other agencies. The ATP has filled this important need. Uniquely in the federal government, it welcomes anyone with a great idea to apply for funding.

Close Scrutiny

The ATP is also unique in the extent to which it has been evaluated. The scrutiny applied to the ATP from outside has combined with its excellent evaluation staff to advance the science of economic evaluation.

“If We Didn't Have It, We'd Need to Invent It”

For all of these reasons, I believe, first, that there is no longer any debate as to the need for a program like the ATP. Second, after having responded and adapted to scrutiny over the past decade, the ATP is extremely well managed. The policy discussion at this point should be about how to carry it forward and continue improving it. If the ATP were not in place, addressing the problem of under-investment in R&D, we would need to invent something much like the ATP.

QUESTIONS FROM THE AUDIENCE

Social Return

Lewis Branscomb observed that private investments in R&D, like public investments, also produce excessive social returns. So what does the government do that is different? In other words, what market failures does the ATP address? The R&D “funding gap”, or under-investment in R&D by the private sector, is the one most often mentioned. Yet there are others. The recent report Managing

24 Council of Economic Advisers, Supporting Research and Development to Promote Economic Growth, Washington, D.C.: Executive Office of the President, 1995. See also National Research Council, Conflict and Cooperation, op. cit., pp. 33-35, Box A.

Suggested Citation:"Panel IV: Assessing Progress: Case Study Cluster." National Research Council. 2001. The Advanced Technology Program: Assessing Outcomes. Washington, DC: The National Academies Press. doi: 10.17226/10145.
×

Page 159

Technical Risk25 identifies a list of institutional failures that may deter such investments even if they may be in the private interest of a firm. These institutional failures include difficulties in communicating between technical and financial people, lack of trust, and the risk-taking inhibitions of large firms. Together these failures provide even greater support for the need for public investments like those made by the ATP.

Funding Enabling Technologies

In response, Rosalie Ruegg explained that the ATP's purpose goes beyond merely compensating for the “funding gap.” In focusing on “enabling technologies,” ATP funds technologies that are expected to have broader than average spillovers. To document this point, she commended to the group several recent studies of the economic basis of the ATP. 26

Proof of Concept

The ATP, said David Ayares of PPL Therapeutics, allowed his company to advance its xenograft technology to “proof of concept,” at which point it was able to file for patents. Private investors regard a strong patent portfolio as important evidence of a small company's ability to go forward toward profitability. PPL has filed for two patents, and expects to file for one or two more. PPL has received no other public funding for this technology.

Todd Watkins picked up Rosalie Ruegg's point that the ATP seeks to fund “high-spillover” projects. The ATP should, he suggested, try to identify the factors that are related to high spillover. The ATP's Economic Assessment Office has commissioned a series of studies of sectors to identify such technologies. 27 He asked if spillovers are built into the selection process as explicitly as they should be.

25 Branscomb, Morse, and Roberts, Managing Technical Risk, op. cit.

26 These included Austin and Macauley, Estimating Future Consumer Benefits for ATP-Funded Innovation, op. cit.; Branscomb, Morse, and Roberts, Managing Technical Risk, op. cit.; and Schachtel and Feldman, Reinforcing Interactions Between the Advanced Technology Program and State Technology Programs, op. cit.

27 For example, Austin and Macauley, Estimating Future Consumer Benefits for ATP-Funded Innovation, op. cit.; and Martin et al., A Framework for Estimating the National Economic Benefits of ATP Funding of Medical Technologies, op. cit.

Suggested Citation:"Panel IV: Assessing Progress: Case Study Cluster." National Research Council. 2001. The Advanced Technology Program: Assessing Outcomes. Washington, DC: The National Academies Press. doi: 10.17226/10145.
×
Page 145
Suggested Citation:"Panel IV: Assessing Progress: Case Study Cluster." National Research Council. 2001. The Advanced Technology Program: Assessing Outcomes. Washington, DC: The National Academies Press. doi: 10.17226/10145.
×
Page 146
Suggested Citation:"Panel IV: Assessing Progress: Case Study Cluster." National Research Council. 2001. The Advanced Technology Program: Assessing Outcomes. Washington, DC: The National Academies Press. doi: 10.17226/10145.
×
Page 147
Suggested Citation:"Panel IV: Assessing Progress: Case Study Cluster." National Research Council. 2001. The Advanced Technology Program: Assessing Outcomes. Washington, DC: The National Academies Press. doi: 10.17226/10145.
×
Page 148
Suggested Citation:"Panel IV: Assessing Progress: Case Study Cluster." National Research Council. 2001. The Advanced Technology Program: Assessing Outcomes. Washington, DC: The National Academies Press. doi: 10.17226/10145.
×
Page 149
Suggested Citation:"Panel IV: Assessing Progress: Case Study Cluster." National Research Council. 2001. The Advanced Technology Program: Assessing Outcomes. Washington, DC: The National Academies Press. doi: 10.17226/10145.
×
Page 150
Suggested Citation:"Panel IV: Assessing Progress: Case Study Cluster." National Research Council. 2001. The Advanced Technology Program: Assessing Outcomes. Washington, DC: The National Academies Press. doi: 10.17226/10145.
×
Page 151
Suggested Citation:"Panel IV: Assessing Progress: Case Study Cluster." National Research Council. 2001. The Advanced Technology Program: Assessing Outcomes. Washington, DC: The National Academies Press. doi: 10.17226/10145.
×
Page 152
Suggested Citation:"Panel IV: Assessing Progress: Case Study Cluster." National Research Council. 2001. The Advanced Technology Program: Assessing Outcomes. Washington, DC: The National Academies Press. doi: 10.17226/10145.
×
Page 153
Suggested Citation:"Panel IV: Assessing Progress: Case Study Cluster." National Research Council. 2001. The Advanced Technology Program: Assessing Outcomes. Washington, DC: The National Academies Press. doi: 10.17226/10145.
×
Page 154
Suggested Citation:"Panel IV: Assessing Progress: Case Study Cluster." National Research Council. 2001. The Advanced Technology Program: Assessing Outcomes. Washington, DC: The National Academies Press. doi: 10.17226/10145.
×
Page 155
Suggested Citation:"Panel IV: Assessing Progress: Case Study Cluster." National Research Council. 2001. The Advanced Technology Program: Assessing Outcomes. Washington, DC: The National Academies Press. doi: 10.17226/10145.
×
Page 156
Suggested Citation:"Panel IV: Assessing Progress: Case Study Cluster." National Research Council. 2001. The Advanced Technology Program: Assessing Outcomes. Washington, DC: The National Academies Press. doi: 10.17226/10145.
×
Page 157
Suggested Citation:"Panel IV: Assessing Progress: Case Study Cluster." National Research Council. 2001. The Advanced Technology Program: Assessing Outcomes. Washington, DC: The National Academies Press. doi: 10.17226/10145.
×
Page 158
Suggested Citation:"Panel IV: Assessing Progress: Case Study Cluster." National Research Council. 2001. The Advanced Technology Program: Assessing Outcomes. Washington, DC: The National Academies Press. doi: 10.17226/10145.
×
Page 159
Next: Panel V: Assessing the ATP Assessment Program: Challenges and Policy Issues »
The Advanced Technology Program: Assessing Outcomes Get This Book
×
Buy Paperback | $80.00 Buy Ebook | $64.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

This report examines the operations of the APT, reviews its extensive assessment program, and provides NRC Committee findings concerning the ATP’s operations and recommendations for potential improvements to the program. The report includes a summary of a major conference held in April 2000 as well as seven papers, including surveys of the industry participants or users of the ATP program, a summary of the results of fifty awards, detailed assessments of major joint ventures, and a description of the current selection process. It is the most comprehensive study to date of the program’s origins, operations, achievements, and assessment. Its conclusion: the program works.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!