Engineering Research and Technology Development on the Space Station (1996)

This committee was asked to “review techniques for assessing benefits of research to U.S. competitiveness that have been used by other federal agencies and, using one or more of these, perform a preliminary assessment of the potential benefits to U.S. competitiveness of engineering research that might be conducted on a space station.” Five principal approaches have been used to assess the benefits of R&D to the economy (Austin, 1993; Hertzfeld, 1992; Tratjenberg, 1990):

• adapting macroeconomic production function models to estimate the impacts of technological change attributed to R&D spending on GNP and other measures

• evaluating returns from specific technologies to the economy through the use of demand and supply curve analyses of consumer and producer surplus

• examining survey data that provide evidence of direct transfers of technology to the private sector

• using patent data to infer the value of R&D investment, the extent of R&D diffusion, the diffusion of technology to other nations, and other metrics

• tracking changes in the stock market value of firms that are granted patents or government funding

However, approaches such as these have typically been unable to measure returns from past investments accurately or to forecast future returns on incremental investments (Hertzfeld, 1992; OTA, 1986). Some methods (such as tracking changes in the stock prices of biotechnology firms) have shown promise in

measuring the benefits of past research in some situations, but none of these methods is capable of predicting the impact of future research.

Failure to quantify links between government R&D programs and the competitiveness of private companies springs from three factors. First, the pathways from public technology to private competitive advantage are idiosyncratic rather than systemic. They depend, in part, on chance and the vagaries of human personality. Second, these pathways may take many years to unfold. The studies are frequently not able to take this long perspective, either because of the impatience of agency sponsors or because data have become unavailable. Third, and surely most important, the phenomena are too complex to be modeled by the simple quantitative links that would provide unambiguous answers.

A recent study commissioned by the U.S. Air Force in which 10 full-time researchers examined industry/government cooperative R&D and joint technology development programs determined that conventional metrics used by government to quantify the results usually “ignore output, distort the success and failure of cooperative technology programs, and serve as a flawed incentive, driving quantity (the total number of agreements) over quality” (Dougherty and Irish, 1995). The National Institute of Standards and Technology (NIST) came to similar conclusions in an assessment of the economic impact of four major NIST programs, including laboratory R&D and the Advanced Technology Program (NIST, 1994). NIST's conclusion was that setting priorities, evaluating performance, and measuring economic impact “cannot be reduced to simple formulas that yield unambiguous, quantitative answers.”

NASA also realizes that quantifying the benefits of its R&D programs to the economy is not feasible. NASA's Commercial Development and Technology Transfer Department, for example, explained to the committee that they could not determine the overall benefits to the economy of their R&D; they could only determine a “minimum” level of economic activity that had been produced by a number of commercial ventures conducting the R&D (Norwood, 1995). According to NASA's Commercial Technology Progress Report (NASA, 1995), NASA's current metrics for measuring the success of their commercial programs are based on the number of commercial activities and the amount of investment being put into the R&D by industry rather than on any assessment of the effects of these programs on the economy or on U.S. competitiveness. NASA is not attempting to make quantitative predictions of the economic impact of future R&D.

In view of the lack of a satisfactory quantitative model, the committee made qualitative assessments of the potential economic benefits of ERTD on the space station to the ISS program itself (primarily in terms of reduced cost and improved capability), to other space programs, and to Earth-based applications. These assessments are summarized in chapter 2 and are detailed in chapter 3. In addition, the committee suggested things NASA could do to increase the likelihood that ERTD on the ISS would convey benefits to the U.S. economy. These appear in chapter 5.

Austin, D. 1993. An Event-Study Approach to Measuring Innovative Output: The Case of Biotechnology. American Economic Review Papers and Proceedings. 83(2): 253–258.

Dougherty, A. and M. Irish. 1995. Shared Investment-Shared Return: Industry/Government Technology Programs . Washington: Economic Strategy Institute.

Hertzfeld, H. Measuring Returns to Space Research and Development. Space Economics. Joel Greenberg and Henry Hertzfeld, eds. Progress in Astronautics and Aeronautics, volume 144. Washington: American Institute of Aeronautics and Astronautics.

NASA (National Aeronautics and Space Administration). 1995. NASA Commercial Technology Progress Report. April, 1995.

NIST (National Institute of Standards and Technology). 1994. Setting Priorities and Measuring Results at the National Institute of Standards and Technology. Washington, D.C.: National Institute of Standards and Technology.

Norwood, R. 1995. Some Indications of Commercial Activity and Economic Benefits Resulting from OSAT Sponsored Technology Development. Presentation to National Research Council Committee on the Use of the International Space Station for Engineering Research and Technology Development. Washington, D.C. February 16, 1995.

OTA (U.S. Congress Office of Technology Assessment). 1986. Research Funding as an Investment: Can we Measure the Returns?: A Technical Memorandum. Washington, D.C.: U.S. Government Printing Office.

Tratjenberg, Manuel. 1990. A Penny for Your Quotes: Patent Studies and the Value of Innovations . RAND Journal of Economics. 21(1): 172–187.