An Assessment of the SBIR Program at the National Aeronautics and Space Administration (2009)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

2 Findings and Recommendations I.  NATIONAL RESEARCH COUNCIL STUDY FINDINGS The NASA SBIR program is making significant progress in achieving the congressional goals for the program. Keeping in mind NASA’s unique mission and the recent significant changes to the program, the SBIR program is sound in concept and effective in practice at NASA. With the programmatic changes recommended here, the SBIR program should be even more effective in achiev- ing its legislative goals. A. Overall, the program has made significant progress in achieving its con- gressional objectives by: • Increasing private-sector commercialization of innovations derived from federal research and development. (See Finding B.) • Using small business to meet federal research and development needs. (See Finding C.) • Stimulating technological innovation. (See Finding D.)   These changes create discontinuities in program goals that complicate assessment. These important changes are described in Chapter 5 on Program Management.   These objectives are set out in the Small Business Innovation Development Act (PL 97-219). In reauthorizing the program in 1992, (PL 102-564) Congress expanded the purposes to “emphasize the program’s goal of increasing private-sector commercialization developed through federal research and development and to improve the Federal government’s dissemination of information concerning small business innovation, particularly with regard to woman-owned business concerns and by socially and economically disadvantaged small business concerns.” 26 

FINDINGS AND RECOMMENDATIONS 27 • Fostering and encouraging participation by minority and disadvantaged persons in technological innovation. (See Finding E.) B. The NASA SBIR program helps its award recipients achieve significant levels of commercialization, as shown by the following metrics: • Market sales. According to the NRC Phase II Survey, nearly half (46 percent) of NASA Phase II projects reach the marketplace and generate revenue. 17.7 percent of projects generate revenues greater than $1 million.  • Substantial revenues. According to the NASA Commercial Metrics Survey, from 1983 to 1996, NASA SBIR projects created goods and services that generated over $2.3 billion in revenues in the private economy. Nearly half (46 percent) of all sales resulting from Phase II awards went to markets other than the federal government. • Commercial success is concentrated.  the NASA SBIR Phase II projects reporting sales greater than $0, For average sales per project were $1,154,156. Commercial success is highly concentrated with about 40 percent of the total sales dollars re- sulting from two NASA projects that had $5,000,000 or more in sales. The highest cumulative sales figure reported was $15,000,000. This skewed pattern is also evident in private-sector early-stage finance. • Balanced sales. Phase III sales by NASA SBIR recipients appear to be roughly bal- anced between the private sector (47 percent) and the federal govern- ment (53 percent). (See Figure 2-1.)   SeeFigure 4-1.   SeeFigure 4-2.  See NASA Commercial Metrics Survey, October 2002, p. 8. Accessed at <http://www.sbir.nasa. gov/SBIR/survey.html>.  See Table 4-1 on Percentage of Sales by Type of Customer.  See Figure 4-2 for the distribution of projects with sales greater than zero dollars.  See John H. Cochrane, “The Risk and Return of Venture Capital,” Journal of Financial Econom- ics, 75(1):3-52, 2005. Drawing on the VentureOne database Cochrane plots a histogram of net venture capital returns on investments that “shows an extraordinary skewness of returns. Most returns are modest, but there is a long right tail of extraordinary good returns. Fifteen percent of the firms that go public or are acquired give a return greater than 1,000 percent! It is also interesting how many modest returns there are. About 15 percent of returns are less than 0, and 35 percent are less than 100 percent. An IPO or acquisition is not a guarantee of a huge return. In fact, the modal or “most probable” outcome is about a 25 percent return.” See also Paul A. Gompers and Josh Lerner, “Risk and Reward in Private Equity Investments: The Challenge of Performance Assessment,” Journal of Private Equity, 1 (Winter 1977):5-12. Steven D. Carden and Olive Darragh, “A Halo for Angel Investors” The McKinsey Quarterly, 1, 2004, also show a similar skew in the distribution of returns for venture capital portfolios. 

28 SBIR AT THE NATIONAL AERONAUTICS AND SPACE ADMINISTRATION Other federal agencies (3%) Export markets (11%) U.S. private sector Prime contractors (36%) (11%) NASA (17%) DoD (22%) FIGURE 2-1  Distribution of NASA SBIR Phase III sales by customer. NASA Figure 2-1.eps SOURCE: NRC Phase II Survey. C. SBIR Phase II projects result in substantially useful results for NASA. 1. SBIR projects create valuable commercial and research outcomes. • According to the NRC Project Manager Survey: Sixty-three percent of surveyed projects were deemed by NASA’s Contracting Officer’s Technical Representatives (COTR) to have significant noncommercial, intrinsic research value.10 34.6 percent of surveyed projects resulted were deemed by NASA COTRs to have resulted in a product or service of commercial value.11 2. SBIR Phase II projects are linked to NASA missions. • According to the NRC Project Manager Survey:12 About a quarter (26 percent) of NASA project managers who re- sponded to the survey reported that the SBIR-funded project “pro- duced results that have been useful to us, and we have tried to   See Project Manager Survey, Appendix D. 10  See Question 14 and Table App-D-20 of the Project Manager Survey. 11  See Table App-D-24 of the Project Manager Survey. 12  See Table App-D-12 of the Project Manager Survey. 

FINDINGS AND RECOMMENDATIONS 29 follow up on the ideas initiated in this SBIR contract by encourag- ing the firm to apply for additional SBIR awards.” Another third of NASA project managers responding to the NRC Program Manager Survey noted that the SBIR project “produced results that have been useful to us, and we have tried to follow up on the ideas initiated in this SBIR contract in other research we conduct or sponsor.” 3.  ASA’s SBIR research is ranked less favorably than traditional N NASA research. • According to the NRC Project Manager Survey:13 COTRs were asked to rank their SBIR projects in terms of quality on a scale of 1 to 10. The mean score for SBIR is 6.98 (with a stan- dard deviation of 1.846) and the mean score for non-SBIR is 7.45 (with a standard deviation of 1.268).14  all, 57 percent of those surveyed said that SBIR Phase II project In produced useful results for the agency.15 4. SBIR projects give NASA value for money: • According to the NRC Project Manager Survey:  little more than two-thirds of COTRs (68 percent) saw that SBIR A spending gave the same or more benefits to the agency mission as other NASA R&D projects.16  majority of COTRs (56 percent) received more quality Phase II A proposals than they could fund.17 Another fifth of COTRs (23 percent) thought that the balance of funding available and good applications is about right.18 D. The NASA SBIR program stimulates collaboration, technological in- novation and generates new knowledge. 1.  he NASA SBIR program has successfully supported the creation T and diffusion of knowledge by small companies: •  Patents.  A quarter of projects responding to the NRC Phase II Survey reported filing at least one related patent; a fifth received at least one patent.19 13  SeeTables 4-7, 4-8, 4-9 and 4-10. 14  While these data indicate that project managers viewed the quality of non-SBIR research more favorably, this result is driven by outliers: 6.1 percent of the respondents rank SBIR projects at a score of three or below compared to 0.0 percent for non-SBIR projects. 15  See Table App-D-12 of the Project Manager Survey. 16  See Table App-D-13 of the Project Manager Survey. 17  See Table App-D-14 of the Project Manager Survey. 18  Ibid. 19  See Table 4-18. 

30 SBIR AT THE NATIONAL AERONAUTICS AND SPACE ADMINISTRATION • Publications.  Respondents to the relevant question in the NRC Phase II Survey reported that peer-reviewed articles were published because of the SBIR-supported work.20 2.  he NASA SBIR program has stimulated links among NASA, small T businesses, and universities. •  Just under a third of NRC Phase II Survey respondents (29 percent) reported having university participation in their projects had the fol- lowing interactions with universities.21  was faculty/adjunct (3 percent). PI Faculty participated as consultants (17 percent). Graduate students participated (15 percent). University facilities/equipment used (13 percent). Technology licensed from university (2 percent). Technology developed at university by participant (4 percent). University was subcontractor on Phase II (16 percent). E. NASA SBIR provides substantial support for small businesses, stimulat- ing entrepreneurship, new business formation and employment. 1. Awards stimulate significant small businesses formation. •  From the NRC Firm Survey, 20 percent of the respondents stated that they were founded at least in part due to SBIR.22 •  Firsthand accounts of this can be seen in several of the NASA-related firms interviewed for case studies. They noted that their firm was founded either because of SBIR and/or attribute significant growth of the firm to the impact of the SBIR award.23 2. Employment growth is positively correlated with SBIR.24  From the NRC Phase II Survey, half of the responding firms reported • that they were in the smallest size group (one to five employees) at the time of their first Phase II award.  Only 22 percent remained at that size at the time of the survey. All • other (larger) size groups have increased in percent of reporting firms. 3.  he NASA SBIR program supports both seasoned entrepreneurs T and academic researchers with little experience in the commercial world. 20  SeeTable 4-19. Some 161 respondents reported 220 articles. 21  SeeNRC Phase II Survey, Question 31 and Table 4-22. 22  See NRC Firm Survey, Question 1. 23  See, for example, the case studies of AeroSoft, Inc., and Deformation Technology Control, Inc., in Appendix E. 24  See Table 4-5. 

FINDINGS AND RECOMMENDATIONS 31 • Based on results from the NRC Firm Survey, over a quarter (28 percent) of the respondent firms had no founders with a business background. • For 26 percent of the firms, at least one founder had an academic background.25 • Private companies (65 percent) and universities or colleges (36 per- cent) are the source of the most recent employment for most of the firm founders.26 • Several of the NASA-related firm case studies are consistent with these survey findings.27 4.  BIR prompts the project initiation decision. S •  Over two-thirds (68 percent) of SBIR Phase II award recipients say that they definitely or probably would not have undertaken the funded research project without the SBIR funding.28 •  While the impact of the award is significant, just under a fifth of the respondents to the NRC Phase II Survey (18 percent) said that the project probably or definitely would have occurred even without the SBIR award.29 5.  BIR enhances the scope and progress of projects. S •  Forty-three percent of respondents stated that the project would have been narrower in scope without the SBIR award.30 •  Over half responded that the project would have been behind the cur- rent schedule in the absence of SBIR funding.31 6.  BIR may also help small businesses obtain additional developmental S funding for their projects. •  significant percent of the survey respondents (44 percent) received A additional funding for their project subsequent to the receipt of the Phase II award.32 •  Non-SBIR federal funds (average of $133,829) and intrafirm funds (average of $100,450) made up the majority of the development fund- ing for the respondents.33 •  Further federal non-SBIR funding for NASA amounted to 48 percent of all additional funding for development.34 25  NRC Firm Survey, Question 2. 26  NRC Firm Survey, Question 3. 27  See, for example, the case study of Mainstream Engineering Corporation in Appendix E. 28  See NRC Phase II Survey, Question 13. 29  Ibid. 30  See NRC Phase II Survey, Question 14. 31  See NRC Phase II Survey, Question 15c. 32  See NRC Phase II Survey, Question 22. 33  See NRC Phase II Survey, Question 23. 34  See NRC Phase II Survey, Question 23. 

32 SBIR AT THE NATIONAL AERONAUTICS AND SPACE ADMINISTRATION • This feature is also documented in the case studies where several firms indicated that SBIR had a positive effect on securing other federal financing.35 F. NASA’s SBIR program supports the participation of minority- and woman-owned small businesses in innovation research. • During the 1997-2004 period, minority-owned firms received 12.18 per- cent of Phase II awards and woman-owned firms received 9.94 percent of Phase II awards.36 • Participation by minority- and woman-owned firms in the SBIR pro- gram did not appear to greatly increase or diminish with time (see Fig- ure 2-2). G. The recent shift in NASA’s technology transfer program from a focus on commercialization (“spin-out”) to a focus on supplying mission needs (“spin-in” or “infusion”) will create significant challenges for the SBIR program. 1.  he new emphasis on spin-in requires the creation of a new regional in- T frastructure focused on technology acquisition, not technology generation and diffusion. 2.  his infrastructure: T •  Needs both advanced knowledge of specific technologies and extensive knowledge of NASA’s needs. •  Must account for a shift in responsibility for achieving mission- purpose use from SBIR to NASA centers and NASA’s scientists and engineers. •  Requires that each NASA center have the capacity for identifying and support the SBIR projects best matched to the center’s high priority technology needs, regardless of where in the country those projects are. •  Must have more coordinated and effective NASA oversight of projects from solicitation through Phase III to ensure that the research done aligns with agency needs. H. Notwithstanding the change in NASA organization and mission, NASA’s solicitation of SBIR proposals is run efficiently and effectively. NASA’s SBIR proposal review process can be improved further. 1.  lectronic Handbook.  Proposals are submitted electronically, using the E 35  See, for example, case study of Space Photonics, Inc., in Appendix E. 36  See NASA awards database. 

FINDINGS AND RECOMMENDATIONS 33 30 Woman-owned Firms Minority-owned Firms 25 Either Percent of All Phase II Awards 20 15 10 5 0 1997 1998 1999 2000 2001 2002 2003 2004 Year FIGURE 2-2  NASA SBIR Phase II awards, by demographic group, 1997-2004. SOURCE: National Aeronautics and Space Administration. NASA Figure 2-2.eps NASA-developed Electronic Handbook. NASA has been an early leader in this regard. 2.  echnical review.  This takes place at each NASA center based on evalu- T ation criteria that considers: • Scientific/Technical Merit and Feasibility. • Experience, Qualifications and Facilities. • Effectiveness of the Proposed Work Plan. • Commercial Merit and Feasibility. •  Alignment with the NASA mission, most recently with the emphasis on “spin-in.” 3.  election.  NASA centers provide a ranked listing to the Program Man- S agement Office, which prepares a selection of options for the Source Selection Official, who make the final decisions. 4.  mproving the review process.  Several of the companies interviewed I for the NASA-related case studies held that NASA’s review process is in need of improvement. This would include improvement in communication between potential applicants and NASA to ensure that it is worthwhile 

34 SBIR AT THE NATIONAL AERONAUTICS AND SPACE ADMINISTRATION for firms to submit proposals and better debriefing process for rejected applications to give a firm a better idea of whether or not to reapply. 37 I. Monitoring and assessment of the program needs improvement. NASA does not provide an appropriate level of resources for assessing the program’s performance and, as a result, SBIR management is not suf- ficiently evidence-based. 1.  iven the size and scope of its SBIR program, NASA does not provide an G appropriate level of resources for assessing the program’s performance. 2.  artly because of lack of sufficient funding, the program is not suffi- P ciently evidence-based. It lacks clear benchmarks and metrics for success. Program evaluation—while recently improved—needs to be enhanced further. J. Understanding multiple-award winners: The path to successful SBIR award outcomes is non-linear and technologies often require multiple awards to reach fruition. 1.  irms receiving more NASA Phase II awards appear to produce bet- F ter commercial outcomes. •  The Commercial Metrics survey suggests that either companies im- prove their commercialization capabilities with practice or certain entrepreneurs have an inherent knack for business (and others do not). It is also consistent with the nonlinear and cumulative influence of technological change and its relation to economic growth.38 •  Evidence from the NASA-related firm case studies supports the notion that multiple awards contribute to the development of a given technol- ogy. The companies interviewed indicated that they made sustained, strategic use of a number of SBIR awards over time.39 2.  nnovative new technologies often take more time and money to de- I velop than a single set of Phase I and Phase II awards provides. •  Not every project leads immediately to a market-ready product. Some small businesses have difficulty raising money from private sources for early-stage technologies, especially for NASA’s limited niche market. The firms, therefore, often rely on multiple SBIR awards to get to the point where they can provide the product to NASA and/or sell it in the commercial market and thus fund their own research. 37  See, for example, the case studies of AeroSoft, Inc., and Luna Innovations, Inc., in Appendix E. 38  See NASA Commercial Metrics Survey, October 2002, op. cit. 39  See the case studies in Appendix E. 

FINDINGS AND RECOMMENDATIONS 35 II.  NATIONAL RESEARCH COUNCIL STUDY RECOMMENDATIONS The recommendations in this section are designed to improve the operation of the SBIR program at NASA. They complement the core findings that the program is addressing its legislative goals—that significant commercialization is occur- ring, that the awards are making valuable additions to nation’s stock of scientific and technical knowledge, and that SBIR is developing products that apply this knowledge to NASA’s missions. A. Improve program processes. 1.  ASA should explore how to increase the flexibility of projects, given N changes in technology and information. •  Address Time Lags: Some of the firms interviewed for this study noted that the period from writing a proposal to completing a Phase II is often too long, compared to the pace of change in the technology. 40 • Increase Contract Flexibility: It would be helpful if firms could change direction of the contract as information changes. Means of increasing flexibility should be examined by NASA. 2.  reate a sense of ownership, including career and financial incentives C for project managers to promote increased technology infusion. •  “Program offices should own SBIR.” This is a quote from Wayne Schober, SBIR program director at JPL. This is a necessary condi- tion to increase infusion or mission use of the technologies developed through the SBIR program. 3.  tudy the funding gap problem and propose and implement a S solution. •  The funding gap between Phase I and Phase II is a major concern, es- pecially for the smaller firms. While larger and more diversified firms can often cope, smaller and newly founded firms often have to scale back significantly or go into debt. A phased approach, such as that adopted by the Navy and NSF, may be applicable here. 4.  evelop program synergies and complementarities. D •  Complementarities between the NASA and DoD SBIR projects should be encouraged. NASA SBIR projects often find homes in DoD during Phase III. Encouraging such complementarities can promote efficient use of technology for society as a whole. 5.  tudy the feasibility of yearly technical conferences to help define S NASA’s technical needs. 40  See, for example, the case study of AeroSoft, Inc., in Appendix E. 

36 SBIR AT THE NATIONAL AERONAUTICS AND SPACE ADMINISTRATION •  yearly technical conference can help NASA generate more appropri- A ate proposals by more clearly defining its technical needs to prospec- tive applicants. B. NASA should reexamine the size of its Phase II awards. 1.  ASA Phase I and II awards are smaller than those of other major SBIR N funding agencies, and had not increased for several years. •  Recently, the maximum Phase I award size was increased from $70,000 to $100,000, but the maximum level of Phase II award remains $600,000. •  Award size affects the portfolio of projects and Technology Readiness Levels that can be supported. 2.  here is a trade-off between award size and the number of awards that T can be funded. •  Case study evidence suggests that while firms generally want larger size awards, many are not willing to accept the smaller number of awards implied by larger awards.41 •  COTRs and Project Managers also both indicated that NASA received more good proposals than it could fund.42 3.  ASA should consider whether pilot programs offering larger (or indeed N smaller) Phase II awards might be useful in some cases. •  The key test will be whether there is an adequate flow of good pro- posals that are substantial enough to meet the agency’s technology needs. C. Encourage program experimentation, followed by evaluation of outcomes. 1.  evelop and evaluate pilot programs. D •  NASA should develop an effective program for developing, deploying, and evaluating pilot initiatives. Pilot programs underway include:  Langley Research Center program to provide matching non-SBIR A money to proposals. This approach could be very helpful in promot- ing spin-in/infusion efforts. The Director’s Fund to provide greater Phase II funding for worthy projects.  Langley Research Center program that helps link SBIR projects to A NASA prime contractors that could become potential customers. 41  For example, see the case of ARACOR in Appendix E. 42  See Table App-D-14 of the Project Manager Survey. 

FINDINGS AND RECOMMENDATIONS 37 2.  xpand and assess the NASA Alliance for Small Business Opportuni- E ties (NASBO). •  NASBO’s regional dimension has the potential to encourage spin-out, spin-in, and dual use outcomes. •  realize this potential, NASBO should be increased in scale and To evaluated to assess its effectiveness. 3.  dopt best practices from other SBIR programs, as applicable. A •  Evaluate other agencies’ approaches to commercialization assistance and adopt the best approaches where applicable.43 4.  xpand the relationship between states and spin-out SBIR projects. E •  This may replace the phased out Regional Technology Transfer Cen- ters (RTTC) approach to tech transfer. D. Additional management resources are needed. 1.  ffective management and evaluation requires adequate funding.44 E An evidence-based program requires high-quality data and systematic assessment. 2.  o enhance program utilization, management, and evaluation, the NASA T SBIR program should be provided with additional funding for manage- ment and evaluation. 3.  ncreased funding is needed to provide effective oversight, including site I visits, program review, systematic third-party assessments, and other necessary management activities. 4.  n considering how to provide additional funds for management and I evaluation, there are three ways that this might be done: •  Additional funds might be allocated internally, within the existing budgets of the services and agencies, as the Navy has done. •  Funds might be drawn from the existing set-aside for the SBIR pro- gram to carry out these activities. •  Congress may consider marginally increasing the set-aside for the program, currently at 2.5 percent of external research budgets, with the goal of providing management resources necessary to maximize 43  Bestpractices include the Navy’s strategic approach to procurement and NSF’s Phase I-B and Phase II-B supplemental awards. 44  As noted above, a recent OECD report, the International Benchmark for program evaluation of large SME and Entrepreneurship Programs is between 3 percent for small programs and 1 percent for large-scale programs. See Organization for Economic Cooperation and Development, “Evaluation of SME Policies and Programs: Draft OECD Handbook,” Paris: Organization for Economic Coopera- tion and Development. 

38 SBIR AT THE NATIONAL AERONAUTICS AND SPACE ADMINISTRATION the program’s return to the nation.45 These increased resources should not be used to create new or separate management but should go to enable existing management to enhance their efforts with respect to the program. The Committee recommends this third option. E. Develop data for evaluation and conduct regular assessments. 1.  evelop data objectives. The NASA SBIR program should develop a D series of specific data objectives—identifying both the data needed to run the program well and the means of acquiring those data. Possible bench- marks include: •  The percentage of companies achieving Phase III. •  The development time of high priority NASA technologies from R&D in Phase I to NASA utilization in Phase III. •  The fraction of important technology used by NASA to advance its own capabilities. •  Benchmarks for spillovers at the regional and national level. 2.  ecording of Phase III outcomes. R •  Benefits of the SBIR program are likely to be significantly underesti- mated unless this is improved. Better data collection on outcomes is necessary, even though it will use some additional resources. •  NASA may also wish to consider whether the DoD model—and tech- nology—for producing a Company Commercialization Report, up- dated each time the company applies for further awards, might be a useful way of generating better data about commercial outcomes. 3.  ummary annual report: Each year, NASA should provide Congress S with a summary report on the SBIR program. This annual report should include descriptive statistics for applications, awards, and outcomes along the dimensions identified in this report, including knowledge creation, 45  Eachof these options has its advantages and disadvantages. For the most part, over the last 25 years, the departments, institutes, and agencies responsible for the SBIR program have not proved willing or able to make additional management funds available. Without direction from Congress, they are unlikely to do so. With regard to drawing funds from the program for evaluation and man- agement, current legislation does not permit this and would have to be modified. This would also limit funds for awards to small companies, the program’s core objective. The third option, involving a modest increase to the program, would also require legislative action and would perhaps be more easily achievable in the event of an overall increase in the program. In any case, the Committee envisages an increase of the “set aside” of perhaps 0.03 percent to 0.05 percent on the order of $35 million-40 million per year or, roughly, twice what the Navy currently makes available to manage and augment its program. In the latter case (0.05 percent), this would bring the program “set aside” to 2.55 percent, providing modest resources to assess and manage a program that is approaching an annual spend of some $2 billion. Whatever modality adopted by Congress, the Committee’s call for improved management, data collection, experimentation, and evaluation may prove moot without the benefit of additional resources. 

FINDINGS AND RECOMMENDATIONS 39 technology innovation, and impact on agency mission, as well as com- mercialization. As part of this process, NASA should produce regular reports from the commercialization database. 4.  egular assessments. The proposed annual report noted above could R become a focus for wider efforts to develop improved internal assessment capabilities that can be used to enhance program operations. It could also tie proposed improvements to evidence-based analysis and, for example, include an evaluation of the predictive power of selection scoring with regard top commercialization and other outcomes. NASA should also commission regular external arms-length evaluations to assess the pro- gram progress and the impact of new initiatives. F. NASA should consider the creation of an independent advisory board that draws together senior agency management, SBIR managers, and other stakeholders as well as outside experts to review current opera- tions and achievements and recommend changes to the SBIR program. 1.  he purpose of such an advisory board is to provide a regular monitoring T and feedback mechanism that would address the need for upper manage- ment attention, and encourage internal evaluation and regular assessment of progress towards definable metrics. 2.  he annual report of the NASA SBIR program, recommended above, T could be presented to the board. The board would review the report that would include updates on program progress, management practices. It would make recommendations to senior agency officials. 3.  he board could be assembled on the model of the Defense Science T Board (DSB) or perhaps the National Science Foundation’s SBIR Advi- sory Board.46 In any case, it should include senior agency staff and the Director’s Office on an ex officio basis, and bring together, inter alia, representatives from industry (including award recipients), academics, and other experts in program management. G. Evaluate the impact of NASA’s reorganization on SBIR.  Following the recent agency restructuring, NASA seeks to make “spin-in” the main priority for the SBIR program. Because management structures at NASA have changed so extensively, data from past projects are of limited relevance in guiding current management.” NASA should study how the new agency orientation towards spin-in will impact SBIR program outcomes. Such studies may include: 46  The intent here is to use the DSB or the NSF SBIR Board as a model, not something necessarily to be copied exactly. 

40 SBIR AT THE NATIONAL AERONAUTICS AND SPACE ADMINISTRATION 1.  he impact of NASA’s reorganization on the availability of SBIR T solicitation topics should be evaluated. •  NASA adjusts to its new mission priorities, the breadth of SBIR As topics may well narrow, discouraging some applicants. The impact of this adjustment should be evaluated and efforts to ensure that the program is effectively integrated should be undertaken. 2.  he new NASA structure and the Innovative Partnership Program T (IPP) should be evaluated in terms of its technology transfer manage- ment goals. •  The challenge to create a successful SBIR program can be appreci- ated by pondering NASA’s own description of its technology transfer management goal: to manage NASA technology transfer from “top- to-bottom, coast-to-coast, and cradle-to-grave.” •  “Top-to-bottom” refers to all levels of management; “coast-to-coast” highlights the importance of the ten NASA centers and NASA’s re- gional infrastructure; and “cradle-to-grave” points out the difficulty of managing SBIR from solicitation through Phase III. 3.  iven the new NASA orientation towards spin-in, the role of risk- G taking in NASA SBIR should be evaluated. •  The stronger orientation towards infusion may push NASA toward opting for lower risk SBIR projects. A balance must be struck between encouraging and funding riskier projects at lower Technology Readi- ness Levels (TRL) that may be very challenging to commercialize or use immediately by NASA and the emphasis on applications for NASA.47 •  The “innovation” focus of the Small Business Innovation Research program suggests that such risk taking should be encouraged. This trade-off should be evaluated fully. 4.  ASA should study how the new agency orientation towards spin-in N will impact SBIR program outcomes. •  NASA may often provide only a limited market for a product. Concen- trating on infusion makes the NASA SBIR program more technology- driven rather than market-driven. •  critical balance in the portfolio of projects needs to be struck and the A impact of this shift on SBIR program outcomes should be evaluated. 5.  ASA should consider how SBIR can be used to maintain technologi- N cal capacity in areas facing declining budgets. 47  Technology Readiness Level (TRL) is a measure used by NASA to assess the maturity of evolv- ing technologies prior to incorporating them into an operational system or subsystem. See John C. Mankins, Technology Readiness Levels: A White Paper, NASA, Office of Space Access and Technol- ogy, Advanced Concepts Office, 1995. 

FINDINGS AND RECOMMENDATIONS 41 • NASA faces substantial change as resources are increasingly focused on the Moon-Mars mission. The result is declining resources for R&D in other parts of NASA, especially aeronautics and earth sciences. •  NASA becomes increasingly dependent on externally-developed As technologies in areas that are being defunded, SBIR could perhaps be a useful and relatively inexpensive way of maintaining technological capacity in these areas, allowing NASA to maintain currency in im- portant technologies at reduced cost.