5
Program Management

5.1
INTRODUCTION: ASSESSING SBIR IN A RESTRUCTURING NASA

As with other parts of NASA, the NASA SBIR program has, experienced sequential waves of reorientation and restructuring. Mission objectives have changed very substantially, far more than at other SBIR agencies.

During NASA’s reorganization of 2003-2004, the agency’s SBIR program became a component of the Advanced Space Technology Program within the Exploration Systems Mission Directorate (ESMD), which is charged with implementing NASA’s planned exploration of Mars and other space exploration projects. In 2006, further reorganization led a change in the balance of management power between the Mission Directorates and the centers, with the former assuming much more direct authority over SBIR topic and award selection.

Because of this churn, any assessment of program management at NASA must deal with a moving target. Extensive changes in management structures mean that data regarding past activities is of limited relevance in guiding current management.

This chapter details how NASA implements its SBIR management strategy.1 It begins by focusing on issues related to the SBIR award cycle, including topic

1

The NASA SBIR/STTR management team during the course of this analysis was led by:

• Carl G. Ray (Code RC)—SBIR/STTR Executive Director; Oversight-Strategic Direction; SBIR/STTR Selection Official, NASA Headquarters, Washington, DC.

• W. Paul Mexcur (GSFC)—SBIR/STTR Program Manager; SBIR/STTR Program Operations; Program Management, Goddard Space Flight Center, Greenbelt, MD.

• Karin Huth (GRC)—SBIR/STTR Procurement Manager; Procurement Oversight.



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 105
5 Program Management 5.1 INTRODUCTION: ASSESSING SBIR IN A RESTRUCTURING NASA As with other parts of NASA, the NASA SBIR program has, experienced sequential waves of reorientation and restructuring. Mission objectives have changed very substantially, far more than at other SBIR agencies. During NASA’s reorganization of 2003-2004, the agency’s SBIR program became a component of the Advanced Space Technology Program within the Exploration Systems Mission Directorate (ESMD), which is charged with imple- menting NASA’s planned exploration of Mars and other space exploration proj- ects. In 2006, further reorganization led a change in the balance of management power between the Mission Directorates and the centers, with the former assum- ing much more direct authority over SBIR topic and award selection. Because of this churn, any assessment of program management at NASA must deal with a moving target. Extensive changes in management structures mean that data regarding past activities is of limited relevance in guiding current management. This chapter details how NASA implements its SBIR management strategy. 1 It begins by focusing on issues related to the SBIR award cycle, including topic 1The NASA SBIR/STTR management team during the course of this analysis was led by: • Carl G. Ray (Code RC)—SBIR/STTR Executive Director; Oversight-Strategic Direction; SBIR/ STTR Selection Official, NASA Headquarters, Washington, DC. • W. Paul Mexcur (GSFC)—SBIR/STTR Program Manager; SBIR/STTR Program Operations; Program Management, Goddard Space Flight Center, Greenbelt, MD. • Karin Huth (GRC)—SBIR/STTR Procurement Manager; Procurement Oversight. 0

OCR for page 105
0 SBIR AT THE NATIONAL AERONAUTICS AND SPACE ADMINISTRATION selection, the evaluation of proposals, and selection of awards. The chapter then examines NASA’s commercialization effort that includes a detailed review of the regional dimension of NASA’s SBIR program. The chapter closes with an analysis of challenges for the future of SBIR at NASA, given the agency’s new organizational structure and mission focus. An annex to this chapter describes the SBIR program at each NASA center. 5.2 MANAGING SBIR AT NASA 5.2.1 Guiding Principles NASA has based the management of its SBIR program on the following four principles: • Aligning research topics to the highest technology priorities of the agency. • Focusing on program effectiveness as measured by Phase III commercialization. • Enhancing program efficiency by using advanced information technology. • Providing opportunity for a cross section of small U.S. business. 2 5.2.2 Program Administration The NASA SBIR program has varied over the years in terms of its degree of centralization. Currently, NASA’s SBIR program is managed at multiple levels. • Level 1—SBIR Program Executive (Headquarters; agency-wide). • Level 1—SBIR program Mission Directorate liaisons (Headquarters, Mission Directorates). • Level 2—SBIR Program and Procurement Policy Managers (agency wide). • Level 3—SBIR Field Center Program Managers (Centers). • Level 4—Contract Officer Technical Representative (COTR) (projects). Leel . Program Executie.3 Located at NASA Headquarters, and supported by a national office, the program executive focuses on overall program administra- tion. Overall program policy, effectiveness, and assessment are the responsibility of the Headquarters Program Executive. The national office, located until recently at Goddard, has now moved to Ames. 2 NASA Program Management Web site, accessed at , July 2005. 3The current program executive is Carl G. Ray. His title is Program Executive, Technology Infusion, Innovative Partnerships Program (IPP) Office.

OCR for page 105
07 PROGRAM MANAGEMENT Leel . Mission Directorates. Each of four Mission Directorates (MDs) has assigned a senior staffer as liaison between the technology programs run by the MD’s and the SBIR program. Following the 2006 restructuring, the Mission Di- rectorates now dominate topic selection, and approve project selection. 4 Leel . SBIR Program and Procurement Policy Managers. The NASA SBIR Program Management Office, at the NASA Ames Research Center, runs SBIR in conjunction with NASA Mission Directorates and centers. The NASA Shared Ser- vices Center provides the overall procurement management for the programs. Leel . Field Centers. Prior to FY2006, program operations were managed at each of the ten NASA Field Centers. Following the FY2006 reorganization, the program will run through only four field centers (Ames, JPL, Glenn, and Langley) At each center, an SBIR Field Center Program Manager administers the program. Contracts are managed by NASA’s Contracting Officer at each center. Leel . The COTR. The Contract Officer Technical Representative (COTR) serves as the primary contact between the project and NASA on a contract’s technology focus and objectives, and handles assessment of project progress. The COTR is a staffer at one of the centers. 5.2.3 Administrative Budget NASA’s budget for administering the SBIR programs is approximately $3.8 million per year. This funding—3.2 percent of the $119 million SBIR budget— comes from separate agency funds (not SBIR). This budget does not include the substantial costs associated with employee time used on the program, including in particular the time needed to develop and approve topics and subtopics, and to evaluate proposals. NASA has not calculated the value of this time, although the recent change to a full-cost accounting approach will make it easier to estimate full administrative costs in the future. 5.2.4 FY2006 Reforms The impact of the 2006 reorganization has been to refocus SBIR on the NASA’s core mission objective, de-emphasizing the previous stress on outside commercialization. Interviews with all of the Mission Directorate liaisons indicate that the bal- ance of power between the centers and Headquarters changed substantially in FY2005-2006. The reorganization helps to address a dissatisfaction with the outcomes of the 4The FY2006 reforms are described in Section 5.2.4.

OCR for page 105
0 SBIR AT THE NATIONAL AERONAUTICS AND SPACE ADMINISTRATION previous approach. The 2002 Commercial Metrics report (covering 1983-1996) found that only about six percent of NASA’s 1,739 SBIR Phase II awards during this period supported technologies that were eventually infused into NASA or other federal programs ia Phase III funding.5 The reorganization also reflects changing needs and priorities within NASA. The addition of new missions and the expansion of existing ones have placed additional demands on Mission Directorates, squeezing funding for basic research. As a result of the reorganization, Mission Directorates are now focusing on aligning research funded through SBIR with specific technologies that can be taken up (or in NASA-speak “infused”) into the their own technology de- velopment programs. Whereas commercialization was the primary priority of NASA’s SBIR program (or, at least, an priority equal to the support for the agency’s mission) the focus of the program since the 2006 reorganization is squarely on support for the NASA mission. For SBIR, this involves finding and developing technologies that can help NASA meet its very specific needs and requirements. Overall, this new clarity of focus appears to be a positive development. As described in some detail in Chapter 4 (Outcomes), the low volume and high de- gree of specificity (e.g. space-hardiness) required to meet NASA’s needs makes it less likely that SBIR funded technologies can spin off into commercial sales. 6 5.3 THE AWARDS PROCESS In this section, we discuss the details of the NASA awards process. This can be disaggregated into the following components: • Topic development, including efforts to align topics with the needs of the agency. • Outreach into the business and technology communities, to help ensure that the best possible proposals reach NASA. • Project selection, including an assessment of commercialization potential. NASA sets aside 2.5 percent of its extramural research and development budget for SBIR awards. Each year NASA identifies various R&D topics, rep- resenting scientific and technical problems that the agency needs to solve, for 5 “Phase III funding” comprises contractual or other monies awarded to a SBIR project for federal agency use of the subject technology after expiration of a SBIR Phase II award. 6 Of course, some companies have made this transition successfully, but overall, there are significant structural impediments standing against successful commercialization from NASA SBIR project—as opposed for example to DoD, where there may be a huge potential market within the agency, or NIH where the private-sector market for SBIR-funded technologies is also potentially enormous.

OCR for page 105
0 PROGRAM MANAGEMENT pursuit by small businesses under the SBIR program. These topics are bundled together into annual NASA "solicitations," which are publicly announced re- quests for SBIR proposals from interested small businesses.7 A small business can identify an appropriate topic that it wants to pursue from these solicitations and, in response, propose a project for an SBIR award. The following sections describe the SBIR award process that was in place at NASA throughout much of the study period for this assessment. This structure, implemented in the mid-1990s, has since been substantially altered by reforms in 2005-2006, as noted above. 5.3.1 Selecting SBIR Topics 5.3.1.1 Aligning Topics with Agency Needs Topic and subtopic development and selection are the primary tools used to ensure that the SBIR program is closely aligned with agency needs. Beginning in 1995, NASA’s SBIR program initiated a series of steps to make the program more consistent with NASA’s mission. The goal was to make the SBIR program a strategic asset for NASA by integrating all aspects of these programs with NASA’s mission. The change involved implementing an organi- zational structure that better supported the technology goals of what were then NASA’s four Mission Directorates: Aeronautics, Exploration Systems, Science, and Space Operations. This set into motion several initiatives that included selecting topics integral to NASA’s overall mission, creating a system that closely tracks each contract for program management, leveraging non-SBIR agency funds to support un- funded high quality projects, and developing metrics for evaluating commercial outcomes. 5.3.1.2 Deciding on Topics A tension exists between encouraging proposals focused on solving very spe- cific problems facing NASA, and making the topics broad enough to encourage enough firms to apply, and the application of innovative solutions. NASA’s staff acknowledges this tension; they note that topic definitions at NASA are designed to walk the necessarily fine line between overly tight specifications and the devel- opment of exciting technologies that are not however needed by the agency. Program management issues guidelines to each of the centers. Centers may then propose subtopics. These are eventually prioritized at NASA Headquarters. To facilitate this decision-making process, NASA holds an annual Solicita- tion Development Workshop. This is attended by both Mission Directorate repre- 7 See NASA solicitation Web page. Accessed at .

OCR for page 105
0 SBIR AT THE NATIONAL AERONAUTICS AND SPACE ADMINISTRATION sentatives and center managers. Centers propose subtopics at the workshop, while the Mission Directorates note which subtopics support their goals. Program management then tries to design solicitations based on the agency’s highest priority technology needs, as reflected by center rankings and Mission Directorate needs. If a technology is needed by more than one center, the topic and subtopic are assigned to the center that ranked it highest. Subsequently, NASA’s SBIR/STTR Program Manager makes prioritizing recommendations about topics to the SBIR/STTR Executive Director, who makes the final decisions. 5.3.2 Agency Outreach 5.3.2.1 Agency Outreach Objectives and Methods Like other agencies, NASA staff note it is important to reach out to the small business community, in order to encourage higher quality proposals from a wide range of applicants. At NASA, this outreach is undertaken mainly through par- ticipation in national and regional SBIR conferences. NASA does not organize these events.8 Representatives from the ten NASA field centers regularly attend these conferences, where they set up briefing charts and displays, and hand out literature about the program. The NASA SBIR/STTR office also works with NASA’s Office of Small and Disadvantaged Business Utilization (known as “Code K” at NASA) to increase participation by small and disadvantaged businesses. As a key part of its outreach, NASA maintains an extensive SBIR Web site. NASA’s application process is entirely Web-based. NASA does not believe that lack of information about the program is a significant barrier for potential applicants.9 5.3.2.2 Agency Outreach Benchmarks NASA receives a large number of applications—from 1,099 companies in 2003. NASA staff note that the agency receives far more high quality applications than it can fund, as evidenced by the scores garnered by applicants. Agency staff also note that about a third of Phase I awards go to firms that have not previously won awards from NASA. NASA has made awards to firms in 48 out of 50 states,10 NASA staff believe that the decentralized nature of NASA, with numerous centers located often in 8 Interview with Paul Mexcur, Program Manager, November 21, 2003. 9 Ibid. 10 North and South Dakota are the two states that have not received a NASA SBIR grant.

OCR for page 105
 PROGRAM MANAGEMENT areas outside the main U.S. research hubs, provides sufficient geographical di- versity for the program.11 While there are no formal benchmarks or metrics for agency outreach, NASA officials point to the influx of new firms and the continuing 8:1 applica- tions to award ratio as evidence that new companies have little difficulty finding out about the NASA SBIR program, and that a significant increase in outreach is not needed. 5.3.3 Submission, Evaluation, and Selection12 5.3.3.1 Proposal Submission Small businesses submit their proposals to NASA through a sophisticated multipurpose online system called Electronic Handbooks and E-Submission (EHB). NASA describes this Electronic Handbook as a “set of Internet-based tools that support the paperless documentation and management of complex distributed processes,” including the SBIR program.13 EHB helps guide users through the program and provides real-time, online, paperless documentation and process management. 5.3.3.2 Evaluation Criteria Once submitted electronically via the Electronic Handbook, NASA screens the proposals to ensure that they are complete before sending them to the NASA center that “owns” the relevant topic for technical review. Evaluation is based on: • Scientific/Technical Merit and Feasibility. • Experience, Qualifications and Facilities. • Effectiveness of the Proposed Work Plan. • Commercial Merit and Feasibility. 5.3.3.3 Peer Review Panels—Membership, Selection, Qualifications Peer review in the NASA SBIR program is done internally by NASA tech- nologists.14 NASA staff scientists are used as technical experts. External re- viewers are also used for many Phase II applications, primarily for reviewing 11 Interview with Paul Mexcur, Program Manager, November 21, 2003. 12This is based on an interview with Carl Ray, Executive Director and Paul Mexcur, Program Manager of the NASA SBIR/STTR Program. The interview was on November 10, 2003. 13 National Aeronautics and Space Administration, The Paperless Solution, p.1, accessed . 14 Interview with Paul Mexcur, Program Manager, November 21, 2003.

OCR for page 105
 SBIR AT THE NATIONAL AERONAUTICS AND SPACE ADMINISTRATION commercial potential. These reviewers include the Research Triangle Institute (RTI), independent experts, retired senior executives, and others. The total cost of all Phase II external reviews is $80,000 to $100,000 annually. All Phase II applications are subjected to a peer review by at least one non-NASA person. It should be noted that some award recipients have questioned the value of the external review, on the grounds that these staff may not be sufficiently familiar with the NASA programs—alignment with which will in the end be decisive for any proposal. To enhance fairness in the selection process, NASA screens for multiple applications from a single company, as well as duplication of proposals or tech- nologies.15 NASA will not accept more than ten Phase I applications from the same firm in a given year, and will make not more than five Phase I awards to a single firm.16 Program management also makes sure that awards are spread ap- propriately across different technical areas and Mission Directorates. Problems are resolved by adjusting the rankings of proposals. 5.3.3.4 Phase I Evaluation and Selection Submitted Phase I proposals must be complete, as evaluators are not expected to seek additional information. Evaluations are performed by NASA scientists and engineers at the center(s) identified in the solicitation as responsible for the applicable subtopic. In some cases, qualified experts from outside NASA (from industry, academia, and other government agencies) may provide additional ad- vice. Applicants should not assume that evaluators are acquainted with the firm, its key individuals, or with any experiments or other information. Any pertinent references or publications should be noted in the technical proposal. NASA gives primary consideration to (a) the proposal’s scientific and tech- nical merit and feasibility and (b) the proposal’s benefit to NASA. According to NASA, each proposal is judged and scored on its own merits using the factors described below: • Factor 1. Scientific/Technical Merit and Feasibility. The proposal is evaluated on whether it offers an innovative and feasible technical approach to the described NASA problem area. Proposals must demonstrate relevance to the subtopic. Specific objectives, approaches, and plans for developing and verify- ing the innovation must demonstrate a clear understanding of the problem and the current state of the art. The applicant must also define risks involved in the proposal. • Factor 2. Experience, Qualifications and Facilities. The technical capabilities and experience of the Principal Investigator or project manager, key 15 Based on interview with Paul Mexcur, Program Manager, November 21, 2003. 16 NASA sdoliciation Web site. Accessed at .

OCR for page 105
 PROGRAM MANAGEMENT personnel, staff, consultants and subcontractors (if any), are evaluated for consis- tency with the research effort and their degree of commitment and availability. The necessary instrumentation or facilities must be shown to be adequate. The proposal should specify if the project will rely on any external sources, such as government furnished equipment or facilities. • Factor 3. Effectiveness of the Proposed Work Plan. The work plan is reviewed for its comprehensiveness, effective use of available resources, cost management, and proposed schedule for meeting Phase I objectives. The methods proposed for achieving each objective or task must be described in detail. • Factor 4. Commercial Merit and Feasibility. The proposal is evalu- ated for any potential commercial applications in the private sector or for use by the federal government. Factors 1, 2, and 3 are scored numerically with Factor 1 worth 50 percent and Factors 2 and 3 each worth 25 percent. The sum of the scores for Factors 1, 2, and 3 comprise the Technical Merit score. The score for Commercial Merit is in the form of an adjectival rating (Excellent, Very Good, Average, Below Average, Poor). For Phase I proposals, Technical Merit carries more weight than Commercial Merit. Each center ranks the proposals recommended for award relative to all other proposals recommended by that center. Center rankings are then forwarded to the Program Management Office for analysis, and are then presented to the Source Selection Official and Mission Directorate Representatives. Final selection decisions take into consideration the center rankings as well as overall NASA priorities, program balance, and available funding. Recommen- dations and relative rankings developed by the centers do not guarantee selection for award. The Source Selection Official has the final authority for choosing the specific proposals for contract negotiation.17 5.3.3.5 Phase II Evaluation and Selection The Phase II evaluation process is similar to the Phase I process. NASA plans to select for award those proposals offering the best value to the agency. Each proposal is reviewed by NASA scientists and engineers, and by qualified experts outside of NASA as needed per the factors identified below. Those proposals with high technical merit are reviewed for commercial merit. NASA uses a peer review panel to evaluate commercial merit. Panel membership includes non-NASA experts in business development and technol- ogy commercialization. 17The list of proposed selections is posted on the NASA SBIR/STTR Homepage. Accessed at . Additionally, all firms receive a formal notification letter.

OCR for page 105
 SBIR AT THE NATIONAL AERONAUTICS AND SPACE ADMINISTRATION • Factors -. The first three selection factors for Phase II are essentially identical to those for Phase I, except that Factor 1 (Scientific/Technical Merit and Feasibility) also addresses the extent to which Phase I objectives were achieved, and the impact of Phase I results on Phase II feasibility. • Factor . Commercial Potential and Feasibility. NASA assesses the proposed commercialization plan in terms of its credibility, objectivity, reason- ableness of key assumptions and awareness of key risk areas and critical business vulnerabilities, as applicable to the following factors: Commercial potential of the technology. Commercial intent of the applicant. Capability of the applicant to realize commercialization. Factors 1, 2, and 3 are scored numerically with Factor 1 worth 50 percent and Factors 2 and 3 each worth 25 percent. The sum of the scores for Factors 1, 2, and 3 comprise the Technical Merit score. Proposals receiving numerical scores of 85 percent or higher are evaluated and rated for their commercial potential using the criteria listed in Factor 4, and by applying the same adjectival ratings described above for Phase I. For Phase II proposals, commercial merit is a critical factor. This sequential evaluation (of technical merit followed by commercialization) is an interesting innovation, allowing NASA to focus scarce resources for assessing commercial- ization only on the most promising application. Once again, each center makes recommendations for awards among those proposals that it evaluates. The center recommendations are forwarded to the Program Management Office for analysis and presented to the Source Selection Official and Mission Directorate Representatives. Final selection decisions con- sider the center recommendations, overall NASA priorities, program balance and available funding, as well as any other evaluations or assessments (particularly pertaining to commercial potential). Recommendations provided by the centers do not guarantee selection for award. The Source Selection Official has the final authority for choosing the specific proposals for contract negotiations. 5.3.4 Funding “Gaps” Funding gaps can be found between the end of financial support under Phase I and the start of support under Phase II. About two thirds of respondents to the NRC Phase II Survey reported a gap between Phase I and Phase II fund- ing at NASA; the average length of the gap was 6 months. Only 3 percent of respondents reported a gap of one or more years.18 NASA cites the efficiency of its Electronic Handbook in minimizing this gap. Nevertheless, several firms interviewed for case studies indicated that the funding 18 NRC Phase II Survey, Question 26.

OCR for page 105
 PROGRAM MANAGEMENT gap remains an important issue, especially for smaller, less developed firms. In addition, three-quarters of NRC Phase II Survey respondents who experienced a gap responded by stopping work on the project.19 It is unclear whether NASA has benchmarked its own funding gap against those of other agencies. NASA does not appear to have adopted any of the “gap-reducing” initiatives initiated at other agencies. These include: • Development of a Phase I “option” that provides bridge funding for selected projects at the end of Phase I (DoD). • Simultaneous application for Phase I and Phase II (the NIH Fast Track). • Work at risk (NIH).20 • Expedited processing of Phase II awards (DoE). Several of the small firms interviewed for this study suggested that it would make sense to find a more standardized way to operate contracts, given that both the funding amounts and time to delivery are essentially fixed. 5.3.5 Other Aspects of Award Selection 5.3.5.1 Reporting Phase II typically requires quarterly reports, submitted electronically. In ad- dition, NASA Phase II awards usually require a prototype as a deliverable. These reports are used by the COTR and the COTR’s management to help identify potential for Phase III.21 The NASA contracting officer is supported by the Contract Officer’s Tech- nical Representative (COTR). The COTR is the firm’s primary contact within NASA on the contract’s technology focus and objectives. Given that one primary goal of the SBIR program at NASA is the eventual infusion of the firm’s tech- nology into NASA’s programs and missions, the interaction with technologists within NASA is critically important, and the COTR is both the link and the fa- cilitator for such interaction. 5.3.5.2 Resubmission Procedures and Outcomes Resubmissions of rejected applications are not allowed.22 However, NASA can select an applicant at any point in time. Although rare, a previous application 19 NRC Phase II Survey, Question 28. 20At NIH, firms scoring well inside the likely Payline may decide to continue work before a Phase II award is made. The Phase II award covers up to three months work of pre-contract expenses incurred for Phase II work. 21 Based on interview with Paul Mexcur, Program Manager, November 21, 2003. 22 Based on interview with Paul Mexcur, Program Manager, November 21, 2003.

OCR for page 105
0 SBIR AT THE NATIONAL AERONAUTICS AND SPACE ADMINISTRATION Annex to Chapter 5 SBIR at the NASA Centers Until the most recent reorganization, basic SBIR processes were common to all ten field centers. Other processes affecting infusion/commercialization op- portunity reflect the missions of NASA’s various field centers. As a result, the centers have had differing definitions of what constitutes successful infusion/ commercialization of SBIR Phase II technologies, making it difficult to compare infusion/commercialization outcomes with accuracy. Below, we examine how SBIR was administered at each of the NASA centers. 5.9 AMES RESEARCH CENTER (ARC)—SAN JOSE, CA The Ames SBIR Program Office primary interface is with the three research directorates at Ames, whose technology foci are not closely aligned with agency Mission Directorates. These research directorates govern the SBIR topic develop- ment process, although Exploration Systems Mission Directorate does participate in subtopic development. SBIR Phase I activity by the Ames Program Office includes a cursory evalu- ation of infusion/commercialization opportunity in the proposal review process, although the Program Office is planning a preparatory workshop for the 2006 Phase I proposals ranking committee that will have an infusion/commercializa- tion appraisal component. In both Phase I and Phase II proposal review processes, the research direc- torates make proposal ranking recommendations to a ranking committee. During Phase II proposal evaluation, outside peer reviews of infusion/commercialization opportunity are considered, but are not decisive in ranking committee funding recommendations. During a Phase II contract, little or no emphasis is placed by SBIR project monitors on identification of infusion/commercialization opportuni- ties by the SBIR awardees. The Ames SBIR Program Office does not monitor either infusion or commer- cialization outcomes of SBIR projects, but evidence of “success stories”—usu- ally obtained anecdotally, and then verified with the SBIR firm—is passed on to NASA SBIR Level 1 and 2 principals. Although the aggregate of such success stories is “. . . a small number,” according to Ames SBIR Program Manager Geoffrey Lee, “. . . some of NASA’s largest Phase III contracts come from Ames SBIR contracts, whose technology was infused into Space Life Sciences Payloads programs.”51 Among the 18 Phase III contracts from Ames documented since 1983, Ames claims the largest Phase III dollar aggregate from SBIR projects among the ten NASA field centers. Its Phase III contracts have totaled approxi- 51 Interview with Geoffrey Lee, February 10, 2005.

OCR for page 105
 PROGRAM MANAGEMENT mately $123,500,000. This includes NASA’s largest individual SBIR Phase III contract—$59,104,971 given to Orbital Technologies, Inc. in 1994 to develop a plant research unit. NASA’s new full-cost accounting environment for all agency functions (i.e., all funding is derived from mission programs) requires field center compliance and reorganization in many cases. Consequently, the Ames SBIR Program Of- fice has had to explore development of an infusion/commercialization assistance component. This component will be derived from best practices of other field centers and discussions with Ames’s Technology Partnerships Office. 5.10 DRYDEN FLIGHT RESEARCH CENTER (DFRC)—EDWARDS, CA Dryden is the sole “flight research Center” among the ten NASA field centers. Its SBIR Program Office is primarily responsible for proof-of-concept aeronautics flight-test support. SBIR Phase I activity at Dryden includes a cursory evaluation of infusion/commercialization opportunity in the Phase I proposal technical review process. Phase II proposal evaluation is limited to a formal out- side peer review of infusion/commercialization opportunity. In both Phase I and Phase II proposal review processes, the research director- ates make recommendations to the DFRC Ranking Committee. During a Phase II contract, SBIR program monitors (from research directorates) and the Program Office do work with awardees to help ensure that the small business aligns its SBIR technology with a NASA acquisition opportunity. This work includes as- sistance in assuring resonance between the SBIR firm’s choice of “key words” 52 and NASA’s base technology taxonomy, so that a search of the NASA SBIR Da- tabase for technologies that could be infused into NASA enterprise work would yield relevant SBIR project files. DFRC SBIR does not formally monitor either infusion or commercialization outcomes of SBIR projects, and evidence of “success stories”—usually obtained anecdotally, and then verified with the SBIR firm—is passed on to NASA SBIR Level 1 and 2 principals. Informally, notes Dryden SBIR Program Manager Rod Bogue, “. . . we try and call our SBIRs at least every two years to ascertain Phase III success, especially NASA infusion of their technologies. To me, the business case of a SBIR project has to focus on Phase III opportunity—but our administrative resources to help SBIR projects in this regard are very limited.”53 DFRC SBIR has no estimate of its Phase III results from SBIR contracts. Accord- ing to the NASA EHB Web site, DFRC has had two Phase III contracts. Two of Dryden’s current SBIR Phase II projects participate in the NASA 52 “Key words” are self-assigned by SBIR awardees to identify their technology or technology domain. The key word chain appears on the cover sheets of most agency SBIR records, including electronic databases, and are searchable by external inquiry. 53 Interview with Rod Bogue, February 16, 2005.

OCR for page 105
 SBIR AT THE NATIONAL AERONAUTICS AND SPACE ADMINISTRATION Commercialization Center incubator. Their performance as incubator tenants is not monitored by DFRC SBIR. 5.11 GLENN RESEARCH CENTER (GRC)—CLEVELAND, OH The GRC SBIR Program Office aggressively pursues infusion opportunities for SBIR technologies. As with all field centers, SBIR Phase I activity includes evaluation of infusion/commercialization opportunity in the Phase I proposal technical review process. Phase II proposal evaluation also includes a formal out- side peer review of infusion/commercialization opportunities. GRC adds unique resources to these processes: • In addition to strongly encouraging Mission Directorate program offices to participate in SBIR topic and subtopic development, the SBIR Program Office holds one-day infusion/commercialization workshops—just prior to the proposal review process—for reviewers and for the subtopic managers who are key to the review process. • The GRC SBIR Program Office has, since 1986, noticed relevant acqui- sition opportunities in the Army and Air Force, and maintains ties with appropri- ate program offices in both service agencies. • GRC markets its SBIR program to small business SBIR candidates at various national and state SBIR conferences, to identify candidates with infu- sion/commercialization potential. To track GRC SBIR “success stories,” GRC SBIR Program Manager Dean Bitler employs a five-column Excel table that notes the SBIR company and loca- tion, the NASA organizational code and COTR; the SBIR project title; the SBIR topic/subtopic number; the year of award; and finally, a succinct infusion/com- mercialization history. This table, updated regularly with SBIR firms, tallies 142 projects whose technologies were successfully infused into federal acquisitions, were success- fully commercialized, or were in transition to Phase III at the time of the update. GRC does not track aggregate revenues. According to the NASA Electronic Handbook Web site, there have been a cumulative 48 Phase III contracts at Glenn. For GRC SBIR Program Office, the key venues for evaluation discussion are the semi-annual meetings of SBIR Level 1, 2, and 3 personnel, where infusion/ commercialization issues are both formally and informally discussed. 5.12 GODDARD SPACE FLIGHT CENTER (GSFC)—GREENBELT, MD The “Center-wide investment strategy” of Goddard’s Technology Manage- ment Office—of which the GSFC SBIR Program Office is a component—ensures

OCR for page 105
 PROGRAM MANAGEMENT that applications needs of the resident Earth Science Enterprise Mission Director- ate (ESEMD) drive all Goddard technology development. The Center’s SBIR Program Office says its role is to assist in managing a balanced technology pipeline at Goddard. “We are part of the ESEMD program offices’ investment strategy, and have been for some time,” says SBIR PM Dr. James Chern. “Eight years ago, we aligned our SBIR program with Goddard’s needs . . . but still, it took at least three years to convince the mission program offices of SBIR benefits. In fact, SBIR technologies are essential to building mis- sion technology portfolios.”54 The GSFC SBIR program emphasizes maturing SBIR projects at high Tech- nology Readiness Level (TRL), as seen in an Astra project strategic technol- ogy/systems model. GSFC deploys the agency-wide SBIR process model. SBIR Phase I activity includes evaluation of infusion/commercialization opportunities in the Phase I proposal technical review process. Phase II proposal evalua- tion also includes a formal outside peer review of infusion/commercialization opportunities. At Goddard, there is also serial interface with ESEMD regarding topic/sub- topic development to ensure that SBIR awards help fill the Center’s technology pipeline. ESEMD group leaders and branch heads play key leading roles in the SBIR process—including aggressive roles as subtopic managers. In these roles, they advocate specific infusion opportunities for SBIR technologies in ESEMD programs. GSFC’s SBIR Program Office pairs subtopic managers with COTRs to promote infusion opportunities. The COTR takes the lead in Phase II, begin- ning with the presentation of infusion opportunities, cited in Phase II proposals, to the review committee. GSFC’s Earth Science Technology Office management strategy map il- lustrates Goddard’s approach to meeting the technology needs of what were the Earth Science Enterprise missions, including the SBIR contribution to those missions. GSFC SBIR Program Office also markets its resources to small business SBIR candidates at various national and state SBIR conferences, and participates in the Center’s annual “Small Business Day.” Due to staff and budget constraints, GSFC SBIR Program Office does not actively track the infusion/commercialization success of its SBIR contractors. GSFC was, however, the first Center to produce and disseminate a Success Sto- ries publication—a function now assumed by NASA SBIR Levels 1-2 through Spin-off and other publications. According to the NASA EHB Web site, Goddard has had 15 Phase III contracts over the years. While the process is informal, evaluation of SBIR practices is continuous at Goddard through discussions with the Technology Management Office. Recom- mendations from these discussions are presented at the semi-annual meetings of Level 1, 2, and 3 personnel. 54 Interview with Dr. James Chern, February 22, 2005.

OCR for page 105
 SBIR AT THE NATIONAL AERONAUTICS AND SPACE ADMINISTRATION Key infusion/commercialization assistance issues for GSFC SBIR include improved alignment of ESEMD element program themes and potential Phase III opportunities, improved identification of SBIR projects with high infusion poten- tial, increased work with COTRs to improve communication with potential tech- nology customers, and better use of the NASA technology transfer network. 5.13 JET PROPULSION LABORATORY (JPL)—PASADENA, CA Owned by NASA and operated by the California Institute of Technology (normally known as “Caltech”), JPL has five resident Directorates: Planetary Flight Projects, Solar System Exploration, Astronomy and Physics, Earth Science Enterprise, and Interplanetary Network. Of these, the Mars mission’s Technology Program Office is key to JPL’s SBIR infusion focus, and is the principal contribu- tor to SBIR topic/subtopic development. Most JPL SBIR awards directly support the field center’s Strategic Technol- ogy Plan, and the SBIR program is closely aligned with future mission needs. This alignment creates infusion opportunities for SBIR projects, but also triggers added JPL management and technical staff support for its SBIR program office. SBIR topic/subtopic managers have technical/management responsibilities at JPL and NASA Headquarters levels. As with all field Centers, SBIR Phase I activity includes evaluation of infu- sion/commercialization opportunities in the Phase I proposal technical review process. Phase II proposal evaluation also includes a formal outside peer review of infusion/commercialization opportunity. As a “national field center,” however, the JPL SBIR proposal review process is a cross-cutting effort in which techni- cal program office representatives from other field centers actively participate in proposal evaluation and ranking committees. Moreover, JPL SBIR awardees in Phase I are often required to go beyond the “proof of principle” work typical of Phase I activity and perform prototyp- ing work usually found in the final year of Phase II activity. “We want to avoid funding research that does not lead to infusion opportunity,” says JPL’s SBIR Program Manager Wayne Schober, “. . . in concert with our belief that the pro- gram offices should own SBIR.”55 At JPL, the lead for SBIR infusion/commercialization activity is the project Technical Monitor (often a technical program office representative) charged with advocating for SBIR awardees to project managers in mission program offices. The Technical Monitor is tasked with ensuring that an SBIR project observes NASA program requirements. During Phase II, the Technical Monitor also serves as a liaison between SBIR contractors and appropriate project managers, and ar- ranges for project representative participation in meetings with the SBIR contrac- tor. In turn, the small business is required to provide status reports to its Technical 55 Interview with Wayne Schober, February 28, 2005.

OCR for page 105
 PROGRAM MANAGEMENT Monitor on the interface with mission projects. This information is then pushed back up to center technology management personnel for evaluation. JPL SBIR Program Office both tracks and documents its Phase II awards for infusion/commercialization success, using a stringent dollar threshold of $100k for defining Phase III success.56 By its definition, of 228 JPL Phase II projects since 1983, 41 have been infused into NASA mission programs (technology and flight hardware), and 66 have found commercial success, with some projects having both infusion and commercialization success. Aggregate revenues are not available.57 While JPL SBIR Program Office uses the agency Success Stories Quad for- mat to record and market its SBIR projects, it has also created a mission-specific format to document infusion of SBIR technologies. Evaluation of SBIR practices is informal and continuous at JPL through dis- cussions within the Technology Management Office and various project offices. This evaluation is informed in part by status reports provided by the SBIR firms as well as input from Technical Monitors. Some recommendations, filtered from these discussions, are presented at the semi-annual meetings of SBIR Level 1, 2, and 3 personnel. 5.14 JOHNSON SPACE CENTER (JSC)—HOUSTON, TX JSC has the largest mission program-based research budget of the field centers. JSC’s SBIR program has a complex interaction with multiple Mission Directorates. ESMD and Space Operations are the principal customers and driv- ers of SBIR topic/subtopic development, with the Explorations and Aeronautics directorates also playing substantive roles. Discussions with directorate program leads and technology element managers align JSC’s SBIR program with the center’s mission priorities. SBIR Program Manager Dr. Kumar Krishen also reviews agency Broad Area Announcements (BAAs) and Intramural Calls for Proposals (ICPs) to parse them for information that can inform the topic/subtopic development process.58 As with other field centers, SBIR Phase I activity includes evaluation of in- fusion/commercialization opportunities in the Phase I proposal technical review process. Phase II proposal evaluation also includes a formal outside peer review of infusion/commercialization opportunities. JSC’s unique practice focuses on interaction between SBIR principals and mission program principals. When subtopic managers brief the ranking com- mittee on Phase II proposals, they must cite evidence of infusion opportunities, including summaries of specific discussions and meetings with directorate tech- nology element leads on SBIR technology viability. 56 Some other field centers reportedly use a lower dollar threshold to determine Phase III success. 57 Itshould be noted that there is only one Phase III contract cited on the NASA EHB Web site. 58 Interview with Dr. Kumar Krishen, February 24, 2005.

OCR for page 105
 SBIR AT THE NATIONAL AERONAUTICS AND SPACE ADMINISTRATION Dr. Krishen reaches beyond NASA in the greater Houston area to advocate for SBIR commercialization and investment in SBIR firms with federal program infusion opportunities. At the time of his interview with NAS program staff in early 2005, his most recent SBIR marketing pitch, for example, had been made to the Houston Area Economic Alliance. JSC SBIR Program Office actively promotes its Phase III successes through its annual Johnson Space Center Spin-Off Awards (which also reward “spin-in” technologies infused into NASA mission directorate programs) and Success Stories. In addition, it annually nominates SBIR candidates for NASA’s Space Technology Hall of Fame—with two inductees in both 2004 and 2005. JSC uses NASA’s Success Stories Quad format of one-page summaries of SBIR projects, for use in infusion/commercialization marketing and Phase III recordation work. Although JSC SBIR Program Office does actively monitor its SBIR portfo- lio of companies to assess Phase III success, tracking statistics are not available except to NASA SBIR Levels 1-2 principals for inclusion in the agency-wide Spin-off and other publications. According to the NASA Electronic Handbook Web site, JSC has had 44 Phase III contracts over the years. JSC SBIR’s main venues for evaluation are the semi-annual meetings of SBIR Level 1, 2, and 3 personnel, and the weekly/monthly electronic confer- ences. Infusion/commercialization issues—including Dr. Krishen’s innovations regarding infusion/commercialization opportunity consideration in Phase I and II proposal rankings—have been reviewed by the NASA SBIR community at the semi-annual events. Internally, informal evaluation discussions of SBIR program effectiveness are held between the SBIR Program Office and local Mission Di- rectorate personnel. In 2005, the principal emergent issue was the need for closer ESMD participation in the subtopic review process. 5.15 KENNEDY SPACE CENTER (KSC)—FLORIDA KSC has a small mission program research budget, and a correspondingly small SBIR program. Mission project offices, especially in Human and Robotic Technology work, drive SBIR infusion/commercialization activity. KSC SBIR shares the basic topic/subtopic development and Phase I-Phase II proposal review processes common to all NASA field centers, but includes a special emphasis on infusion opportunity and infusion planning by the SBIR contractor in both Phase I and Phase II proposal review. The responsible party is the SBIR project monitor, or COTR, who is tasked in the Phase II proposal presentation format with identifying NASA technology user community “gate- keepers.” Gatekeepers include project office program managers from the Expend- able Launch and Space Station mission groups who provide infusion leadership for SBIR technology once the COTR has obtained buy-in from that gatekeeper through discussions and meetings.

OCR for page 105
7 PROGRAM MANAGEMENT The KSC SBIR Program Office also monitors the participation of one of its SBIR firms resident in the NASA Commercialization Center incubator. Citing budgetary and staff constraints, the KSC SBIR Program Office does not actively track and update information on its Phase II projects regarding Phase III activity, apart from its interest in tracking SBIR-derived intellectual property. According to the NASA EHB Web site, there have been nine Phase III contracts over the years at KSC. As KSC SBIR is aligned with the center’s Human and Robotic Technology Program and that program’s Element Plan, the SBIR Program Office holds pe- riodic evaluative discussions with the infusion “gatekeepers” from Expendable Launch and Space Station organizations. Priority issues arising from these discussions in 2004-2005 are: closer align- ment of SBIR subtopics with KSC’s Human and Robotic Technology Program Element Plan; availability of demonstration venues to Phase II SBIR projects to ensure higher TRL levels upon Phase II completion; more formal infusion plan- ning by SBIR firms; and infusion incentives for these firms. These incentives might include cost-plus, award-fee contracts where the award-fee is heavily de- pendent on progress with the technology infusion process. No formal KSC plans have yet been developed to deal with these issues. 5.16 LANGLEY RESEARCH CENTER (LARC)—HAMPTON, VA Langley was founded in 1920 at the genesis of the aviation industry. The nation’s first aeronautics lab, Langley is dedicated to aeronautics and engineering technologies, as well as atmospheric science, with Earth Sciences responsibilities. Its $700 million budget in 2005 marks it as the smallest field center. However, Langley is co-located with Langley Air Force Base, the Jefferson National Ac- celerator Laboratory, Northrop Grumman Newport News naval shipyard and the Virginia Institute of Marine Science. As these institutions have a history of partnering to pursue technologies of mutual interest, numerous small firms with advanced technology skills can be found on LaRC’s perimeter. Since 2000, Vir- ginia has averaged the nation’s second-highest number of SBIR awards by state, with the Hampton Roads region surrounding LaRC as the state’s second most SBIR-productive area.59 At LaRC, SBIR Phase I activity includes evaluation of infusion/commercial- ization opportunities in the Phase I proposal technical review process. Phase II proposal evaluation also includes a formal outside peer review of infusion/com- mercialization opportunities. With LaRC’s emphasis on aeronautics and engineer- ing technologies, and atmospheric science, NASA mission directorate program offices do not appear to be solicited for participation in SBIR topic and subtopic 59 Hampton Roads Market Alliance, Hampton Roads: The Adanced Technology Center (presenta- tion), 2004.

OCR for page 105
 SBIR AT THE NATIONAL AERONAUTICS AND SPACE ADMINISTRATION development as frequently as with other field Centers. STTR topic development is influenced by LaRC’s Center for Excellence for Structures and Materials. The LaRC SBIR Program Office markets its resources to small business SBIR candidates at state SBIR events, its own annual “Research Development and Technology Conference” (in collaboration with Virginia’s Center for Inno- vative Technologies), and periodic small business mentoring events. Beginning in 2004, the LaRC SBIR Program Office leveraged its ties with regional tech- nology partners noted above, co-producing SBIR events in July and September, 2004. These two events launched the LaRC SBIR PO’s Technology Partnership Initiatie. The “Northrop Grumman SBIR Technology Transition Conference” on 28 July, 2004, introduced a pool of 30 SBIR Phase II awardees from the NASA LaRC, Navy SBIR, and Department of Energy SBIR programs to Northrop Grumman shipyard engineers through the yard’s VASCIC Technology Develop- ment Center. SBIRs were preselected according to their congruity with shipyard technology needs in 11 technology areas (e.g., data control, sensors and monitor- ing, power systems, etc.) for the Navy’s Virginia-class submarine and CVN-21 aircraft carrier. The College of William & Mary’s follow-on “SBIR Mentoring Day” on Sep- tember 1, 2004, gave five firms from the same SBIR Phase II pool the opportunity to present their business cases to an experienced group of technology investors, for identification and analysis of risk factors. LaRC SBIR Program Office plans to continue the Technology Partnership Initiatie at other corporate venues. LaRC SBIR Program Office does not appear to make available information on its Phase II projects regarding Phase III activity. However, according to the NASA EHB Web site, there have been 29 Phase III contracts over the years at Langley. As regards the Technology Partnership Initiatie, according to Northrop Grumman Newport News sources, in December, 2004 the shipyard was pursuing discussions about further collaboration with six of the SBIR projects that pre- sented at the July, 2004 conference. No further updates are available. Scrutiny of the NASA LaRC Web site does not reveal “Success Stories” or other awards accruing to LaRC SBIR awardees. For LaRCs SBIR PO, the principal venue for evaluation is the semi-annual meeting of SBIR Level 1 and 2 personnel with all ten NASA field center SBIR program managers. Infusion/commercialization issues can be both formally and informally discussed among the NASA SBIR peer community at these events. No other information is available about evaluation of the LaRC SBIR program or its Technology Partnership Initiatie. 5.17 MARSHALL SPACE FLIGHT CENTER (MSFC)—HUNTSVILLE, AL MSFC SBIR practice observes the basic NASA field center formula. SBIR Phase I activity includes evaluation of infusion/commercialization opportunities

OCR for page 105
 PROGRAM MANAGEMENT in the Phase I proposal technical review process. Phase II proposal evaluation also includes a formal outside peer review of infusion/commercialization opportuni- ties, as well as internal data on the alignment of candidate SBIR technologies with ESMD and/or SMD directorate needs. At MSFC, however, these needs are expressed by MSFC’s Chief Tech- nologist and the Center Technology Council, from subtopic development prior to Phase I onwards. The SBIR proposal review Ranking Committee’s Phase I recommendations are pushed back up to the Center Technology Council, which then assigns project Technical Monitors from appropriate program offices—or, in some cases, research labs. Infusion opportunity is anticipated by this direct interface in SBIR Phases I and II between the SBIR firm and an appropriate program office. Subtopic managers continue to play a role in SBIR work through the Phase II review process. Phase III activity is tracked somewhat formally at MSFC using a six-column Excel template. The center’s current tracking file cites 200 Phase IIIs (from ap- proximately 300 SBIR Phase II projects since 1989) which have a non-SBIR rev- enue aggregate of $117,531,532.60 Another ten SBIR projects are in the process of being added to this list, for an added $10,494,000. “Although we are staff and budget constrained,” says MSFC PM Lynn Garrison, “. . . we have learned to rely on procurement records—in addition to direct follow-up with SBIR awardees—as an accurate and comprehensive way of getting Phase III data we otherwise might miss. This was an MSFC decision, that the SBIR Program Office needed expert support to accurately track infusion/commercialization success.”61 Among the ten NASA field centers, MSFC approach to recording Phase III results is unique. Like other field Centers, MSFC evaluates its SBIR infusion/commercializa- tion practices informally, and it participates in discussions at the semi-annual meetings of field center SBIR PMs with NASA SBIR Level 1 and 2 staff. In ad- dition, MSFC SBIR Program Office has used strategic shifts at the agency-wide level—such as the recent merger of SBIR activity with ESMD—as an opportu- nity to reevaluate core practices. The recent decision to align center SBIR work through MSFC’s Chief Technologist and the Center Technology Council, instead of directorate POCs, is one such reform, as is the decision to track Phase III oc- currences through procurement sources. 5.18 STENNIS SPACE CENTER (SSC)—MISSISSIPPI SCC’s primary role is that of a test facility, and it averages six or fewer SBIR Phase II projects annually. Apart from observing the basic SBIR topic development and proposal review process, no special emphasis is placed on infu- sion/commercialization opportunity in the SSC SBIR program. 60According to the NASA EHB Web site, Marshall has had 113 (the largest by far of any of the 10 centers) Phase III contracts over the years. 61 Interview with Lynn Garrison, February 28, 2005.

OCR for page 105
0 SBIR AT THE NATIONAL AERONAUTICS AND SPACE ADMINISTRATION SSC has recorded four instances of Phase III infusion/commercialization. Although SBIR PM Jim Bryant believes that at least two of these four SBIR tech- nologies has achieved multiple Phase IIIs, no detailed tracking data is available. According to the NASA EHB Web site, SCC has had eight Phase III contracts over the years.