National Academies Press: OpenBook

SBIR at the Department of Defense (2014)

Chapter: 7 Findings and Recommendations

« Previous: 6 Program Management
Suggested Citation:"7 Findings and Recommendations." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×

7

Findings and Recommendations

The findings and recommendations in this chapter address the Statement of Task and are focused on improving the performance of the SBIR program at the Department of Defense against the four Congressional objectives for the Small Business Innovation Research Program (SBIR) program.1 These objectives have been reiterated in the 2011 program reauthorization and in the subsequent Small Business Administration (SBA) policy Directive that guides program implementation at all agencies. Section 1c of the SBA Directive states program objectives as follows:

“The statutory purpose of the SBIR Program is to strengthen the role of innovative small business concerns (SBCs) in Federally-funded research or research and development (R/R&D). Specific program purposes are to: (1) Stimulate technological innovation; (2) use small business to meet Federal R/R&D needs; (3) foster and encourage participation by socially and economically disadvantaged small businesses (SDBs), and by women-owned small businesses (WOSBs), in technological innovation; and (4) increase private sector commercialization of innovations derived from Federal R/R&D, thereby increasing competition, productivity and economic growth.”2

From the perspective of the Department of Defense (DoD), the second objective is by far the most important, even though it is often conflated in practice with the fourth objective. Many at DoD believe that meeting agency needs in the form of technologies that can help warfighters is commercialization.3 The committee’s findings, summarized below, identify accomplishments and issues with regard to each of these program objectives at

__________________

1See Box 1-2 and the discussion in Chapter 1 of the Committee’s task.

2SBA SBIR Policy Directive, October 18, 2012, p. 3.

3See Chapter 5.

Suggested Citation:"7 Findings and Recommendations." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×

DoD, as well as some specific aspects of program management. The committee’s recommendations follow these findings.

NRC STUDY FINDINGS

The SBIR program at DoD is meeting three of its four legislative and mission-related objectives. The program funds the development of mission-critical technologies that meet the specific needs of DoD components; it supports the development of products that reach the market at an appropriate rate and scale; and it encourages the development of transfer of new technical knowledge while connecting companies to universities and research organizations. At the same time, it is clear that DoD has failed to meet the important Congressional objective of increasing the involvement of woman- and minority-owned small businesses.

I. Commercialization

A. SBIR projects at DoD commercialize at a substantial rate. With regard to commercialization, projects funded by the SBIR program are reaching the market at, what is in the Committee’s judgment, an appropriate rate, and are also attracting substantial amounts of follow-on investment, which in many cases is a necessary next step toward commercialization.

  1. The percentage of Phase II projects reporting sales continues to be greater than 45 percent, based on responses to the National Research Council (NRC) Survey.4 This rate tracks closely with data from previous surveys and from DoD databases and is appropriate for projects focused on early-stage applied research. An additional 26 percent of projects reported that they anticipate future sales. If these expectations are even partially correct, then this suggests that a majority of SBIR Phase II projects will eventually reach the market.
  2. Data from the DoD commercialization database suggest that over time about 70 percent of Phase II projects at DoD reach the market.5
  3. Projects with very large commercial successes continue to occur but remain rare: about 1.5 percent of Phase II projects reported sales of more than $20 million. This type of skew is not unusual in early-stage finance.6

B. Total commercialization continues to be under-reported both in the DoD databases and through the survey instrument.

__________________

4See Table 3-1.

5See Table 3-18.

6See Table 3-32.

Suggested Citation:"7 Findings and Recommendations." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×
  1. Much commercialization by dollar value occurs in areas not covered by the DoD Federal Procurement Data System (FPDS), which does not track subcontracts executed through prime contractors (primes) at DoD. Nor is FPDS set up to track SBIR projects as they commercialize within DoD, beyond the first Phase III contract.
  2. Data from surveys, including the NRC survey, also have limitations. In particular, all outcome surveys capture data at the time of the survey, but SBIR projects typically have a relatively long product cycle, and hence the bulk of sales may take place after the date of the survey.

C. Substantial commercialization also occurs in the private non-defense sector.

Although not a high priority for DoD, this activity nonetheless addresses a core program objective.

  1. About a one-quarter of sales by value are to the domestic private-sector or export markets, according to the survey responses.7
  2. In some cases, these awards have had a highly leveraged impact on entire industries: for example, the massive commercial success of Qualcomm8 (which still leads the global market for handset chips) is attributed by a founder in part to the acquisition of SBIR awards at a key inflection point, funding critical research and providing the firm credibility vis-à-vis private investors. Similarly, iRobot executives noted that their contribution to an entire new sector of personal robotics (as well as advanced DoD applications) is based on research funded by early SBIR awards.9

D. Further investment in SBIR technology is another metric for value.

1.   Subsequent investment provides further evidence that SBIR projects generate significant commercial value.

a.   More than 60 percent of Phase II survey respondents reported additional investment funding, which is up slightly from the 2005 NRC survey (54 percent).10

b.   Substantial additional funding has been provided by internal company resources and by non-SBIR federal sources (in large part acquisition programs at DoD).

__________________

7See Table 3-3.

8See the Qualcomm case study, Appendix F.

9See the iRobot case study, Appendix F.

10See Table 3-6.

Suggested Citation:"7 Findings and Recommendations." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×

E. SBIR awardees indicated that overall the SBIR program had a profoundly positive effect on their companies.

Commercialization in the long run requires sustainable companies, and the SBIR program has supported the development of an ecosystem of small innovative companies in the United States.

1.   The NRC survey provided SBIR companies with the opportunity to report the overall impact of the SBIR program on the company, as well as to identify specific kinds of impacts.

a.   Twenty percent of Phase II winners reported that the program had a “transformative” effect on their company. Another 57 percent said that it had a “substantial positive long-term effect.”11

b.   Of the 649 detailed comments received, 4 reported negative effects, and 157 reported transformative positive effects. Widely differing kinds of impact were reported, summarized in Box 7-1.

F. SBIR is associated with modest job growth.

  1. NRC survey data indicate that the median size of firms grew from 17 employees at the time of award to 24 employees at the time of survey.12
  2. However, the data do not suggest that SBIR should be viewed as a substantial direct job creation program.13

II. Meeting Agency Needs

A primary objective of the SBIR program is to use SBIR-funded technologies to meet agency mission needs.

A. SBIR projects at DoD are in broad alignment with mission needs of the agency.

1.   There is substantial evidence that outputs from the program are taken up by federal agencies and in particular by DoD and by its primes.

a.   Sales are strongly focused on defense. About 60 percent of reported sales by value from SBIR projects go to either DoD directly or to DoD primes (NRC survey data). Of sales recorded in

__________________

11See Table 3-22.

12See Table 3-4.

13Further research is needed to gauge indirect employment effects.

Suggested Citation:"7 Findings and Recommendations." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×

BOX 7-1

Different Ways in Which SBIR Awards Helped to Transform Companies

  • Provided first dollars, funding company formation
  • Funded product areas where VC and other funders were not interested
  • Created connections to acquisition programs
  • Opened doors to many potential stakeholders in specific technologies, including agencies, primes, investors, suppliers, subcontractors, and universities
  • Helped address niche markets too small for primes and other large companies
  • Funded technology development
  • Enabled projects with high levels of technical risk
  • Supported adaptation of technologies to new uses, markets, and industry sectors
  • Provided resources for more diversified expertise, allowed hiring of specialists
  • Substituted for private capital funding during economic downturns
  • Attracted and developed young researchers
  • Redirected company activities to new opportunities
  • Developed connections to primes
  • Reduced costs
  • Helped address needs that require high tech at low volume and relatively low cost
  • Moved technology up to Technology Readiness Level (TRL) 7-9 (at which point acquisition funding becomes more likely)
  • Provided new companies with greater credibility
  • Encouraged researchers to enter business full time
  • Helped university researchers manage Intellectual Property (IP) and Information Technology Acquisition Review (ITAR) problems
  • Transformed company culture to become more market oriented
  • Drove researchers to focus on technology transition
  • Supported feasibility testing for high-risk/high-payoff projects (Phase I)

__________________

SOURCE: Analysis of company responses to 2011 NRC Survey.

Suggested Citation:"7 Findings and Recommendations." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×

      the DoD commercialization database, 63 percent go to DoD and DoD primes.14

b.   SBIR technologies are currently in use by the federal government. More than one- fifth of Phase II respondents reported that their technologies are currently in use by a federal system or program.15 This portion is higher than the 12 percent figure reported in the 2005 survey.

c.   Numerous case studies indicate ways in which specific SBIR projects have made a substantial difference to DoD capabilities, costs, or both. SBIR projects have led directly, for example, to new torpedoes at Navy, to significant components of the Joint Strike Fighter, and to bomb-disarming robots (funded by the Defense Advanced Research Projects Agency [DARPA] and other DoD components).16

B. DoD components have made substantial efforts to further align SBIR and mission needs.

1.   Components have reformed topic selection procedures to develop better linkages between the SBIR and acquisitions programs within DoD.

a.   DoD policy guidance now strongly emphasizes the need to align SBIR programs with agency needs. DoD calls for 50 percent of topics to be sponsored by acquisition offices.

b.   Topic selection now broadly requires substantial input from Program Executive Offices (PEOs). Navy has pioneered efforts to forge closer linkages; today, more than 90 percent of SBIR topics are sponsored by PEOs or Headquarters directorates.

2.   Components have implemented other initiatives aimed at the further alignment of SBIR and acquisitions:

a.   New initiatives (e.g., Phase II.517 at the Navy and the Air Force) indicate that some DoD components are experimenting with different ways of enhancing alignment and eventually transition.18

__________________

14See Table 3-3.

15See Table 3-3.

16Examples are drawn from extensive discussions with program staff at Navy, Air Force, DARPA, and the Special Operations Command (SOCOM), as well as company case studies, agency success stories, and other sources. Unfortunately, examples are not available from Army, which did not contribute to this study.

17Phase II.5 is discussed in Chapter 5. It is a new program designed to provide bridging funds between the end of Phase II and the start of full acquisitions funding. It is typically co-funded between the component SBIR office and an acquisitions program or office.

Suggested Citation:"7 Findings and Recommendations." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×

b.   Metrics are now heavily focused on transition. For example, Navy measures both the number of transitions and the dollar amount of Phase III contracts; Air Force focuses on the number of transitions and the percentage of transitions from Phase II awards, as well as the leverage generated through Phase II.5 contracts.

C. Phase III contracts provide direct evidence of value, and total Phase III funding continues to grow.

  1. Data drawn from FPDS indicated that in fiscal year (FY) 2009 approximately $650 million in Phase III contracts were signed with SBIR companies.
  2. Total Phase III contracts, excluding those at Navy (the Phase III leader), more than doubled between FY2004 and FY2009 (see Figure 7-1).
  3. Navy commercialization increased sharply in FY2010, coinciding with the introduction of new commercialization initiatives. 19 (see Figure 7-1).
  4. More than one-half of all additional investment reported by NRC respondents came from federal non-SBIR sources. This reflects in part efforts to attract funding using new matching fund requirements: the Air Force program had attracted a total of $337 million in additional investment through FY2013, in part, a response to its own investment of $93 million in SBIR program funds for Phase II.5 transitions.

D. Further investment in SBIR projects beyond Phase II continues to grow.

  1. This investment reflects the value of projects that may still not be ready for the market.
  2. Both Navy and Air Force now devote 20 percent of SBIR funding to Phase II.5 projects, most of which requires 1:1 matching funding from acquisition programs.

E. Providing execution-year flexibility and the potential for rapid deployment

1.   DoD has recognized the need for rapid technology development and deployment in the face of suddenly changing conditions. Previous reports noted SBIR contributions in this area. These have recently been strengthened.

__________________

18These experiments are discussed in some detail in Chapter 4.

19See the description in Chapter 5.

Suggested Citation:"7 Findings and Recommendations." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×

images

FIGURE 7-1 Phase III contracts reported through the Federal Procurement Data System, by total value, FY1999-2011.
SOURCE: Navy SBIR Program Office.

2.   A number of DoD initiatives outside the SBIR program have addressed the need for rapid deployment. For example, the Rapid Innovation Fund (RIF) has provided new ways to address high-urgency needs.

III. Fostering the Participation of Women and Other Under-Represented Groups in the SBIR Program20

A. Current participation is low and not increasing.

1.   Levels of participation are low and not rising.

a.   Data from DoD indicate that during the study period approximately 15 percent of awards went to woman-owned Small Businesses (WOSBs) and 7 percent to minority-owned Small Businesses (MOSBs).21

b.   The share of Phase I awards to WOSBs increased slightly in the early part of the past decade but have since remained flat.22

__________________

20Legislative language and the SBA policy directive both focus on a slightly different way of addressing essentially the same issue, using the term “members of socially and economically disadvantaged groups.” SBA in turn then defines members of these groups (see Chapters 4 and 6 for further discussion). In order to make the report more readable, the committee here utilizes the more easily understood notion of woman-owned small businesses (WOSBs) and minority-owned small businesses (MOSBs).

21See Table 6-22.

22See Table 6-26.

Suggested Citation:"7 Findings and Recommendations." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×

c.   The share of awards to MOSBs has declined since peaking in 2005. MOSB Phase I applications succeed at a lower rate than do non-MOSB applications.23

d.   The number of applications from WOSBs and MOSBs has not increased according to data provided by DoD. Phase I applications from WOSBs declined by about 10 percent during the period covered by DoD data (FY2007-2011).24

e.   DoD does not maintain data on woman and minority Principal Investigators (PIs). Data from the NRC survey indicates that these numbers are also low and not rising.

2.   DoD has no separate data on Black- and Hispanic-owned small businesses.

a.   The NRC survey indicated that Black- and Hispanic-owned small businesses are themselves a very small share of MOSBs overall. Black-owned small businesses accounted for approximately 0.5 percent of all respondents; Hispanic-owned firms, about 1 percent.25

b.   In the NRC survey, companies reported that 11 percent of PIs were minority (using the broad definition approved by SBA) (the same as the 2005 survey). However, further analysis indicates that only 1 percent of PIs were Hispanic, and less than 0.5 percent were African American.26

3.   Differences in the shares of Phase I and Phase II awards should be explored further.

a.   For each of the 6 years for which DoD provided detailed demographic data, and for both WOSBs and MOSBs, award rates were smaller than application rates—overall by 1.8 percentage points for WOSBs and 3.5 percentage points for MOSBs.27

b.   In the NRC survey, companies reported that they had a woman as PI on 10 percent of Phase I awards and only 6 percent of Phase II awards (compared with 4 percent for the 2005 survey).28

B. DoD has not made sustained efforts to “foster and encourage” the participation of woman- and minority-owned small businesses.

__________________

23See Chapter 6.

24See Chapter 6.

25See Table 6-22.

26See Table 6-20.

27See Chapter 6.

28See Table 6-25.

Suggested Citation:"7 Findings and Recommendations." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×

1.   As of August 2013, no component had in place a plan for outreach to these communities.

2.   The committee identified limited evidence of activity focused on this objective:

a.   Efforts at Team Subs (Navy) from 2001 to 2008 to reach out to Historically Black Colleges and Universities (HBCUs) and Minority-Serving Institutions (MSIs) appear to have been discontinued.

b.   No similar efforts appear to have been made to reach out to WOSBs within DoD components.

c.   The committee was not provided with any documentation on any other efforts in this direction, beyond the addition in 2013 of a panel on WOSBs and MOSBs at annual SBIR conferences.

3.   DoD does not report on or sufficiently track participation by WOSBs and MOSBs.

a.   DoD does not provide an annual report that covers either data on participation or efforts to foster and encourage participation at the different components.

b.   DoD does not appear to track MOSBs at a level sufficient to meet Congressional intent.

c.   DoD does not appear to track the participation of woman and minority PIs in the program.

IV. Stimulating Technological Innovation

A. The SBIR program at DoD supports the development and adoption of technological innovations.

1.   Selection of topics and individual projects for funding maintains a strong focus on developing innovative technologies.

a.   Topic selection is monitored to ensure that it focuses sufficiently on innovation. Topics are routinely rejected by DoD-level review for this reason.

b.   Scoring for individual projects is weighted toward technological innovation: 40 percent of the score for an application at Navy, for example, is based on the innovative characteristics of the project.

2.   Close integration with acquisition requirements does require careful monitoring to avoid over-focus on short-term needs, which could reduce the likelihood of breakthrough innovations funded in the past (e.g., in telecommunications, robotics, logistics).

Suggested Citation:"7 Findings and Recommendations." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×

a.   A number of survey respondents and case study interviewees indicated that they believed DoD was becoming progressively less interested in high-risk/high-reward research, focusing instead on projects that could be reasonably expected to transition within a fairly short period.

b.   Although DoD does not currently have any formal targets for transition, it is generally assumed that more transition is better. It should be recognized that there is, at least at the margin, a trade-off to be considered with higher-risk/higher-value research. Managers need to remain conscious of these trade-offs, although formal targets seem unlikely to be useful.

c.   No agency efforts exist to identify and highlight exceptionally innovative projects on a systematic basis.

B. SBIR is increasing connections between companies and universities.

1.   Survey data indicate that DoD SBIR projects continue to utilize universities in a variety of ways.

a.   More than one-third of DoD Phase II survey respondents reported a link to a university for the surveyed project. In about 20 percent of projects, universities were subcontractors, while almost as many had faculty or graduate students working on the project.29

b.   Survey respondents identified 211 different universities as project partners; 28 were mentioned by more than 10 respondents.30

c.   About 60 percent of SBIR companies reported at least one academic founder, and about one-quarter reported that the most recent prior employment of the founder was a university.31

d.   Data from the survey for linkages with universities, including use of faculty as PIs, use of graduate students as researchers, licensing of technology from universities, and use of a university as a subcontractor, all increased from the 2005 survey, suggesting growing university linkages with the DoD SBIR program.

C. SBIR projects generate knowledge-based outputs such as patents and peer-reviewed publications.

1.   Patenting remains an important component of knowledge diffusion (and protection).

__________________

29See Table 3-16.

30See Table 3-17.

31See Table 3-16.

Suggested Citation:"7 Findings and Recommendations." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×

a.   Slightly less than 60 percent of Phase II respondents reported filing at least one patent related to the surveyed project.32

b.   Slightly greater than 20 percent reported filing five or more related patents.33

2.   Publication of peer-reviewed articles remains the primary currency of scientific discourse, and despite the need to protect ideas in the commercial environment of small businesses, SBIR firms continue to contribute actively to scientific publication.

a.   About three-quarters of surveyed projects reported at least one resulting peer-reviewed publication.34

b.   One-quarter reported more than three publications resulting from the surveyed project.35

c.   Many of the companies interviewed for case studies made a point of indicating that they take a great deal of pride in the number of peer-reviewed publications developed by their scientists and engineers, both within and outside of the SBIR program.

V. Challenges for the DoD SBIR Program

A. Challenges within services and components

1.   The “SBIR as tax” mentality among Program Executive Officers (PEOs)

a.   Because of its unique funding structure as a percentage of extramural R&D, the SBIR program has often been viewed by R&D program managers in DoD (and at other agencies) as a tax imposed by supporters of small business.

b.   Continuing efforts to change this perspective, at both DoD and individual components, have had mixed success as reflected in interviews with agency staff and small businesses. Some components have been more successful than others.

2.   Difficulties in integrating SBIR into long-term technology maps

a.   While primes have large groups working on these issues, access for small businesses is available only sporadically and requires a

__________________

32See Table 3-14.

33See Table 3-13.

34See Table 3-15.

35See Table 3-15.

Suggested Citation:"7 Findings and Recommendations." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×

      degree of forward planning that may not be simple or cost-effective for small businesses.

b.   There is no single voice at the table for SBIR companies during map development processes; they must rely either on champions within primes or supporters within the agency staff community.

3.   Contracting and auditing issues.

a.   Contracting remains a challenge because Phase III contracts are unusual. They differ in important ways—especially the sole-source option and the SBIR data rights—from standard procurement contracts.

 

i.   Contracting officers may be reluctant to certify Phase III contracts in part because they confer important data rights on the company, which are by statute not negotiable.

ii.  Contracting officers may also be reluctant to accept the sole-source rights conferred under Phase III contracts, especially when DoD is under pressure to ensure that contracts are awarded as competitively as possible. Sole sourcing requires contracting office approval.

iii.  Recent funding cuts have, according to agency staff and recipient interviews, negatively affected SBIR contracting.

 

b.   The Defense Acquisition University (DAU) is the primary source of training for acquisitions and contracting officers. DAU does not currently require training in SBIR rules and contracting procedures, nor does it provide a regularly scheduled elective class on the SBIR program. Such a course could contribute to a better understanding of the opportunities and processes associated with SBIR funding for small firms.

c.   Defense Contract Audit Agency (DCAA) audits are both an overall barrier to entry into DoD programs, and a particular problem for small contractors, especially in some regions during some periods.36 Although audits are of course necessary to prevent fraud and abuse, DCAA does not appear to have effective procedures in place for addressing the problems its audits continue to cause for SBIR firms, even when no wrongdoing emerges. Among the issues raised by SBIR recipients are:

 

i.   Audits provide no useful feedback. Companies either pass or fail, and they may not be informed about reasons for failure.

__________________

362011 NRC Survey and Case Studies.

Suggested Citation:"7 Findings and Recommendations." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×

ii.   Audits take a long time. Several companies noted that audits of a small company, which they believe could be completed by a competent accountant in 2 weeks, took 18 months or more at DCAA.37

iii.  Penalties are very severe for small businesses, which are entirely blocked from further federal funding until problems are resolved. This can be catastrophic for small firms dependent on SBIR awards or other federal contracts.

iv.  DCAA does not appear to have in place an effective conflict-resolution process for small businesses. Companies report no office within the agency that provides support or advice.

 

4.   Continuity problems are caused by the regular rotation of agency liaisons.

a.   Agency and company interviews suggest that eventual project success is closely tied to the existence of effective project supporters within DoD.

b.   However, the NRC survey indicates that about one-third of Phase II projects had their Technical Point of Contact (TPOC) replaced during the course of the award. Given that the commercialization transition period (prior to handoff to a PEO) usually extends for at least another year beyond Phase II, it seems plausible that a very substantial portion of all TPOCs are rotated out before a project is completely transitioned.

c.   Continuity problems are multiplied by the additional need to train TPOCs before they can act effectively as project liaisons to the acquisition programs. Survey data suggest that fewer than half of TPOCs are “very helpful” or “somewhat helpful” in connecting projects to acquisition funding.

d.   Agency and company interviews suggested that, in some cases at least, the SBIR TPOC assignment is regarded as somewhat burdensome and is assigned where possible to the most junior staff available.

B. Challenges in working with the primes

1.   SBIR data rights—though clearly necessary and important—involve special considerations.

a.   DoD contracts typically require that primes provide all data rights to the contracting agency. SBIR regulations retain rights for the

__________________

37One case study company is in 2014 completing an audit of labor rates for 2008. See Appendix F.

Suggested Citation:"7 Findings and Recommendations." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×

      small company, sometimes making it harder to align its rights with the different rights conferred via the larger project.

b.   Primes have no incentives to respect SBIR company-data rights.

2.   Primes sometimes work in direct competition with SBIR companies; others cooperate effectively with SBIR companies.38

a.   While some companies have a track record of cooperating effectively with multiple primes contractors over a long period that may last decades, others report negative experiences, asserting that primes used SBIR technologies to win contracts and then failed to include them in subsequent work.

b.   There is no quantitative evidence to measure either of these outcomes. It does appear that in cases where there is no direct competition (e.g., Daniel H. Wagner’s specialized development of mathematical tools and algorithms), the relationship is likely to be better and more sustained than in cases when SBIR firms provide products or services that the prime could develop or has already developed itself.

3.   Primes respond to incentives, but they have not been incentivized to work with SBIR companies.

a.   The committee has not identified any documented cases in which primes were provided with financial incentives to work with SBIR companies.

b.   The Navy found that financial incentives for primes to work with selected demographics on Virginia-class subs had a substantial impact on subcontracting practices.

C. Challenges in tracking outcomes effectively and in reporting

1.   There are broad challenges to tracking commercialization for DoD SBIR programs, especially at the company and project levels. Companies move in and out of the program, and tracking in some cases stops when they leave (see below). More generally, commercialization may come years after an award and may involve multiple awards plus considerable additional funding. All of this makes it difficult to assert that any specific outcome “results from” an SBIR award. But there are also specific challenges with existing tracking tools.

2.   The FPDS is the primary source of program data: it captures all federal contracts larger than $3,000, but remains limited in important ways.

a.   Data entry related to Phase III awards remains somewhat erratic. Contracting officers must be trained to recognize and designate

__________________

38Other competitors for funds are DoD Labs, FFRDC, and UARCs.

Suggested Citation:"7 Findings and Recommendations." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×

      Phase III contracts, which differ from standard contracts and hence require affirmative recognition as such.

b.   Phase III designation is limited primarily to the first contract after the end of Phase II and possibly to the second such contract. However, downstream contracts may be very substantial and are not captured by FPDS.

c.   FPDS does not capture SBIR commercialization via subcontracts to primes. Evidence from the survey suggests that this accounts for a quarter of all commercialization.

d.   FPDS does not capture non-defense or export sales. These too account for about a quarter of all commercialization.

3.   The Company Commercialization Report (CCR) is the second major source of data on commercialization. It captures self-reported data from companies that continue to participate in the SBIR program. However, although it covers subcontracts and sales outside the defense sector, it too has important limitations.

a.   Being self-reported, CCR requires periodic cross-checks for accuracy. It is not clear whether these are currently funded by DoD.

b.   Companies not currently participating in the SBIR program have no incentive to participate in CCR and likely do not. DoD has not analyzed how and why companies cease CCR reporting.

c.   CCR provides only aggregate data at the point of updating. It is therefore not useful for longitudinal analysis.

4.   DoD does not effectively track the participation of women and underrepresented minorities in the SBIR program.

a.   Data collection is limited to self-certified MOSBs and WOSBs. There are no data collected on the demographics of PIs.

b.   Data on the participation of under-represented subgroups—notably African American- and Hispanic-owned businesses and PIs—are not tracked.

c.   Analysis of participation by women and minorities is limited to the reporting of aggregate numbers. There appears to have been no systematic analysis of these data with a view to identifying sources of applications from these groups as well as possible barriers.

5.   Even though contract reporting mechanisms allow DoD to track the award of sole-source contracts based on SBIR data rights, DoD does not in practice appear to track the incidence of these contracts in Phase III. This could be a useful data point for understanding how one of the

Suggested Citation:"7 Findings and Recommendations." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×

      more important characteristics of the SBIR program is used in practice; the data to do so may be available via FPDS.

6.   DoD does not differentiate between small business Phase III awards and legacy Phase III awards to former SBIR companies now either acquired or sized out of the program.39 Annual and other current reports are limited and provide minimal guidance for program management. The current annual report on the program provided by DoD is limited to a set of numbers that reflect annual awards. This is insufficient for a program of this size, scope, and importance.

7.   However, new reporting required under reauthorization imposes significant burdens. DoD staff indicate that following reauthorization the annual preparation of several new reports is required annually, some of which substantially overlap.40

D. Challenges in addressing under-represented populations

  1. The awards data described in Chapter 2, and the tracking deficits identified in the preceding section, suggest that DoD has not effectively addressed the mandate to foster the participation of women and other under-represented populations.
  2. Efforts to address these deficits do not require the adoption of quotas or other approaches that would reduce program effectiveness.

a.   DoD has not made a concerted effort to develop focused or targeted outreach programs.

b.   DoD has not developed benchmarks for appropriate levels of participation, based on a range of factors that might include the incidence of technically qualified PIs, the number of qualified companies that could apply, or repeat applications from previous winners.

E. Challenges caused by unanticipated effects of the reauthorization language

1.   In developing the Policy Guidance, SBA addressed section 5111 of the Reauthorization Act, concerning multiple Phase II awards.41 Under the

__________________

39Team Subs is the single largest PEO at DoD in terms of Phase III transitions, accounting for more than $1.5 billion in validated Phase III contracts. However, a review indicated that about 30 percent of that amount was accounted for by two legacy companies that were acquired, and another 30 percent went to a single small business that reached the size limits in 2010 (but continues, correctly under SBA and Federal Acquisition Regulation (FAR) rules, to receive Phase III awards based on previous work).

40See the description of Program Management in Chapter 6.

41The legislative language is contained in the SBIR/STTR reauthorization,

Suggested Citation:"7 Findings and Recommendations." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×

      new language, section 4(b)(5) of the Policy Directive reads: “(5) A Phase II awardee may receive one additional, sequential Phase II award to continue the work of an initial Phase II award.”42

This has been interpreted in ways that introduce substantial and possibly unanticipated inflexibility into program operations. The legislation imposes several limits on the award of SBIR Phase II funding:

a.   Only two Phase II awards can be made sequentially to a single company on a single technical topic.

b.   Total funding for all Phase II and II.5 awards cannot exceed $3 million without an explicit waiver from SBA.

c.   The legislation does not address matching non-SBIR funds, which could presumably be any amount. Air Force data indicate 3.6:1 leverage for SBIR funding (FY2007-2012), which suggests that $1 million on Phase II.5 SBIR funding has been matched by $3.6 million of non-SBIR funding (presumably without requiring an SBA waiver).

2.   Agency staff have in discussions indicated that this means that:

a.   Only two Phase II awards can be made per topic. This sometimes prevents “reachback”—laudable agency efforts to identify previous SBIR awards that developed technologies of current use, even if they were not transitioned at the time of the award. As a result, agencies other than the original funder (and even other Services or components within DoD) can be prevented from picking up a technology for transition. There is nothing in the legislative language to suggest that this interpretation is correct: the language discusses a second award to a company, not a second award per individual topic.

b.   The second award must follow immediately from the first. There is nothing in the legislative language to suggest this, but it is apparently the interpretation that currently governs within DoD.

____________________

“Section 5111: SEC. 5111. ADDITIONAL SBIR AND STTR AWARDS.

 

   Section 9 of the Small Business Act (15 U.S.C. 638), as amended by this title, is further amended by adding at the end the following:

      ‘‘(ff) Additional SBIR and STTR Awards.--

           ‘‘(1) Express authority for awarding a sequential phase II award.--A small business concern that receives a Phase II SBIR award or a Phase II STTR award for a project remains eligible to receive 1 additional Phase II SBIR award or Phase II STTR award for continued work on that project.” Public Law 112-81, 112th Congress

42SBA Policy Directive, section 4(b)(5).

Suggested Citation:"7 Findings and Recommendations." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×

c.   DoD Phase II.5 awards must be treated as Phase II awards for the purposes of this policy. This interpretation is understandable: reauthorization still does not permit agencies to use SBIR funds for Phase III, only for Phase II—so, almost by definition, Phase II.5 must be treated as a kind of Phase II; otherwise it would not be permissible to use SBIR funding to make these awards. However, this determination has the potential effect of essentially nullifying even the limited additional flexibility intended under the legislation. As a result, either:

 

i.   companies that have received only one Phase II are eligible for Phase II.5, but cannot then be awarded a sequential Phase II either from the original agency or from any other agency or component that wishes to adapt their work to new needs; or

ii.   companies that receive a Phase II.5 are ineligible for both sequential Phase II funding and any subsequent Phase II.5 funding for their technology.

 

d.   Legislative intent may be misinterpreted. The language was designed to expand the flexibility of the program by allowing agencies to add an additional Phase II award for projects that were not ready to transition but that still held promise (in ways effectively captured by Phase II.5). There is no evidence that Congress intended to limit reachback, prevent agencies from picking up prior awards from other agencies or components, or limit the provision of additional SBIR awards per topic.

F. Challenges in protecting small business data rights

1.   Contracting officers are often unfamiliar with SBIR contracts and data rights, especially as DAU provides no systematic training on this subject.

a.   Unless a contract is officially certified as a Phase III contract, data rights are not protected as required under the legislation.

b.   Contracting officers can be reluctant to sign off on what is to them an unusual form of contract.

c.   Some PEOs are much more diligent than others in ensuring that data rights are protected in the contracting process.

2.   There is no systematic reporting on the protection of data rights, and no effort to acquire systematic information.

a.   As noted elsewhere, there are no efforts to acquire granular information from SBIR awardees through regular surveys or other

Suggested Citation:"7 Findings and Recommendations." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×

      feedback mechanisms. In particular, there is no provision for the acquisition of anonymized information from awardees about program operations.

3.   Procedures for addressing problems with data rights are not sufficient.

a.   None of the components appears to have in place systematic mechanisms through which companies can raise difficulties and concerns.

b.   No component-level staff are explicitly responsible for addressing issues related to small business data rights.

c.   SBA has mechanisms in place that can be used, but they are slow moving and in some instances, at least, have proved ineffective in addressing problems even when SBA formally steps into contract disputes after upholding a complaint.

G. Challenges in identifying and transferring best practices

1.   There are no formal procedures for identifying and transferring best practices within DoD.

a.   Each component appears to operate almost entirely independently. Air Force is to be commended for recently visiting NAVAIR to seek a better understanding of its approach to SBIR, but this is unusual.

b.   Minimal attention is paid to identifying best practices even within larger components. There do not appear to be formal processes in place to evaluate program management practices, nor to transfer them between PEOs.

2.   Transferring best practices is a growing major challenge.

a.   The decision to end the annual SBIR program managers’ meeting leaves agency SBIR staff without any organized opportunity to connect in person to similar staff at other agencies and components.

b.   The decision not to hold a further “Beyond Phase II” conference at DoD means that even within the agency, knowledge transfer between components will be more limited.

c.   There are no program manager email lists or online forums that can connect staff on a day-to-day level either within DoD or across all agencies.

d.   There is to our knowledge no newsletter or other publication focused on SBIR program management—again either at individual components, at DoD, or across all agencies. The absence of

Suggested Citation:"7 Findings and Recommendations." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×

      conferences where recent experience and best practices are exchanged is a significant limitation on the effective operation of the program.

RECOMMENDATIONS

As noted in the NRC Study Findings section above, the DoD SBIR program continues to generate substantially positive outcomes and has been improving in a number of areas and on a number of metrics, although the lack of progress in meeting Congressional objectives for the participation of women and minorities remains a concern.

As with the Findings, the Recommendations section is organized around the four core Congressional objectives, with an additional section focused on general program management issues.

I. Commercialization

A. Improve Phase III transition43

DoD has made significant strides to put in place new mechanisms to encourage Phase III transition. The Phase II.5 program adopted by Air Force and Navy is a particularly helpful innovation, bringing together funding from the SBIR program and program offices (acquisitions) in a new matching-fund format, operating at a scale that could be sufficient to make a significant difference to eventual outcomes.

There are, however, a number of areas where improvement could be made.

1.   Aligning incentives for primes. The latter are a critical component of successful transition in that they are the primary pathway into major programs of record.

a.   Primes have few incentives—and many disincentives—to work with SBIR companies. There are some cases in which these partnerships work, and others in which they do not—and many other cases in which primes proceed as though SBIR does not exist at all.

b.   DoD should consider experimenting with different kinds of incentives to encourage primes to work more effectively—and more often—with SBIR firms. There is evidence that such financial incentives do affect the behavior of primes.44

__________________

43See Finding II and Finding IV-B.

44See the Chapter 6 discussion of incentives used by Team Subs.

Suggested Citation:"7 Findings and Recommendations." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×

2.   Improving outreach to PEOs. It has become apparent that PEOs and Program Contracting Officers (PCO) who have positive views of the SBIR program—and utilize it to address their mission needs—have much more satisfactory transition profiles. However, many PEOs and PCOs are not fully aware of the advantages provided by the SBIR program.

a.   Many PEOs continue to regard the SBIR program as more of a tax than a potential source of solutions to challenging technical problems.

b.   New administrative funding should be used in part to develop better briefing materials for PEOs.

c.   DoD should consider developing a briefing program for all PEOs and PCOs, and should in particular focus on briefings for new PEOs and PCOs.

d.   DoD should consider developing SBIR information modules for integration into DAU courses, particularly those designed for PEOs and PCOs.

B. Continue to explore ways to bring primes and SBCs together45

1.   Programs to bring primes and small businesses together should be encouraged.

a.   The recent efforts to develop Technology Transition Agreements at Air Force are one example of potentially useful approaches.

b.   Navy has operated the Navy Opportunity Forum for a number of years, bringing together selected SBIR companies and numerous representatives from the primes.

c.   Air Force operates an online showcase of SBIR companies and technologies.

2.   Where such efforts prove to be effective, DoD should proactively seek to spread the use of such approaches more widely, perhaps by providing a pot of funding for initiatives that target such activities.

C. Use of financial incentives for components46

1.   DoD should encourage components to experiment with financial incentives for the adoption of SBIR technologies. Given the substantial disincentives that exist, it is reasonable to address a market failure with corrective incentives.

__________________

45See Finding IV-B.

46See Findings IV-A, II-A, and V-A.

Suggested Citation:"7 Findings and Recommendations." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×

a.   DoD should experiment with explicit use of financial incentives in the form of participation bonuses, similar to those used in the Team Subs program in the middle of the previous decade to enhance utilization of small business concerns (SBCs) and WOSBs.

b.   Even where financial incentives are not available, DoD should consider encouraging components to add explicit targets to prime contracts, in the same way that targets for the participation of small businesses more generally have been added to contracts.

2.   DoD should ensure that these initiatives are systematically tracked and evaluated (see section below on Tracking).

a.   For this approach to be effective, DoD will need to greatly advance its ability to track the use of SBIR companies as subcontractors to primes.

b.   There are long lags between the initiation of incentives and eventual outcomes (Navy’s experiments required a 5-year period between contract agreement and payment on incentives). Thus, tracking must be both consistent and persistent.

D. Highlight exceptionally innovative projects47

1.   DoD does not have a process for highlighting exceptionally innovative and effective projects. One potential model for informed assessment of scientific/technological impact is the practice of editorial boards for some leading scientific journals that review the articles published by their journal over the previous year and collectively identify the “most influential” few articles of each type featured. These project awards would be separate from agency programs to recognize sustained success by SBIR/STTR companies such as the National Science Foundation’s Tibbetts Award or the Department of Energy’s SBIR/STTR Small Company of the Year Award.

a.   At each agency or sub-agency program, SBIR/STTR staff and the TPOCs who monitored recently completed Phase II products would identify and evaluate most scientifically/technologically innovative projects in various categories. The “best” one or few projects per category would be identified as Innovation Leaders and recognized with a suitable certificate, a description on the agency’s SBIR/STTR website, and a press release. Project categories might include best scientific breakthrough; best

__________________

47See Findings IV-A and V-G.

Suggested Citation:"7 Findings and Recommendations." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×

      hardware development and engineering; best software development and engineering; best commercial potential; etc.

b.   In order to stimulate staff engagement, moderate cash awards to the TOPCs who monitored the recognized projects and the author(s) of the solicitation topics addressed by these projects could be provided.

II. Addressing Under-Represented Populations

A. DoD should substantially enhance efforts to address the clear Congressional mandate to foster the participation of under-represented populations in the SBIR program.48

1.   Significant effort and resources will need to be committed if change is to occur as it must. Given the highly disaggregated nature of the program, component-level activity will be required.

2.   DoD should not develop quotas for the inclusion of selected populations into the SBIR program. Such an approach is not necessary to meet Congressional intent and is likely to reduce program effectiveness.

B. DoD should develop new benchmarks and metrics.49

1.   Improve participation metrics: The SBIR/STTR program office should work with NSF and draw on work of the NSF indicators group to develop much improved metrics for benchmarking the participation of underserved populations, developing and publishing clear benchmarks based on a defensible analysis of existing data.

2.   Disaggregate Benchmarks: Measures of the participation of socially disadvantaged groups must be disaggregated by ethnicity, and attention focused on the clear Congressional intent to support “minority” participation. We do not believe a focus on the current SBA definition of “socially and economically disadvantaged” is in any way sufficient to meet this objective.

3.   Customize Benchmarks: Points of reference should be developed separately (though perhaps drawing on a shared methodology) for women and minorities. These benchmarks should be shared with other SBIR agencies. Benchmarks should address key questions that would include, for both SBIR and STTR:

a.   Shares of applications from companies majority-owned by women and minorities.

__________________

48See Finding III-A.

49See Finding III-B.

Suggested Citation:"7 Findings and Recommendations." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×

b.   Shares of applications with woman and minority principal investigators.

c.   Share of Phase I awards.

d.   Shares of Phase II awards.

4.   Track Related Program Operations: Metrics should also track related program operations including outreach efforts. (See below.)

5.   Components should be required to report annually to OSB on outreach plans designed to meet Congressional mandates.

C. DoD should develop an outreach and education program focused on expanding participation of under-represented populations.50

This will require the provision of agency resources and senior staff time, and should be a high priority for the program. DoD and the DoD components will need to make concerted efforts in this area.

1.   Develop Outreach Strategy: DoD should develop a coherent and systematic outreach strategy that provides for cost effective approaches to enhance recruitment of both woman- and minority-owned companies and female and minority PI’s, developed in conjunction with other stakeholders and with experts in the field.

2.   Integrate Outreach Effort: DoD should ensure that outreach to selected populations is an integral part of its overall outreach.

3.   Review Selection Processes: DoD should review internal award and selection data and processes to address questions arising from disparities between Phase I and Phase II awards to selected populations.

4.   Provide Management Resources: DoD should provide significant management resources as improving participation is likely to be both difficult and a long term effort.

5.   Designate Staff: DoD should designate a senior staffer to work exclusively on participation issues, providing for both improved reporting and the deployment of new initiatives laid out in the new strategy identified in 1) above.

6.   The DoD Office of Small Business (OSB) should improve tracking and metrics against which to benchmark component activities in relation to this Congressional objective.

a.   Metrics should address all aspects of the source-selection process, including the percentages of applications and awards for each subgroup.

b.   Metrics should assess the extent and impact of outreach activities that target under-represented populations.

__________________

50See Findings III-B and V-D.

Suggested Citation:"7 Findings and Recommendations." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×

c.   DoD should improve tracking accuracy by adding a second option to the checkbox for self-certification as a minority-owned firm, which asks respondents to declare which under-represented group or groups they belong to.

7.   Metrics and other reporting should be collated annually and included in the proposed revised annual report (see III.A.4 below).

III. Tracking, Data Collection, and Adoption of Best Practices

High-quality data, collected systematically and in a timely manner, are at the core of developing data-driven management. Currently, DoD tracking has significant weaknesses. This matters because high-quality data and analysis are required for the identification and adoption of best practices.

A. Further developing a culture of monitoring, evaluation, and assessment predicated on enhanced information flows.51

Now that additional administrative funding has become available through reauthorization, it is very important that DoD address the need for better alignment of data collection, agreed metrics, and utilization of effective evaluation and assessment tools to guide program management. The committee anticipates that the development of more careful monitoring and more sophisticated analysis of key variables would substantially improve outcomes over the medium term.

1.   DoD spent $1.05 billion in 2012 on SBIR funding.52 It must deploy appropriate levels of resources to determine how to spend these funds to generate the maximum return. Detailed analysis of outcomes could help to answer many operational questions. For example:

a.   Is it possible to systematically identify topics that transition more effectively?

b.   What are the year-to-year trends in transition achievement?

c.   Is SBIR best suited for certain kinds of technologies or sectors?

2.   SBIR companies—like “customers” in other markets—are an important source of information about program strengths and weaknesses. This knowledge is almost entirely excluded from formal program evaluation.

__________________

51See Findings I-B, III-A and V-C.

52Department of Defense, DoD Annual SBIR Report Summary, <http://www.dodsbir.net/annualreport/annrpt.html>, accessed February 3, 2014.

Suggested Citation:"7 Findings and Recommendations." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×

a.   DoD should develop pathways to provide ongoing feedback from companies about program activities and operations. These could include electronic communication tools such as wikis, listservs, and other emerging social media.

b.   Similarly, DoD should consider developing or expanding mechanisms through which SBCs can share information about SBIR projects, helping them to find technical or marketing partners and to navigate the often-complex regulatory and technical environment of DoD programs.

c.   DoD should consider introducing annual surveys to probe SBC knowledge about the program more directly. Such surveys can also be an important source of suggestions for program improvement, and can flag obscure but important problems—such as those reported with DCAA—in a systematic way.

3.   DoD should develop a more sophisticated approach to analyzing the data that it already has. For example, perhaps limited by the available resources, there have been few systematic efforts within DoD to evaluate factors that tend to encourage successful transitions between Phases, into Phase II.5, and then into Phase III and beyond.

a.   DoD should seek to develop a more consistent approach to the application of quantitative metrics to program management.

b.   DoD should also identify and adopt a much more systematic approach to the current use of success stories, which are little more than promotional material for the companies and the agency. Once improved, success stories could be analyzed to highlight trends and to identify variables playing a key role in successes, as well as help stakeholders understand the extent of program successes.

4.   Reporting requirements. We recommend that DoD simultaneously address the gaps in its current reporting and the burdens imposed by the numerous requirements imposed through reauthorization by providing a single much more comprehensive annual report that could, after appropriate consultations, be used to satisfy the reporting requirements of numerous Congressional sponsors. The report should

a.   include up-to-date data on awards and applications by phase, state, and component,

b.   include narrative that described developments and initiatives in the program on an annual basis,

c.   summarize outcomes data from FPDS and CCR,

d.   address the take-up of SBIR-funded technologies within DoD,

e.   address efforts to increase participation among under-represented populations, and

Suggested Citation:"7 Findings and Recommendations." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×

f.   reflect any specific additional requirements of sponsors.

B. FPDS must be improved in some significant ways.53

1.   The accuracy of data recorded in FPDS needs to be improved. Experience at Navy indicates that erroneous data entry is a significant problem.

a.   Coverage of Phase III contracts can be improved by training contracting officers more effectively in the rules governing Phase III.

b.   An additional layer of (incentivized) reviewers can be added by requiring that the SBC be provided with ongoing opportunities to review its FPDS entries, including its Phase III status.

2.   FPDS Phase III reporting does not extend to cover all commercialization from SBIR.

a.   DoD should consider ways to track SBIR technologies through FPDS downstream through follow-on contracts: currently, the more successful a technology is, and the more additional contracts it generates, the less likely these are to be certified as Phase III awards.

C. DoD should improve tracking of SBIR company subcontracts through the primes.54

1.   Under reauthorization, primes are now required to start tracking SBIR subcontracts.55

a.   Large prime contracts are, under the reauthorization legislation, required to develop a plan for SBIR technology insertion and to provide reports that reflect both the plan and the extent to which it is realized. DoD should work to ensure that these goals are in place as soon as possible.

__________________

53See Findings V-C and I-B.

54See Findings I-B and V-C.

55Public Law 112-81 112th Congress, Section 5122 a (7)

Insertion incentives.--For any contract with a value of not less than $100,000,000, the Secretary of Defense is authorized to--

(A) establish goals for the transition of Phase III technologies in subcontracting plans; and

(B) require a prime contractor on such a contract to report the number and dollar amount of contracts

Suggested Citation:"7 Findings and Recommendations." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×

b.   DoD should develop standard language for insertion into large prime contracts that reflects the intent of the law by providing both a goal for SBIR insertion and a standard reporting mechanism and timeline.

2.   CCR provides unique data and should continue to be used to track commercialization via self-reported data from SBIR companies.

a.   Until the proposed SBA Commercialization Database is fully operational, DoD should continue to utilize CCR.

b.   When the SBA database becomes operational, DoD should ensure that data collected in CCR are seamlessly and completely included in the new database.

IV. Program Management and Agency Mission Objectives

Recommendations in this section are not explicitly designed to address one of the four legislative objectives for the program. They are designed to improve program operations in ways that should enhance the program’s ability to address some or all of these objectives.

A. Review and if necessary amend Policy Guidance.56

Changes should be made to address guidance at SBA, DoD, or component levels that impose unnecessary rigidity on program operations. Specifically, amendments should be made to

  1. encourage rather than obstruct “reachback” to previously funded research; and
  2. support the infusion of additional funding via programs similar (but not limited) to Phase II.5.

B. Improved use of TPOCs57

1.   DoD should identify ways to ensure that the knowledge and enthusiasm of sponsoring TPOCs is not lost to the project. For example, DoD might consider

a.   encouraging wider adoption of programs similar to the Air Force’s Technology Agent (TA) strategy—after a review and evaluation of the TA system in operation. This would put in place permanent liaison officers.

__________________

56See Finding II-B and V-C.

57See Finding V-A.

Suggested Citation:"7 Findings and Recommendations." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×

b.   considering ways to support ongoing engagement by TPOCs in projects after they have formally handed them on at the end of a rotation.

c.   considering mechanisms to improve the transfer of knowledge during handoffs.

2.   Given that TPOC rotation is part of DoD structure, DoD should improve training for TPOCs. It should consider

a.   identifying and implementing ways to better train TPOCs about the operation of the SBIR program;

b.   identifying and implementing ways to better train TPOCs in the operation of acquisition programs so that they can become more effective liaisons between SBIR projects and companies and acquisition programs; and

c.   ensuring that SBIR-related programming is in place at the DAU, which should include two elements:

 

i.   curricula and programs for TPOCs; and

ii.  parallel curricula as part of standard training programs for acquisitions officers, to ensure that all are familiar with the operation of and opportunities provided through the SBIR program.

C. Protecting small business data rights.58

Although the reauthorization provided full support to existing provisions with regard to SBIR company data rights, some companies reported cases in which these rights were breached with only limited recourse.

1.   DoD should establish a link on its website for companies to report what they believe to be breaches of data rights.

2.   DoD should track reported breaches (whether reported through the website or via other means). OSB should seek ways to

a.   track the incidence of reported breaches on an annual basis; and

b.   generate a section of the proposed annual report on data breaches, which would include information about alleged breaches disaggregated to the component level, along with a short narrative that explains what OSB has done to explore and if necessary remedy these alleged breaches.

__________________

58See Findings V-A, V-B, and V-F.

Suggested Citation:"7 Findings and Recommendations." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×

3.   DoD should work with SBA to explore mechanisms through which SBIR data rights could be protected more effectively.

D. Identifying and transferring best practices within DoD.59

Some parts of DoD are experimenting vigorously with the new tools provided in part through re-authorization and previous adjustments to the program. They should be strongly commended for doing so. However, the point of experimentation is in part to identify best practice and then to encourage its adoption more widely. Neither occurs widely enough within DoD or even within individual components, where activities are often decentralized.

1.   DoD should develop a process for tracking experimentation within the SBIR program. This is likely to track the different elements of program management. Tracking should include

a.   annual reports on program initiatives from SBIR component program offices to OSB (which could derive from internal reporting within components that is currently not shared with OSB); and

b.   reports that cover activities down to the appropriate level of experimentation. In some cases—for example some parts of Navy—this occurs even below the level of the PEO.

2.   Given the long lags in some cases between experiment and outcome, DoD will need to find mechanisms to measure effectiveness even before quantitative outcomes become available.

a.   These should include input data that reflect the extent of rollout for any given experiment and specific objectives, as well as specific identification of topics and offices involved.

b.   DoD should explore a variety of tools for measuring effectiveness, including surveys of participants (both within DoD and in the primes and small businesses), as well as the application of other possible tools, including the use of social media.

3.   DoD needs to focus attention on the development of a comprehensive toolset of mechanisms for transferring both formal and informal knowledge about best practices.

a.   OSB should support the scheduling of annual program manager conferences in Washington, DC, as a means of supporting face-to-face interaction between components across agencies.

__________________

59See Finding V-G.

Suggested Citation:"7 Findings and Recommendations." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×

b.   OSB should deploy online tools that can be used to help share information among SBIR offices, acquisition offices, and supporting functions such as contracting offices and even DCAA.

c.   DoD should reinstate the “Beyond Phase II Conference” or provide a similar opportunity for in-person knowledge transfer.

V. Contracts and Audits

A. Auditing: Auditing issues, although technical and only tangentially related to the SBIR program as such, can cause devastating damage to small business and anecdotally appear to represent a significant barrier to program participation for new applicants and to the timely award of contracts by DoD program managers.60 DoD should address DCAA on issues of concern to SBIR companies as a matter of high priority at senior levels. DoD should:

1.   explore with DCAA (and perhaps relevant Congressional staff) the possibility of developing less onerous and more effective auditing procedures for small businesses that can be completed in a more timely manner. This would of course also reduce the burden on auditing staff; and

2.   provide all SBIR awardees with up-to-date information about redress procedures within DCAA as part of initial contract information deployment. If these are not satisfactory, then OSB should work with DCAA to improve them.

3.   Based on enhanced feedback from SBIR recipients, DoD should include a section in the proposed annual report on auditing concerns.

B. Contracting: SBIR contracts are unusual within the world of DoD contracting. They are accordingly not always handled correctly, and SBIR companies note that contracting issues are sometimes very challenging, especially given recent cuts in the resources devoted to contracting at DoD.61

1.   DoD should ensure that comprehensive training on SBIR contracting is available to contracting staff at all components.

a.   OSB should work directly with DAU to ensure that a course is developed to cover contracting issues with the SBIR program and that this course is made available to all contracting officers at least annually.

b.   All units that service SBIR contracts should ensure that they have on staff at least one employee who has been certified as

__________________

60See Finding V-A.

61See Finding V-B.

Suggested Citation:"7 Findings and Recommendations." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×

      completing SBIR contracts training provided by DAU or an equivalent, to act as an expert resource.

2.   Contracting practices vary widely within DoD, and small businesses have little recourse to address problems at the component level. DoD should:

a.   provide opportunities for SBCs to raise concerns about contracting practices at the component level, by mandating that components provide feedback opportunities and by providing its own feedback mechanisms for small businesses related to contracting; and

b.   consider developing an “expert group” of senior contracting officers who can meet virtually and can provide direct advice and support for small businesses in the event of contracting difficulties.

Suggested Citation:"7 Findings and Recommendations." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×

 

 

APPENDIXES

Suggested Citation:"7 Findings and Recommendations." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×

This page intentionally left blank.

Suggested Citation:"7 Findings and Recommendations." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×
Page 200
Suggested Citation:"7 Findings and Recommendations." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×
Page 201
Suggested Citation:"7 Findings and Recommendations." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×
Page 202
Suggested Citation:"7 Findings and Recommendations." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×
Page 203
Suggested Citation:"7 Findings and Recommendations." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×
Page 204
Suggested Citation:"7 Findings and Recommendations." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×
Page 205
Suggested Citation:"7 Findings and Recommendations." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×
Page 206
Suggested Citation:"7 Findings and Recommendations." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×
Page 207
Suggested Citation:"7 Findings and Recommendations." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×
Page 208
Suggested Citation:"7 Findings and Recommendations." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×
Page 209
Suggested Citation:"7 Findings and Recommendations." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×
Page 210
Suggested Citation:"7 Findings and Recommendations." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×
Page 211
Suggested Citation:"7 Findings and Recommendations." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×
Page 212
Suggested Citation:"7 Findings and Recommendations." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×
Page 213
Suggested Citation:"7 Findings and Recommendations." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×
Page 214
Suggested Citation:"7 Findings and Recommendations." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×
Page 215
Suggested Citation:"7 Findings and Recommendations." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×
Page 216
Suggested Citation:"7 Findings and Recommendations." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×
Page 217
Suggested Citation:"7 Findings and Recommendations." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×
Page 218
Suggested Citation:"7 Findings and Recommendations." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×
Page 219
Suggested Citation:"7 Findings and Recommendations." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×
Page 220
Suggested Citation:"7 Findings and Recommendations." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×
Page 221
Suggested Citation:"7 Findings and Recommendations." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×
Page 222
Suggested Citation:"7 Findings and Recommendations." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×
Page 223
Suggested Citation:"7 Findings and Recommendations." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×
Page 224
Suggested Citation:"7 Findings and Recommendations." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×
Page 225
Suggested Citation:"7 Findings and Recommendations." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×
Page 226
Suggested Citation:"7 Findings and Recommendations." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×
Page 227
Suggested Citation:"7 Findings and Recommendations." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×
Page 228
Suggested Citation:"7 Findings and Recommendations." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×
Page 229
Suggested Citation:"7 Findings and Recommendations." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×
Page 230
Suggested Citation:"7 Findings and Recommendations." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×
Page 231
Suggested Citation:"7 Findings and Recommendations." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×
Page 232
Suggested Citation:"7 Findings and Recommendations." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×
Page 233
Suggested Citation:"7 Findings and Recommendations." National Research Council. 2014. SBIR at the Department of Defense. Washington, DC: The National Academies Press. doi: 10.17226/18821.
×
Page 234
Next: Appendix A: Overview of Methodological Approaches, Data Sources, and Survey Tools »
SBIR at the Department of Defense Get This Book
×
Buy Paperback | $64.00 Buy Ebook | $49.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

Created in 1982 through the Small Business Innovation Development Act, the Small Business Innovation Research (SBIR) program remains the nation's single largest innovation program for small business. The SBIR program offers competitive awards to support the development and commercialization of innovative technologies by small private-sector businesses. At the same time, the program provides government agencies with technical and scientific solutions that address their different missions.

SBIR at the Department of Defense considers ways that the Department of Defense SBIR program could work better in addressing the congressional objectives for the SBIR program to stimulate technological innovation, use small businesses to meet federal research and development (R & D) needs, foster and encourage the participation of socially and economically disadvantaged small businesses, and increase the private sector commercialization of innovations derived from federal R&D. An earlier report, An Assessment of the Small Business Innovation Research Program at the Department of Defense, studied how the SBIR program has stimulated technological innovation and used small businesses to meet federal research and development needs. This report builds on the previous one, with a revised survey of SBIR companies. SBIR at the Department of Defense revisits some case studies from the 2009 study and develops new ones, and interviews agency managers and other stakeholders to provide a second snapshot of the program's progress toward achieving its legislative goals.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!