National Academies Press: OpenBook
« Previous: 1 Introduction
Suggested Citation:"2 Background." National Academies of Sciences, Engineering, and Medicine. 2021. DoD Engagement with Its Manufacturing Innovation Institutes: Phase 2 Study Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/26149.
×
Page 6
Suggested Citation:"2 Background." National Academies of Sciences, Engineering, and Medicine. 2021. DoD Engagement with Its Manufacturing Innovation Institutes: Phase 2 Study Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/26149.
×
Page 7
Suggested Citation:"2 Background." National Academies of Sciences, Engineering, and Medicine. 2021. DoD Engagement with Its Manufacturing Innovation Institutes: Phase 2 Study Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/26149.
×
Page 8
Suggested Citation:"2 Background." National Academies of Sciences, Engineering, and Medicine. 2021. DoD Engagement with Its Manufacturing Innovation Institutes: Phase 2 Study Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/26149.
×
Page 9
Suggested Citation:"2 Background." National Academies of Sciences, Engineering, and Medicine. 2021. DoD Engagement with Its Manufacturing Innovation Institutes: Phase 2 Study Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/26149.
×
Page 10
Suggested Citation:"2 Background." National Academies of Sciences, Engineering, and Medicine. 2021. DoD Engagement with Its Manufacturing Innovation Institutes: Phase 2 Study Interim Report. Washington, DC: The National Academies Press. doi: 10.17226/26149.
×
Page 11

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

2 Background DOD MANUFACTURING INSTITUTES DoD has a strong tradition of innovation development and implementation in support of next- generation defense systems. Within DoD’s technology development enterprise, there is constant attention to the maturation of technologies that support the challenging performance goals of future military systems. It is recognized that, in order to effectively mature and transition DoD science and technology advances into production, access to a robust and responsive U.S. industrial base is essential. Ideally, the industrial base would be equipped with advanced manufacturing technologies, which could deliver critical products and systems affordably and rapidly. To better support the need for timely, effective manufacturing technology development and transition, DoD has established nine Manufacturing Innovation Institutes through its Defense-wide Manufacturing Science and Technology (DMS&T) program element within the DoD Manufacturing Technology (ManTech) program. The institutes are considered by DoD to be crucial facilitators that bring together innovative often non-traditional ecosystems in key technology and market sectors in the United States, enabling their vibrancy and robustness.1 The DoD Manufacturing Innovation Institutes are industry-led, with a dual public–private benefit, providing large commercial market potential while meeting key U.S. defense needs. The mission of the nine DoD-established institutes addresses both commercial and defense manufacturing needs within specific, defense-relevant technology areas. FIGURE 2.1 The DoD Manufacturing Innovation Institutes are cost-shared public private partnerships focused on bringing new technologies to U.S. warfighters. SOURCE: Department of Defense, 2021, Fiscal Year 2020 Industrial Capabilities Report to Congress, p. 157. 1 See Department of Defense (DoD), 2017, Department of Defense Manufacturing USA Strategy, Version Date September 8, 2017, Director DoD Manufacturing Technology Program, OUSD(R&E) Strategic Technology Protection and Exploitation. PREPUBLICATION COPY – SUBJECT TO FURTHER EDITORIAL CORRECTION 2-1

As is shown in Figure 2.1, DoD has committed approximately $1.12 billion to date for these nine institutes,2 and non-DoD partners have contributed approximately $1.93 billion. As noted in Fiscal Year 2020 Industrial Capabilities Report to Congress,3 DoD views the MII public–private partnerships as a means to achieve significant advances in important manufacturing technologies in a cost-constrained environment. However, budgeted core funding for the institutes is planned to decrease from the FY 2020 appropriated level of $167 million to about $64 million in FY 2022-2025,4 which will require difficult decisions by DoD and the institutes on how to accommodate the drop in future funding. The mission of the DoD MIIs is consistent with the responsibilities for manufacturing technology assigned to the USD(R&E) and overseen by the OSD ManTech office.5 That office recognized the need for a coherent and effective strategy to guide the establishment and sustainment of MIIs, and also to ensure their value to DoD and the nation, by setting five strategic goals for its Manufacturing USA institutes:  Goal 1: Drive impactful advanced manufacturing research and development.  Goal 2: Encourage the creation of viable and sustainable institute business plans.  Goal 3: Maintain an optimal program design to maximize value delivery.  Goal 4: Maximize stakeholder understanding of DoD’s Manufacturing USA institutes.  Goal 5: Effectively support a capable workforce.6 The OSD ManTech office provides oversight of the DoD institutes and has performed annual reviews of all of the MIIs since the inception of the institutes. EVALUATION CRITERIA AND METRICS FOR THE DOD MIIS The federal government has long recognized the need for periodic program assessments to assure the performance, cost, and effectiveness of taxpayer-funded programs. Government agencies, such as the Office of Management and Budget (OMB) and the Government Accountability Office (GAO), and professional societies, such as the American Evaluation Association and the National Academies, have performed a number of studies on best practices and strategies for assessment of a wide variety of federal programs.7,8,9 DoD was an early adopter and strong proponent of formal program evaluation processes. In 1961, DoD established the Office of Systems Analysis to perform the program evaluation function. Since then, requirements for program evaluation have been included in DoD policies and guidebooks for a wide range of defense programs and organizations. One example is the Comprehensive Review Guidelines for DoD federally funded research and development centers (FFRDCs).10 2 DoD, 2021, Fiscal Year 2020 Industrial Capabilities Report to Congress, p. 157, https://media.defense.gov/2021/Jan/14/2002565311/-1/-1/0/FY20-INDUSTRIAL-CAPABILITIES-REPORT.PDF. 3 Ibid. 4 Budget Exhibit R-2 for PE 060368D8Z: Defense Wide Manufacturing Science and Technology, Feb 2020 5 DoD, 2018, “DoD Directive 5200.14: Manufacturing Technology (ManTech) Program,” Change 1, October. 6 Ibid. 7 Phase 4 Implementation of the Foundations for Evidence-Based Policymaking Act of 2018: Program Evaluation Standards and Practices, https://www.whitehouse.gov/wp-content/uploads/2020/03/m-20-12.pdf Accessed on 11/23/2020. 8 An Evaluation Roadmap for a More Effective Government, https://www.eval.org/evaluationroadmap, accessed on 11/23/2020. 9 NASEM, 2017, Principles and Practices for Federal Program Evaluation: Proceedings of a Workshop, Washington, DC: The National Academies Press, https://doi.org/10.17226/24831. 10 USD(AT&L) Policy Memo, Subject: FFRDC Management Plan and Associated “How To” Guides, 2 May 2011. PREPUBLICATION COPY – SUBJECT TO FURTHER EDITORIAL CORRECTION 2-2

The evaluation framework that has been developed by DoD for the MIIs is based on one that DoD has used successfully in review of FFRDCs and University Affiliated Research Centers (UARCs). The success of the evaluation framework is demonstrated by the fact that the DoD has used the framework to complete 5-year, strategic evaluations of 24 DoD FFRDC and UARC organizations over a 20+ year period. The evaluations performed, based on this framework, are used by senior DoD officials as the basis for decisions to renew the FFRDC or UARC, to reopen the competition in the core competency areas, or to end the activity. This framework is consistent with the models of evaluation processes that have been proposed and used by many federal agencies as well. For example, a National Academies study that examined major National Science Foundation (NSF) program recompetitions and suggested that the two key questions for reviewers to assess were the following: 1. Is this still a worthwhile activity or has it become obsolete in the face of scientific and technological advances?; and 2. If the activity is deemed worthwhile, is the current grantee doing a good job that merits continuation or should the award be opened to competition?11 The OUSD R&E evaluation process takes a similar approach but has additional considerations which are relevant for the more mission-focused DoD MIIs. The DoD will assess the MIIs in four key question areas, as seen in Figure 2.2, which is taken from the MII Strategy and Assessment document.12 FIGURE 2.2 Key questions to be addressed in the Department of Defense (DoD) 5-year assessment process for the Manufacturing Innovation Institutes (MIIs). SOURCE: MII Strategy and Assessment Process. 11 National Academy of Sciences, National Academy of Engineering, and Institute of Medicine, 1994, Major Award Decisionmaking at the National Science Foundation, Washington, DC: The National Academies Press, https://doi.org/10.17226/2268. See Chapter 5, “Recompetition of Awards,” p. 110. 12 Table from the MII Strategy and Assessment_ManTech_Oct2020_cleared PowerPoint, received on November 9, 2020, from ManTech. PREPUBLICATION COPY – SUBJECT TO FURTHER EDITORIAL CORRECTION 2-3

OUSD R&E plans to use the Joint Manufacturing Defense Council (JDMC) as an independent source to perform the 5-year strategic evaluations of the MIIs, as seen in Figure 2.3.13 The JDMC, chartered in 2020, is an OSD-led organization with 18 organizational members within DoD; it is tasked with coordination and strategic guidance of advanced manufacturing activities across DoD. The mission of the organization is: OUSD(R&E) and OUSD(A&S) charter the Joint Defense Manufacturing Council (JDMC) to collaborate and make recommendations to improve the alignment of resources, prioritize areas of challenge across the Department, facilitate information sharing on advances and needs in manufacturing, and ultimately shape the manufacturing capabilities of the defense industrial base. The success of these activities has profound implications for the readiness and cost-effectiveness of existing, emerging, and new systems. Important goals of the JDMC include the following: alignment of defense research, manufacturing, maintenance, and supply capabilities; increased collaboration and efficiencies in DoD manufacturing activities; development of strategies for a robust organic and commercial manufacturing base; and development of DoD’s human capital plans in coordination with other national programs. Since the MIIs address these goals, it is appropriate that the JDMC serve as the evaluator for this substantial DoD investment in advanced manufacturing. In addition, the JDMC is well positioned to not only assess the performance of the MIIs but also to facilitate increased visibility of the MIIs within DoD and enhance the impact of the institutes for DoD. The JDMC has been formally charged with organization and execution of the periodic, 5-year reviews of the MIIs. The JDMC will put together an evaluation team to perform the review of an MII. The team will be responsible for the assessment of the value and performance of the MII in support of Department mission goals. The team membership will include all of the expertise required to complete the assessment of performance in the critical elements of the MII (e.g., manufacturing technology, education and workforce development, and ecosystem). The team will primarily consist of personnel from the JDMC organizations but may be supplemented with additional experts as needed. The deliverable from the JDMC evaluation team will be a recommendation to OSD ManTech and OSD R&E on the continuation of core funding to the MII. Senior DoD leadership will then incorporate JDMC evaluation team’s recommendations into broader strategic considerations within the Department and make the decision on the future of the MII. OSD ManTech will manage the evaluation process for all of the MIIs. These reviews will be performed on a rotating basis and are scheduled to start in 2021. In addition to the development of a strategy and approach for periodic independent evaluation of the institutes, DoD has made significant progress in the development of metrics for assessing MII performance. DoD has produced the “2020 DoD Metrics List” and the “Final Metrics List for the DoD MIIs.”14 These documents, representing the output from the latest activity on metrics development for the DoD MIIs, began with activities sponsored by the Advanced Manufacturing National Program Office (AMNPO),15 which examined metrics for all of the manufacturing institutes sponsored by federal agencies. As the MIIs have matured, the metrics used to assess performance have grown from the initial, organizational development-focused metrics to the output focused-metrics and impact-focused metrics Based on these metrics lists, OSD ManTech currently collects a considerable amount of data on the MIIs on a weekly, quarterly, semi-annual, or annual basis. The information topics include: Ecosystem (e.g., patents filed and licensed, successful technology transitions, extent of member partnering, state 13 JDMC Charter, https://www.DoDmantech.com/JDMTP/Files/JDMTP_SubPanels_Charter_2011.pdf, accessed January 15, 2021. 14 Excel books: “Final DoD MII Performance Metrics_cleared” and “2020 MII Complete Metrics List_cleared,” both received from ManTech on December 11, 2020. 15 National Network of Manufacturing Institutes, 2015, Guidance on Institute Performance Metrics, https://www.manufacturingusa.com/reports/guidance-institute-performance-metrics-national-network- manufacturing-innovation. PREPUBLICATION COPY – SUBJECT TO FURTHER EDITORIAL CORRECTION 2-4

involvement in MII), EWD ( e.g., EWD projects underway and completed, number of manufacturing workforce participants in EWD activities, number of participants in Institute-aligned training, apprenticeship, or certification programs), Financial ( e.g., core and non-core funding levels (from DoD, federal agencies, state agencies, industry, Foundations, etc.)) and Operations ( e.g., projects completed, projects meeting objectives, number of small and large companies involved in projects, number of Government directed projects ), An example of the information gathered via the metrics lists is shown in Figure 2.4. The JDMC will benefit from all of the data gathered by OSD ManTech in the completion of their independent 5-year assessment of the MIIs. FIGURE 2.3 The Department of Defense (DoD) assessment process. SOURCE: MII Strategy and Assessment Process. PREPUBLICATION COPY – SUBJECT TO FURTHER EDITORIAL CORRECTION 2-5

  titu om Ins ed fr y te res Me i c  ry te enc tri c ega ego cif a su ed qu Spe gr Ne Cat Me F re Ag ta  Da Successful Tech Transitions Number of tech transitions in a commercial  #  Annual # product or defense program or system Patents Filed (Optional)  # Annual # Patents filed (#) Patents Used (Optional)  Number or percentage of patents that are (a) cited,  % Annual # (b) licensed and (c) implemented on the factory  floor E c o s y s t e m Members on Funded Projects # Annual # Unique members on funded projects (#) Extent of Member Partnering # Annual # Total members on funded projects (#) Ecosystem Participation Number of events and  # Annual # number of persons attending Institute events  (total) State Contributions $ Annual $ State contributions ($) Members Cash vs. In‐Kind Contributions Members in‐kind reported ($) $ Annual Ratio Member cash collected ($) States Involved in Institute Number and list of state governments that are  # Annual actively involved as members, investors, nodes, or  providing cost share DoD Subject Matter Experts Engaged Semi‐ DoD SMEs engaged (#) throughout the year # # annual FIGURE 2.4 An example of a section of the metrics for evaluating the Ecosystem currently gathered by OSD ManTech. SOURCE: Final DoD MII Performance Metrics received from ManTech on December 11, 2020. PREPUBLICATION COPY – SUBJECT TO FURTHER EDITORIAL CORRECTION 2-6

Next: 3 Discussion »
DoD Engagement with Its Manufacturing Innovation Institutes: Phase 2 Study Interim Report Get This Book
×
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

To better support the need for timely, effective manufacturing technology development and transition, the Department of Defense (DoD) has established nine Manufacturing Innovation Institutes (MIIs) through its Defense-wide Manufacturing Science and Technology program element within the DoD Manufacturing Technology program. The Institutes are considered by DoD to be important facilitators that bring together innovative ecosystems in key technology and market sectors in the United States. The DoD MIIs are industry-led, with dual, public-private benefit, providing large commercial market potential while also meeting key U.S. defense industrial needs. The mission of the nine DoD-established Institutes addresses both defense and commercial manufacturing needs within specific, defense-relevant technology areas.

The National Academies of Sciences, Engineering, and Medicine was asked to perform a follow-on study to its 2019 report Strategic Long-Term Participation by DoD in Its Manufacturing USA Institutes, which examined DoD's strategic relationship with the DoD Manufacturing Innovation Institutes. The Committee on DOD Engagement with Its Manufacturing USA MIIs Phase 2 Study was formed to provide strategic guidance in the following topics that were identified in the 2019 report for further exploration: protocols for conducting long term engagement assessments of the MIIs including evaluation metrics, best practices for MII education and workforce development programs, and development of strategies for better connecting MIIs to the broader DoD community. This interim report focuses on the protocol assessment topic in a timeframe that would allow its perspectives to be used in the execution of the first DoD 5-year MII evaluations planned for 2021.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  6. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  7. ×

    View our suggested citation for this chapter.

    « Back Next »
  8. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!