National Academies Press: OpenBook
Suggested Citation:"Front Matter." National Academies of Sciences, Engineering, and Medicine. 2016. The Role of Experimentation Campaigns in the Air Force Innovation Life Cycle: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/23602.
×

The Role of Experimentation
Campaigns in the Air Force
Innovation Life Cycle

Proceedings of a Workshop

Committee on the Role of Experimentation Campaigns
in the Air Force Innovation Life Cycle

Air Force Studies Board

Division on Engineering and Physical Sciences

images

THE NATIONAL ACADEMIES PRESS
Washington, DC
www.nap.edu

Suggested Citation:"Front Matter." National Academies of Sciences, Engineering, and Medicine. 2016. The Role of Experimentation Campaigns in the Air Force Innovation Life Cycle: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/23602.
×

THE NATIONAL ACADEMIES PRESS 500 Fifth Street, NW Washington, DC 20001

This is a report of work supported by Grant FA9550-14-1-0127 with the U.S. Air Force. Any opinions, findings, conclusions, or recommendations expressed in this publication do not necessarily reflect the view of any organization or agency that provided support for the project.

Digital Object Identifier: 10.17226/23602

Limited copies of this report are available from:

Air Force Studies Board
National Academies of Sciences, Engineering, and Medicine
500 Fifth Street, NW
Washington, DC 20001

Copyright 2016 by the National Academy of Sciences. All rights reserved.

Printed in the United States of America

Suggested citation: National Academies of Sciences, Engineering, and Medicine. 2016. The Role of Experimentation Campaigns in the Air Force Innovation Life Cycle: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/23602.

Suggested Citation:"Front Matter." National Academies of Sciences, Engineering, and Medicine. 2016. The Role of Experimentation Campaigns in the Air Force Innovation Life Cycle: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/23602.
×

Image

The National Academy of Sciences was established in 1863 by an Act of Congress, signed by President Lincoln, as a private, nongovernmental institution to advise the nation on issues related to science and technology. Members are elected by their peers for outstanding contributions to research. Dr. Marcia McNutt is president.

The National Academy of Engineering was established in 1964 under the charter of the National Academy of Sciences to bring the practices of engineering to advising the nation. Members are elected by their peers for extraordinary contributions to engineering. Dr. C. D. Mote, Jr., is president.

The National Academy of Medicine (formerly the Institute of Medicine) was established in 1970 under the charter of the National Academy of Sciences to advise the nation on medical and health issues. Members are elected by their peers for distinguished contributions to medicine and health. Dr. Victor J. Dzau is president.

The three Academies work together as the National Academies of Sciences, Engineering, and Medicine to provide independent, objective analysis and advice to the nation and conduct other activities to solve complex problems and inform public policy decisions. The Academies also encourage education and research, recognize outstanding contributions to knowledge, and increase public understanding in matters of science, engineering, and medicine.

Learn more about the National Academies of Sciences, Engineering, and Medicine at www.national-academies.org.

Suggested Citation:"Front Matter." National Academies of Sciences, Engineering, and Medicine. 2016. The Role of Experimentation Campaigns in the Air Force Innovation Life Cycle: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/23602.
×

Image

Reports document the evidence-based consensus of an authoring committee of experts. Reports typically include findings, conclusions, and recommendations based on information gathered by the committee and committee deliberations. Reports are peer reviewed and are approved by the National Academies of Sciences, Engineering, and Medicine.

Proceedings chronicle the presentations and discussions at a workshop, symposium, or other convening event. The statements and opinions contained in proceedings are those of the participants and have not been endorsed by other participants, the planning committee, or the National Academies of Sciences, Engineering, and Medicine.

For information about other products and activities of the Academies, please visit nationalacademies.org/whatwedo.

Suggested Citation:"Front Matter." National Academies of Sciences, Engineering, and Medicine. 2016. The Role of Experimentation Campaigns in the Air Force Innovation Life Cycle: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/23602.
×

COMMITTEE ON THE ROLE OF EXPERIMENTATION CAMPAIGNS IN THE AIR FORCE INNOVATION LIFE CYCLE

LESTER L. LYLES (Gen., USAF, Ret.), NAE,1 Independent Consultant, Co-Chair

ALEX MILLER, University of Tennessee, Co-Chair

TED F. BOWLDS (Lt. Gen., USAF, Ret.), The Spectrum Group

CHARLES R. “CR” DAVIS (Lt. Gen., USAF, Ret.), Seabury Aerospace & Defense

BLAISE J. DURANTE, U.S. Air Force (Ret.)

ANTONIO L. ELIAS, NAE, Orbital Sciences Corporation

IVY ESTABROOKE, Utah Science Technology and Research Agency

DAVID E. HAMILTON, JR., Eagle Aerie Inc.

BERNADETTE JOHNSON, Massachusetts Institute of Technology Lincoln Laboratory

WILLIAM JOHNSON, WMJ Associates LLC

JOSEPH LAWRENCE, National Defense University

ROBERT ANDREW KIRK MITCHELL, NAE, Independent Consultant

BENJAMIN P. RILEY, Georgia Tech Research Institute

JOEL SERCEL, ICS Associates Inc.

DANIEL WARD, Dan Ward Consulting, LLC

Staff

JOAN FULLER, Director, Air Force Studies Board

GEORGE J. COYLE, Senior Program Officer

MARGUERITE E. SCHNEIDER, Administrative Coordinator

STEVEN G. DARBES, Research Assistant

DIONNA C. ALI, Research Assistant

___________________

1 NAE, National Academy of Engineering.

Suggested Citation:"Front Matter." National Academies of Sciences, Engineering, and Medicine. 2016. The Role of Experimentation Campaigns in the Air Force Innovation Life Cycle: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/23602.
×

AIR FORCE STUDIES BOARD

DOUGLAS M. FRASER, Doug Fraser, LLC, Chair

DONALD C. FRASER, NAE,1 Charles Stark Draper Laboratory (retired), Vice Chair

BRIAN A. ARNOLD, Peachtree City, Georgia

ALLISON ASTORINO-COURTOIS, National Security Innovations, Inc.

TED F. BOWLDS, The Spectrum Group

STEVEN R.J. BRUECK, University of New Mexico

FRANK J. CAPPUCCIO, Cappuccio and Associates, LLC

BLAISE J. DURANTE, U.S. Air Force (Ret.)

BRENDAN B. GODFREY, University of Maryland, College Park

MICHAEL A. HAMEL, Lockheed Martin Space Systems Company

DANIEL E. HASTINGS, Massachusetts Institute of Technology

RAYMOND E. JOHNS, JR., Flight Safety International

ROBERT H. LATIFF, R. Latiff Associates

NANCY G. LEVESON, NAE, Massachusetts Institute of Technology

MARK J. LEWIS, Institute for Defense Analyses Science and Technology Policy Institute

ALEX MILLER, University of Tennessee

OZDEN OCHOA, Texas A&M University

RICHARD V. REYNOLDS, The VanFleet Group, LLC

STARNES E. WALKER, University of Delaware

DEBORAH WESTPHAL, Toffler Associates

DAVID A. WHELAN, NAE, Boeing Defense, Space, and Security

REBECCA WINSTON, Winston Strategic Management Consulting

MICHAEL I. YARYMOVYCH, NAE, Sarasota Space Associates

Staff

JOAN FULLER, Director

ALAN H. SHAW, Deputy Director

GEORGE J. COYLE, Senior Program Officer

CARTER W. FORD, Program Officer

ANDREW J. KREEGER, Program Officer

DIONNA C. ALI, Research Assistant

STEVEN G. DARBES, Research Assistant

CHRIS JONES, Financial Manager

MARGUERITE E. SCHNEIDER, Administrative Coordinator

___________________

1 NAE, National Academy of Engineering.

Suggested Citation:"Front Matter." National Academies of Sciences, Engineering, and Medicine. 2016. The Role of Experimentation Campaigns in the Air Force Innovation Life Cycle: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/23602.
×

Preface

The U.S. Air Force Office of the Assistant Secretary for Science, Technology and Engineering asked the Air Force Studies Board of the National Academies of Sciences, Engineering, and Medicine to define and assess the current use of experimentation campaigns within the Air Force, evaluate barriers to their use, and make recommendations to increase their use.

To accomplish this work, the Committee on the Role of Experimentation Campaigns in the Air Force Innovation Life Cycle was established to (1) organize a workshop and (2) complete a consensus study with recommendations on how to best institutionalize experimentation within the Air Force. (Appendix A contains the committee’s statement of task, and Appendix B contains biographical sketches of the committee members.) The goal of the workshop was to bring together practitioners of experimentation methodologies from a broad cross-section of the community with the intention of using these inputs to guide the subsequent consensus study activity.

The Workshop on the Role of Experimentation Campaigns in the Air Force Innovation Cycle was held in Washington, D.C., on January 27-29, 2016. Participants at the workshop presented a broad range of issues, experiences, and insights related to experimentation, experimentation campaigns, and innovation. These proceedings provide a summary of the workshop presentations; however, no attempt was made to develop consensus findings and recommendations. The agenda for the workshop appears in Appendix C, and Appendix D lists the workshop participants. A list of acronyms and abbreviations is provided in the front matter.

The Air Force Studies Board and the committee recognize the contributions of the workshop participants and appreciate this opportunity to provide input on a subject of importance to the Air Force.

A NOTE ABOUT THE ABSTRACTS

The abstracts in these proceedings are those provided by or approved by the speakers; presentations shown on the agenda but without a corresponding transcript are those for which the speaker did not provide an abstract or declined the offer to include an abstract. The abstracts have been lightly edited for clarity.

Page viii Cite
Suggested Citation:"Front Matter." National Academies of Sciences, Engineering, and Medicine. 2016. The Role of Experimentation Campaigns in the Air Force Innovation Life Cycle: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/23602.
×

This page intentionally left blank.

Suggested Citation:"Front Matter." National Academies of Sciences, Engineering, and Medicine. 2016. The Role of Experimentation Campaigns in the Air Force Innovation Life Cycle: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/23602.
×
Suggested Citation:"Front Matter." National Academies of Sciences, Engineering, and Medicine. 2016. The Role of Experimentation Campaigns in the Air Force Innovation Life Cycle: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/23602.
×

Acronyms

ACTD Advanced Concept Technology Demonstration
AF Air Force
AFLCMC Air Force Life Cycle Management Center
AFMC Air Force Materiel Command
AMRAAM Advanced Medium-range Air to Air Missile
ATP authorization to proceed

C4ISR

Command, Control, Communications, Computers, Intelligence, Surveillance, and Reconnaissance

COG Cyber Operations Group
COS Cyber Operations (or Ops) Squadron
COTS commercial off-the-shelf (most common); Commercial Orbital Transportation Services (specialized space usage)
CW Cyber Wing

DARPA

Defense Advanced Research Projects Agency

DIUx Defense Innovation Unit-Experimental
DoD Department of Defense
DP Development Planning
DT&E Developmental Test and Engineering

FAR

Federal Acquisition Regulation

HMS

His (Her) Majesty’s Ship

IA

information assurance

ID/IQ Indefinite Delivery/Indefinite Quantity

JPL

Jet Propulsion Laboratory

Suggested Citation:"Front Matter." National Academies of Sciences, Engineering, and Medicine. 2016. The Role of Experimentation Campaigns in the Air Force Innovation Life Cycle: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/23602.
×
MVP minimum viable product

NASA

National Aeronautics and Space Administration

NATO North Atlantic Treaty Organization
NDAA National Defense Authorization Act

OSD

Office of the Secretary of Defense

OT other transactions
OTA Other Transactions Authority

PEM

program element monitor

PEO Program Executive Office
PM program manager
POR program of record
PPP public-private partnership

RCO

Air Force Rapid Capabilities Office

RPO Rendezvous Proximity Operations

SaaS

software as a service

SAF/AQ USAF Office of the Assistant Secretary (Acquisitions)
SAF/AQR USAF Office of the Assistant Secretary for Science, Technology and Engineering
SBIR Small Business Innovation Research
SIMAF Simulation and Analysis Facility

TRL

technology readiness level (e.g., TRL 3)

UAV

unmanned air vehicle

UFP unit flyaway price
USAF U.S. Air Force
USC U.S. Code

VC

venture capital

Suggested Citation:"Front Matter." National Academies of Sciences, Engineering, and Medicine. 2016. The Role of Experimentation Campaigns in the Air Force Innovation Life Cycle: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/23602.
×
Page R1
Suggested Citation:"Front Matter." National Academies of Sciences, Engineering, and Medicine. 2016. The Role of Experimentation Campaigns in the Air Force Innovation Life Cycle: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/23602.
×
Page R2
Suggested Citation:"Front Matter." National Academies of Sciences, Engineering, and Medicine. 2016. The Role of Experimentation Campaigns in the Air Force Innovation Life Cycle: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/23602.
×
Page R3
Suggested Citation:"Front Matter." National Academies of Sciences, Engineering, and Medicine. 2016. The Role of Experimentation Campaigns in the Air Force Innovation Life Cycle: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/23602.
×
Page R4
Suggested Citation:"Front Matter." National Academies of Sciences, Engineering, and Medicine. 2016. The Role of Experimentation Campaigns in the Air Force Innovation Life Cycle: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/23602.
×
Page R5
Suggested Citation:"Front Matter." National Academies of Sciences, Engineering, and Medicine. 2016. The Role of Experimentation Campaigns in the Air Force Innovation Life Cycle: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/23602.
×
Page R6
Suggested Citation:"Front Matter." National Academies of Sciences, Engineering, and Medicine. 2016. The Role of Experimentation Campaigns in the Air Force Innovation Life Cycle: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/23602.
×
Page R7
Page viii Cite
Suggested Citation:"Front Matter." National Academies of Sciences, Engineering, and Medicine. 2016. The Role of Experimentation Campaigns in the Air Force Innovation Life Cycle: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/23602.
×
Page R8
Suggested Citation:"Front Matter." National Academies of Sciences, Engineering, and Medicine. 2016. The Role of Experimentation Campaigns in the Air Force Innovation Life Cycle: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/23602.
×
Page R9
Suggested Citation:"Front Matter." National Academies of Sciences, Engineering, and Medicine. 2016. The Role of Experimentation Campaigns in the Air Force Innovation Life Cycle: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/23602.
×
Page R10
Suggested Citation:"Front Matter." National Academies of Sciences, Engineering, and Medicine. 2016. The Role of Experimentation Campaigns in the Air Force Innovation Life Cycle: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/23602.
×
Page R11
Suggested Citation:"Front Matter." National Academies of Sciences, Engineering, and Medicine. 2016. The Role of Experimentation Campaigns in the Air Force Innovation Life Cycle: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/23602.
×
Page R12
Next: 1 Introduction »
The Role of Experimentation Campaigns in the Air Force Innovation Life Cycle: Proceedings of a Workshop Get This Book
×
Buy Ebook | $9.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

The Workshop on the Role of Experimentation Campaigns in the Innovation Cycle was held in January 2016 to define and assess the current use of experimentation campaigns within the Air Force, evaluate barriers to their use, and make recommendations to increase their use. Participants at the workshop presented a broad range of issues, experiences, and insights related to experimentation, experimentation campaigns, and innovation. This publication summarizes the presentations and discussions from the workshop.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!