4
Better Experimentation in the Air Force: Barriers and Levers
MAKING EXPERIMENTATION PART OF THE CULTURE
Lt. Gen. Steven Kwast Commander and President, Air University, Maxwell Air Force Base
On January 28, 2016, General Kwast presented his remarks to the workshop on “Making Experimentation a Part of the Culture in Air Force Developments.”
LIFE CYCLE MANAGEMENT CENTER PERSPECTIVES ON BARRIERS
Lt. Gen. C.D. Moore (Ret.) Executive Vice President, Dayton Aerospace, Former Commander Air Force Life Cycle Management Center
With the standup of the Air Force Life Cycle Management Center (AFLCMC) in 2012, nearly all non-space acquisition and product support responsibilities within the Air Force were consolidated into a single organization. In hindsight, this initiative had both positive and negative implications on the culture of experimentation within the Air Force’s acquisition and product support community.
Under this reorganization, AFLCMC took on the consolidated responsibility of more than 3,000 programs managed at 9 large continental United States locations and more than 70 worldwide sites. In order to build a consistent and more effective way of doing business across these many locations, AFLCMC embraced three key principles: speed with discipline, trust and confidence, and unity of purpose. In addition, AFLCMC established several key objectives to guide the stand-up and unifying actions of the organization. These objectives helped build common ways of doing business across the organization and facilitated breaking down us-versus-them attitudes, what was eventually embraced as a “geo-agnostic” operating model. The two AFLCMC objectives with most direct relevance to organizational experimentation were
- Launch high confidence sustainable programs, and
- Standardize and continuously improve center processes.
The focus on process discipline produced both the desired effects as well as some unintended consequences, particularly on previous experimentation tendencies within certain programs of record (PORs) across the center. With program offices working full time and resource stressed to deliver specific, approved requirements while operating under greater pressure to employ process discipline, experimentation has become less likely to “fit” within a cost/schedule/performance POR framework. On the other hand, the focus on launching high confidence programs helped build a re-energized development planning (DP) process intended to facilitate a renewed culture of experimentation. This initiative had some early success and has proven to be a step in the right direction with respect to experimentation.
With these two objectives in mind, there were several challenges that needed to be overcome. For one, a push for process standardization and greater process discipline, accompanied by expanded layers of business oversight, has in hindsight tended to stymie process tailoring in support of faster-to-field experimentation within PORs. In addition, the push for “owning the technical baseline” has energized a more rigorous system engineering approach and greater U.S. government involvement in the design review process. Applying this enhanced rigor too broadly and deeply seems to have caused a deleterious effect on experimentation within PORs where innovation and “speed to field” were historically part of the culture.
With the focus on launching high confidence programs, the center worked to re-grow a DP process with targeted experimentation using enhanced analytical live virtual constructive tools and greater collaboration across the center’s wide (i.e., aircraft, munitions, C4I [Command, Control, Communications, Computing, and Intelligence]) program responsibilities. Although a step in the right direction, limited resources dedicated to DP have restricted the number of concepts that could be evaluated in a robust experimentation environment and, subsequently, constrained the ability to evaluate new, innovative warfighting approaches. The DP chokepoint is dedicated resources—manpower and money.
With this backdrop, experimentation barriers could be addressed with some recommended actions. For starters, there should be a push for and support of more process tailoring to facilitate experimentation within PORs. The responsibility and authority to approve this process tailoring should be as low as possible to allow prudent risk taking at the appropriate level (i.e., program executive office, program manager). Furthermore, “speed” should be equally as important and balanced with “discipline” in program execution, particularly where experimentation has the potential of introducing significant new capabilities to the field. In addition to process tailoring within PORs, experimentation would be enhanced with an expansion of dedicated DP capability through greater resource application (manpower and funds), preferably targeted at functional/technical teams (i.e., SIMAF [Simulation and Analysis Facility], capability planning teams, product support enterprise initiatives) dedicated to experimentation in partnership with industry, laboratories, and program offices. These dedicated execution teams are optimally situated and motivated to bridge between science and technology and PORs, and best positioned to take advantage of rapidly evolving analytical tools to demonstrate the value of innovative concepts via experimentation. Finally, with greater resources dedicated to the DP process, there would be more opportunities to pull in ideas from industry, both large and small, as well as from U.S. government laboratories. In taking these steps, the Air Force would be better positioned to enhance the role of experimentation in the innovation cycle.
PERSPECTIVE ON ACQUISITION BARRIERS TO INNOVATION AND EXPERIMENTATION
Camron Gorguinpour Director of Transformational Innovation to the Assistant Secretary of the Air Force for Acquisitions (SAF/AQ)
The job of the Transformational Innovation office for the Air Force (transform.af.mil) is to identify, experiment with, and ultimately implement new ideas for how the Air Force (AF) can do business with contractors to buy products and services. Acquisitions include not only cutting-edge technologies but also legacy systems. The current process is widely seen as broken. We are trying to break down barriers to acquisition efficiency that have been placed with the good intention of protecting the government’s interest but, unfortunately, thwart better end results. Our office works on a variety of projects, both experimentation- and non-experimentation-related.
Cultural Barriers to Innovation: Fear of Failure
Innovators in Silicon Valley and elsewhere often say that innovating requires a fearless acceptance of failure. However, there is an intense fear of failure within the Department of Defense (DoD). No one wants to see AF leadership dragged before Congress after a major failure. So, it is not unreasonable that fear of failure can be debilitating. There is a rich “oral history” of failed acquisition reform within DoD. You are warned of failure: “I tried that 15 years ago.” You ask, “What happened? What did you try? Why didn’t it work?,” but there usually is not a satisfactory reply. What we need is more rigor to explain failings. Every experimentation effort, including ours in acquisition reform, has failures. Some of our failures include not getting to meaningful acquisition milestones fast enough, but our record of failures is limited because we are still a new organization.
Barriers and Work-Arounds to Sourcing External Innovation: Special Contracting Vehicles
DoD has launched an initiative to source innovation from Silicon Valley and has created Defense Innovation Unit-Experimental (DIUx). It is not that the AF cannot identify or access innovation wherever it is—it is not a “geographic problem”—but the AF lacks efficient business processes to acquire these innovations. There are other contracting vehicles, such as In-Q-Tel (for innovations in the Central Intelligence Agency) and On Point (for the U.S. Army). These are essentially government-backed venture capital firms that do good work. However, these have not created AF projects. Our office tried to create a comparable entity for the AF, but eventually decided to leverage the existing entities. In fact, the AF has all the constructs it needs to change its acquisition system to efficiently source innovation. Separate from how the AF might acquire innovative technology through In-Q-Tel, and others, is the question of whether the innovations are likely to benefit the AF and offer a strategic return on investment beyond potential financial gain.
The challenge is to engage the “non-traditional defense contractors” who do not normally do business with DoD because it is too difficult—long timelines to get to funding, lengthy application process to compete for business, restrictive intellectual property rules, security rules, auditing, and international trade restrictions. One solution we are experimenting with is the Other Transaction Authority (OTA). In fact, In-Q-Tel and On Point were created using other transactions (OTs). These can be used for any type of research, development, test, and engineering activity, up to low-rate initial production. DoD has the legal right to enter into what is explicitly not a contract or agreement but just a transaction with a company, starting with a clean slate and drawing up a circumstance-specific transaction. Standard Federal Acquisition Regulations (FARs) do not apply. There are some restrictions with respect to type of company that can participate, but the rules are flexible by design. We just need to use them. One can execute a project from technology readiness level (TRL)-1 through low-rate initial production all under one OT, and the OT also becomes a sole-source justification triggered by meeting technical and cost milestones. While OTs have been discouraged since 2000, there is currently little disincentive to use of OT. Most people do not, however, understand the basic rules to structure OTs. In fact, the recent award of RD-180 replacement leveraged OTs because the purchase was for a specific purpose—to achieve an industry cost-share.
There are many obstacles to acquiring innovation: contracting, financial management, legal—it takes but one person to kill a project. A pro-active contracting officer, especially a head of contracting, is like gold. Congress is expanding authority for OTs in the 2016 National Defense Authorization Act (NDAA). The authority has been made permanent, and the limit for transactions has been raised from $100 million to $250 million without Office of the Secretary of Defense (OSD) approval; higher amounts require OSD approval. The transition from OT to production contract has also been made more flexible. In our OT, we trimmed the intellectual property clauses to resemble more compact commercial intellectual property rights (IPR) contracts. Government auditing rules have vanished because they do not apply to the non-traditional OTs. We will be championing more use of OTs in the coming year and expanding the contract vehicle that we created to use it. Another even more esoteric authority, which is seldom used, is experimentation authority 10 U.S.C. 2373 for the purchase of equipment for experimental purposes in certain broad subject areas, including aerospace, space, transportation, energy, signals, etc. Our general counsel will advise on its applicability and any restrictions. Another authority, NDAA Section 804,
also waives all sorts of Joint Capabilities Integration and Development System and other requirements at the front end, for rapid delivery, and for prototyping and experimentation. There is further opportunity to make additional changes in NDAA 2017, as needed.
Lack of use of these authorities may stem from general management’s aversion to risk of potentially being reprimanded for “bending the rules,” if the project fails. However, current AF leadership is strongly supportive of these types of contracting improvements. Early career professionals are also enthusiastic, but middle management is risk averse. With this open vehicle, we can make a legitimate pitch and honestly demonstrate that the revised process expands competition, lowers cost, and gets acquisition done in 3 weeks instead of 2 years. The vast majority of people in government are well-intentioned folks, trained to accept as a given that the process, as it is, is working as intended, which is to limit risk and get things done and to make incremental progress. Other things sound nice and exciting until you try to do them, and then they are prone to failure; that is where people stop paying attention. So we have to create a space where people feel comfortable trying new things, and we have had some success in that.
Strategies to Communicate/Expand AF “Open Systems” Approach
“Open architecture” is the right approach to innovation development. This may not apply to legacy or unique systems, but we believe that more than 50 percent of AF programs should move toward some form of open architecture, to avoid lock-in to a single vendor and potentially high cost, limited technical innovation, and delays in innovation capability reaching the warfighter. However, if the standard contracting process takes 2 years and the information assurance (IA) process another 2 years, the value of open architecture is wiped out. We, therefore, combined an Army acquisition model called the “Other Transaction Consortium,” which has been in use for over a decade. The Army Contracting Command B issues a transaction to a consortium instead of an individual company, with the consortium evolving in company constituents over time. A company can join by completing a one-page online application, paying a $500 joining fee, and then viewing and responding to government requirements, if it chooses. The Army’s process requires a white paper as well, followed by a down-selection, proposal, and award. The entire process from white paper to award takes on average less than 2 months.
We then combined another acquisition model called the Plug Fest, an industry event hosted by OSD where vendors plug their equipment into an open architecture with published requirements to demonstrate performance and interoperability. Vendors are scored against a baseline, and a winner is selected. However, in the current Plug Fest, a winner may not receive a contract award. Our change allows for a contract award to the winner within 1 month; all that is needed is a constant-schedule proposal, and not even a technical write-up. Such a non-FAR-based OT award cannot be protested. IA may take another 6 months, but even that would be better than the status quo. Furthermore, the open architecture system itself may have a certified security layer that alleviates the IA burden. We are working with AFLCMC to implement an expedited authorization and accreditation process that runs in parallel to the acquisition process. Metrics for performance evaluation are now milestones in the Developmental Test and Engineering (DT&E) process, so that the acquired product is not a finished product, and some work can be performed post-award.
Our demonstration involved 19 companies, of which 14 were non-traditionals. They naturally formed teams as we sat back and watched with interest. They were developing on their own dime. About 12 teams ultimately formed, 6 made it to the final acquisition event, and 3 demonstrated end-to-end products. We awarded to two, and the reason we could award to two was that costs were so much less than our estimates; it cost only a bit more to fund both to see how they would work. Time from award to product was perhaps 4 months, of which 1 month was for paperwork at one-third of current cost, primarily for “customer hand-off” documentation. We did not meet our 3-week target, in part because we relied on an unfamiliar Army contract vehicle. In the end, we got better capability at lower cost from a greater variety of vendors, including non-traditional contractors, all at greater speed. The Air Force Research Laboratory has now created its own AF OT consortium vehicle. There are nuances to be worked through, but we at least have a contract vehicle with which to work. There is an effort to acquire major weapons systems using this OT vehicle. FAR creep may limit its use, but a call for reform favors our efforts.
The Army has multiple OT consortia, organized by subject area: Ordnance, C4ISR [Command, Control, Communications, Computers, Intelligence, Surveillance, and Reconnaissance], Cyber, and so on. The AF plans to have just one consortium for all manner of “open architecture” systems. If ultimately successful, each program will create its own OT consortium to better address requirements and competition. There are other similar AF vehicles: Combat Air Operations Center-Experimental, Information Sharing Analysis Center at Warner-Robbins Air Logistics Center, and the Cyber Innovation Center at Hanscom Air Force Base, etc. That said, some people prefer more traditional yet efficient acquisition approaches (e.g., multi-award ID/IQs [Indefinite Delivery/Indefinite Quantity] which are also effective). However, they are more applicable to a limited set of vendors (5-50) who will provide something specific. OT consortia vehicles support an infinite number of companies.
