endless iteration is possible, except that the components of such systems are likely all evolving concurrently. In these circumstances other techniques are needed. Rather than trying to refine and reduce the risk in any given area, research is needed into design precepts that support adaptivity77 and agility to allow ultrascale systems to evolve.
The research questions should be evident from the foregoing. What are the particular kinds of evidence that might be produced? How does this evidence relate to the modeling formalisms that are important for architecture, design, and assurance (as noted in the previous two area descriptions)? How can iterative processes be defined so they are feasibly executed for mainstream defense software development? How must these processes be adapted for the constant development cycles of ultrascale (or system-of-systems) programs? This may appear to be a management topic, but there is in fact substantial technical challenge in the identification and evaluation of the particular kinds of models that can be used to frame the evidence that is created.
The foregoing overview of three research topics provides the committee’s interim feedback to DoD on research that holds promise for improving the producibility of software-intensive systems. These and additional research areas that emerge from committee deliberations will be addressed in more depth in the final report for this study.
Although a clear message in this letter report is that additional, focused research is necessary and has the potential to alleviate many difficult problems, merely funding additional research will not completely solve the software producibility problem for DoD. A multifaceted, carefully managed, ongoing approach is necessary. This is because software technology, unlike other engineering disciplines rooted in physical artifacts, is essentially unbounded. Indeed, it is characteristic of both software technology and defense program requirements that it impossible to completely solve this problem or even fully routinize the core value creation in software development. We can, however, and will (with proper support) make genuine progress. Continuous progress is in fact a necessity given the growing need for software, the value contribution of software, and thus the demands placed on expertise and experience in software producibility. Through this growth cycle there will likely always be some critical dimensions of software engineering that appear to be primary sources of cost and risk. Understanding the nature of progress in software technology and producibility, as contrasted with progress in other engineering disciplines, is essential to create appropriate expectations for how these risks can be managed and, more generally, how the risk/value balance can be addressed as requirements and technology continue to evolve.
As part of an overall research program, the issues that the DoD faces must be addressed in the short to medium term as well. These intermediate-term results are a risk mitigator for the overall research investment, and the feedback from those results can provide midcourse guidance for the longer-term work. The issues that research funding alone cannot solve include institutional as well as technical problems, such as industry incentive structures, the quality and quantity of technical talent within DoD and its software industrial base, and development practices. Addressing these issues, as well as issues of research management and strategy, may have as much or more impact as specific technical breakthroughs in a research project. Particularly with respect to industry, a complementary approach that adds development-stage investments for industrial-base and system-of-systems toolsets, knowledge, and capabilities would be beneficial. These and related considerations will be covered in the committee’s final report.