A key conclusion from these scenarios is the high importance of three factors: (1) The extremely high value of incorporating assurance considerations (including security considerations—see Box 4.1) into the full systems lifecycle starting with conceptualization, throughout development and acceptance evaluation, and into operations and evolution. (2) The strong influence of technology choices on the potential to succeed with assurance practices. (3) As a consequence, the value to DoD software producibility that comes from enhancements to critical technologies related to assurance, including both what is delivered (programming languages, infrastructure) and what is used during development (models and analytics, measurement and process support, tools and environments).
Recommendation 4-1: Effective incentives for preventive software assurance practices and production of evidence across the lifecycle should be instituted for prime contractors and throughout the supply chain.
This includes consideration of incentives regarding assurance for commercial vendor components, services, and infrastructure included in a system.
As illustrated in the scenario, when incentives are in place, there are emerging practices that can make significant differences in the outcomes, cost, and risk of assurance. The experience at Microsoft with the Lipner-Howard Security Development Lifecycle (SDL)30 reinforces this—the lifecycle not only leads to better software but also incentivizes continuous improvement in assurance technologies and practices.
When ecosystems, vendor components, open-source components, and other commerical off-the-shelf (COTS) elements are employed, assurance practices usually necessitate the DoD to constantly revisit selection criteria and particular choices. The relative weighting among the various sourcing options, from an assurance standpoint, will differ from project to project, based on factors including transparency of the development process and of the product itself, either to the government or to third-parties. This affords opportunity to create incentives for commercial vendor components to include packaged assurance-related evidence somewhere between the two poles of “as is” and “fully Common Criteria certified.” Advancement in research and practice could build on ideas already nascent in the research community regarding ways that the evidence could be packaged to support quality claims and to protect trade secrets or other proprietary technology embodied in the components.
Recommendation 4-2: The DoD should expand its research focus on and its investment in both fundamental and incremental advances in assurance-related software engineering technologies and practices.
This investment, if well managed, could have broad impact throughout the DoD supply chain. When both recommendations are implemented, a demand-pull is created for improved assurance practices and technologies.
Recommendation 4-3: The DoD should examine commercial best practices for more rapidly transitioning assurance-related best practices into development projects, including contracted custom development, supply-chain practice, and in-house development practice.