role and should focus on competently extending SLAD’s technical, engineering, and scientific analyses within appropriate operational and organizational bounds.

Changes and Accomplishments

Changes and accomplishments noted during this review cycle include the following:

• SLAD improved the definition of scope for S4. The focus of S4 modeling applications has been reduced from brigade level to company level and below.

• SLAD recently instituted a formal software development process. This process includes:

— Betterarticulation and definition of the SLAD vision for system-of-systems analysis,

— Formulation of a verification and validation strategy and required documentation of results,

— Participation of the Los Alamos National Laboratory in the evaluation of S4,

— Completion of S4 proof-of-principle efforts for the Army Evaluation Center (AEC) and the Program Executive Office for Integration, and

— Plans by AEC to accredit the S4 model.

• SLAD expanded collaboration efforts. Collaborations include Stanford Research Institute, Sandia National Laboratory, Army Research Office, Air Force Research Laboratory, Air Force Office of Scientific Research, Naval Postgraduate School, and Army organizations both within and external to ARL (i.e., Army Test and Evaluation Command, Army Evaluation Center, CISD).

• SLAD reported that the Army Test and Evaluation Command has concluded that SLAD’s system-of-systems analyses provide insight beyond current modeling and simulation capabilities and address measures not answered in tests or other modeling and simulation applications.

• SLAD has trained software development staff in capability maturity model integration (CMMI) tenets and practices of process improvement. SLAD has also contracted qualified experts in CMMI to overlay its framework on SLAD’s systems and to help to formulate a plan for improving the rigor of its processes as necessary. However, the SLAD scope is limited to self-assessment, without going through formal certifications.

Opportunities and Challenges

SLAD should consider a tactical pause to review and more carefully define S4’s role and mission. This review should include consideration of collaborative efforts with DoD agencies responsible for tactical and operational issues as well as SLAD’s role in providing high-fidelity analysis and tools focused on the physics of new and emerging survivability designs. Although SLAD has made some improvements in S4, the software and SLAD’s application of it have exhibited flaws, and progress has not yet been significant enough to warrant endorsement of the changes as meaningful. As SLAD plans and implements a systematic software development effort, more substantive improvements are expected. In addition, SLAD should carefully reexamine its technical plan for collaborating with and supporting the efforts of other modeling activities within the DoD (such as TRAC), focus its system-of-systems analysis (SoSA) modeling on cases that demonstrate the usefulness of S4 to others, and align its resources with its plan.

SLAD should prepare a detailed flowchart to determine the structure and capabilities of the current instantiation of the model to enable a more structured, well-thought-out, long-term plan for where this program should go to support the SLAD mission and strategic plan. SLAD should modify its model and software development process to become compliant with software development standards employed elsewhere in the acquisition and analysis community, such as the Software Engineering Institute’s

The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement