• DoD did not use a consistent framework for developing and articulating program objectives, capacities, and metrics and did not prioritize among its metrics. As a consequence, the DoD Metrics Report mixes project management measures with higher-level program performance metrics for some of the CTR programs and weights equally metrics that are critically important and others are not.
• DoD plans to leverage other U.S. Government agencies’ experience, capabilities, and assets as CTR expands to new countries and as it continues existing programs. DoD also needs to communicate, coordinate, and cooperate with relevant agencies. At the same time, most of DoD’s metrics do not obviously draw on or tie into metrics developed by other agencies for similar or related purposes and related programs.
• The DoD Metrics Report deliberately does not consider future missions or changes in objectives, and for some programs it does not take into consideration or explicitly discuss planned and unplanned change over time. Projects have different phases that make different kinds of progress toward the program’s impact and effectiveness. Also, circumstances change, including changes in the threats, the political environment, the program’s priorities, and the funding available. A project’s performance itself feeds back into the management and decision-making process and can lead to change. It will be difficult to address change and sustainability without considering and incorporating these factors explicitly.
The practical consequences of some of these shortcomings, such as not articulating the connection to threat or risk, might not be large for projects in progress under longstanding agreements. But they are important for good management, new projects, and especially for new partnerships.
The committee has several recommendations to DoD on how it can improve its metrics for the CTR programs and reports on CTR. The committee does not recommend specific alternative metrics (DoD has to develop its own metrics with its partners), but does recommend what it considers a more effective approach for DoD to take to developing metrics. In the report, the committee provides an example of its recommended approach applied to the CBEP.
Objectives and Partnership:
1. For each program in the DoD Metrics Report, DoD should include a concise statement of its objectives and of how the program is intended to reduce threat or risk.
2. Objectives for projects and the overall CTR Program in a partner country are developed jointly between the United States and the partner country. An agreed set of metrics should also be built into projects from the outset. They may change, but the parties responsible for the projects should know at any given time the metrics that will be used to measure impact and effectiveness.