For Evaluating Effectiveness

  1. Is there a clearly stated purpose for the information-based program?

    • Is that objective worthwhile?

    • Is it legally appropriate?

    • Is there a demand or need for it?

    • Is it already being accomplished or could it be accomplished through less intrusive or less costly means?

  1. Is there a sound rational basis for the information-based program and each of its components?

    • Is there a scientific foundation for the system?

  1. Is there a sound experimental basis for the information-based program and each of its components?

    • Does the system work to achieve its stated purpose?

    • Has the new system been shown to work in simulations or laboratory settings or has it been field-tested?

    • Did the test conditions take into account real-world conditions?

    • Has it been applied to historical data to determine if it accurately accomplished its objective?

    • Have experimental successes been replicated to demonstrate that they were not coincidence?

    • Has the system been subjected to critical analysis, challenge, and likely countermeasures (for example, through “red-teaming”)?

  1. Is the information-based program scalable?

    • Has it been tested on a data set of adequate size to predict its scalability?

    • Has it been tested against likely countermeasures or changes in technologies, threats, and society?

  1. Is there a clearly stated set of operational or business processes that comprehensively specify how the information-based program should operate in the organization?

  2. Is the information-based program capable of being integrated in practice with related systems and tools?

    • Does the system interact effectively with the sources of information on which it relies?

The National Academies of Sciences, Engineering, and Medicine
500 Fifth St. N.W. | Washington, D.C. 20001

Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement