step is to develop data reporting and analysis plans, including how data will be collected, submitted, managed, analyzed, and reported. Thomas said that the CDC builds evaluation capacity into the grant awards as it must be able to report back to Congress to justify continued funding of the program. As a result of the Pandemic and All-Hazards Preparedness Act, there are legislative mandates that require benchmarks be met. Failure to meet any benchmarks will result in funding cuts, a challenge of primary concern to awardees, Thomas said.
In selecting points to measure, the CDC must consider what is core to public health versus what is under the control of EMS or healthcare delivery. There is also need for measures between and among systems, to gain a better understanding of where they fit and how they work together. Measures must be scalable. The system is designed to collect data on routine events that, Thomas explained, serve as proxies for how the public health system might function in public health emergency response. Finally, potential bottlenecks that affect timely delivery of services are identified.
Many of the measures are time based, assuming that time is a proxy for quality of response. One could measure, for example, time to notify preidentified staff to fill public health agency incident management roles, or time to complete a draft of an after-action report/improvement plan. Other parameters, such as quality of the response and whether the right decisions were made, are more difficult to measure. There are not much data to guide such measures, but the CDC is addressing this as the program moves forward, Thomas said.
The core science and technology mission of the Department of Homeland Security (DHS) is to strengthen America’s security and resiliency by providing innovative science and technology solutions for the Homeland Security enterprise, explained James Grove, regional director of the Interagency and First Responder Programs Division in the DHS Office of Science and Technology. One of the approaches to achieving this mission is a First Responder Capstone Integrated Product Team (IPT) established in 2009. After identifying needs-based input from first responders, the program makes investments in technologies and solutions that could potentially fill the gaps identified.
Grove highlighted several methods of evaluating concepts and metrics. Usually, in an emergency management community, he explained, the process of fixing problems starts with developing concepts. Then standard