In assessing the adequacy of data for specific performance measures, the panel concluded that there are few available data sources that are ideal for performance monitoring. Understanding the limits of available data is important if appropriate inferences are to be drawn. Many federal efforts to collect health-related data, for example, provide national rates, but do not collect data that provide state-level rates. Even when data are available at the state level, if comparisons are to be made among states, attention must be paid to the effect of different data collection methods on the comparability of results. Other issues that need to be considered include whether or not specific populations of interest are included in samples from which data are drawn and whether data are collected sufficiently often, or are made available soon enough, to be useful in the monitoring process.
It is important to note that many of the performance measures presented in this report can, and should, be subdivided to focus on specific high-risk populations in a state. These populations may be defined demographically, such as minorities, children, or elderly persons; by conditions, such as not having health insurance or being homeless; or by geographic area, such as central cities, high-risk neighborhoods, or rural communities. Specific subpopulations of interest vary across states. Rather than create multiple submeasures for each proposed measure, the panel chose, in most cases, to identify broad population measures that can be tailored by each state to focus on its specific population group priorities.
Despite their widespread use and intuitive appeal, health outcome measures are insufficient by themselves for monitoring the efforts of a given program in reducing complex public health problems. Many measures that are recognized as valid for tracking health outcomes are affected by many factors (inputs or processes), so changes in outcomes cannot be attributed only to specific program effectiveness. Attribution of responsibility for outcomes becomes even more difficult when the services in question are supported by multiple funding sources or multiple provider organizations. The panel concludes that performance monitoring must make use of process and capacity measures to complement available measures of outcomes. The panel recommends that each process and capacity measure be accompanied by reference to published clinical guidelines or other professional standards that describe the relationship between the process measure or capacity measure and the desired health outcome.
Given the current and potential uses of performance measurement in public health, substance abuse, and mental health, the panel recommends that a combination of measures of health outcome, process, and capacity be used in the agreements between the federal government and states. Because in some cases actual health status outcomes are impractical to measure or because there are many factors that affect the ultimate health outcome, the panel recommends using ''intermediate" outcome measures, such as risk status, for which there is general consensus that the result being measured is related to the health status outcome. The panel uses the following definitions in this report: