of information obtained about applicants for hospital privileges or how often review committees agree in their recommendations. To determine the reliability of a method to detect quality problems, one might calculate how often chart reviewers identify the same adverse events. Reliability in correcting problems is more theoretical, but one might envision measuring whether comparable corrective action plans (e.g., continuing education courses or reading designated literature) consistently improve tested knowledge.

Validity in this context refers to whether a method acts as intended. For one to consider board specialization a valid method of ensuring high quality, for instance, one would look for proof that those who are board certified provide a demonstrably higher quality of care than those who are not. Likewise, the validity of an outcome measure of quality could be assessed by determining whether patients with poor outcomes received deficient care and whether the deficiency produced the poor outcome. To demonstrate the validity of a method of correcting problems one would look for evidence that a specific intervention brought about the desired change. For instance, required consultation with a colleague before treating certain cases should result in fewer problem cases.

Assessment methods may be valid in that they detect real problems in quality, but even valid tools may be inefficient (if they detect a great many events that are not quality problems) or ineffective (if they fail to detect many important quality problems).

For virtually no method of assessment do we know the effect on provider behavior or the effect of practitioner change in behavior on patient outcomes. These are the ultimate tests of validity. Although methods may be accurate at identifying problems, they are valuable for quality assurance only if, or to the extent that, identification leads to changed behavior and to improved patient outcomes. Measures of these two demanding but critical factors are almost nonexistent, and this shortfall must temper any recommendations for specific approaches.

Assessment methods have important attributes other than reliability and validity. These include their practicality, ease of application, lack of unintended negative effects, inclusion of patient views and preferences, and ability to detect poor technical quality, overuse, and underuse. It is also useful to consider whether various methods of assessment provide timely information to improve performance and whether they yield information that accords with ideas about how professionals learn.


Accreditation and Licensure for Organizations

Hospitals, ambulatory care facilities, managed care organizations, and home health agencies can be accredited on a voluntary basis by the Joint Commission on Accreditation of Healthcare Organizations (Joint Commis-

The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement