Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Using Outcome Measures to Improve Care Delivered by Physicians and Hospitals Eugene C. Nelson The question "What works in the practice of medicine?" is very impor- tant. It is largely methodological and focuses on measurement. Yet an even more critical question is this: What works to improve the practice of medicine? It is one thing to use measurement to find out what works, but it is quite another thing to know what to do to improve that work. "If you always do what you always did, you will always get what you always got." This simple saying, spoken by a factory worker to W. Edwards Deming, the father of continuous improvement, makes that point (1~. Improvement in outcomes requires change upstream in the process. Measurement is part of a process of change it can help the process get started in the right direction and monitor the effect of efforts, but measurement alone will not create improvements. If effectiveness is to be increased, process improvement thinking must be included while constructing outcomes measurement systems. The challenge is not to create outcomes measurement systems, but to construct outcomes measurement/improvement (MI) systems for use by clinicians, hospitals, and other health care organizations. In this chapter, I will cover four points briefly. First, I describe two MI systems for medical practices. Second, I introduce two MI systems for hospitals. Third, I highlight the hallmarks of these systems, and fourth, I offer guidelines for using outcomes measures to make improvements. Before moving to point number one, I wish to illustrate the concept of an outcomes measurement/improvement system. Figure 1 illustrates an MI system for individual patients. The cycle begins with a patient visiting the physician or entering the hospital. The patient's baseline health outcomes are measured (disease-specific measures, general health status indicators, and patient expectations for care) and assessed by the clinician; the patient's 201
202 EFFECTIVENESS AND OUTCOMES IN HEALTH CARE Patient Visit/Stay 1 Measure Outcomes Implement and Foilow-Up Plan Regimen Assess Status FIGURE 1 A Measurement/lmprovement System for Individual Patients regimen is planned; care is implemented and follow-up is instituted; and outcomes measures are periodically gathered. The cycle continues making adjustments as the patient's status changes. OUTCOMES MEASUREMENT/IMPROVEMENT SYSTEMS FOR MEDICAL PRACTICES THE COOP CHARTS The first two systems I describe might be thought of as early attempts to develop MI systems for doctors' offices. One of these is the Dartmouth COOP Chart system. The following are the vital facts about the system: What? ~ ~ ~ - ~ ~ ~ How? Illustrated posters of health status Patient rates health Patient scores self Resource guide for clinician to prompt a regimen Benefits? Better communication Discovery of important problems Ease of use The COOP Charts (Figure 2) are similar to the Snellen charts that physi- cians have used in their offices for decades to test vision quickly. In fact, the Snellen charts were the inspiration for the COOP Charts (2~. The idea was to construct simple charts that could be used to measure some 10 key dimensions of overall health rapidly-physical function, mental health, so- cial function, pain, quality of life, and so on. Recently, we have begun to link COOP measurements with a functionally oriented resource guide. The
203 oo oo o C) z O cn a) I ~: O a _ c - CJ) O C] z o - > a) o ~) ~ E ~ a) <1 ~n ~ ~ Q (O _ ~ :~_ ~ C ._ 2 I ~ ~ a' ~ a) 0, ·_ D (L) Ct 4- ~ ·~ D `,, .' m-~) C ~ ~n ~n ~n a) O ~) Q >`E a) _ ~ ,~, D 0,, Q . - o =_ X CO ~ C Co O ~ O Q O ~ Q I a, C C o o o C' o .~ > C~- ~n ~, ~ .c ,~ U) ~CM Q u' o a) ~ (4 1~=3(,j,~': I C) ·_, o o o - au - o o o c~ ca r ~ - pH o u . . o u) ~;~1 ~ W~ o I E1_~ O ~ - ~ O O _, ' . ~ ~. 0 ~ ~ ~ ~ v) ~ a >~C(O [ 7L~ O ~0 ~ ~0 > ~ ~ ~ C)
204 EFFECTIVENESS AND OUTCOMES IN HEALTH CARE objective is to link measurement of the patient's functioning with sugges- tions for improvements that the physician can use to plan the regimen, thereby building improvement into the process of patient care delivery. Studies of the COOP Charts show that they are very easy to use in busy medical practices, that they are reliable and valid, and that use of them has several benefits (3,4~. Both patients and physicians believe that the charts improve communication and frequently lead to the discovery of important problems that would otherwise be missed. In a study conducted in about a dozen medical practices, physicians said that when the COOP Charts are used for case-finding, new, important information is produced for approximately 25 percent of patients; physicians also said that this leads to new treatment in two of five of these patients, providing a better fit between the patient's problems and the physician's plan of treatment. In addition to case-finding, COOP Charts can be used to monitor the overall functioning of patients with serious chronic diseases. Research suggests that the charts are able to show what impact discrete medical events, such as falls and adverse drug reactions, have on the patient's basic physical and mental function. Thus, use of the charts may help the doctor to understand better the effect of disease on the "whole" patient and thus to deliver more comprehensive care. The COOP Chart system for measuring and improving health outcomes holds great promise. The Henry J. Kaiser Family Foundation is sponsoring a large randomized trial at the Harvard Community Health Plan to document the system's case-finding utility in clinical practice, and the charts are being field tested in 20 countries to determine their value in other parts of the world. THE RUBENSTEIN FUNCTIONAL HEALTH STATUS APPROACH The second MI system for use in medical practice was developed by Lisa Rubenstein and her colleagues at the University of California, Los Angeles. The following are vital facts about the system: What? Questionnaire on health status How? Patient rates health Computer scores and profiles patient Resource guide for clinician Benefits? Better mental health Better social function The development of the Rubenstein functional health approach is an in- teresting and important story for anyone interested in improving outcomes. Several years ago, Dr. Rubenstein and colleagues at UCLA, BIAC (Beth Israel Ambulatory Care Center), RAND, and Harvard collaborated on a
APPLICATION TO CLINICAL PRACTICE 205 randomized trial. Their goal was to show that functional assessment of elderly patients visiting the offices of general internists would improve outcomes of care (5~. The measurement strategy was based largely on the short-form general health status tools developed by John Ware and his col- leagues at RAND. Two rather large randomized trials were conducted, one in Los Angeles and the other in Boston (6,7~. The results were disappoint- ing. Measurement of functioning of the internists' patients did nothing to improve outcomes a year later. In analyzing the reasons for these negative results, the investigators dis- covered that the MI cycle had been broken. Patients' baseline functioning had been measured and the results placed in the medical records; however, there was very little evidence that physicians had used this new information to add to their assessment or to plan treatment. As noted earlier, measure- ment of outcomes alone may produce no gains: "If you always do what you always did, you will always get what you always got." Dr. Rubenstein conducted a second randomized trial using the same measurement tool, but this time adding a function-oriented resource guide to the system. The resource guide was designed to link the patient's prob- lem with specific treatments that would be appropriate and effective. It provided site-specific "tips" on what the physician might do for an elderly patient with a physical disability such as poor balance or a mental health condition such as depression. The results from this second randomized trial, which included more than 76 physicians and 571 patients, were positive. Patients in the test group had significantly better mental health and social activity scores than patients in the control groups who received customary care after one year (8~. This time, the entire measurement/improvement cycle had been completed, and patients' outcomes had improved. MEASUREMENT/IMPROVEMENT SYSTEMS FOR HOSPITALS The first MI system for hospitals that I will discuss is being used in my organization, Hospital Corporation of America (HCA), and other hospitals around the country. PICA PATIENT JUDGMENT SYSTEM Here are the vital facts on the Hospital Quality Trends (HQT) Patient Judgment System: What? Random sample of patients rating hospital quality and health status on questionnaire How? Patient rates quality and health status Computer scores and profiles hospitals Results show improvement opportunities
206 EFFECTIVENESS AND OUTCOMES IN HEALTH CARE Benefits? Trends in quality over time Benchmarks across hospitals Linked to process improvement method (FOCUS-PDCA) The HQT Patient Judgment System was developed by a multidisciplinary design team that included practicing physicians, hospital administrators, nurses, and quality research leaders such as Paul Batalden, Donald Berwick, and John Ware from HCA, Harvard, and RAND, respectively. The system was tested in eight hospitals in 1987 and is now in use in approximately 100 hospitals. About 65 of these hospitals are owned by HCA; the others are a mix of large and small voluntary hospitals. An article describing the sys- tem was published in the June 1989 issue of Quality Review Bulletin and a monograph summarizing the development work is in press at Medical Care (9,10~. The aim of the system is to provide hospitals with valid, reliable, and useful trends in hospital quality, based on the voice of the patient. A random sample of discharged patients judges 10 dimensions of hospital quality (for example, admissions, nursing, physicians, information, daily care, and discharge) that are measured with a 68-item questionnaire. Patients also evaluate their health benefit from the stay and complete selected COOP Charts showing postdischarge functioning. Each hospital receives reports twice a year. The reports use graphic techniques to reveal longitudinal trends in quality. Hospitals use the reports to monitor trends and to identify (or focus on) high priority areas for improvement. These areas can then be addressed by Quality Improvement Teams using a structured improvement method, FOCUS-PDCA, that takes advantage of the scientific method in planning and managing process improvement (11~. An example of how this MI system is used can be found at West Paces Ferry Hospital in Atlanta, Georgia. The senior leadership team there iden- tified those aspects of quality that were most important to patients-clinical outcome, nurse response time and caring, nurse skill, the admitting process, and the discharge process-and that were candidates for improvement. The leadership then "chartered" several Quality Improvement Teams (composed of members from different departments involved in the process). It challenged every department in the hospital to identify which of its processes influence these key areas of quality and to begin FOCUS-PDCA on one or more of them. West Paces will use the HQT Patient Judgments System to monitor the overall success of its quality improvement efforts. In October 1989, 50 hospitals began using the HQT system for adult psychiatric patients. In addition to everything in the system described above, it includes the clinician's assessment of mental health at admission and discharge plus the patient's rating of his or her own physical function, mental health, and quality of life at admission, discharge, and one month after discharge.
APPLICATION TO CLINICAL PRACTICE SOUTH SHORE HOSPITAL GERIATRIC ASSESSMENT AND PLANNING PROGRAM 207 The final example of a measurement/improvement system is the Geriatric Assessment and Planning (GAP) program. A thumbnail sketch follows: What? How? Functionally oriented hospital record system for managing and following elderly patients Nurse rates functioning at admission, midstay and discharge Functional ratings are linked to treatment plan Benefits? Replaces nursing notes Basis for comprehensive discharge planning Better match between patient function and treatment plan Frail patient follow-up after discharge The GAP system was developed by leaders at South Shore Hospital, Carolee DeVito and William Zubkoff, with the assistance of external consultants in functional assessment such as Paul Densen and Charlotte Hamill (12~. The purpose of the GAP program is to provide a standard method of comprehensive patient assessment that will enable the hospital to improve the match of its services to the changing needs of elderly patients (13~. Starting in about 1983, South Shore began modifying the processes for admitting, nursing, and discharge planning to include full assessment of the patient's clinical and functional status at admission, midstay, and discharge. The GAP program involves all patients age 65 and older admitted to the hospital. Assessment includes standard data on such aspects of health as clinical parameters; Activities of Daily Living; Instrumental Activities of Daily Living; social, emotional, and cognitive function; and continuing care needs after discharge. The entire caregiving team physicians, nurses, discharge planners, and home health professionals-builds and uses the assessment/ management form to update the patient's status and to match services to patient needs. Patients with continuing care needs who are discharged to their homes are checked to see if the ordered services are being delivered, if their needs have changed, and if they need to be "relinked" with services. The GAP system is being extended and applied to new areas. For ex- ample, it serves as the backbone of a major demonstration program sponsored by the Centers for Disease Control to prevent falls leading to hip fractures in frail elderly patients. HALLMARKS OF MEASUREMENT/IMPROVEMENT SYSTEMS It is probably fair to say that none of the MI systems discussed above possess all of the desired features needed to be as good as it could possibly be. It is a fact that all systems can and should-be improved continuously
208 EFFECTIVENESS AND OUTCOMES IN HEALTlI CARE (14~. Nevertheless, these systems share certain characteristics that medical practices, hospitals, health maintenance organizations, and other providers could use to both measure and improve outcomes. Chief among them are the following: commitment by leaders in the provider organization to foster improvement, valid and reliable measures of outcomes. systematic, repeated assessment of outcomes, easy to fit into day-to-day pattern of care delivery, ease of administration, scoring, and interpretation of measures, directly linked between outcomes measures and improvement efforts, direct benefit to individuals and groups of patients, high value placed on system's utility by patients and clinicians ability to pass information "up-line" and to aggregate it for multisite efficacy studies and appropriate comparisons, and to use measurement . ability to compare outcomes against those of other providers. These features, when combined into a working system that is part and parcel of the caregiving routine, can be very powerful. Such a system creates a new way of processing and using measures to manage and improve outcomes. In the right environment one that promotes cooperation on quality improvement clinicians can work together to improve the system. USE OF OUTCOMES MEASURES TO BENEFIT A PATIENT POPULATION The use of outcomes measures in the aggregate to benefit an entire pa- tient population, as opposed to benefiting an individual patient, produces special challenges. The improvement cycle for a patient population is illustrated in Figure 3. The cycle begins with a population of patients with a selected health problem or condition. Measurements of structure, process, and outcomes are taken and then the relationships among them are analyzed to attempt to determine the "best" upstream settings (elements of structure) and the "best" upstream actions (processes) that appear to yield the "best" downstream results (outcomes). A field trial of the "best" upstream conditions is conducted to determine if they will produce the desired results in multiple settings. Finally, if the results are positive, this new information is disseminated to providers. Even a casual comparison of this cycle with the simpler one for indi- vidual patients (Figure 1) shows that it is far easier to make improvements for an individual patient than for a population of patients. It is still harder to construct an outcome MI program that can help improve an entire system of care composed of autonomous health care providers.
APPLICATION TO CLINICAL PRACTICE Disseminate to Field Conduct Trial of "Best" Vet Select Problem 1 Measure, Structure, Process and Outcomes Identif y "Best" Process and Settings Analyze Relationships FIGURE 3 A Measurement/Improvement System for a Population of Patients 209 Recognizing that the challenge that is, how best to use outcomes mea- sures for improvement- is very great, one might be wise to look outside the health care industry for guidance. There one would find a new way of thinking about what quality is and how best to improve it that stresses continuous improvement of processes (15~. One tool that is being used widely in quality improvement circles is an activity called "benchmarking." A recent book by R.C. Camp, an executive at Xerox, describes what benchmarking is and how to practice it (16~. Camp defines benchmarking this way: "Benchmarking is the search for industry's best practices that lead to supe- rior performance." The term "best practices" is equivalent to the term "best processes" and the term "superior performance" is analogous to "superior outcomes." Hence the purpose of benchmarking is to search upstream for the best processes that lead to superior outcomes. Note that the aim is not to find out who is best able to achieve superior ends. Rather, the goal is to spot superior outcomes as a way of flagging providers who employ outstanding processes that might be adapted for use in one's own organization. Benchmarking, in my opinion, could be a powerful vehicle for improve- ment in health care if it is a voluntary, provider-based, "from-the-bottom- up" activity. Benchmarking could succeed if it is undertaken with zeal by physicians, hospitals, and other providers as a search for the conditions and processes that are most likely to produce the best outcomes. Benchmarking, however, is unlikely to be helpful in health care if it is imposed from the top down. In fact, such a strategy for benchmarking might be counterproductive. Why? There are many reasons: (1) top-down benchmarking does not begin with the genuine need felt by most providers
210 EFFECTIVENESS AND OUTCOMES IN HEALTH CARE to find a "better way"; (2) it is likely to produce fear, a desire to protect one's own position and to discredit the information and its source; (3) the focus will be on the ends-the outcomes as opposed to the process and the means for achieving the end; and (4) top-down benchmarking is likely to foster blind competition among providers rather than useful cooperation. With these thoughts about the potential power of benchmarking and some sense of the pitfalls if it is launched in the wrong way, I would like to offer a few guidelines on how to use outcomes measures for improvement. · When measuring outcomes over time, one must measure related upstream conditions in order to understand the outcomes measures. · It is essential to separate the technical results (often termed clinical end points or parameters) from the benefits desired or achieved by patients. · Strive to understand all relevant upstream conditions (settings, processes, practices, and events) when interpreting outcomes measures. Identify the key features of the upstream conditions most likely to yield superior outcomes and conduct a trial to determine if the new way is more efficacious than the old. CONCLUSION Measuring outcomes is important. Improving outcomes is even more important. Outcomes can be improved by developing dual-purpose measurement/ improvement systems that are useful for individual patients, physicians, and other providers of care. These systems should link measurement of health outcomes directly with the care-giving process. They can best be assembled using a bottom-up, rather than a top-down, approach. This will be more likely to stimulate the curiosity of providers to make constructive clinical comparisons and thereby discover better ways for continuously improving patient care. REFERENCES 1. Deming, W.E. Out of the Crisis. Cambridge, MA: MIT Center for Advanced Engineering Study, 1988. 2. Nelson, E.C., Conger, B., Douglass, R., et al. Functional Health Status Levels of Primary Care Patients. Journal of the American Medical Association 249:3331-3338, 1983. 3. Nelson, E.C., Wasson, J.H., and Kirk, J.W. Assessment of Function in Routine Clinical Practice: Description of the COOP Chart Method and Preliminary Findings. Journal of Chronic Diseases 40(Supplement 1~:55S-63S, 1987. 4. Nelson, E.C., Landgraf, J.M., Hays, R.D., et al. The COOP Function Charts: A System to Assess Functional Health Status in Physicians' Offices. Final report to
APPLICATION TO CLINICAL PRACTICE 211 the Henry J. Kaiser Family Foundation. Hanover, NH: Dartmouth Medical School, 1987. 5. Jette, A., Davis, A., Cleary, P., et al. The Functional Status Questionnaire: Reliability and Validity When Used in Primary Care. Journal of General Internal Medicine 1:143-149, 1986. 6. Rubenstein, L.V., Calkins, D.R., Young, R.T., et al. Improving Patient Func- tional Status: Can Questionnaires Help? Clinical Research 34:835a, 1986. 7. Calkins, D.R., Rubenstein, L.V., Cleary, P.D., et al. The Functional Status Questionnaire: Initial Results of a Controlled Trial. Clinical Research 34:359a, 1986. 8. Rubenstein, L.V., McCoy, J.M., Cope, D.W., et al. Improving Patient Func- tional Status: A Randomized Trial of Computer-Generated Resource and Manage- ment Suggestions. Paper presented the annual meeting of the American Federation of Clinical Research, Washington, D.C., May 1989. 9. Nelson, E.C., Hays, R.D., Larson, C., et al. lithe Patient Judgment System: Reliability and Validity. Quality Review Bulletin 15: 1 85 - 1 9 1, 1 989. 10. Meterko, M., Nelson, E.C., and Ruben, H.R. Patient Judgments of Hospital Quality: Report of a Pilot Study. Medical Care, in press. 11. Batalden, P.B. and Buchanan, E.D. Industrial Models of Quality Improvement. Pp. 133-159 in Providing Quality Care: The Challenge to Clinicians. Goldfield, N. and Nash, D.B., eds. Philadelphia: American College of Physicians, 1989. 12. W.K. Kellogg Foundation. Patient Assessment for Continuing Care: Execu- tive Summary. Westchester Patient Assessment Program. Battle Creek, MI: W.K. Kellogg Foundation, 1987. 13. DeVito, C.A. and Zubkoff, W. Discharging the Frail Elderly: One Hospital's Model Program. Continuing Care 42:26-31, 1989. 14. Berwick D.M. Continuous Improvement as an Ideal in Health Care. New England Journal of Medicine 320:53-6, 1989. 15. Walton, M. The Deming Management Method. New York: Dodd, Mead & Company, 1986. 16. Camp, R.C. Benchmarking: The Search for Industry's Best Practices that Lead to Superior Performance. Milwaukee, WI: ASQC Quality Press and White Plains, NY: Quality Resources, 1989.
