Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
1 r Measuring Patient Function and Well-Being: Some Lessons from the Medical Outcomes Study John E. Ware, Jr. Among the important developments in the health care field during the past decade is the recognition that the patient's point of view, in monitoring the quality of medical care outcomes, is central. Indeed, the goal of medi- cal care today for most patients is the achievement of a more "effective" life (1) and the preservation of function and well-being (2,3,4,5~. The patient is the best source on the achievement of these goals. However, information about patients' experiences of disease and treatment is not rou- tinely collected in clinical research or medical practice. This information is not part of the medical record and consequently is not typically available for analysis in the current health care data base. We are entering a new era in which information from patients about functional status, well-being, and other important health care concepts will be added to the health care data base. Included are data bases used to compare costs and benefits of various financial and organizational aspects of health care services, by organizational managers who try to provide the best value for health care dollars, by clinical investigators who evaluate new treatments and technologies, and by practicing physicians and other providers who try to achieve the best possible outcomes for their patients. The primary source of this information will be from standardized patient surveys that have served research well over the past decade. The most efficient way to monitor functional status and well-being for most adults is via scoring of carefully constructed sets of survey questions. Advances in assessment and measurement, particularly in terms of surveys of patient perspectives, have facilitated this kind of data collection (see, for example, 6 and 7), although their use on a large scale has not been practical. It is clear that the field of health care needs more cost-effective ways to obtain new data about patient outcomes. The methods must be practical and 107
108 EFFECTIVENESS AND OUTCOMES IN HEALTH CARE they must satisfy the most crucial psychometric standards. The trade-off between practical considerations and psychometric standards has led to a rethinking of measurement strategy. Better measurement is measurement that has information one absolutely has to have, and no more. I am going to emphasize practical issues as much as precision and reliability and validity, and I plan to do so without using any numbers whatsoever. Numbers provide some form of authority, but they also can be restrictive. CONCEPTS AND DATA SOURCES Health care providers collect data about functioning for virtually every body organ, but none of these measures tells about the function of the entire individual-which is certainly affected by disease and treatment (see Figure 1~. Further, these measures of biologic phenomena cannot be used to characterize human phenomena. There simply are not good algorithms for combining diverse biologic information to predict functioning, and such algorithms are doomed to leave too much about quality of life unexplained. The most comprehensive models I have seen to date might explain 10-25 percent or so of the reliable variance in, for example, physical functioning. Thus, biologic indicators are not adequate proxies for measures of functional sta- tus or well-being or to changes in these variables over time. Biologic indicators must be supplemented if we are to use outcome data to achieve the goal of providing the best value for the health care dollar. Specifically, we must consider how individuals experience disease as well as treatment. The current data base also has information about death. How- ever, to quote Jack Elinson, formerly of the National Center for Health /BEHAVIOR FUNCTIONING - / - - - FIGURE 1 Health Status Concepts WELL BEING - BIOLOGIC FUNCTIONING \
COLLECTION OF PRIMARY DATA 109 Statistics, there isn't very much information about the health of a population from mortality data in a developed country. Consider heart surgery: for patients with heart disease the mortality rates are approximately 5 percent or less. Thus, for nearly all patients, that particular indicator provides no information about variations in outcomes. Essentially, we need a new data base. In addition to some other things, the new data base should add two types of information: patients' experience of health care, and the patients' experience of health outcomes. Our task is to find ways to incorporate this information into the total health care equa- tion. In this regard, I believe that the effectiveness initiative (8) involves more than going from efficacy to effectiveness. It really involves going from one relatively limited set of variables that has been used in judging efficacy to a completely different set of variables not traditionally used to evaluate alternate treatments and technologies. We have not routinely assessed the effect of treatment on quality of life, or functioning, or well-being from the patients' point of view. DISEASE-SPECIFIC VERSUS GENERIC MEASURES Before proceeding, let me define what I mean by generic measures. They measure concepts that are relevant to everyone. They are not specific to any age, disease, or treatment group. Generic measures focus on such basic human values as emotional well-being and the ability to function in everyday life. Should we use disease-specific or generic measures? The overwhelming answer should be to use both and to use them together. We should not reject one data base in favor of the other. We went through a period in the mid-1960s during which the validity of a patient rating a generic health concept was questioned when it did not agree with what was in the record or with what the provider said. The logic of validity has since been turned around. We are now entering an era in which the same findings are accepted as evidence for the necessity of including patient assessments as part of the evaluation process. The record and provider judgments are not valid prox- ies for patient ratings of functioning, well-being, or other aspects of the quality of life. We should not always expect assessments of different health components or clinical versus generic measures to agree, and often they do not. One example comes from a study of the effects of antihypertensive therapy on quality of life (9~. Therapies shown to be equally efficacious in terms of medical efficacy (i.e., blood pressure control) had significantly different quality of life profiles. In other words, it is possible to work with a patient in therapy to achieve a better quality of life outcome without compromising biologic function. There is also evidence accumulating that shows that
110 EFFECTIVENESS AND OUTCOMES IN HEALTlI CARE differences in biologic function often have quality of life implications. These two concepts are distinct; they are affected by different processes and they interact with each other. To understand patient health outcomes, different health components need to be measured and interpreted separately and in combination (the latter when trade-offs are involved). The greatest progress is going to occur, not by substituting one measurement or assessment strategy for another, but by mastering them in concert. We should not underestimate the power of a data base that includes clinical measurements familiar to medical providers, measurements that they believe in because they have clinical validity, in parallel with other measures, such as measures of generic health concepts, not typically linked with such measures in clinical practice or research. This is the most powerful strategy for analyzing and understanding outcomes and for diffusing recent advances in methods for assessing patient outcomes. A MINIMUM SET OF GENERIC HEALTH CONCEPTS I would like to take this opportunity to recommend what a minimum set of generic health concepts might look like. At the risk of oversimplifying the past 40 years of health assessment research, I think most health measures can be classified into one of three major categories: functional status, well- being, and general health perceptions. I have defined these categories elsewhere and have illustrated them with sample questionnaire items from widely used measures (10~. Functional status, which includes disability assessment, refers to behavioral dysfunctions due to health problems. It is the concrete, observable, tangible, and objective category of health measures. Measures in this category use a standard external to the individual, such as usual role activity, walking at a certain rate, or customary self-care behaviors. This is the functional status axis in a multidimensional conceptualization of health. It is the concept that has been preferred and best understood until now. There are a number of well-developed measures of functioning available. Interestingly, almost completely orthogonal to the functional status axis is the well-being axis, which includes psychological distress, psychological well-being, and life satisfaction. In most populations we observe all levels of each of these axes at all levels of the other. In fact, in most populations, the correlation between them is only 0.20 or less (assuming confounding of measures across axes has been removed). The implication is that we cannot know how people feel by observing what they are doing. Consider two people sitting on a fencepost, for example. One may be experiencing a lot of pain and may have difficulty just sitting there. The other person may sit in ecstasy. In order to know, we have to ask them. It used to be thought that well-being could not be measured reliably. We have learned that quite reliable scores for this continuum can be obtained and that they add a
COLLECTION OF PRIMARY DATA 111 completely different perspective to that gained from functional status as sessment. Finally, there is a third axis, which cuts across the other two and brings still another perspective beyond both of the other two axes; that is the category of general health perceptions. It includes measures that are per- sonal evaluations of health, based on whatever health means to the respon- dent. This category of measures brings each person's own health values to the equation. He might be a mental-health-oriented person or a physical- health-oriented person. Health perceptions represent the third axis or category that I would recommend for inclusion as a minimum standard for generic health measures. THE MEDICAL OUTCOMES STUDY A hallmark of the Medical Outcomes Study (MOS) is its reliance on a broad array of outcome measures, including parallel assessments of disease- specific clinical endpoints traditionally measured by clinicians (biologic functioning in Figure 1) as well as generic measures of functional status, well-being, and satisfaction with health care as reported by patients. This more encompassing assessment of outcome increases the likelihood of detecting the consequences to patients of policies that modify the structure of the health care system or the process of care. Measuring disease-specific end points, as well as a common set of generic health outcomes for various conditions, will also contribute a new data base that will allow physicians to inform patients about the trade-offs involved in different treatment. I am focusing here on health outcomes. Patients should also be involved in assessing the quality of the medical care process (11~. I am going to give you a brief summary of some of our experiences to date in the MOS (12~. One of our intentions in the MOS was to test the feasibility of implementing the same primary data collection system in very diverse systems of care for purposes of monitoring the results of that care over time. By design, we included very different health care settings and very different patient popu- lations. We sampled different health care settings to vary the structure and pro- cess, and we are measuring variations in the outcomes of care. Structural features of care include, for example, whether the provider is an HMO, an insurance plan, a subspecialist, or a more generally trained physician. These traditionally stable attributes of the health care system are now among the many tools of cost containment. People are experimenting with such structures in efforts to reduce medical expenditures. In the MOS we are looking at how structural differences affect the process of care in the two major categories, technical process and interpersonal process (12~. Sponsors of the MOS are undoubtedly interested in whether the expenditure
112 EFFECTIVENESS AND OUTCOMES IN HEALTH CARE for the study, which is approaching $12 million, can also be justified in terms of addressing whether different ways of organizing and financing health care affect patient outcomes. What is the best way to organize, finance, and deliver care? If you have hypertension, for example, does it matter whether you are treated by a cardiologist or a family practitioner? Does it matter whether depressive disorders are detected and treated (13~? Again, one of our primary interests has been to work towards advancing the state-of-the-art in outcome assessment methods. One lesson we have learned is that it is feasible to create the new data base defined above and to add to it routinely on a rather large scale in very diverse health care settings. MOS analyses in progress will be quite informative about the more cost- effective ways of creating such a database and which variables are most important for what kinds of analyses. Before commenting on some of the MOS lessons to date, let me discuss briefly some study design features. Additional details are given elsewhere (12, and in some references cited there). The study was done in three sites. At each site we sampled physicians and patients from three different kinds of organizations: traditional pre- paid group practice form of health maintenance organizations, multispecialty groups, and solo practices. From the latter two we sampled both fee-for- service and prepaid patients treated by the same physicians. The result is five "systems of care" that differ in organization and financing. We sampled 523 physicians trained in family practice, general internal medicine, endo- crinology, cardiology, and psychiatry. Other mental health providers were also sampled (13~. These are the specialties that treat the MOS tracer conditions: hypertension, diabetes, heart disease, and depressive disorders. The conditions were chosen primarily because they are prevalent, costly, treatable with variations in practice style, and have an impact on the outcomes of interest. We looked at adult patients who were seen during a nine-day period in these physicians' offices. We gathered screening data from both the patient and the doctor at that time. We then took approximately a 10 percent random subsample of those patients who had one or more of our chronic tracer conditions. As of October 1988 we had followed these patients for over two years. We hope to continue to follow them. We look at the care they receive and we monitor transitions in their clinical status as well as functional status and well-being, the latter at six-month intervals. FEASIBILITY AND COST Much of the expense of the MOS was attributable to the cost of identifying and recruiting providers and patients, designing instruments and data collection methods, and making sure that they would work. We spent nearly two
COLLECTION OF PRIMARY DATA 113 thirds of the study's total funds before the panel started. Measuring and analyzing patient outcomes over time has not been the major expense. Clearly what one concludes about what things cost depends upon what is charged to a given cost category. If primary data collection is considered a marginal cost in a framework for monitoring patient outcomes, the marginal cost would be relatively small. If other costs are charged to this category, including for example, defining what diabetes is, determining whether somebody has it, determining how to sample doctors and patients, dealing with differences in patient case mix, the cost per patient followed is much higher. Again, once we had identified both patients and providers and had measured differences in patient case mix, the cost of following patients over time was relatively small. With state-of-the-art short forms and processing methods, the cost to process a patient health assessment in a doctor's office is less than the least expensive lab test. There has been at least one other lesson about feasibility. We oversampled Medicare patients because of the policy relevance of that group. We hypothesized that they would be treated differently in different practice settings as a result of oversampling Medicare. The median age in our longitudinal sample was about 60. All of them had one or more chronic conditions. This population is very sick relative to, for example, the population we followed in the Health Insurance Experiment where we also used self-administered questionnaires as a primary data collection tool in comparing outcomes across different systems of care (14~. How well did these methods work in the MOS? When we conducted our two-year follow-up survey two years after enrollment, we again used a self- administered survey, a booklet with about 250 questionnaire items. Our response rate was over 80 percent for those who self-administered the full- length questionnaire. We used telephone interviews and the MOS short form for those who did not and raised the overall response rate to over 90 percent of those who were contacted and still alive. Whereas dollars can be saved early on by using self-administration, you must be willing to spend some of these savings for follow-up (e.g., by telephone) for people who do not complete a self-administered form. Nearly 70 percent of the panel has completed all surveys during the course of the study. The typical completed questionnaire has 1 percent or fewer of the items missing. Thus, it is possible to get high completion rates for long questionnaires even in a relatively elderly and relatively sick population. This experience has made me more enthusiastic than I had been after the Health Insurance Experiment (and I was enthusiastic then) about the feasibility of standardized surveys and self-administration as a primary data collection strategy for monitoring patient outcomes. MOS providers were roughly evenly divided between group and solo practice settings. We found that it is more efficient to monitor outcomes in
114 EFFECTIVENESS AND OUTCOMES IN HEALTH CARE group practices. Solo practitioners do not have equivalent support person- nel. Thus our sampling rates were higher in groups (12~. Our completion rates were roughly the same across practice settings once people had enrolled in the study. Thus, with centralized data collection and standardized forms and methods, the completeness and quality of the database need not vary by practice setting. This leads to the notion of centralized health assessment laboratories. With support from the John A. Hartford Foundation of New York, we are now developing and testing this concept. Again, the MOS is a methodological study, and with support from the Henry J. Kaiser Family Foundation, my colleagues and I are now comparing the briefest forms of measurement, e.g., the best single-item measure, with longer but still short multi-item scales, and with full-length research versions of these scales. Our question is how well do shorter measures work relative to much longer measures used in research? Not surprisingly, preliminary findings indicate that longer measures do better. However, the question should be: "Do shorter measures do well enough?" The answer is very important because the most psychometrically elegant instrument is useless if it is impractical to use. Thus, we should be very interested in how briefer measures do in these comparisons. STANDARDS FOR EVALUATING MEASURES On what basis should measures be compared and how do we construct them in the first place? A number of things are wrong with what we traditionally do in psychometrics. Take reliability as an example. Reliability is important, but we learned quickly that although reliability is a prerequisite, other attributes of a score (scale) are equally or more important. In comparing scales, most important are tests that most closely approximate the intended use of the measure. Unfortunately, traditional reliability and validity coef- ficients have little or no relationship to actual applications of measures. Some attributes of measures typically ignored include things like how many different scores are possible. An enumeration system that puts people into one of four levels or categories with a reliability of 0.90 is not as valuable as is one that puts people into 10 categories with the same reliability. This particular attribute of measurement may not prove critical in a cross- sectional analysis comparing things as different as Volkswagens and trucks. The latter is analogous to comparing disease groups that differ a lot. Most measures do well in that kind of comparison. When we start measuring change in health over time, however, this issue becomes crucial. How much change can occur within a given health category before the person changes to the next category? The number of levels of measurement is a very important attribute or measure. This attribute is almost never discussed in books on health assessment.
COLLECTION OF PRIMARY DATa 115 Another important and related issue is simply how many people get the lowest or the highest possible score for a given measure. If 90 percent of the people in a long-term care facility have the worst possible score before a cost-containment strategy is implemented, you do not have precision for showing a worsening in their condition as a result. If 80 percent of the people earn the highest possible score on a physical health index (as they did in the Health Insurance Experiment), and we randomly assign them to free care, we do not have much chance of determining, using that particular measure, whether there is any benefit of free care. Fortunately, this was not the only measure used in that experiment (151. I suggest that, when measures are published, we routinely report how many people get the lowest and the highest possible scores when measures are published as well as the number of scores possible, in addition to reliability coefficients. The same logic should apply to results regarding validity. It is extremely important that the kind of analysis used to judge the validity of a measure approximate as closely as possible the intended use of the measure in medical practice, a clinical trial, or a policy study. Much published evidence bears little or no relationship to most intended applications. One example of what I mean is the issue of whether a given questionnaire is sensitive to the extent and nature of differences in functional status and well-being across groups of patients with different chronic conditions. My colleagues and I recently reported an example of such comparisons using the 20-item MOS short-form survey (16,17~. Figure 2 presents examples of profiles for patients with four different chronic conditions at a point in time when the study began. Each profile is expressed as standard score deviations from the averages for well patients (represented by the horizontal dotted line). The first three data points (columns) for each disease are defined by functional status scales, the last three by well-being scales. We have connected the points across scales for each disease to help identify a particular disease profile. These are scored so that the lower the profile on the scales the worse the profile. Figure 2 generally confirms clinical wisdom about the impact of these diseases. Not surprisingly, patients with hypertension (the top profile in Figure 2) function no differently than well patients. The only significant decrement was their score for health perceptions. Patients with hypertension tend to believe their health is worse. Arthritis has the most pain. Physical, role, and social functioning is poor for survivors of myocardial infarction (MI) and tends to be as bad as, if not worse than, any of the nine chronic conditions we have studied to date. One lesson from this is that, on average, the patient point of view is valid. Further, even very brief measures can be used to measure differences in health across groups of patients. The questionnaire used to estimate scores in Figure 2 was administered to about 12,000 patients while waiting in a doctor's office, in about three and a half minutes each.
116 EFFECTIVENESS A1VD OUTCOMES IN HEALTH CARE 0.20 -O. 20 a' a -0.40 -0.60 -0.80 - 1 .00 Hypertension Arthritis ,~ Ml / Physical Role Social Mental Health Bodily Functioning Functioning Functioning Health Perceptions Pain FIGURE 2 Health Profiles for Patients with Four Conditions. Dotted line indicates patients win no chronic conditions; GI, gastrointestinal disorder; MI, myocardial infarction. Relative to available full-length research instruments, including our own, this short-form has one-fifth to one-tenth the number of questionnaire items. Yet it produced a pattern of results that make sense from a clinical point of view. One surprising finding (not shown in Figure 2) is the very low profile of scores for patients with depression, including those with a psychiatric diagnosis and those suffering with symptoms of depression. They scored very low on these scales relative to other chronic diseases, suggesting that the burden of depression may have been underestimated to date (13~.1 Of course, other kinds of tests are necessary before conclusions are drawn about candidate measures. How well does a questionnaire distinguish differences in functional status and well-being across groups differing in severity within a diagnostic category? Research in progress within the MOS is encouraging in this regard. For example, in preliminary analyses of MOS data, average functional status scores for diabetics differing in severity (e.g., with or without renal failure) show ordinal consistency in relation to clinical severity. Thus, these measures may be sensitive to differences in severity within a . . c lagnostlc group. This kind of analysis does not prove that if treatment moved people from severity level five to level three, we would see a corresponding change in functioning. This example is based on a cross-sectional analysis. We are currently linking measures of actual change in disease severity over time with measures of change in functional status and well-being. Again, pre- liminary results are encouraging. iFor another description of the MOS and results pertaining to depression, see Chapter 19 of this volume (20~.
COLLECTION OF PRIMARY DATA 117 USE OF HEALTH SURVEY DATA The potential of generic functional status and well-being scales, even short forms, to be used successfully across a wide range of purposes is illustrated by the vitality scale used in the MOS, a 4-item scale that takes about one minute to complete. It measures a continuum of energy versus fatigue. Its history, which is documented in part elsewhere (17), includes successful use in a population health survey about 15 years ago (18), the Health Insurance Experiment (17), a more recent clinical trial comparing antihypertensive therapies (9), and the MOS (19~. Its track record defies the notion that completely different measures are needed for different applications. This one-minute vitality scale, for example, has been used successfully in describing the young and the old in the U.S. population, the sick and the well, and in measuring outcomes across homogeneous groups of patients receiving different treatments in a randomized trial. With my recent move to the New England Medical Center, I have had occasion to contrast my own background and training with the needs of the next era of health assessment. I was trained in measurement theory and methods and for the prior 15 years I had been in a full-time research setting where we evaluated measures in traditional psychometric terms and used them for purposes of research. Now I focus much more on the information needs of health care delivery organizations and look at measurement and the use of data from a different perspective. Two years in a research institute in a health care delivery organization have convinced me that a new data base with information about patient outcomes is not the solution to the problem, it is just the next step. The challenge of implementing outcomes management or an effectiveness initiative is not a problem of measurement, and it is certainly not a problem of assessing outcomes from the patients' view. The methodological problems include determining (1) a coherent sampling strategy; (2) techniques for case-mix measurement and statistical control; (3) a meaningful schedule of assessments for different diagnostic groups; (4) analytic strategies for displaying results in a meaningful way; and (5) recognizing that conclusions are sensitive to these and other choices. Finally, the real challenge is the creation of a decision-making process capable of using data about patient outcomes. Indeed, the collection of outcomes data from patients is one of the simplest steps ahead. ACKNOWLEDGEMENTS The MOS has been sponsored by grants from the Henry J. Kaiser Family Foundation, the Robert Wood Johnson Foundation, the John A. Hartford Foundation, the Pew Charitable Trusts, the Agency for Health Care Policy and Research, the National Institute on Aging, and the National Institute of
118 EFFECTIVENESS AND OUTCOMES IN HEALTH CARE Mental Health, and by The RAND Corporation and the New England Medi- cal Center from their own research funds. The author gratefully acknowledges Albert P. Williams, at The RAND Corporation and especially the MOS staff and consultants, including Sharon Arnold, Sandra H. Berry, M. Audrey Burnam, Maureen Carney, Allyson Ross Davis, Sheldon Greenfield, Ron D. Hays, Elizabeth McGlynn, Eugene C. Nelson, Lynn Ordway, Judith Perlman, Edward B. Perrin, William Rogers, Cathy Sherbourne, Anita L. Stewart, Alvin R. Tarlov, Kenneth B. Wells, and Michael Zubkoff, and the secretarial and administrative support of Kathy Clark. REFERENCES 1. McDermott, W. Absence of Indicators of the Influence of Physicians on a Society's Health. American Journal of Medicine 70:833-843, 1981. 2. Cluff, L.E. Chronic Disease, Function and the Quality of Care. Journal of Chronic Diseases 34:299-304, 1981. 3. Tarlov, A.R. Shattuck Lecture. The Increasing Supply of Physicians, the Changing Structure of the Health-Services System, and the Future Practice of Medi- cine. New England Journal of Medicine 308:1235-1241, 1983. 4. Schroeder, S.A. Outcome Assessment 70 Years Later: Are We Ready? New England Journal of Medicine 216:160-162, 1987. 5. Ellwood, P.M. Shattuck Lecture. Outcomes Management: A Technology of Patient Experience. New England Journal of Medicine 318:1549-1556,1988. 6. Lohr, K.N. and Ware, J.E. Advances in Health Assessment, Special Issue. Journal of Chronic Diseases 40:Supplement l:lS-193S, 1987. 7. Lohr, K.N. Advances in Health Status Assessment: Overview of the Confer- ence. Medical Care 27~3) Supplement:Sl-Sll, 1989. 8. Roper, W.L., Winkenwerder, W., Hackbarth, G.M., et al. Effectiveness in Health Care: An Initiative to Evaluate and Improve Medical Practice. New England Journal of Medicine 319:1197-1202, 1988. 9. Croog, S.H., Levine, S.M., Testa, M.L., et al. The Effects of Antihypertensive Therapy on Quality of Life. New England Journal of Medicine 314:1657-1664, 1986. 10. Ware, J.E. Standards for Validating Health Measures: Definition and Con- text. Journal of Chronic Diseases 40:473 -480, 1987. 11. Davies, A.R. and Ware, J.E. Involving Consumers in Quality of Care Assessment: Do They Provide Valid Information? Health A~airs 7:33-48, 1988. 12. Tarlov, A.R., Ware, J.E., Greenfield, S., et al. The Medical Outcomes Study: An Application of Methods for Monitoring the Results of Medical Care. Journal of the American Medical Association 262:925-930, 1989. 13. Wells, K.B., Stewart, A., Hays, R.D., et al. The Functioning and Well-Being of Depressed Patients: Results from ~e Medical Outcomes Study. Journal of the American Medical Association 262:914-919, 1989. 14. Ware, J.E., Brook, R.H., Rogers, W.H., et al. Comparison of Health Out- comes at a Health Maintenance Organization with Those of Fee-for-Service Care. Lancet i(8488~: 1017-1022, 1986.
COLLECTION OF PRIMARY DATA 119 15. Brook, R.H., Ware, J.E., Rogers, W.R., et al. Does Free Care Improve Adults' Health? Results from a Randomized Controlled Trial. New England Jour- nal of Med icine 309: 1426-1434, 1983. 16. Stewart, A.L., Greenfield, S., Hays, R.D., et al. Functional Status and Well- Being of Patients with Chronic Conditions: Results from the Medical Outcomes Study. Journal of the American Medical Association 262:907-913, 1989. 17. Ware, J.E., Johnston, S.A., Brook, R.H., et al. Conceptualization and Mea- surement of Health for Adults in the Health Insurance Study: Volume III, Mental Health. R-1987/3-HEW. Santa Monica, Calif.: RAND Corp. 1979. 18. Dupuy, H.J. The Psychological Section of the Current Health and Nutrition Examination Survey. U.S. Dept. of Health, Education, & Welfare Publication. No. (HRA). 74-1214. Washington, D.C.: Government Printing Office, 1974. 19. Stewart, A.L. and Ware, J.E., eds. Measuring Functional Status and Well- Being: The Medical Outcomes Study Approach. Forthcoming. 20. Burnam, M. A. Studying Outcomes for Patients with Depression: Initial Findings from the Medical Outcomes Study. Pp 159-168 in Electiveness and Out- comes in Health Care. Heithoff, K.A. and Lohr, K.N., eds. Washington, D.C.: National Academy Press, 1990.
