|
Items in cart [0] |
Questions? Call 888-624-8373 |
| Copyright © 2009. National Academy of Sciences. All rights reserved. Terms of Use and Privacy Statement |
Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 107
1 r
Measuring Patient Function and
Well-Being: Some Lessons from the
Medical Outcomes Study
John E. Ware, Jr.
Among the important developments in the health care field during the
past decade is the recognition that the patient's point of view, in monitoring
the quality of medical care outcomes, is central. Indeed, the goal of medi-
cal care today for most patients is the achievement of a more "effective"
life (1) and the preservation of function and well-being (2,3,4,5~. The
patient is the best source on the achievement of these goals. However,
information about patients' experiences of disease and treatment is not rou-
tinely collected in clinical research or medical practice. This information is
not part of the medical record and consequently is not typically available
for analysis in the current health care data base.
We are entering a new era in which information from patients about
functional status, well-being, and other important health care concepts will
be added to the health care data base. Included are data bases used to
compare costs and benefits of various financial and organizational aspects
of health care services, by organizational managers who try to provide the
best value for health care dollars, by clinical investigators who evaluate
new treatments and technologies, and by practicing physicians and other
providers who try to achieve the best possible outcomes for their patients.
The primary source of this information will be from standardized patient
surveys that have served research well over the past decade. The most
efficient way to monitor functional status and well-being for most adults is
via scoring of carefully constructed sets of survey questions. Advances in
assessment and measurement, particularly in terms of surveys of patient
perspectives, have facilitated this kind of data collection (see, for example,
6 and 7), although their use on a large scale has not been practical.
It is clear that the field of health care needs more cost-effective ways to
obtain new data about patient outcomes. The methods must be practical and
107
OCR for page 108
108
EFFECTIVENESS AND OUTCOMES IN HEALTH CARE
they must satisfy the most crucial psychometric standards. The trade-off
between practical considerations and psychometric standards has led to a
rethinking of measurement strategy. Better measurement is measurement
that has information one absolutely has to have, and no more. I am going to
emphasize practical issues as much as precision and reliability and validity,
and I plan to do so without using any numbers whatsoever. Numbers provide
some form of authority, but they also can be restrictive.
CONCEPTS AND DATA SOURCES
Health care providers collect data about functioning for virtually every
body organ, but none of these measures tells about the function of the entire
individual-which is certainly affected by disease and treatment (see Figure
1~. Further, these measures of biologic phenomena cannot be used to characterize
human phenomena. There simply are not good algorithms for combining
diverse biologic information to predict functioning, and such algorithms are
doomed to leave too much about quality of life unexplained. The most
comprehensive models I have seen to date might explain 10-25 percent or
so of the reliable variance in, for example, physical functioning. Thus,
biologic indicators are not adequate proxies for measures of functional sta-
tus or well-being or to changes in these variables over time.
Biologic indicators must be supplemented if we are to use outcome data
to achieve the goal of providing the best value for the health care dollar.
Specifically, we must consider how individuals experience disease as well
as treatment. The current data base also has information about death. How-
ever, to quote Jack Elinson, formerly of the National Center for Health
/BEHAVIOR FUNCTIONING -
/
-
-
-
FIGURE 1 Health Status Concepts
WELL BEING -
BIOLOGIC
FUNCTIONING
\
OCR for page 109
COLLECTION OF PRIMARY DATA
109
Statistics, there isn't very much information about the health of a population
from mortality data in a developed country. Consider heart surgery: for
patients with heart disease the mortality rates are approximately 5 percent
or less. Thus, for nearly all patients, that particular indicator provides no
information about variations in outcomes.
Essentially, we need a new data base. In addition to some other things,
the new data base should add two types of information: patients' experience
of health care, and the patients' experience of health outcomes. Our task is
to find ways to incorporate this information into the total health care equa-
tion. In this regard, I believe that the effectiveness initiative (8) involves
more than going from efficacy to effectiveness. It really involves going
from one relatively limited set of variables that has been used in judging
efficacy to a completely different set of variables not traditionally used to
evaluate alternate treatments and technologies. We have not routinely assessed
the effect of treatment on quality of life, or functioning, or well-being from
the patients' point of view.
DISEASE-SPECIFIC VERSUS GENERIC MEASURES
Before proceeding, let me define what I mean by generic measures. They
measure concepts that are relevant to everyone. They are not specific to
any age, disease, or treatment group. Generic measures focus on such basic
human values as emotional well-being and the ability to function in everyday
life.
Should we use disease-specific or generic measures? The overwhelming
answer should be to use both and to use them together. We should not
reject one data base in favor of the other. We went through a period in the
mid-1960s during which the validity of a patient rating a generic health
concept was questioned when it did not agree with what was in the record
or with what the provider said. The logic of validity has since been turned
around. We are now entering an era in which the same findings are accepted
as evidence for the necessity of including patient assessments as part of the
evaluation process. The record and provider judgments are not valid prox-
ies for patient ratings of functioning, well-being, or other aspects of the
quality of life.
We should not always expect assessments of different health components
or clinical versus generic measures to agree, and often they do not. One
example comes from a study of the effects of antihypertensive therapy on
quality of life (9~. Therapies shown to be equally efficacious in terms of
medical efficacy (i.e., blood pressure control) had significantly different
quality of life profiles. In other words, it is possible to work with a patient
in therapy to achieve a better quality of life outcome without compromising
biologic function. There is also evidence accumulating that shows that
OCR for page 110
110
EFFECTIVENESS AND OUTCOMES IN HEALTlI CARE
differences in biologic function often have quality of life implications. These
two concepts are distinct; they are affected by different processes and they
interact with each other. To understand patient health outcomes, different
health components need to be measured and interpreted separately and in
combination (the latter when trade-offs are involved).
The greatest progress is going to occur, not by substituting one measurement
or assessment strategy for another, but by mastering them in concert. We
should not underestimate the power of a data base that includes clinical
measurements familiar to medical providers, measurements that they believe
in because they have clinical validity, in parallel with other measures, such
as measures of generic health concepts, not typically linked with such measures
in clinical practice or research. This is the most powerful strategy for
analyzing and understanding outcomes and for diffusing recent advances in
methods for assessing patient outcomes.
A MINIMUM SET OF GENERIC HEALTH CONCEPTS
I would like to take this opportunity to recommend what a minimum set
of generic health concepts might look like. At the risk of oversimplifying
the past 40 years of health assessment research, I think most health measures
can be classified into one of three major categories: functional status, well-
being, and general health perceptions. I have defined these categories elsewhere
and have illustrated them with sample questionnaire items from widely used
measures (10~. Functional status, which includes disability assessment,
refers to behavioral dysfunctions due to health problems. It is the concrete,
observable, tangible, and objective category of health measures. Measures
in this category use a standard external to the individual, such as usual role
activity, walking at a certain rate, or customary self-care behaviors. This is
the functional status axis in a multidimensional conceptualization of health.
It is the concept that has been preferred and best understood until now.
There are a number of well-developed measures of functioning available.
Interestingly, almost completely orthogonal to the functional status axis
is the well-being axis, which includes psychological distress, psychological
well-being, and life satisfaction. In most populations we observe all levels
of each of these axes at all levels of the other. In fact, in most populations,
the correlation between them is only 0.20 or less (assuming confounding of
measures across axes has been removed). The implication is that we cannot
know how people feel by observing what they are doing. Consider two
people sitting on a fencepost, for example. One may be experiencing a lot
of pain and may have difficulty just sitting there. The other person may sit
in ecstasy. In order to know, we have to ask them. It used to be thought
that well-being could not be measured reliably. We have learned that quite
reliable scores for this continuum can be obtained and that they add a
OCR for page 111
COLLECTION OF PRIMARY DATA
111
completely different perspective to that gained from functional status as
sessment.
Finally, there is a third axis, which cuts across the other two and brings
still another perspective beyond both of the other two axes; that is the
category of general health perceptions. It includes measures that are per-
sonal evaluations of health, based on whatever health means to the respon-
dent. This category of measures brings each person's own health values to
the equation. He might be a mental-health-oriented person or a physical-
health-oriented person. Health perceptions represent the third axis or category
that I would recommend for inclusion as a minimum standard for generic
health measures.
THE MEDICAL OUTCOMES STUDY
A hallmark of the Medical Outcomes Study (MOS) is its reliance on a
broad array of outcome measures, including parallel assessments of disease-
specific clinical endpoints traditionally measured by clinicians (biologic
functioning in Figure 1) as well as generic measures of functional status,
well-being, and satisfaction with health care as reported by patients. This
more encompassing assessment of outcome increases the likelihood of detecting
the consequences to patients of policies that modify the structure of the
health care system or the process of care. Measuring disease-specific end
points, as well as a common set of generic health outcomes for various
conditions, will also contribute a new data base that will allow physicians to
inform patients about the trade-offs involved in different treatment.
I am focusing here on health outcomes. Patients should also be involved
in assessing the quality of the medical care process (11~. I am going to give
you a brief summary of some of our experiences to date in the MOS (12~.
One of our intentions in the MOS was to test the feasibility of implementing
the same primary data collection system in very diverse systems of care for
purposes of monitoring the results of that care over time. By design, we
included very different health care settings and very different patient popu-
lations.
We sampled different health care settings to vary the structure and pro-
cess, and we are measuring variations in the outcomes of care. Structural
features of care include, for example, whether the provider is an HMO, an
insurance plan, a subspecialist, or a more generally trained physician. These
traditionally stable attributes of the health care system are now among the
many tools of cost containment. People are experimenting with such structures
in efforts to reduce medical expenditures. In the MOS we are looking at
how structural differences affect the process of care in the two major categories,
technical process and interpersonal process (12~.
Sponsors of the MOS are undoubtedly interested in whether the expenditure
OCR for page 112
112
EFFECTIVENESS AND OUTCOMES IN HEALTH CARE
for the study, which is approaching $12 million, can also be justified in
terms of addressing whether different ways of organizing and financing
health care affect patient outcomes. What is the best way to organize,
finance, and deliver care? If you have hypertension, for example, does it
matter whether you are treated by a cardiologist or a family practitioner?
Does it matter whether depressive disorders are detected and treated (13~?
Again, one of our primary interests has been to work towards advancing
the state-of-the-art in outcome assessment methods. One lesson we have
learned is that it is feasible to create the new data base defined above and to
add to it routinely on a rather large scale in very diverse health care settings.
MOS analyses in progress will be quite informative about the more cost-
effective ways of creating such a database and which variables are most
important for what kinds of analyses. Before commenting on some of the
MOS lessons to date, let me discuss briefly some study design features.
Additional details are given elsewhere (12, and in some references cited
there).
The study was done in three sites. At each site we sampled physicians
and patients from three different kinds of organizations: traditional pre-
paid group practice form of health maintenance organizations, multispecialty
groups, and solo practices. From the latter two we sampled both fee-for-
service and prepaid patients treated by the same physicians. The result is
five "systems of care" that differ in organization and financing. We sampled
523 physicians trained in family practice, general internal medicine, endo-
crinology, cardiology, and psychiatry. Other mental health providers were
also sampled (13~. These are the specialties that treat the MOS tracer
conditions: hypertension, diabetes, heart disease, and depressive disorders.
The conditions were chosen primarily because they are prevalent, costly,
treatable with variations in practice style, and have an impact on the outcomes
of interest.
We looked at adult patients who were seen during a nine-day period in
these physicians' offices. We gathered screening data from both the patient
and the doctor at that time. We then took approximately a 10 percent
random subsample of those patients who had one or more of our chronic
tracer conditions. As of October 1988 we had followed these patients for
over two years. We hope to continue to follow them. We look at the care
they receive and we monitor transitions in their clinical status as well as
functional status and well-being, the latter at six-month intervals.
FEASIBILITY AND COST
Much of the expense of the MOS was attributable to the cost of identifying
and recruiting providers and patients, designing instruments and data collection
methods, and making sure that they would work. We spent nearly two
OCR for page 113
COLLECTION OF PRIMARY DATA
113
thirds of the study's total funds before the panel started. Measuring and
analyzing patient outcomes over time has not been the major expense.
Clearly what one concludes about what things cost depends upon what is
charged to a given cost category. If primary data collection is considered a
marginal cost in a framework for monitoring patient outcomes, the marginal
cost would be relatively small. If other costs are charged to this category,
including for example, defining what diabetes is, determining whether somebody
has it, determining how to sample doctors and patients, dealing with differences
in patient case mix, the cost per patient followed is much higher. Again,
once we had identified both patients and providers and had measured differences
in patient case mix, the cost of following patients over time was relatively
small. With state-of-the-art short forms and processing methods, the cost to
process a patient health assessment in a doctor's office is less than the least
expensive lab test.
There has been at least one other lesson about feasibility. We oversampled
Medicare patients because of the policy relevance of that group. We hypothesized
that they would be treated differently in different practice settings as a
result of oversampling Medicare. The median age in our longitudinal sample
was about 60. All of them had one or more chronic conditions. This
population is very sick relative to, for example, the population we followed
in the Health Insurance Experiment where we also used self-administered
questionnaires as a primary data collection tool in comparing outcomes
across different systems of care (14~.
How well did these methods work in the MOS? When we conducted our
two-year follow-up survey two years after enrollment, we again used a self-
administered survey, a booklet with about 250 questionnaire items. Our
response rate was over 80 percent for those who self-administered the full-
length questionnaire. We used telephone interviews and the MOS short
form for those who did not and raised the overall response rate to over 90
percent of those who were contacted and still alive. Whereas dollars can be
saved early on by using self-administration, you must be willing to spend
some of these savings for follow-up (e.g., by telephone) for people who do
not complete a self-administered form. Nearly 70 percent of the panel has
completed all surveys during the course of the study. The typical completed
questionnaire has 1 percent or fewer of the items missing. Thus, it is
possible to get high completion rates for long questionnaires even in a
relatively elderly and relatively sick population. This experience has made
me more enthusiastic than I had been after the Health Insurance Experiment
(and I was enthusiastic then) about the feasibility of standardized surveys
and self-administration as a primary data collection strategy for monitoring
patient outcomes.
MOS providers were roughly evenly divided between group and solo
practice settings. We found that it is more efficient to monitor outcomes in
OCR for page 114
114
EFFECTIVENESS AND OUTCOMES IN HEALTH CARE
group practices. Solo practitioners do not have equivalent support person-
nel. Thus our sampling rates were higher in groups (12~. Our completion
rates were roughly the same across practice settings once people had enrolled
in the study. Thus, with centralized data collection and standardized forms
and methods, the completeness and quality of the database need not vary by
practice setting. This leads to the notion of centralized health assessment
laboratories. With support from the John A. Hartford Foundation of New
York, we are now developing and testing this concept.
Again, the MOS is a methodological study, and with support from the
Henry J. Kaiser Family Foundation, my colleagues and I are now comparing
the briefest forms of measurement, e.g., the best single-item measure, with
longer but still short multi-item scales, and with full-length research versions
of these scales. Our question is how well do shorter measures work relative
to much longer measures used in research? Not surprisingly, preliminary
findings indicate that longer measures do better. However, the question
should be: "Do shorter measures do well enough?" The answer is very
important because the most psychometrically elegant instrument is useless
if it is impractical to use. Thus, we should be very interested in how briefer
measures do in these comparisons.
STANDARDS FOR EVALUATING MEASURES
On what basis should measures be compared and how do we construct
them in the first place? A number of things are wrong with what we
traditionally do in psychometrics. Take reliability as an example. Reliability
is important, but we learned quickly that although reliability is a prerequisite,
other attributes of a score (scale) are equally or more important. In comparing
scales, most important are tests that most closely approximate the intended
use of the measure. Unfortunately, traditional reliability and validity coef-
ficients have little or no relationship to actual applications of measures.
Some attributes of measures typically ignored include things like how
many different scores are possible. An enumeration system that puts people
into one of four levels or categories with a reliability of 0.90 is not as
valuable as is one that puts people into 10 categories with the same reliability.
This particular attribute of measurement may not prove critical in a cross-
sectional analysis comparing things as different as Volkswagens and trucks.
The latter is analogous to comparing disease groups that differ a lot. Most
measures do well in that kind of comparison. When we start measuring
change in health over time, however, this issue becomes crucial. How much
change can occur within a given health category before the person changes
to the next category? The number of levels of measurement is a very
important attribute or measure. This attribute is almost never discussed in
books on health assessment.
OCR for page 115
COLLECTION OF PRIMARY DATa
115
Another important and related issue is simply how many people get the
lowest or the highest possible score for a given measure. If 90 percent of
the people in a long-term care facility have the worst possible score before
a cost-containment strategy is implemented, you do not have precision for
showing a worsening in their condition as a result. If 80 percent of the
people earn the highest possible score on a physical health index (as they
did in the Health Insurance Experiment), and we randomly assign them to
free care, we do not have much chance of determining, using that particular
measure, whether there is any benefit of free care. Fortunately, this was not
the only measure used in that experiment (151. I suggest that, when measures
are published, we routinely report how many people get the lowest and the
highest possible scores when measures are published as well as the number
of scores possible, in addition to reliability coefficients.
The same logic should apply to results regarding validity. It is extremely
important that the kind of analysis used to judge the validity of a measure
approximate as closely as possible the intended use of the measure in medical
practice, a clinical trial, or a policy study. Much published evidence bears
little or no relationship to most intended applications. One example of what
I mean is the issue of whether a given questionnaire is sensitive to the
extent and nature of differences in functional status and well-being across
groups of patients with different chronic conditions. My colleagues and I
recently reported an example of such comparisons using the 20-item MOS
short-form survey (16,17~. Figure 2 presents examples of profiles for patients
with four different chronic conditions at a point in time when the study began.
Each profile is expressed as standard score deviations from the averages
for well patients (represented by the horizontal dotted line). The first three
data points (columns) for each disease are defined by functional status scales,
the last three by well-being scales. We have connected the points across
scales for each disease to help identify a particular disease profile. These
are scored so that the lower the profile on the scales the worse the profile.
Figure 2 generally confirms clinical wisdom about the impact of these
diseases. Not surprisingly, patients with hypertension (the top profile in
Figure 2) function no differently than well patients. The only significant
decrement was their score for health perceptions. Patients with hypertension
tend to believe their health is worse. Arthritis has the most pain. Physical,
role, and social functioning is poor for survivors of myocardial infarction
(MI) and tends to be as bad as, if not worse than, any of the nine chronic
conditions we have studied to date.
One lesson from this is that, on average, the patient point of view is
valid. Further, even very brief measures can be used to measure differences
in health across groups of patients. The questionnaire used to estimate
scores in Figure 2 was administered to about 12,000 patients while waiting
in a doctor's office, in about three and a half minutes each.
OCR for page 116
116
EFFECTIVENESS A1VD OUTCOMES IN HEALTH CARE
0.20
-O. 20
a'
a
-0.40
-0.60
-0.80
- 1 .00
Hypertension
Arthritis
,~
Ml /
Physical Role Social Mental Health Bodily
Functioning Functioning Functioning Health Perceptions Pain
FIGURE 2 Health Profiles for Patients with Four Conditions. Dotted line indicates
patients win no chronic conditions; GI, gastrointestinal disorder; MI, myocardial infarction.
Relative to available full-length research instruments, including our own,
this short-form has one-fifth to one-tenth the number of questionnaire items.
Yet it produced a pattern of results that make sense from a clinical point of
view. One surprising finding (not shown in Figure 2) is the very low
profile of scores for patients with depression, including those with a psychiatric
diagnosis and those suffering with symptoms of depression. They scored
very low on these scales relative to other chronic diseases, suggesting that
the burden of depression may have been underestimated to date (13~.1
Of course, other kinds of tests are necessary before conclusions are drawn
about candidate measures. How well does a questionnaire distinguish differences
in functional status and well-being across groups differing in severity within
a diagnostic category? Research in progress within the MOS is encouraging
in this regard. For example, in preliminary analyses of MOS data, average
functional status scores for diabetics differing in severity (e.g., with or
without renal failure) show ordinal consistency in relation to clinical severity.
Thus, these measures may be sensitive to differences in severity within a
. .
c lagnostlc group.
This kind of analysis does not prove that if treatment moved people from
severity level five to level three, we would see a corresponding change in
functioning. This example is based on a cross-sectional analysis. We are
currently linking measures of actual change in disease severity over time
with measures of change in functional status and well-being. Again, pre-
liminary results are encouraging.
iFor another description of the MOS and results pertaining to depression, see
Chapter 19 of this volume (20~.
OCR for page 117
COLLECTION OF PRIMARY DATA
117
USE OF HEALTH SURVEY DATA
The potential of generic functional status and well-being scales, even
short forms, to be used successfully across a wide range of purposes is
illustrated by the vitality scale used in the MOS, a 4-item scale that takes
about one minute to complete. It measures a continuum of energy versus
fatigue. Its history, which is documented in part elsewhere (17), includes
successful use in a population health survey about 15 years ago (18), the
Health Insurance Experiment (17), a more recent clinical trial comparing
antihypertensive therapies (9), and the MOS (19~. Its track record defies
the notion that completely different measures are needed for different applications.
This one-minute vitality scale, for example, has been used successfully in
describing the young and the old in the U.S. population, the sick and the
well, and in measuring outcomes across homogeneous groups of patients
receiving different treatments in a randomized trial.
With my recent move to the New England Medical Center, I have had
occasion to contrast my own background and training with the needs of the
next era of health assessment. I was trained in measurement theory and
methods and for the prior 15 years I had been in a full-time research setting
where we evaluated measures in traditional psychometric terms and used
them for purposes of research. Now I focus much more on the information
needs of health care delivery organizations and look at measurement and
the use of data from a different perspective. Two years in a research
institute in a health care delivery organization have convinced me that a
new data base with information about patient outcomes is not the solution
to the problem, it is just the next step. The challenge of implementing
outcomes management or an effectiveness initiative is not a problem of
measurement, and it is certainly not a problem of assessing outcomes from
the patients' view. The methodological problems include determining (1) a
coherent sampling strategy; (2) techniques for case-mix measurement and
statistical control; (3) a meaningful schedule of assessments for different
diagnostic groups; (4) analytic strategies for displaying results in a meaningful
way; and (5) recognizing that conclusions are sensitive to these and other
choices. Finally, the real challenge is the creation of a decision-making
process capable of using data about patient outcomes. Indeed, the collection
of outcomes data from patients is one of the simplest steps ahead.
ACKNOWLEDGEMENTS
The MOS has been sponsored by grants from the Henry J. Kaiser Family
Foundation, the Robert Wood Johnson Foundation, the John A. Hartford
Foundation, the Pew Charitable Trusts, the Agency for Health Care Policy
and Research, the National Institute on Aging, and the National Institute of
OCR for page 118
118
EFFECTIVENESS AND OUTCOMES IN HEALTH CARE
Mental Health, and by The RAND Corporation and the New England Medi-
cal Center from their own research funds.
The author gratefully acknowledges Albert P. Williams, at The RAND
Corporation and especially the MOS staff and consultants, including Sharon
Arnold, Sandra H. Berry, M. Audrey Burnam, Maureen Carney, Allyson
Ross Davis, Sheldon Greenfield, Ron D. Hays, Elizabeth McGlynn, Eugene
C. Nelson, Lynn Ordway, Judith Perlman, Edward B. Perrin, William Rogers,
Cathy Sherbourne, Anita L. Stewart, Alvin R. Tarlov, Kenneth B. Wells,
and Michael Zubkoff, and the secretarial and administrative support of Kathy
Clark.
REFERENCES
1. McDermott, W. Absence of Indicators of the Influence of Physicians on a
Society's Health. American Journal of Medicine 70:833-843, 1981.
2. Cluff, L.E. Chronic Disease, Function and the Quality of Care. Journal of Chronic
Diseases 34:299-304, 1981.
3. Tarlov, A.R. Shattuck Lecture. The Increasing Supply of Physicians, the
Changing Structure of the Health-Services System, and the Future Practice of Medi-
cine. New England Journal of Medicine 308:1235-1241, 1983.
4. Schroeder, S.A. Outcome Assessment 70 Years Later: Are We Ready? New
England Journal of Medicine 216:160-162, 1987.
5. Ellwood, P.M. Shattuck Lecture. Outcomes Management: A Technology of
Patient Experience. New England Journal of Medicine 318:1549-1556,1988.
6. Lohr, K.N. and Ware, J.E. Advances in Health Assessment, Special Issue.
Journal of Chronic Diseases 40:Supplement l:lS-193S, 1987.
7. Lohr, K.N. Advances in Health Status Assessment: Overview of the Confer-
ence. Medical Care 27~3) Supplement:Sl-Sll, 1989.
8. Roper, W.L., Winkenwerder, W., Hackbarth, G.M., et al. Effectiveness in
Health Care: An Initiative to Evaluate and Improve Medical Practice. New England
Journal of Medicine 319:1197-1202, 1988.
9. Croog, S.H., Levine, S.M., Testa, M.L., et al. The Effects of Antihypertensive
Therapy on Quality of Life. New England Journal of Medicine 314:1657-1664, 1986.
10. Ware, J.E. Standards for Validating Health Measures: Definition and Con-
text. Journal of Chronic Diseases 40:473 -480, 1987.
11. Davies, A.R. and Ware, J.E. Involving Consumers in Quality of Care
Assessment: Do They Provide Valid Information? Health A~airs 7:33-48, 1988.
12. Tarlov, A.R., Ware, J.E., Greenfield, S., et al. The Medical Outcomes
Study: An Application of Methods for Monitoring the Results of Medical Care.
Journal of the American Medical Association 262:925-930, 1989.
13. Wells, K.B., Stewart, A., Hays, R.D., et al. The Functioning and Well-Being
of Depressed Patients: Results from ~e Medical Outcomes Study. Journal of the American
Medical Association 262:914-919, 1989.
14. Ware, J.E., Brook, R.H., Rogers, W.H., et al. Comparison of Health Out-
comes at a Health Maintenance Organization with Those of Fee-for-Service Care.
Lancet i(8488~: 1017-1022, 1986.
OCR for page 119
COLLECTION OF PRIMARY DATA
119
15. Brook, R.H., Ware, J.E., Rogers, W.R., et al. Does Free Care Improve
Adults' Health? Results from a Randomized Controlled Trial. New England Jour-
nal of Med icine 309: 1426-1434, 1983.
16. Stewart, A.L., Greenfield, S., Hays, R.D., et al. Functional Status and Well-
Being of Patients with Chronic Conditions: Results from the Medical Outcomes
Study. Journal of the American Medical Association 262:907-913, 1989.
17. Ware, J.E., Johnston, S.A., Brook, R.H., et al. Conceptualization and Mea-
surement of Health for Adults in the Health Insurance Study: Volume III, Mental
Health. R-1987/3-HEW. Santa Monica, Calif.: RAND Corp. 1979.
18. Dupuy, H.J. The Psychological Section of the Current Health and Nutrition
Examination Survey. U.S. Dept. of Health, Education, & Welfare Publication. No.
(HRA). 74-1214. Washington, D.C.: Government Printing Office, 1974.
19. Stewart, A.L. and Ware, J.E., eds. Measuring Functional Status and Well-
Being: The Medical Outcomes Study Approach. Forthcoming.
20. Burnam, M. A. Studying Outcomes for Patients with Depression: Initial
Findings from the Medical Outcomes Study. Pp 159-168 in Electiveness and Out-
comes in Health Care. Heithoff, K.A. and Lohr, K.N., eds. Washington, D.C.:
National Academy Press, 1990.
Representative terms from entire chapter:
primary data
