Health Outcomes for Older People: Questions for the Coming Decade (1996)

To improve the scope and quality of health outcomes research, several basic infrastructure and methodologic or analytic issues need to be addressed. As noted earlier, although the field of health outcomes research has seen a great deal of progress over the past few years, more work is still needed. Greater sophistication in data analysis and interpretation has given researchers, practitioners, and policymakers new insight, understanding, and ways to improve health care, but it has also raised more questions.

The sense of the committee is that significant breakthroughs on a variety of these questions are possible over the next 10 years. However, the committee has concerns about the current health outcomes research infrastructure—the actual workforce of health outcomes researchers and the methodologic and analytic tools used in research. The committee has made five recommendations aimed at strengthening that infrastructure. They relate to:

• Workforce Issues

• Conduct of Research

• Data Quality

• Data Management Systems

• Methodological and Analytic Issues

• The successful implementation of an outcomes research agenda depends not only on the persuasive identification of important issues and the securing of funding but also on the availability of well-trained and committed researchers.

• Although the private sector will provide some support, governmental and foundation aid is needed to ensure that the health outcomes research workforce is adequate in terms of overall size, appropriateness of training, and sensitivity to issues involving older individuals.

The health outcomes research workforce has two distinct components. It includes:

1. investigators and researchers who originate, design, supervise, and report basic and applied research; and

2. individuals who analyze health outcomes information and apply certain tools of health outcomes research in management, policy, and service delivery settings.

A recent IOM committee (IOM, 1995) concluded that well-trained health services researchers with practical experience in health care organizations (e.g., integrated health care systems, insurance companies) and in managing research units appear to be in short supply. A survey of employers of health services researchers, commissioned as part of that study, found that employers stated that they have the greatest difficulty in recruiting researchers with expertise in the area of “outcomes/health status measurement.” Employers also indicated that within the next five years they expected to recruit more researchers in this area than in any other. However, although employers and private health plans do cover some of the costs and arrange or provide some of the care for older individuals, their primary focus is on younger age groups. Even taken as a whole, these efforts by private organizations are unlikely to substitute for more than a portion of government- and foundation-supported research and training in magnitude, coherence, scope, or concern for long-term consequences.

The Agency for Health Care Policy and Research provides some support of education and training through dissertation grants and National Research Service Awards. Other government agencies, including some components of the

National Institutes of Health, the Bureau of Health Professions, the Department of Defense, and the Department of Veterans Affairs, support varying amounts of research training that may include some attention to outcomes research. Some foundations, including the Pew Charitable Trusts and the Robert Wood Johnson Foundation, have supported education in health policy and outcomes research. More often, support from these sources comes indirectly through grants for research at academic institutions that hire investigators in training to assist in the research and learn skills on the job. Formal training support from private industry is sometimes available. For example, some private organizations sponsor students on semester or year-long internships to expose them to how health outcomes research is carried out in the for-profit environment. However, such training has typically been given to students in specific areas, such as pharma-coeconomics.

Because of their experience, expertise, and leadership positions, research training support for clinicians, especially geriatricians, will be useful in advancing the field of health outcomes research for older people. Including training in outcomes assessment as an integral part of practitioners’ (physicians, nurses, dentists, etc.) initial and continuing education will also be beneficial.

• Fellowships, pre- and postdoctoral traineeships, dissertation grants, grants to institutions offering health outcomes research training, loans, and other financial supports for aspiring health outcomes researchers and educational institutions.

• Health outcomes research training programs for health care professionals, especially geriatricians.

• All researchers must hold themselves to a high ethical standard of quality and integrity to maintain public confidence in the validity and integrity of their work.

• Efforts are needed to ensure that the results of health outcomes research are made available to the broader scientific community and the population at large.

The committee believes the field of health outcomes research would benefit from a statement of “principles” regarding the conduct of research. As is true in many other fields of research, concerns about the validity of the research and integrity of the researcher are bound to arise. The relationship between industry funders and independent researchers is a particular concern because of fears that industry sponsors might censor negative research results that could damage their business interests. A similar call for a professional code of ethics was issued at the 1996 annual conference of the Association for Pharmacoeconomics and Outcomes Research (Medical Outcomes and Guidelines Alert, 1996).

Ethical codes have been developed by some research organizations (e.g., the RAND Corporation and the Mayo Clinic) and other research professions. Epidemiologists, for example, have a code spelling out the researcher’s obligations to the subjects of research, society, funders and employers, and colleagues. Issues covered include the need to protect the welfare of research subjects, obtain informed consent, and maintain confidential information. More generally, researchers should avoid conflicts of interest and partiality to potential sponsors, promote scrutiny of their work by describing their methods and reporting their results fully enough to limit misinterpretation, report results regardless of whether they are favorable to the sponsor’s interests, take care that they choose methods appropriate to the research question, protect privileged information, and refrain from premature publication of findings. These obligations and principles should be clearly specified to funders and employers.

Resolving ethical issues will not be easy, even with the proposed code. For example, researchers at the Mayo Clinic conducted a retrospective study through the use of chart review to examine the outcomes and complication rates of silicone breast implants. They found no complications associated with the implants. Shortly after the study findings were released, attorneys for breast

implant patients asked that the charts be made public so they could contest individual cases. The request has been turned down. The example highlights the conflicting demands placed on researchers. The study provided the public and other professional colleagues with important epidemiologic data on an issue with significant health and monetary implications. However, the researchers and their institution also had to protect patient confidentiality and limit the institution’s liability.

A “statement of principles” that governs the conduct of health outcomes researchers needs to be developed and widely disseminated to health outcomes researchers and to sponsors and users of such research.

• Policymakers, consumers, managers, and others need confidence in the soundness of health outcomes information and analyses.

• Data quality varies enormously depending on the source and collection method used.

• These variations threaten the validity, reliability, and comparability of outcomes research. An understanding of data limitations and measurement problems can help users of research interpret findings with due regard for their limitations.

Although the quality of health outcomes research data is important regardless of its uses, the increasing use of outcomes data to compare health care institutions, practitioners, health plans, and services has focused much more attention on the quality of these data and the deficiencies in outcomes comparisons. Particular shortcomings relate to the sufficiency of the clinical measures of patient characteristics (including severity of illness and other risk factors), the accuracy of recording and coding patient and other data, poor identification of older persons living in rural and underserved urban areas, and the representativeness of the patient population. Because older individuals are underrepresented in some databases, attention to the generalizability of findings is also important.

Similarly, the increased interest in patient satisfaction measures as indicators of quality has prompted concerns about the improper use of well-validated measurement instruments and the use of internally developed instruments whose reliability and validity have not been tested. This threat may be particularly acute for studies undertaken and publicized by the entity that is being evaluated, especially when methods and findings are selectively reported. This has been an issue in some nursing facilities that tout internal surveys indicating high rates of satisfaction among their residents for marketing purposes, even when they also have been found to have serious and life-threatening deficiencies by state regulators.

The use of large administrative databases is very attractive for researchers because of its convenience (e.g., no primary data collection is needed); but growing concern about their limitations (e.g., the level of clinical detail available) is, on the one hand, directing health outcomes researchers to other data sources (in particular, new kinds of clinical trials) and, on the other hand, en-

couraging greater efforts to improve the accuracy and completeness of administrative databases and to develop techniques to compensate for their deficiencies.

Identify and categorize problems with the reliability, validity, and availability of data used in health outcomes research for older individuals.

• Many different systems for data management that support outcomes research have been developed.

• Each of these systems has advantages and disadvantages. Some of these systems are very expensive. Users need objective information on which to base their purchasing decisions.

The emergence of computerized data and information systems has begun to revolutionize the way health outcomes research is conducted and its results disseminated. Advanced technological applications and tools along with powerful computer software and hardware offer the possibility of conducting health outcomes research that is extremely sophisticated. For example, computerized patient records allow for gathering and utilizing more and better self-reported health-related quality-of-life information on social, economic, and spiritual issues. This type of data can then be used routinely or selectively in clinical care to monitor individual outcomes as well as in large-scale epidemiological studies. Large, data-intensive, population-based studies of health outcomes can now be conducted with newly developed technologies for collecting, linking, modeling, and analyzing data.

Researchers and users alike are confronted with a seeming overabundance of technology to assist them. Some of this technology is being very heavily marketed. Yet very little work has examined—in an unbiased manner—the advantages and disadvantages of these different technologies. The committee recommends that a Consumers Report-type evaluation of these various systems for data management be performed that would compare and contrast their different features. Individual purchasers could then make a more informed and objective decision about the features and products they most desire.

Perform independent appraisals of systems for data management to support health outcomes research. This includes evaluations of:

1. specific tools and applications (e.g., voice recognition systems, bar codes, laser wands, reminder technology);

2. informatics, databases; and

3. basic telecommunications infrastructure.

• Although significant progress has been made in developing new types of measurements and in improving techniques of analyzing and reporting health outcomes, much more work remains.

• With the exception of governmental support, this type of research has traditionally been difficult to fund.

• Without continued development of research methods and techniques of outcomes assessment that are in the public domain, the field of health outcomes research will be substantially impaired.

Most of the issues related to methodology and analysis have been mentioned earlier in this report. However, the committee believes it is important to highlight some of the most needed and complex methodological and analytic issues in a separate recommendation. To date, the Agency for Health Care Policy and Research (AHCPR) has been one of the most active supporters of such methodological and analytic advances. The committee is concerned that, because of recent and threatened cuts to AHCPR, this type of research will be even more difficult to fund.

• Develop ways to integrate data elements across settings and over time.

• Improve methods of risk adjustment and severity measurement. These measures need to take into account patient, social, and system characteristics; they may also need to account for past treatment and patient history.

• Develop better statistical models for controlling for selection bias.

• Compare the advantages and disadvantages of different study designs (e.g., retrospective, prospective, randomized, staged qualitative).

• Determine how different scales or measures should be calibrated to increase their comparability.

• Compare the advantages and disadvantages of different approaches to gathering outcomes information (e.g., patient self-report, computer-assisted interview, chart review, review of administrative data, use of proxy information, observation of nonverbal clues, ethnographic observation). Increase the comparability of data using different approaches.

• Expand techniques to obtain information from people who are cognitively impaired.

• Determine when outcomes should be measured. Establish a standard definition of an “episode” of care. Examine how time of measurement affects the outcome result (e.g., measuring satisfaction each time services are provided and averaging them versus measuring it once a year).

• Develop better statistical models for determining selection bias.

• Study how measures of utility and patients’ preferences relate to summary measures of health-related quality of life.

IOM (Institute of Medicine). Health Services Research: Workforce and Educational Issues. M.J.Field, R.E.Tranquada, and J.C.Feasley, eds. Washington, D.C.: National Academy Press, 1995.

Medical Outcomes and Guidelines Alert. At First APOR Meeting, Challenges and Controversies. 4(10):6–8, 1996.

This page intentionally left blank.