Click for next page ( 14


The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 13
Chapter 2 HEALTH SERVICES RE SEARCH DEFINED The term "health services research" is commonly used to ret er to a broad and heterogeneous set of activities, but there is no consensus as to its precise meaning. In a review of the literature, the committee f ound no def inition that both (1 ) provides criteria to i dentify studies as health services research and distinguish them from other types of inquiry and (2) subsumes the entire range of studies that are conventionally classif fed as health services research. f or such a def inition is two-f old. As a practical matter The need ~ the committee required an understanding of the characteristics of health services research in order to establish the scope of its study. Beyond this, because health services research has become an administrative category f or support of research and training, off icials in the federal gover-~- ment need a definition upon which to base research funding and related policy decisions and with which to devise an effective division of responsibilities for research among the several agencies involved in health care. This chapter reviews existing def initions, explains the one adopted by the committee, and discusses and illustrates various types and us es of health services res earch . Existing Def initions Most existing definitions describe characteristics of the field of health services research but do not specify the features of studies that distinguish health services research f rom other types of inquiry. Such - - - ~ - - - - - statements typically emphasize the variety ot disciplinary perspectives and methods employed in the f ield of health services research and note its broad substantive concerns and purposes. For instance, the often cited definition of the Panel on Health Services Research and Development of the President 's Science Advisory Committee characteriz es health services research and development as a "broad scientif ic field, the overall ob jective of which is to improve the provision of health services," [l] and illustrates its scope with lists of participating disciplines and "representative questions. " ~ 2 ~ 13

OCR for page 13
14 There are several problems with definitions of this sort. First, they imply that studies in health services research are inherently multidisciplinary and directly focused on specific programmatic or policy questions. While many studies do in fact incorporate theoretical perspectives of several disciplines and are aimed at solving particular problems, it is equally true that others that should be classified as health services research employ the conceptual frameworks of particular disciplines or address problems that have no direct implications for program or policy decisions. Second, to define the scope of a field by listing its principal specific concerns both presumes that issues endure and risks omission of emerging topics. Priorities for health services research change in response to new information and changing definitions of problems in health care. Therefore, illustrations of the core concerns of the field of health services research based on lists of current issues are likely to become outdated. Finally, definitions and critiques of the field frequently use the terms "health services research and development" and "health services research" interchangeably. As the former connotes a strategy of research coupled with systematic interventions in the delivery system, it should not be confused with research, which may or may not be focused on planned or conscious changes. "Health Services Research" Health services research is inquiry to produce knowledge about the structure, processes or effects of personal health services . A study is classified as health services research if it satisfies two criteria: It deals with some features of the structure, processes, or effects of personal health services. At least one of the features is related to a conceptual framework other than that of contemporary applied biomedical science.* *This definition is similar to that proposed by the National Research Council Committee on National Needs for Biomedical Research Personnel.~3] The purposes of the IOM and NRC studies differed and the definitions of health services research were developed to meet the specific study objectives. In the NRC study a definition was employed that more directly related to the health policy aspects of health services.

OCR for page 13
15 The first criterion defines the core focus of health services research as a personal health service, which the committee understands to be a transaction between a provider of health services and a client for the purpose of promoting the health of the client. Providers include licensed health care professionals and ancillary personnel as well as "marginal" and lay practitioners. Hence, health services research deals with both formal and informal systems of health care. In this definition, health services are understood to include direct applica- tions of medical knowledge and technologies and the provision of advice and assurance. In addition, health services encompass the full range of personal health care, including dental and mental health services. As concepts of health and health services change, the scope of health services research will change accordingly. The second criterion places in the category of health services research all studies of personal health services that focus on at least one feature of their structure, processes, or effects defined in terms of some conceptual framework other than that of contemporary applied biomedical science.* That framework views the human organism in terms of its anatomical structure and physiological processes, and identifies, classifies, and explains diseases, which usually are defined as struc- tural malformations, chemical lesions, or behavioral abnormalities.~4] These two criteria permit one to distinguish the principal emphases of health services research from other related types of inquiry. The committee emphasizes, however, that the boundaries of health services research are neither fixed nor sharply distinct. Similarly, research in this area draws upon concepts and methods from various fields of inquiry, frequently attempting to integrate their knowledge and tech- niques and to investigate their implications for the organization, processes, and effects of personal health services. Research on Environmental Health Services The first criterion differentiates the principal focus of health services research from that of research on environmental health services, which concentrates on services that attempt to promote the health of popula- tions by treating their environments rather than by treating specific individuals. Knowledge from studies of environmental causes of health problems is *"Basic" biomedical science is concerned with development of knowledge about the fundamental life process.

OCR for page 13
16 obviously important in health services research, because it gives insight into the kinds of health problems for which people seek care and the types of services that must be provided by the personal health services industry. Furthermore, research aimed at assessing the rela- tive effects of environmental factors and personal health services on the health of populations combines information from both fields of study. Studies of effects of fluoridating water supplies on the incidence of dental caries, for example, would not be thought of as health services research. However, investigations of the relative costs or effective- ness of reducing caries by fluoridating community water supplies versus applying topical fluorides to individuals would be. Behavioral Research A portion of the field of health-related behavioral research is con- cerned with understanding factors influencing individuals' life styles that, in turn, are associated with their health. Research in this area draws upon knowledge from epidemiological studies that identify behavioral determinants of illness, such as diet and smoking habits, and examines their social and psychological components. As do studies on environmental health services, behavioral research often overlaps with health services research. Behavioral studies of the determinants of smoking behavior, for instance, are not health services research, according to the committee's criteria. However, behavioral and health services research interests come together in studies of effects of life styles on the use of personal health services and in research on the effects of personal health services on individual's health-related habits. Biomedical Research The second criterion differentiates health services research from contemporary applied biomedical research. Within the conceptual frame- work of biomedical research, no explicit attention is given to matters other than therapeutic interventions and disease processes. Indeed, a major assumption of the randomized clinical trial is that all factors that might both influence an organism and be associated with the inter- vention under investigation are controlled by randomization. To the extent that this assumption is tenable, the randomized clinical trial is able to assess the effects of interventions on an individual's disease free from the disturbing influences of extraneous matters, such as the characteristics of physicians and hospitals. Although the randomized clinical trial is considered the ideal method for assessing the safety and efficacy of therapeutic interventions,

OCR for page 13
17 in practice, relatively few are carried out. In consequence, most information about the efficacy of medical procedures is from studies done in practice settings in which the conditions of the randomized clinical trial cannot be assumed. Nevertheless, these studies are not considered health services research unless they take explicit account of factors other than interventions and outcomes conceptualized in terms of the framework of biomedical science. Studies of the rela- tive effectiveness of coronary bypass surgery versus drug therapy on reducing chest pain, for example, are extensions of biomedical research. Studies that examine characteristics of hospitals or physicians that affect differential outcomes of surgery fall into the realm of health services research. The distinction between biomedical and health services research becomes somewhat blurred when the outcomes of therapeutic interventions are conceptualized in terms that are not strictly medical. Studies in this area seek to evaluate technologies in terms of their efficacy, safety, and implications for the organization and costs of care. Evaluations of a surgical procedure might, for example, take into consideration lengths of recovery time required by patients. Such studies of stays in hospitals following surgery are in the realm of clinical (rather than strictly biomedical) research. Because such questions are more closely aligned with other problems of interest to those who do health services research than with the principal concerns of biomedical research, the committee is inclined to make a strict interpretation of its second criterion and to consider such clinically-oriented studies instances of health services researche For the example given, one need only change the research question to a study of differences in postsurgical lengths of stay of patients in acute-care hospitals versus others in extended care facilities to place the inquiry squarely in the domain of health services research.* When outcomes of medical interventions are defined in terms of costs of care, patient satisfaction, or other matters of interest to the social sciences, studies are clearly in the realm of health services research. Epidemiological Research Most contemporary research within the discipline of epidemiology falls outside the boundaries of health services research. Epidemiology is *The same reasoning would lead to classifying as health services research studies of effects of medical interventions on patients' functional or general health status.

OCR for page 13
18 generally viewed as the "study of the distribution and determinants of disease frequency in man"~5] in which explanatory factors are drawn principally from individuals' physical, biological, and social environ- ments and their life style and behavioral patterns. Epidemiological studies that include features of the structure or processes of personal health services among their explanatory factors are instances of health services research if these features satisfy the second criterion. Investigations of effects of populations' use of health services on their mortality rates are examples of this type of research. However, research on the effects of inoculations against smallpox on the incidence of the disease in populations would not be classified as health services research. Although such studies would meet the first criterion of dealing with a personal health service (i.e., having an inoculation), they would not satisfy the second, because the provision of an inoculation is a medical intervention. By contrast, studies employing epidemiological methods to assess the impacts of particular medical interventions on general health status or other outcomes that are not defined in strictly medical terms would be classified as health services research. Levels of Health Services Research Studies of health services may be categorized according to the four general levels of problems they address: clinical, institutional, systemic, or environmental. These levels are depicted in Figure 1 in order of increasing generality, proceeding from the core of studies of clinical practice to the most global level of research on relationships between characteristics of the health services system and events in the larger social, political, and economic environment that affect them. Clinically-Oriented Studies Clinically-oriented studies attempt to discover the characteristics of providers and patients and combinations of resources employed in practice settings that affect the processes and outcomes of care. Unlike applied biomedical research, which concentrates on developing procedures and testing their efficacy and safety under controlled conditions, clinically-oriented studies recognize that the effective- ness of health services is influenced by conditions in which they are provided and that criteria other than strictly medical considera- tions are relevant to outcomes. Accordingly, they deal explicitly with the circumstances of medical practice and take into considera- tion a broader range of outcome criteria, including patient satisfac- tion and the costs of care as well as the effectiveness of treatment.

OCR for page 13
19 FIGURE 1 LEVELS OF HEALTH SERVICES RESEARCH Health Problems Environmental Systemic / I Institutional \ I' ,~, Clinical \ it: Social, Political, Economic Institutions J /

OCR for page 13
20 Studies in this category usually concentrate on characteristics of providers or patients. They deal with such matters as the influences of physicians' ages, training, and work loads on the quality of their practice; the effects of using ancillary personnel or various computer-based techniques on the efficiency and complete- ness of medical history taking; and the cost-effectiveness of alterna- tive treatment modalities. Studies focusing on patients investigate, for instance, characteristics associated with adherence to medical regimens, use of particular preventive or diagnostic services, and lengths of hospital stays. Institutionally-Oriented Studies While institutionally-oriented studies share many of the concerns of clinically-oriented research, they are distinguished by their focus on organizational and administrative features of settings in which services are delivered. Research at this level examines, for instance, the productivity and quality of care rendered by physicians in solo practice compared with those working in group practices, differences in average lengths of stay in hospitals of varying sizes, ownership, and complexity, and the costs of custodial care in extended care facilities compared with those provided in patients' homes. Studies of populations focus on such matters as why families choose prepaid group practices and the determinants of utilization patterns of various providers of services. Systemic Studies System studies deal with features of the health services system that affect the inter-relationships among providers and health care institutions and the population's aggregate demand for health services. At this level, attention is given to the influences of financing mechanisms, regulatory programs, and other features of the system on such matters as the capital expenditures of hospitals, choices of specialties and practice sites by physicians, the development of group practices, and expenditures for various types of personal health services. Environmental Studies Environmental studies seek to understand the circumstances and events in the larger social, political, and economic contexts that shape the health services system and define its societal functions. These include research on the implications of tax policy for the flow of capital into the health services industry, studies of the legal

OCR for page 13
21 and ethical responsibilities of health care institutions, and research on the population's preferences for and expectations of health services. Studies at this level usually deal with matters quite remote from clinically-oriented research. Whether they are considered health services research depends ultimately upon their inter-relationships with other studies at the systemic, institutional, and clinical levels. The question of tax policy, for instance, becomes relevant to health services research when it can be shown to have implications for the ways in which health care is organized, delivered, or used. Types of Health Services Research The definition of health services research presented above states that it is inquiry to produce knowledge. Inquiry refers to the series of stages ordinarily associated with empirical research, including problem formulation and conceptualization, measurement and data collection, and analysis and interpretation. While each of these stages is a necessary part of the process of inquiry, the crucial ingredients of research are the analysis and interpretation of data for the purpose of answering a question. The collection of data to administer programs does not constitute research unless it is directed toward answering some question that applies to groups of units. The Health Care Financing Administration, for instance, manages the Medicare and Medicaid programs for the federal government. In this capacity the agency assembles vast amounts of information about beneficiaries' use of health services and charges for them in order to determine whether particular services and charges are eligible for reimbursement. This activity does not constitute research, however, because it is not directed toward answering broader questions that apply to groups of Medicare or Medicaid beneficiaries. In general, program monitoring activities, in which interest is focused on individual units, are not research. The nature of the questions or problems that occasion inquiry define two types of research, descriptive and analytic. Descriptive research addresses questions of the form "how many (or what) Xs are in A?" The objective of quantitative descriptive inquiry is to estimate parameters that apply to groups of cases. For instance, descriptive data may depict trends over time or compare geographic areas and their popula- tions. A great deal of useful health services research is descriptive. The series of estimates of national health care expenditures produced by the Social Security Administration (and now by the Health Care Financing Administration) provide invaluable information on the amounts and

OCR for page 13
22 categories of public and private expenditures for health care. Similarly, the various series of publications from the National Health Survey (conducted by the National Center for Health Statistics) yield useful no tional estimates of the prevalence of illness and the use of health services. Data from these types of studies identify trends and variations that raise theoretical and policy questions that invite further analysis to reveal their correlates and causes. This is the objective of analytic research. Analytic research attempts to answer cause-effect questions or to - make projections into the future. Descriptive research deals with questions of "how many" or "how much," but analytic research is concerned with answering "why" or "what will be" questions. This type of research is inherently more difficult and abstract than descriptive research. It requires the use of theoretical models and designs of proof to demonstrate that causal interpretations (or projections) are logically consistent with what is already known and that statements (or assumptions) about the effect of particular variables are not spurious (i.e., due to circumstances not explicitly included in the analyses). Several circumstances combine to complicate analytic research efforts in health services research, including (1) the complexity of problems addressed in this area and their variations in time and place; (2) the states-of-the-art of the theories and methods of disciplines that contribute to health services research; (3) the specific data avail- able for research; and (4) the difficulty of establishing truly experi- mental situations, which frequently requires relying on nonexperimental research designs. The logical and practical necessities that set the limits of analytic studies encourage investigators, working from different theoretical perspectives, to focus on selected aspects of problems and to disregard others. As no conceptual framework takes into account all aspects of a problem or is inherently superior to others, health services research encompasses a great variety of perspectives. Research, for instance, relating to the Health Systems Agencies created under the current health planning act would employ the theories and methods of economics to assess their effects on hospital cost inflation, the perspectives and approaches of sociology or political science to study their influences on community organization, and a variety of disciplines ranging from clinical medicine to economics to determine the effects of awarding a certificate of need for the installation of an expensive new technology. Because health services research takes place in particular places and periods and is focused on different levels of and interests in a particular , ___,, ~ ,e often are not generalizable to other settings, because of circumstances that are peculiar to the site in which the study was done. For the same reasons, data from national studies often do not apply to local situations. generality, particular studies cannot satisfy the needs of all potential audiences. Findings from a study done health cart ;n.~:ti t~~t;nn hi tar fir =~=

OCR for page 13
23 Since much of health services research is based on the theories and methods of the social sciences, its ability to explain events is limited by those sciences' levels of development. Even a cursory reading of the literature in these fields reveals considerable uncertainty and debate about the meanings and appplications of fundamental concepts and the validity of basic propositions. These problems are compounded by the concerns of much of health services research with such elusive and judgmental issues as quality and economic value of life, the general health status of populations, and the humaneness of health services. Data for health services research are drawn principally from population surveys, records and documents, and direct observation. Each of these methods admits various biases and unreliability that militate against clearcut description and analysis. Answers to such seemingly straight- forward questions as precisely how many hospital beds or physicians there are in the United States or how much the population spends on various types of services are not readily found from existing data sources, and special studies to determine these numbers are expensive and time-consuming. Furthermore, the protection of privacy afforded individuals and institutions by law and the economic and political advantages that accrue to some from concealing certain types of information frequently lead to incomplete and biased data that limit the validity of analyses. The classic experimental design remains the ideal foundation on which to conduct research. With few exceptions, however, studies in health services research are based on nonexperimental designs. As a conse- quence, it is seldom possible to draw strong conclusions regarding cause and effect, such as those drawn in the laboratory sciences. The practical and ethical obstacles that prevent investigators from con- trolling events and circumstances that are extraneous to their principal research problems introduce errors into analytic studies whose magnitudes often cannot be estimated. Because of these problems, analytically- oriented health services research relies heavily upon the comparative approaches of studying so-called natural experiments and of applying complex statistical procedures to historical data to adjust for charac- teristics of cases and situations that are known or presumed to be related to the question under investigation. These difficulties inherent in health services research account to a large extent for the seeming inconclusiveness of much of the research, and underscore the need for studies aimed at improving research methods and for replications using various perspectives and methods. As the field has developed, significant advances in knowledge have been achieved through the application and integration of theoretical perspectives and methods that either were unknown or undeveloped two decades ago. Advances in computerized multivariate analyses, for example, opened possibilities for research on certain types of questions that previously could not be addressed. Likewise, the availability of national data on use of health services has given impetus to comparative studies

OCR for page 13
24 of national and international scope. Thus, there is unmistakable evidence of progress within the field and need for continued efforts for improvement. While that occurs, however, those who sponsor health services research should recognize the limitations of current theore- tical and methodological approaches and encourage replications to validate and extend findings of studies. Uses of Health Services Research Most definitions and critiques of the field of health services research imply that studies in this area have or should have direct implications for action. The field is characterized as an applied endeavor whose products should be assessed primarily in terms of their usefulness to people with decision making responsibilities, whether they be clini- cians, administrators of health care institutions or government pro- grams, or officials charged with formulating national health care policy. Although the committee agrees that these are legitimate expectations and grounds for assessing health services research, it notes that discus- sions of the usefulness of studies in this area are often clouded by simplistic analogies to research and development in the physical and natural sciences, misunderstandings of decision making processes in various settings, and narrowly defined conceptions of the audiences for health services research. In this section, various potential uses of health services research by decision makers are described when purely rational models of decision making are assumed. The following section critiques these models and discusses others. Evaluation of Technologies and Innovations The logical sequential linkage of biomedical and health services research occurs at the point where a technology* is tested in selected clinical settings, indicated by the "transfer" stage in Figure 2. At this point, technologies whose efficacy and safety have been established by trials carried out in the application stage are placed in clinical settings for further testing. The first step in the transfer stage involves designing suitable arrangements for the use of the techno- logy. Once these are developed, the innovation is tested for clinical effectiveness and cost-effectiveness in terms of the institutions *Although this discussion refers to biomedical technology, it applies equally to other types of technology employed in the health services industry, for instance the uses of computers in institutional management, geocoding and other computer-based technologies employed by planners, and architectural innovations. The term technology encompasses know- ledge and procedures as well as materials and equipment.

OCR for page 13
25 F IGURE 2 EVOLUTION AND DIFFUSION OF HEALTH CARE TECHNOLOGY RESEARCH STAGE OBJECTIVE EVALUATION CRITERIA . Basic research Develop knowledge about New knowledge z fundamental life ~ processes o SAP lied research APP1Y knowledge from New knowledge about ~ basic to research on causes and/or processes 3 particular diseases of particular diseases I ~ I (Application Develop and test Efficacy, safety relative technologies to to existing technologies ~ I diagnose, prevent, cure, or contain particular diseases 2 `~ | |Design and test systems Acceptability, efficiency I for use of technologies of application and effec- / \in clinical settings tiveness and safety within Transfer ~ test sites u: 1 ~ 1 u, / ~ ~ iffusion u, :r: En System impact \ iTest cost-effectiveness, patient and provider acceptability and effects on institutions Develop knowledge about the diffusion of tech- nologies Knowledge about cost- effectiveness, patient and provider accepta- ~ and institution- bility wide impacts Knowledge about why technologies diffuse; effects of prices, institutional resistance, etc. Develop knowledge about Knowledge about impacts impacts of technologies of technologies on due to their diffusion system-wide demand for and applications services, costs, rela- tionships between their supplies and needs of populations, etc.

OCR for page 13
26 in which it is applied. During this transfer stage, development and research activities often become intertwined. In fact, much of health services research involves demonstrations in which developmental and research activities are closely integrated. The subsequent stages of research on the diffusion of technologies and its effects on health services systems are clearly health services research. At this point, however, attention turns from questions of efficacy and cost-effectiveness of technologies to studies at the institutional and systemic levels of health services research. Investi gations of the features of individuals or institutions that adopt technologies and of the effects of their adoption are similar to market research conducted in other industries. At the systemic level, studies in health services research focus on impacts of such matters as the influences of financing and regulatory programs on the diffusion of technologies. Also, they examine the effects of their organization on the demand for services, per capita expenditures for health services, and inter-organizational arrangements among health care and other institutions. Problem Solving The use of health services research in assessment of technologies at the clinical level is a situation in which both the question and the deci- sion are clearly identifiable and the need for decision making is rela- tively predictable. At issue is whether a particular technology should be adopted by a particular individual or institution. The problem arises from the "knowledge-driven" processes of biomedical research and development. Most health services research does not come about in this sequential, predictable way. Instead, studies in this area are occasioned by existing problems identified by societal groups and decision makers. Research is aimed at solving the problems. Rational problem-solving involves several stages, each of which requires a particular type of knowledge. These stages and corresponding types of health services research are shown in Figure 3. Problem identification typically involves descriptive research to mea- sure the extent or seriousness of a problem and to locate subgroups of people, institutions, or geographic areas that are most affected by it. Findings from such studies occasionally may bring "new" problems to the attention of decision makers. More commonly, however, they pro- vide more precise and systematic information about problems previously identified by affected groups. As these types of studies describe the nature and extent of a problem, their findings usually are not amenable to direct translation into decisions about desirable inter- ventions. The studies of the Committee on the Costs of Medical Care

OCR for page 13
27 FIGURE 3 RELATIONSHIPS BETWEEN STEPS IN THE DECISION PROCESS AND TYPES OF RESEARCH STEPS IN THE DECISION PROCESS Recognition of a Problem Establish Causes of the Problem Establish Alternative Solutions Select and Implement Decision Monitor Effects of Program TYPE OF RESEARCH Problem Identification Problem Specification Assessment of Alternative Interventions < . . < Evaluation of Implementation Process ~ Evaluation of Effects

OCR for page 13
28 carried out in the late 1920s and early 1930 s are widely cited examples of problem identification research. The Committee's studies of use of personal health services provided the first quantitative infor- mation in the United States on the distribution of medical services among income groups. The series of descriptive reports of the National Center for Health Statistics and of the Center for Health Administration Studies of the University of Chicago on the use of health services and those of the Office of Policy, Planning, and Research of the Health Care Financing Administration (formerly of the Office of Research and Statistics of the Social Security Administration) on expenditures for health care are notable contemporary examples of this type of research. Problem specification moves beyond description to identify causes of problems. It is analytic insofar as it seeks to quantify the relative importance of various factors presumed to contribute to a problem. Studies of effects of health insurance on hospital utilization and costs, research on factors affecting physicians' choices of practice locations, and inquiries about determinants of hospital capital expenditures il- lustrate this type of research. Assessment of alternative interventions is similar to problem specifica- ~ . . . . . . . . . . Lion in that it attempts to quantify the relative effects of factors on a problem. It differs from problem specification in the explicit attention it gives to effects of alternative potential interventions. In effect, this type of research attempts to quantify the costs and benefits associated with various intervention strategies. Comparative studies of hospital use by health maintenance organizations and insured populations, and cost-benefit studies of alternative modes of improving a population's health status are of this type. Evaluations of implementation processes attempt to assess the degree to .. which a program operates as intended and to identify causes and con- sequences of deviations. Unlike the types of research described above, evaluations follow choices of implementation strategies. Therefore, researchers have access to plans of action that, in principle, specify intended modes of implementation that can be used as standards against which to compare performance. Research of this type can be descrip- tive, aimed at determining whether the actual process conforms to the plan, or it can be analytically oriented, aimed at identifying causes and consequences of the ways in which intervention strategies are implemented. For instance, studies of the compositions of boards of directors of Health Systems Agencies and of physicians' conformance to standards of practice established by Professional Standards Review Organizations are descriptively oriented evaluation research. Exten- sions of these aimed at identifying causes of observed behaviors would be examples of analytically oriented evaluations. Evaluations of effects attempt to measure the extent to which inter- ventions attain stated objectives, to identify their unanticipated consequences, and to explain their causes. Minimally, such research

OCR for page 13
29 describes whether (or the extent to which) a stated objective is at- tained. More elaborate evaluations examine features of interventions or their environments that account for observed results. The series of stages in this process constitute health services research and development when applied at the clinical or institutional levels and policy research at the systemic level. Health services research and development refers to a strategy of cycles of interven- tions combined with research and evaluation. Its purposes, like research on the transfer of technology, are to develop and test innovations in test sites. The problem solving stages described above are sometimes referred to as "cyclical policy analysis" to distinguish it from the more general meaning of policy analysis.~7] Cyclical policy analysis refers to an orderly strategy of research aimed at providing knowledge based on experience. It rests on empirical study of situations or events as they currently exist. Findings from such studies are employed in policy analysis, which assembles information with which to design alternative options for action and anticipates the likely effects of alternative intervention strategies. Because policy analysis usually takes into account a broader range of criteria in evaluating alterna- tives than is considered in particular studies, it typically involves synthesizing findings of existing research and extrapolating their implications. Limitations of Rational Models The foregoing schemes were intended to identify potential uses of health services research in decision making, not to describe what actually occurs. It should be noted, however, that these schemes are based on assumptions that rarely are met in situations for which health services research is conducted. Furthermore, they overlook uses and users of health services research other than decision making by decision or policy makers. The models of technology evaluation and of problem solving described above make several assumptions about decision making situations and the relationship of information from research to decisions: a person has identified a problem to be solved, has formulated consistent criteria for valuing alternative solutions, has the will and -means to implement solutions, and will base decisions exclusively on results of the research; the research was completed before the decision was taken, dealt with the problem as identified by the person and

OCR for page 13
30 incorporated all of his evaluation criteria, and was accurately communicated to, understood, and accepted by the person. Given these assumptions, the utility of research can be assessed in objective terms by comparing the problem situation before and after implementation of the solution. Absent any of these, the utility of research must be judged on subjective grounds, such as its informative value to the person. In formulating its assessment of health services research, the committee took these assumptions as problems to be addressed. The principal difficulty in assessing the uses of health services re- search is that few decision making situations fit the assumptions of rational problem-solving models. Decisions to adopt or not to adopt particular technologies are usually made before completion of cost- effectiveness and institutional studies, and decision makers are influenced by several factors other than the results of empirical studies. Because new technologies are developed and marketed through the private sector, decisions about their adoption or nonadoption are influenced by competitive pressures and custom, as well as by cost- effectiveness and other so-called rational criteria. The same is true of the uses of findings from health services research in policy making. Policy decisions are a blend of factual information, values, and expectations about the future effects of alternative courses of action. Research may point up issues, measure their extent and seriousness, suggest the likely effects of alternative interventions, and influence the context and quality of policy debate. It cannot, however, substitute entirely for the political process through which value choices and judgments about possible outcomes are explicitly and implicitly incorporated into policy decisionse Rational decision models also are limited sources of criteria for assessing the utility of research because they concentrate exclusively on formal decision making and official decision makers. Research findings have informative value whether or not they lead to identifiable decisions. They are the substance of formal education in health care administration and much of clinical medicine and a source of know- ledge, attitudes, and expectations on the part of the public. More- over, as government involvement in the health care industry expands and the scope of political decisions affecting health care enlarges, information from health services research, if effectively transmitted, becomes an increasingly crucial ingredient of the public's abilities to choose the types of health services it desires and to hold providers, planners, and government accountable for what is delivered.

OCR for page 13
31 REFERENCES 1 2 4 5 6 Executive Office of the President, Office of Science and Technology, Improving Health Care Through Research and Development, Report of the Panel on Health Services Research and Development of the President's Science Advisory Committee (March 1972~: 1. Ibid. Personnel Needs and Training for Biomedical and Behavioral Research: flee told Report of the Committee on a Study of National Needs for - Biomedical and Behavioral Research Personnel. Commission on Human Resources, National Research Council, Washington, D.C. (1978~: 113. Alvan R. Feinstein, "Scientific Methodology in Clinical Medicine II. Classification of Human Disease by Clinical Behavior," Annals of Internal Medicine (October-December 1964~: 757-781. Brian MacMahon and Thomas F. Pugh, Epidemiology: Principles and Methods (Boston: Little, Brown and Company, 1970~: 1. Carol H. Weiss, Using Social Research in Public Policy Making (Lexington, Massachusetts: Lexington Books, 1977~: 13. Charles O. Jones, "Why Congress Can't Do Policy Analysis (or words to that effect)," Policy Analysis (Spring 1976~: 251-264.

OCR for page 13