Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
A Pilot Process for Setting National Assessment Priorities The Council on Health Care Technology and its pnonty-seuing group opted for a two-stage strategy to address its congressional mandate to set priorities for health care technology assessment. Although the group recognized the need for a national pnonty-setting process for the assess- ment of medical practice, it also became acutely aware of Be lack of widely accepted pnonty-setting methodologies and adequate primely data to meet this need. Recognizing these methodologic difficulties, the council undertook this pilot effort to set a framework for national pnonty- setting, outline national pnonty-setting cntena, and use a consensus process to identify a first-cut list of priority areas for the assessment of clinical conditions and medical technologies. An expanded effort pro- posed by the council would build on what was reamed from the pilot work, available data, and funkier methodolog~c development. This report is the record of me first, pilot effom ELEMENTS OF A NATIONAL APPROACH The national approach for setting assessment priorities proposed here is composed of Tree key elements: I. Explicit and well-accepted criteria for priority-sefflng that are am plicable at the national level. 2. A conceptual framework that accommodates both clinical condi- tions and medical technologies as assessment pnonties. 15
16 NATIONAL ASSESSME=PR]ORTTlES 3. An accountable process for pnonty-setting that involves a broad range of assessment interests. Two premises underline these he elements. First, a national mecha- n~;rn to set priorities for the evaluation of medical practice must account for the vaned needs and perspectives of different pardes involved in heath care. This pilot effort entailed broad-based involvement on Me part of the organizations providing data about potential priority areas and the members of We group who selected from among these areas. Those involved were representatives of health care sectors having different types of assessment needs. Thus, the council believes that the list of priorities identified in this pilot study represents assessment areas that are of na- tional ~mporunce. Even so, such a list of general assessment priorates might be useful in meeting the needs and pnonty-sefflng cr~tena of patients, providers, researchers, payers, and regulators. Certainly' individual organizations win continue to set their own pr~onties based on their roles in health care delivery. Second, medical technologies and practices should be considered in their clinical context and in comparison with over clinical management altematives. Early in its deliberations, the council decided to expand its charge fimm identifying only priority heath care technologies to deter- mung Be coccal coalitions and medical problems Cat should also be taken into account. In this pilot study, therefore, the council used both a clinical condition approach and a technology approach to set pr~onties. This combined clinical condition/technology framework considers tech- nolog~es and other medical practices in their clinical context. It considers me use, or non-use, of technologies for He management of clinical condi- dons that may affect different patient populations. NATIONALLY APPLICABLE PRIORITY-SETTING CRITERIA The following pnon~-se~ng criteria identified by the Council on Health Care Technology are based upon important factors commonly recognized by the multiple heath sectors represented In me council, including academia' government, health care provided, health care manu- facture~, and payers. The criteria were used to choose He 20 assessment areas identified in this pilot report and to facilitate He comparison of the pnonty-serdng criteria used by some of Be organizations providing infor- madon for this pilot study (see Appendix By. The use of these prionty- setting criteria assumes Hat a well-conducted assessment of a phoney clinical condition or medical technology could be used to unprove medi- cal practice and patient quality of life.
A PILOT PROCESS 17 The primal and secondary criteria provide means of comparing candi- date assessment areas in terms of the types of impact that may result from an tnvesOnent in evaluating that area. For the purposes of this study, primary criteria account for important and readily quantifiable character- istics of national pnor~ties. Secondary criteria account for other factors that should be consider in pnonty-sett~ng decisions, but that may be more difficult to quantify because of their more general or subjective nature. Together, these enters represent a spectrum of factors and issues that are important to decision-makers in setting pnonties. The order in which the criteria are listed does not imply that one criterion is more important than another, or mat aD of the criteria shown should be given equal weight in setting pnonties. The 20 priority assessment areas chosen in this pilot study fulfill all or most of the primary criteria and one or more of the secondary cntena. Primary Criteria Potential (for an Assessment) to Improve Individual Patient Outcome An improved patient outcome refers to the health benefit or reduction of health risk for Me individual patient. Implementation of me findings of an assessment may lead to changes in the use of technologies for managing clinical conditions that can improve patient outcomes. Patient Outcomes resulting from We use of different technologies (or Me use of no technol- ogy) may be measured by the risks to Me patient (e.g., mortality, morbid- ity, complications, or side effects), the improvement of health status' and Me patient's satisfaction with the intervention (e.g., his or her perception of improved emotional and physical weB-being). The relevance of differ- ent patient outcomes may vary according to Me patient's clinical condi- lion, age, sex, and cultural and socioeconomic background. The percep- tion of what constitutes an improved patient outcome may be different for patients, providers, payers, health care product makers, or policymakers. Potential to Affect a Large Patient Population The potential impact of an assessment may be enhanced if the prevalence of Me clinical condition or the frequency of use of the technology is high. Measures of frequency provide a factor for scaling possible benefits from me standpoint of Me individual patient (e.g., the potential to improve Me outcome for Me individual patient and to reduce the cost per patient) ~
18 NATIONAL ASSESSMENT PRIORITIES that of an aggregate patient population. Thus, an inexpensive procedure might not be considered important in the individual case, but might be a high priority in the aggregate if it is a widely used procedure with an un- determined health benefit. Potential to Reduce Unit or Aggregate Cost The results of a high-priority assessment may lead to changes in medical practice that may reduce costs or increase savings while achieving an equivalent or improved effect on patient outcomes. Unit cost may be estimated by the average expenditure of resources (capital, manpower, maintenance, etc.) associated with the use of technologies for one clinical condition for one patient for a specified penod of care. A period of care may be defined as a single procedure, a sequence of interventions for an acute clinical episode, or the lifetime treatment of a chronic disease. Aggregate cost may be estimated by the product of unit cost and preva- lence in the case of a clinical condition, and by the product of unit cost and frequency of use in the case of a technology. The unit or aggregate cost may include, but not be limited to, expenditures ~ncuned by patients, providers, or payers. Potential to Reduce Unexplained Variat~ons in Medical Practice Unexplained variations refer to differences in the use of technologies for a clinical condition, or the associated patient outcomes, Hat cannot be explained by differences in identifiable patient characteristics such as disease seventy or sociodemographic factors. The results of an assess- ment may lead to changes in medical practice that would reduce unex- pla~ned variations in the use of drugs, devices, or procedures. Deviations from appropriate utilization may lead to undesirable consequences such as high costs from the overuse of ineffective technologies or false savings from the underuse of effective technologies. Unexplained van ations In practice patterns and patient outcomes may be detected along a number of dimensions, such as geographical region, clinical practice setting (e.g., inpatient versus outpatient), or type of heath care provider. Secondary Criteria Potential to Address Social and Ethical Implications The introduction of some technologies may be associated with concerns about social and ethical values; examples are those technologies used to prolong life in an incurable state or alter reproductive patterns. The
A PILOT PROCESS 19 attitudes of the patient, provider, or society toward such technologies may be better understood and potentially changed as a result of an assessment. Fur~ennore, some technologies may have broad ~mplicadons for eco- nomic or legal systems, such as me potential impact Cat a health care technology might have on the productivity of the work force or on makers of provider liability. Issues such as equity of access and potential radon- ing of services Even limited resources should also be considered. Potential to Advance Medical Knowledge An assessment may be of high priority if it could sificandy improve knowledge about a clinical condition or a particular technology's effec- tiveness in preventing' diagnosing, or treating a conical condition. This criterion would likely apply to clinical conditions for which the scientific base is limited or for which few or no technologies exist. Similarly, new and emerging technologies may produce a "bmalcthrough" in current medical Linking and practice and have a greater potential to advance medical knowledge. The potential benefit of emerging ~rugs' devices, or procedures that are not yet being used for a large number of patients may be enhanced as a result of an assessment Potent~al to Affect Policy Decisions Well-conducted assessments of the medical practices in question might elucidate policy options for me avocation of national resources, He for- mulabon of regulatory and reimbursement policies, and me management of health benefit programs. Evaluations of Be relative effectiveness of altemative technologies may provide Me scientific justification for prac- tice guidelines for specific coca conditions or technologies. In addi- tion, assessments may be conducted to respond in a timely fashion public demand for the resolution of a controversial issue. Potential to Enhance the Natzor~ Capacity for Assessment An enhanced capacity refers to better use and furler development of me nation's information, methodologic, economic, and human resources avail- able for medical practice and technology assessment. An assessment may help train researched or serge as a mode} to improve mesons. Evalu- ations of specific topics may ser`,e as prototypes for certain types of analyses and could lead to advances In methodology.
20 NATIONAL ASSESSAfENTPRlORmES Patters to Be Reads Conducted The ability to conduct an assessment is dependent on the adequacy of available data, analytical methods, and resources needed for the appropn- ate analyses and formulation of recommendations. Perceived infeasibility should not prevent better methods from being developed and resources from being commit to assessments of the highest-pnon~ areas. How- ever, if two topics have the same potential impact by aU the other cntena' the potential to be readily conducted may determine which topic is of greatest pnonty. A CONCEPTUAL FRAMEWORK FOR PRIORITIES To evaluate medical technologies and practices, the clinical conditions for which these are used and the needs and characteristics of the patient population of interest should be taken into account. Assessment concerns may also be raised about particular technologies or groups of technolo- g~es. Accordingly, in setting priorities among potential assessment top- ics, Be pnonty-setting group used both a clinical condition approach and a technology approach. While the council favors the clinical condition approach to evaluating medical practice, it recognizes that a single ap- proach should not dominate and that a national process for setting pnon- ties must accommodate the needs of those oriented to different approaches. The conceptual framework for priorities accommodating clinical con- ditions and technologies was developed after reviewing the approaches used by a selection of different types of health care organizations (see Appendix C). This unifying framework was used to compile the initial, broad list of clinical conditions and technologies considered in this study. Appendix A shows how the potential assessment topics suggested by the source organizations and by the members of the pnonty-seuing group were considered as altemative technologies and practices, or combina- tions of these, Cat may be used for the prevention, screening, diagnosis, or treatment of particular clinical conditions. The Clinical Condition Approach The clinical condition approach to identifying potential assessment topics focuses on the patient's clinical condition for which different tech- nolog~cal altematives exist. The term clinical condition denotes a set of identifiable clinical signs and symptoms affecting the health status of an
A PILOT PROCESS 21 individual.! Rather than emphasizing specific attributes of a single tech- nology, the clinical condition approach is concerned with the differences in patient outcomes2 that may be related to the use of altemative technolo- g~es, different combinations or sequences of technologies, or no technol- ogy for a coca condition. The clinical condition approach to pnonty-setting is compatible with the expanding interest in "patient outcomes" and "effectiveness research" in evaluating medical practices. These evaluations compare the relative effectiveness of altemative technologies to prevent, diagnose, or treat a clinical condition in actual everyday practice In contrast to measures of their efficacy in controlBed clinical research settings. The emphasis may be on widely used procedures and practices and on questions of the most appropriate treatment for a given set of symptoms in the individual patient. An this context, measures of relative effectiveness reflect bow the decisions that clinicians must make in the care of patients and the differ- ent values or utilities individual patients may have for the outcomes associated with alternative interventions. Dunng its early deliberations, the evaluation panel of the Council of Health Care Technology determined mat the clinical condition approach was more appropriate than a focus on a particular device, drug, or proce- dure (Institute of Medicine, 1987a). The coca condition approach re- fleets more appropnately the context within which alternative drugs, devices, or procedures (or no technologies) are used to treat patients in the everyday practice of medicine. The clinical condition approach to identi- fying assessment priorities was used primarily by, but not Innited to, the individual clinicians and physician organizations contacted in this pilot study (see Appendix C). In proposing a national Effectiveness Initiative the Health Care Financing Administration (HC:E;A), which administers the Medicare program and the federal portion of the Medicaid program, chose the conical condition approach, as demonstrated by its aforemen- ~ The health status of an individual may be assessed in teens of the traditional anatomic aM physiologic measures of disease, as well as in ter ns of the impairments in normal functional, emotional, and mental condition (Lohr and Mock, 1989~. The use of the tenn clinical condition emphasizes the importance of considering other dimensions of health status, beyond the more traditional clinical measures of disease, in evaluating medical technologies and practices and their relationship to patient outcomes. 2 Patient outcomes are defined as "the range of results Mat proceed from (or are presumed to be associated with) the provision of health care services." Lee concept of patient outcomes includes physiologic, functional, emotional mental, and social dimen- sions of health (Lohr, 1988~.
22 NATIONAL AS.9F-~;M~PHORm~ tioned request to the Institute of Medicine for advice on priority cI'n~cal conditions Destitute of Medicine, 1989~.3 The Technology Approach Another approach to identifying assessment priorities concerns par- ticular medical technologies. Medical technologies are me drugs, devices, procedures, support systems, arid organizational systems used in health care.4 This broad definition of medical technologies includes Al medical practices and processes of care. For example, interventions such as a routine physical exam, a diagnostic laboratory test, or laser surgery are types of procedures; and administrative technologies such as infonnation systems and payment mechanisms are types of support and organizations systems. A technology may be in an emerging, new, established, or obsoletes stage of development, diffusion, or acceptance. The focus of many medical technology assessment activities is consis- tent with a technology approach. Such assessments are driven by Me need to answer questions about the properties of specific technologies, such as safety, efficacy, indications for use, and cost, as wed as social, economic, and ethical consequences, whether intended or unintended (Institute of Medicine, 198Sa). The focus may be on single and relatively expensive new, high technologies such as procedures or devices. The technology approach to identify priorities was used by some of the groups involved in the payment or purchasing of health care services that provided background information for this project (see Appendix C). For example, private third-party payers identified assessment needs in terms Of the CPT46 procedure codes used in insurance claims (Blue Cross and Blue Shield Association, 1989b; Metropolitan Life Insurance Program, 1989~. Similarly, organizations representing the interests of hospitals were concerned with technologies, such as devices or equipment-embod- ied procedures, that are important in providing state-of-the-art medical services (Amencan Hospital Association, 1989; ECRI, 198S,1989e). Thus, a technology approach may better represent the assessment needs of some organizations, such as payers or institutional providers of care, faced win decisions about the cost and reimbursement of specific new technologies. 3 See footnote 6 in chapter 1. ~ The definitions for medical technologies, originated by Me congressional Office of Technology Assessment (1978), were expanded by Me Institute of Medicine Committee on Assessing Medical Technologies (Institute of Medicine, 1985a). In this report the terms are used as defined in the Glossary. 5 The stages of diffusion of a medical technology are defined in the Gloss~y. 6 Physicians' Current Procedural Terminology, Fourth Edition (CPI~4) (American Medical Association, 1987~.
A PILOT PROCESS Integrating the Clinical Condition and Technology Approaches 23 Medical technologies may be used for the prevention, screening, ~iag- nosis, and treatments of clinical conditions. The clinical condition ap- proach for setting priories emphasizes the clinics context within which altemative technological interventions should be assessed. The technol- ogy approach may be used Go identify He alternative technologies that apply to a particular stage of a clinical condition. For example, lithotnpsy is a treatment technology used to destroy body stones. Rather than con- sidenng it alone as a specific technology, lithotnpsy can be compared with other treatment technologies, such as surgical removal and dissolu- tion by chemical means, to form a group of technological alternatives for the treatment of the clinical conditions of gallstones or kidney stones. The technology approach may also be used to identify cross-cutting technologies that apply to a large number of conditions at different stages. For instance, imaging technologies such as computed tomography, mag- netic resonance imaging, and positron emission tomography are diagnos- tic technologies that may be used to detect the anatomic abnom~alides associated with many cI=cal conditions, such as many types of cancer and neurolog~c injuries. Because groups of cross-cueing technologies may raise common assessment issues such as We appropriateness and sequencing of multiple testing win different diagnostic imaging modali- ties a technology approach may be more appropriate for identifying them as assessment priorities. However, such technologies must be as- sessed with respect to a particular patient outcome or clinical indication that may serve as a prototype for He issues raised by He group of technologies. In identifying the set of 20 assessment priorities discussed in Chapter 3, this pilot study integrated the clinical condition and the technology am preaches for setting pr~onties. For high-pr~or~ty clinical conditions, the adtemative technologies Hat may be used for prevention, screening, diag- nosis, and treannent were considered. For high-pnonty, cross cueing technologies the range of alternative preventive, screening, diagnostic, or annent indications was taken into account These interrelated am preaches can be used to represent the concerns of those oriented to a technology approach, as well as the concerns of those oriented to a clinical condition approach. 7 These types of intervention are defined in the Gloss~y.
24 METHOD OF THIS PILOT STUDY Overview NATIONAL A~SESSMENTPRlORmES The council and its pnonty-setting group treated this pilot effort as an experiment in developing a prionty-setting process involving a broad range of assessment interests. Based on He preliminary experiences of He council's evaluation pane} win modeling and consensus approaches to pr~onty-setting, He group realized that the development of a formal pnonty-sefflng metrology Hat would integrate the best of these am preaches and use the necessary primary data was not possible within the scope of time and resources available for this pilot effort. The pnor~ty-seuing group decided to move forward win a two-round modified Delphi process to choose a first set of priorities for further study from a larger list of potential topics. The two-round component of this process was used to generate a broad, initial list of potential topics. Candidate topics listed in Appendix A were added to the list in two iterations, based on the background information provided by diverse organizations and the expertise of He members of He group. By means of a modified Delphi homework exercise, the pnor~ty-sefflng group chose from this initial list a narrower field of potential topics to be consider dunug a later meeting of the group. The homework exercise was used to ensure the most efficient use of meeting time by limiting the number of topics to be consider and discussed. In summary, He 20 recommended assessment priorities discussed in Chapter 3 were chosen by the pr~onty-sefflng group from the composite list of 496 topics listed in Appendix A through a two-round modified Delphi process and a meeting of me full group. Round ~ of the Modified Delphi Process The goal of Round ~ was to develop a broad list of potential assessment topics. Staff compiled this composite list after consulting individuals andJ or publications from the 14 organ~zabons descnbed in Appendix C. Each potential topic was considered sequentially and Men reclassified using a clinical condition or a technology approach. When a technology was readily identifiable as an altemative monadic used for the preven~don, screening, diagnosis, or treaunent of a specific coca condition, it was listed under Aim clinical condition. Men a technology applied to mul- dple clinical conditions, it was listed as a cross-cutting technology. Members of the pnonty-setting group reviewed this composite list and
A PILOT PROCESS 25 added topics Rat they thought should be considered. Appendix A dis- plays how Me 496 potential topics on the initial list were reduced to 73 groups of clinical conditions and associated technologies (see Table AM and 36 groups of other technologies (see Table A.2) using the clinical condition/technology framework. This composite list was reconsidered by the group during Round IT of the modified Delphi process. Round ~ of the Modified Delphi Process During Round IT the group generated a preliminary list of assessment priorities by Me following process. Staff sent to committee members a questionnaire asking them to list their top 10 priorities from the topics in Appendix A. Group members were asked to choose cI~rucal conditions or technologies and to justify each of their 10 choices in tempts of the pnonty-sethng criteria listed in this chapter. Staff analyzed Me results of this homework exercise in terms of Me frequency and types of topics se- lected, and the distribution of the pnonty-setting criteria used to justify each chosen topic was noted. These results were presented to the group dunng its meeting, together with a tabulation of an the numencal data on prevalence, cost, etc., provided by Me source orgaruzabons as described in Appendix C. The Priority-Setting Group's Meeting Dunug its meeting on May ~ I, 1989, at the National Academy of Sci- ences in Washington, D.C., the pnonty-setting group reconsidered me preliminary list of priorities they had chosen through the modified Delphi exercise. The group went though several rounds of discussion of Me proposed list of national assessment priorities In relation to the pr~ority- seuing criteria defined. At each stage, group members could move to add or delete topics from the list. After several rounds of elim~nadon based on voting and mnking exer- cises, Me group concluded that the lack of readily accessible unfond data to justify rankings of all the candidates by each of the criteria meant that a methodologically sound quantitative tanking would not be possible for this pilot effort. Throughout its deliberations, the priority-setting group, however, felt that it was important to limit the final list to a manageable number of assessment areas. After considering several rar~ing options for the priorities, the group decided on Me set of 20 priority assessment areas described in Chapter 3. The group determined that, because all
26 NATIONAL ASSESSMENTPRIORmES areas were nationally important and that data for them were not consis- tently available, it would not be possible to assign atop substantiate rar~c- ings among these 20 priority assessment areas. Activities Following the Priority-Setting Meeting In consultation with the prior~ty-setting group, the surf drafted the report, including the descriptions of Me 20 assessment areas chosen by the group. Staff incorporated information provided by the source orgaIiiza- tions and from current literature into these descnphons, which were arranged according to the applicable critena. Fur~ennore, selected ex- perts in the chosen priority areas reviewed the individual descriptions of each of the 20 priority assessment areas identified. The draft report was reviewed and approved by the prionty-setting group, the Council on Heath Care Technology, and external peer reviewers according to the standard procedures of the National Academy of Sciences.
