National Academies Press: OpenBook

Assessment of Diagnostic Technology in Health Care: Rationale, Methods, Problems, and Directions (1989)

Chapter: 6. A National Program for Assessing Diagnostic Technology

« Previous: 5. Costs and Sources of Funding
Suggested Citation:"6. A National Program for Assessing Diagnostic Technology." Institute of Medicine. 1989. Assessment of Diagnostic Technology in Health Care: Rationale, Methods, Problems, and Directions. Washington, DC: The National Academies Press. doi: 10.17226/1432.
×
Page 120
Suggested Citation:"6. A National Program for Assessing Diagnostic Technology." Institute of Medicine. 1989. Assessment of Diagnostic Technology in Health Care: Rationale, Methods, Problems, and Directions. Washington, DC: The National Academies Press. doi: 10.17226/1432.
×
Page 121
Suggested Citation:"6. A National Program for Assessing Diagnostic Technology." Institute of Medicine. 1989. Assessment of Diagnostic Technology in Health Care: Rationale, Methods, Problems, and Directions. Washington, DC: The National Academies Press. doi: 10.17226/1432.
×
Page 122
Suggested Citation:"6. A National Program for Assessing Diagnostic Technology." Institute of Medicine. 1989. Assessment of Diagnostic Technology in Health Care: Rationale, Methods, Problems, and Directions. Washington, DC: The National Academies Press. doi: 10.17226/1432.
×
Page 123
Suggested Citation:"6. A National Program for Assessing Diagnostic Technology." Institute of Medicine. 1989. Assessment of Diagnostic Technology in Health Care: Rationale, Methods, Problems, and Directions. Washington, DC: The National Academies Press. doi: 10.17226/1432.
×
Page 124
Suggested Citation:"6. A National Program for Assessing Diagnostic Technology." Institute of Medicine. 1989. Assessment of Diagnostic Technology in Health Care: Rationale, Methods, Problems, and Directions. Washington, DC: The National Academies Press. doi: 10.17226/1432.
×
Page 125
Suggested Citation:"6. A National Program for Assessing Diagnostic Technology." Institute of Medicine. 1989. Assessment of Diagnostic Technology in Health Care: Rationale, Methods, Problems, and Directions. Washington, DC: The National Academies Press. doi: 10.17226/1432.
×
Page 126
Suggested Citation:"6. A National Program for Assessing Diagnostic Technology." Institute of Medicine. 1989. Assessment of Diagnostic Technology in Health Care: Rationale, Methods, Problems, and Directions. Washington, DC: The National Academies Press. doi: 10.17226/1432.
×
Page 127
Suggested Citation:"6. A National Program for Assessing Diagnostic Technology." Institute of Medicine. 1989. Assessment of Diagnostic Technology in Health Care: Rationale, Methods, Problems, and Directions. Washington, DC: The National Academies Press. doi: 10.17226/1432.
×
Page 128

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

6 A National Program for Assessing Diagnostic Technology A MULTICENTER CONSORTIUM The evaluation of diagnostic technology should be national in scope. The purpose of this chapter is to propose a multicenter consortium for conducting cooperative teals of diagnostic technology. This program could solve many of the problems that have been described in the preced- ing chapters. All studies will adhere to predefined principles and will focus on patients and technologies of interest to policymakers and to clinicians. Competing technologies will be compared, and He data needed for clinical decisionmaking win be obtained. The consortium win be organized according to Be principles discussed in Chapter 7. This proposal is based closely on a plan that appeared in A Forward Plan for Medicare Coverage and Technology Assessment (Roe et al. 1987~. Two different methods of technology assessment will be used by He proposed centers: · Primary technology assessment: the process of assessing technol- ogy by collecting data from patients. This category includes randomized clinical trials and studies that measure the accuracy of diagnostic tests. The centers should emphasize primely technology assessment. In Chap- ter 3, we described an approach to studying diagnostic technology; the centers should adopt this or a similar approach. · Secondary technology assessment: the process of using previously published studies to evaluate technology. Secondary assessment is widely 120

ASSESSING DIAGNOSTIC TECHNOLOGY 121 used by government agencies, professional organizations, and individual investigators. The centers should develop models and methods for help- ing physicians to make decisions concerning individual patients and should do cost-effectiveness analyses of competing technologies. A multicenter approach to evaluating diagnostic technology has sev- eral advantages. Although seldom used in past studies of diagnostic technology, the multi-institutional cooperative study addresses two ~m- portant problems related to the conclusions reached and to study methods. The validity of the conclusions of prior studies of patient care is often questioned, because too few patients have been studied. With a coopera- tive effort involving many centers, studies can be large enough to meet the requirements of statistical analysis. The conclusions of earlier studies have also often had limited applicability because of bias in patient selec- tion and because the study was earned out at a single institution. Mul- ticenter collaborative studies can involve many types of hospitals, from HMOs to university medical centers. Another potential benefit is improvement in study methods. The de- sign of the clinical trial with randomized controls has undergone consid- erable refinement in He past two decades, in part because experienced in- vestigators from different institutions have worked together on issues of study design. There have been few such advances in studies of diagnostic technology. One advantage of a multi-institutional consortium would be to focus the attention of many different experts on study design. CHARACTERISTICS OF THE CENTERS The purpose of the centers is to serve as a standing resource that can perform technology evaluation. For new technologies, the centers will help to determine how the test can best be used and how fast it should diffuse into general use. For existing technologies, they will define the role, if any, of a technology in comparison with newer, competing tech- nologies. Each center should be located at a major teaching hospital, with access to many patient groups and to a strong supporting faculty. To capitalize fully on these advantages, the faculty of the center must convince their colleagues to cooperate. To do so will not be easy, because teaching hospitals are busy, complex organizations that are under considerable stress. Conceptually, the teaching hospital must be organized as a technology

122 vsrnNG V COMMIlTEE |— CORE SUPPORT GROUPS l ECONOMICS SOCIOLOGY STATISTICS EPIDEMIOLOGY ENGINEERING LAW BEHAVIORAL SCIENCES ASSESSMENT OF DIAGNOSTIC TECHNOLOGY r ~ I | ADMINISTRATION ~ I M EDICAL SCHOOL ~ = 1 HOSPITAL I ADMINISTRATION I ~ DIRECTOR _ HEALTH POLICY / PROGRAM - 1 CENTER OF TECHNOLOGY ASSESSMENT l ~ DIRECTOR [IGURE 6.1 Components of a center of technology assessment. EXECUTIVE COMMITTEE OF THE HOSPITAL PROGRAM COORDINATOR RESPARCH ASSISTANTS INVESTIGATORS HOSPITAL ADMINISTRATION MEDICINE SURGERY RADIOLOGY PATHOLOGY NEUROLOGY ANESTHESIA CLINICAL PHARMACOLOGY OBSTETRICS ~ GYNECOLOGY PSYCHIATRY PEDIATRICS assessment laboratory (Figure 6.~. The cast of characters in this labora- tory is complex; it includes the director of the center, coinvestigators, the core support groups, research assistants, and the medical school and hospital administration. The director of the center must have a resource- ful program coordinator. For the technology assessment laboratory to be able to respond rapidly to new technological innovation, all of the major clinical deparunents must be involved and represented on its staff. Beyond the primary group of investigators, there must be a support structure consisting of faculty in biostatistics, economics, sociology, epi- demiology, engineenng, law, and me behavioral sciences. The evaluation of technology is neither more nor less difficult Man human biomedical

ASSESSING DIAGNOSTIC 7ECHNOLOGY 123

124 ASSESSMENT OF DIAGNOSTIC TECHNOLOGY Primary technology assessment. Several primary technology assess- ment studies can be performed simultaneously at ad centers. The number of primary technology assessments mat can be performed at one time win depend on me rate of enrobing patients and on Me level of fiscal support for We centers. Secondary technology assessment. With modest additional support, each center can perform several secondary technology assessments each year. The remainder of this section is a descophon of some of He fiscal and logistical aspects of naming Be proposed program in technology assess- ment. Responding to Requests for a Technology Assessment The centers win respond to requests from venous sources, including He federal government, third-party payers, and professional organ~za- tions. These requests might focus on different kinds of problems, such as: · emerging technologies that have not yet become generally available. Evaluation at this stage of diffusion can prevent indiscnminant adoption of a new technology; · questions that had not been answered in prior studies (for example, about cost-effectiveness. evaluation in new patient groups, changes in the technology); · existing technologies that have been found to be effective in past studies but whose usefulness is now being questioned; and · technologies that have not been studied in a population of patients of special importance for health policy, such as He elderly. Upon receiving a request, several months may be required to determine the feasibility of doing the study, to identify me need for supplemental funding, and to estimate the time required to accrue enough patients. Choice of a Study Method Once a study topic has been decided upon, one individual will be assigned primary responsibility for developing He study protocol. Repre- sentatives from each center win meet to refine the protocol. Estimated Timetable The time required from receiving a request to completing a patient care

ASSESSING DIAGNOSTIC TECHNOLOGY 125 study could be as short as 18 months, depending on He rate at which new patients can be enrolhed. The ability to complete an assessment rapidly, before the new technology has diffused into practice, is one of me pnnci- pal advantages of a multicenter consortium. Review of Results an`] Recommerutations Expert review before disseminating study results and recommenda- tions win be an integral element in the cooperative studies. The prelimi- nary report of a study will, therefore, be sent to a panel of subspecialty experts for review. The pr~ncip~ investigator will be charged with responding to the comments and incorporating appropriate suggestions and critical analyses into the final report. S=y Personnel Several types of study personnel will be required. The exact mix of these is difficult to establish in advance, but experience with clinical studies suggests the foBow~ng configuration: Principal investigator. An investigator from one of the centers will become the principal investigator for a specific study. This person will take primal responsibility for designing, analyzing, and reporting the study, although all investigators will contribute to these phases. The principal investigator will be responsible for making certain that all cen- ters adhere to the study protocol and contribute their expected share of patients. Vesting one investigator with ultimate responsibility for a study is an important principle. Study co-investigators. Each center that participates in a study will designate an individual to be responsible for ensuring that the center fulfills its obligations to the study. This person will be in charge of the study at the center and will represent Be center on the executive commit- tee for the study. Collaborating subspecialistts). Subspecialists have the best knowl- edge of a variety of technologies and the best access to many types of patients. They are essential for studies of diagnostic tests, and their cooperation is necessary to implement protocols for uniform test inte~pre- tation. The amount of subspecialist effort will vary from study to study. The budget must include support for these individuals.

126 ASSESSMENT OF DMGNOSTIC TECHNOLOGY Analysts. Each center should have the part-time services of a biostatis- tician, a decision analyst, and a health economist to advise the principal investigator. Research assistant. A research assistant wiD perfonn foBow-up stud- ies on patients, keep records, and enforce unfond patient data collection procedures. The number of research assistants would depend on the number of studies in progress. Each center should have at least two at aU tunes. Additional research assistants can be recruited as needed. Nurse practitioner. Considerable clinical data must be collected on aD patients, including a history of the patient's problem and a directed physical examination. There should be at least one study nurse practitio- ner at each of the center sites. The nurse practitioner win enroll patients and perfonn the necessary clinical data collection. The number of nurse practitioners win depend on He number of studies in progress. Data entry clerk. This individual's assignment win be data entry and data checking. The number of data entry clerks wild depend on the number of studies in progress. Each center should have at least one at ad times. Additional clerks can be recn~ited as needed. Adrninistrat~ve staff. The administrative needs of a center include a full-time secretary to handle correspondence and filing and to assist in data entry. If many studies are being done simultaneously, a full-time study coordinator win be required. Budget To estimate a budget for each participating center, we assume that each center will provide partial salary support for the principal investigator, several coBaboradng subspecialists, a statistician, a decision analyst, and a health economist. The center will support full-time positions for two nurse clinicians, two research assistants, one data entry clerk, one secre- tary, and one study coordinator. We estimate that each center's annual budget would be between $500,000 and $600,000 in 1988 dollars. Multicenter Studies J!4 Caution For all the irreplaceable advantages of multicenter studies, there are many pitfalls to be avoided in running the studies. Chapter 7 contains a

ASSESSING DIAGNOSTIC TECHNOLOGY 127 discussion of the logistics of multicenter studies and the potential prob- lems that they present. ONGOING TECHNOLOGY ASSESSMENT BY MEDICAL INSTITUTIONS Quite apart from the problem of assessing new diagnostic technologies, He unde~tand~ng of how methods that have already diffused into practice actuary perfonn is frequently deficient The diagnostic laboratories and imaging deparunents of a medical center expend considerable effort on standardizing their product. Accreditation of a hospice depends in part on ongoing quality control of He diagnostic technologies offered within the institution. These activities can be extended in a very important way if an institution makes a commitment to measure the sensitivity and specificity of diagnostic tests as performed on its pa~entpopu~atzon. The performance of a technology depends pardy on the patients on whom it is used and pardy on the health professionals who carry out the diagnostic studies. (This point is illustrated dramatically by Table 2.1 In Chapter 2, which describes the results of a series of Or studies in patients with lung cancer. ~ The sensitivity and specificity of a diagnostic test are likely to be site- specific. One reason is that different sites have different types of patients. For example, the sensitivity of a test in a primary care population is likely to be lower than in a population of patients that have been defend for treatment of advanced disease. A test generally is able to detect advanced disease more easily than the early disease that is likely to be seen in primary care practice. Variation in test performance can also arise from variability in bow equipment and the competence of He health providers. There have been few intennstitutional comparisons of test performance characteristics in which the institutions have been broadly representative. An exception is a recent study in which samples from a single person were sent to many medical centers for serum cholesterol measurement. The institutions varied widely in He concentration of total cholesterol that they reported (College of American Pathologists 1987~. A study of exercise testing came to much the same conclusion (Philbrick en al. 19821. In these studies, the intennstieutional differences in test performance imply large differences in study populations and technique. Under these circumstances, pooling of several studies to obtain a single number for sensitivity and for specificity may be inappropriate. The data suggest, on the contrary, that each institution is unique. The solution is not to use

128 ASSESSMENT OF DIAGNOSTIC TECHNOLOGY pooled data but to measure the performance of Me test in the institution. This assignment win become increasingly feasible as hospital computer data bases become more available to clinicians. In planning a technology assessment, physicians define the gold-stan- dard tests that win be used to determine the true state of the patient. These may vary from surgery, autopsy, or biopsy to We results of another test, or even the results of long-term surveillance to detect disease missed by the index test. Each patient getting a test is entered in a log book. Severe months later, their charts are reviewed to see if Hey had one of the prospectively defined gold-standard tests. If they did, the result is noted, together with He results of He index test. If not, He patient's medical record is flagged, so that he or she can be followed clin~caDy for develop- ment of the disease, using clinical criteria Hat were prospectively defined. The task of identifying medical records would be time-consum~ng win paper-based medical records, but He increasing use of computers to store coded clinical data win reduce this burden to a straightforward routine. How are He data from such a program used? The sensitivity and specificity of me test can be calculated on a periodic basis to see if the technology is changing. The sensitivity and specificity of the test cart be published in institutional newsletters. The physicians who interpret the test (for example, radiologists or nuclear medicine physicians) can incor- porate the institution-specific sensitivity and specificity into probabilistic inte~pretabons of test results that appear In the report to the patient's · · . primacy p nys~c~an. REFERENCES College of American Pathologists. Comprehensive Chemistry 1987 Sur- vey. Skokie,111., College of American Pathologists, 1987. Inouye, S.K., and Sox, H.C. A comparison of computed tomography and standard tomography neoplasms of the chest. Annals of Intemal Medicine 105:906-924, 1986. Philbnck, J.T., Horwitz, R.~., Feinstein, A.R., et al. The limited spectrum of patients studied in exercise test research: Analyzing the tip of the iceberg. Joumal of the American Medical Association 248:2467- 2470, 1982. Roe, W., Anderson M., Gong, J., et al. A Forward Plan for Medicare Coverage and Technology Assessment. Volume IT: Supporting Documentation. Washington, D.C., U.S. Department of Heath and Human Services, 1987. , ~ - - - ~

Next: 7. Problems of Multi-Institutional Studies »
Assessment of Diagnostic Technology in Health Care: Rationale, Methods, Problems, and Directions Get This Book
×
Buy Paperback | $50.00
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

Technology assessment can lead to the rapid application of essential diagnostic technologies and prevent the wide diffusion of marginally useful methods. In both of these ways, it can increase quality of care and decrease the cost of health care. This comprehensive monograph carefully explores methods of and barriers to diagnostic technology assessment and describes both the rationale and the guidelines for meaningful evaluation. While proposing a multi-institutional approach, it emphasizes some of the problems involved and defines a mechanism for improving the evaluation and use of medical technology and essential resources needed to enhance patient care.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  6. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  7. ×

    View our suggested citation for this chapter.

    « Back Next »
  8. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!