Click for next page ( 48


The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 47
5 Strengths and Weaknesses of Health Insurance Data Systems for Assessing Outcomes LESLIE L. ROOS, NORALOU P. ROOS, ELLIOTT S. FISHER, and THOMAS A. BUBOLZ Health care data bases of varying scope and quality exist in a number of dif- ferent settings: research groups, hospitals, insurers, and governmental agencies. Of particular interest are the data generated by health insurance systems in Norm America, Europe, Australia, and New Zealand. Because health care data collected for administrative purposes are evermore available and less expensive to analyze, it is not surprising that such data bases are increasingly used in tech- nology assessment and health policy research (1,2,3~. Moreover, their use is explicitly advocated in the Patient Outcomes Research Team approach, estab- lished by the Agency for Health Care Policy and Research. What kind of information from administrative data bases is useful for clinical analyses? Many American data bases, such as Medicare, commonly provide He following data from hospital discharge abstracts: *Some of the material in this paper has appeared in: Roos LL, Sharp SM, Cohen MM, Wajda A. Risk adjustment in claims-based research: The search for efficient approaches. Journal of Clinical Epidemiology 1989;42:1193-1206; and Roos LL. Nonexperimental data systems in surgery. International Journal of Technology Assessment in Health Care 1989;5:341-386; and Rutkow IM (ed). Socioeconomics of Surgery. St. Louis: C.V. Mosby, 1989. This paper was supported by the Institute of Medicine, by Career Scientist Awards from Health and Welfare, Canada (to Leslie L. Roos and Noralou P. Roos), and by grants from Health and Welfare, Canada (6607-1197-44) and from the National Center for Health Services Research (5 R18 HS-05745~. 47

OCR for page 47
48 Patient-identifying information: Date of Birth Sex Place of Residence Identifying Number (individual or family) Other items for analysis include: Discharge Diagnoses (several) Procedures Performed in Hospital (several) Hospital Date of Admission Date of Discharge Discharge Code (death, another hospital, home, etc.) Secondary items include: Admitting Physician Identifying Number Physician Performing Each Procedure (identifying number) LESLIE L. ROOS ET AL. Physician claims typically identify the patient, the service rendered, date of the service, and the physician. The major data bases are designed to describe patient characteristics, diagnoses, and treatments. One reason for incomplete- ness of data is that hospitals lack motivation to record information that does not have an immediate impact on reimbursement. An ideal data base would have the following characteristics: System-wide coverage of an entire population. Government-organized insurance systems are typically individual-based. Such coverage includes care received at a wide variety of institutions and from the whole universe of health care providers. Coverage of an entire population permits study of utilization from an epidemiologic perspective, attributing use to individuals according to place of residence, no matter where the services are provided. Subgroups or whole populations can be compared to see how much of any given resource is used. Such population-based data can be adjusted for age, sex, and other char- acteristics to facilitate comparisons. Unique identifying number (or combination of identifiers). When each person is identified in this manner, usage can be cumulated for each person, wherever care is received. This data base should record all contacts with the health care system for each individual, with the unique identifier available to facilitate tracing. Ideally, the data base would record all hospital care, both inpatient and outpatient, services in free-standing surgery centers, activities in physician offices, entry to nursing or personal care home, health care received at home, and prescription drug use. Thus, an individual having surgery in one setting who is readmitted to a second institution will have both contacts cap- tured by the system.

OCR for page 47
HEALTH INSURANCE DATA SYSTEMS 49 ~ Enrollment or registryfile. A file specifying when and why each individ- ual's coverage begins and ends is very useful. Such a file is necessary to tell whether an individual with no recorded contact with the health care system resided in the jurisdiction and indeed had no contact; left the jurisdiction; or died. This type of file helps to determine the percentage of individuals enjoying intervention-free survival survival without any contact with the health care system. Comprehensiveness. Data bases can be characterized by their comprehen- siveness. Some aspects of comprehensiveness can determine the design of any study, from relatively simple to relatively complex. At the simplest level of administrative data bases (Level 3), only hospital discharge abstracts are needed (4~. Level 3 data can support studies of length of stay and in-hospital mortality; when combined with coverage of a population, such information permits analy- ses of utilization across medical market areas. At the intermediate level, Level 2 data require consistent individual identifiers on hospital discharge abstracts. Hospital claims can be sorted by date and identifying number to generate hospi- talization histories for each individual. Level 2 data can thus be used for short- term outcome studies of readmissions and complications after surgery. Such research on quality assurance and cost control can provide timely feedback to health care institutions. The most comprehensive Level 1 data bases possess all the features of the Level 2 and 3 files and include an enrollment file with dates for startup, death, and leaving the insurance plan. Longitudinal studies can fol- low individuals' health care utilization through time (see Table 5.1~. A Level 1 system offering complete coverage for a population can often pro- vide large samples and impressive follow-up capabilities, whether the care be ambulatory, community, or hospital based. The proportion of individuals enjoy- ing intervention-free survival can also be ascertained. The ability to develop individual longitudinal histories (before and after an event or index hospitaliza- tion) permits identifying first-time occurrences in a population. These incident cases present a more homogeneous group for study; a second operation or recurrence of a condition can be distinguished from new events. Alternative treatments and different hospitals can be compared and analyses carried out across medical market areas on a per-person basis. STRENGTHS System-Wide, Population-Based Data System-wide coverage allows us to monitor the effectiveness of clinical treatments. Since administrative data bases are not limited to specific institu- tions, Hey include poor health outcomes which occur following discharge from an institution. This makes possible comparative studies of outcomes from insti- tutions with very different lengths of stay. Because administrative data bases

OCR for page 47
so LESLIE L. ROOS ET AL. TABLE 5.1 Data requirements and types of studies using hospital data Data Requirements Types of Studies SimpleLevel 3 Need hospital discharge abstracts Intermediate Level 2 Need hospital discharge abstracts and consistent individual identifiers ComprehensiveLevel 1 Need hospital discharge abstracts, consistent individual identifiers, and enrollment file In-hospital Mortality Volume-outcome comparisons, monitoring of individual hospitals Length of Stay Small-Area Analyses Changes over Time Timely Longitudinal Research Short-Term Readmissions Volume-Outcome Comparisons Monitoring of Individual Hospitals Quality Assurance and Cost Control Highest Quality Longitudinal Research Shortest-Term and Long-Term Outcome Studies Identification of Incident Cases Volume-Outcome Comparisons Monitoring of Individual Hospitals Choice of Treatment Small-Area Analysis by Person SOURCE: Rutkow IM (ed), Socioeconomics of Surgery. St. Louis: C. V. Mosby, 1989. cover care received by multiple providers, complications which might not be picked up in any individual practice can be detected. For example, almost half (42.6 percent) the Manitoba surgeons performing repeat resections were not the physicians who had performed the original prostatectomies (S). Patients may not return to a physician if they are dissatisfied or have poor outcomes on a treatment he or she has prescribed; without system-wide follow-up, physicians may overestimate the positive aspects of their treatment. Efficacy versus Effectiveness . . . . . This problem can be stated simply: treatments that produce excellent out- comes in a research setting (efficacious) may not be beneficial (effective) when applied to a different spectrum of patients in clinical situations. Community hospital practices and medical care outcomes may differ widely from those pub- licized by researchers at academic centers. Research on efficacy of procedures or the results of the so-called "best" situ- ation (generally a teaching hospital) are usually reported in studies of technolo- gy assessment (6~. But technology assessment is not well developed; a lack of

OCR for page 47
HEALTH INSURANCE DATA SYSTEMS 51 information on the efficacy of many procedures (7) may make physicians uncer- tain about choice of treatment (81. The rarity of a condition (9) almost always presents problems in assessing efficacy by randomized clinical trials. Non-experimental research may show clinical trials to be "so difficult to organize or so costly as to be impractical" (101. Even when clinical trials have been performed, non-experimental data bases can play a valuable role. For example, population-oriented data bases facilitate long-term follow-up of clinical trials. Claims research can also help specify the relevance of clinical trials. If clinical trials have too stringent crite- ria for entry, actual physician practice may be so different that the results are only partially applicable. Evaluation of quality of care in both types of settings can be made easier by population-based data, since studies of efficacy and effectiveness can use the same non-experimental data systems. Effectiveness studies that present out- come results from representative samples of all hospitals and all physicians are rare. Administrative data can be particularly valuable for such research. Large Numbers and Time Series An added benefit of the system-wide coverage characteristic of administra- tive data bases is the large numbers of cases and controls which can typically be identified. Administrative data bases help expand the number and type of out- comes traced. In other words, if the mortality rate is too low to permit a statisti- cally strong analysis, we can study additional poor outcomes, including complications reflected in hospital readmissions and patterns of physician visits. If there are insufficient cases in a given year, additional years can be exam- ined. Thus, a population can be tracked over a longer period to accumulate enough events to permit analyses. This is especially useful with rare conditions, such as infective endocarditis (111. Ongoing health insurance systems add a new set of observations every year. Potentially, analysts can go back to the beginning to find those items of infor- mation which are routinely recorded. These long series of data allow retrospec- tive cohort studies. For example, in 1989 a researcher can go back to surgery cases recorded in 1979 and do a 10-year follow-up. Long-term studies of health outcomes can give very different assessments of the efficacy of a given procedure. A workshop convened by the National Institutes of Health (12) suggested that after transurethral prostatectomy, "the need for further operative treatment is uncommon"; however, the cumulative eight-year probability of having a second operation was recently found to be 20.2 percent (131. Administrative data can also be used before an event of interest to define incident cases. A study of infective endocarditis listed all Manitoba patients hospitalized with the condition from April 1, 1979, to March 31, 1985. Then

OCR for page 47
52 LESLIE L. ROOS ET AL. "incident cases were identified by eliminating those individuals with previously diagnosed infective endocarditis in He April 1, 1976 - March 31, 1979 period" (11). In similar fashion, data on histories can help create clean comparison groups. For example, in a study of whether tubal ligation increases a woman's risk of having a hysterectomy, Cohen (14) identified as her control group a random sample of women aged 25 to 44 and eliminated all individuals who had a hys- terectomy prior to July 1974 or tubal ligation from 1970 through 1982. The time series characteristics of data bases can also be used to characterize individuals by health care usage/morbidity patterns to develop measures of case-mix adjustment. This application of administrative data bases is treated in detail in the section on risk adjustment. Events Unaffected by Recall We know that patient reports of drug exposure, hospitalizations, physician visits, and medical conditions are subject to recall biases. Ray and Griffin (15) note that a primary "strength of Medicaid data for pharmacoepidemiology is the availability of detailed pharmacy records from which drug exposure history can be constructed." Most evidence suggests that events such as hospitalizations and physician visits are well recorded in health insurance systems. As dis- cussed later, diagnoses recorded in the administrative data have limitations, which are often related to characteristics of medical practice; two physicians seeing the same patient will sometimes diagnose different entities. Overall, diagnoses recorded in the claims system are physician-originated and likely as accurate as patient self-reports. Accurate recording of past health events is critical in developing lifetime estimates of an exposure (such as x-ray usage) or when timing of an event is important. Thus, in assessing the effectiveness of influenza vaccine, it is impor- tant to know whether the vaccine was delivered and if it was given during the appropriate period. Unobtrusive Nature A great advantage of administrative data is that they permit relatively unob- trusive research. Because studies using these data are done as statistical analy- ses, patient consent is not sought. There are no biases because persons refuse to participate or because patients, providers, or data collectors know about the study. This is important. The biases that arise when subjects know to which group they have been randomized, or even when participants know they are involved in a study, have been discussed elsewhere (16~. Hertzman (17), for example, has shown that information on health status from an occupational group explicitly under study may differ from that obtained from a population unaware of the purpose of the study.

OCR for page 47
HEALTH INSURANCE DATA SYSTEMS 53 Multiple Comparisons The fact that individuals are not randomly assigned to comparison groups raises questions as to the comparability of individuals and hospitals being stud- ied. Such questions arise regardless of the method of risk adjustment admin- istrative data, chart review, physical examination, etc. Administrative data gen- erally give several ways to test the consistency of findings after risk adjustment. Hypotheses often can be tested among a number of subgroups in the popula- tion. In a recent study of prostatectomy, N. Roos et al. (2) found higher mortali- ty among men having transurethral prostatectomies (the more accepted opera- tion) than among those having open prostatectomies (the older operation). The risk-adjusted results from one Manitoba teaching hospital held for all men hav- ing prostatectomies and for a subgroup of the healthiest men. Testing across populations is also helpful. Comparisons using administrative data from four countries confirmed the findings of differential mortality after transurethral and open prostatectomies. Statistical models can also be compared. When several covariates are avail- able, a number of regression models can be tested for consistency. If relative risk of mortality or another dependent variable does not change as covariates are entered or deleted, faith in the findings is increased (18,19~. Design Flexibility Researchers designing cohort and case-control studies must deal with critics, friendly and otherwise, who suggest changes in the design of their study. One great advantage of administrative data is design flexibility, the ability to alter a research design with little difficulty. For example, changes in definition of exposure may involve: (a) altering the time period during which an intervention (such as immunization for flu) is seen to be relevant, and (b) redefining control groups to make them parallel with the group receiving a treatment. The vari- ables used for matching purposes can be easily changed. Finally, several designs can be used with the same administrative data base. For example, a planned study of the efficacy of influenza vaccination will utilize both cohort and case-control designs constructed from the Manitoba data base. Similar design flexibility should be possible using the Medicare data. Potential for Multiple Projects Because administrative data systems are not designed for specific studies, they can be valuable for multiple projects. For example, a set of files originally developed to examine the short-term outcomes associated with cholecystectomy were subsequently used in a study to develop computerized methods for moni- toring readmissions following surgery, changes over time in quality of care over

OCR for page 47
54 LEST 1E L. ROOS ET AL. a 10-year period, quality of care, and health care outcomes in the native and non-native communities. More recently, the literature suggests there may be an elevated risk of heart disease following cholecystectomy; these files will again be reassessed. The flip side of having a data set with the potential for multiple analyses is to be accused of being theoretical and opportunistic in one's research. Because health care data bases are not closely restricted in subject matter and because there are limits to the type of data they make available, studies should be tai- lored to their strengths. For example, coding systems do not identify laterality, so studying outcomes of procedures which can be performed only on one part of the body (prostatectomy) is much easier than studying conditions where a sec- ond procedure will not necessarily represent a complication but may be a new event (total hip replacement or cataracts). Focus on Risks Administrative data banks, by their focus on health care interventions, make possible more accurate assessment of risks associated with treatment (mortality; readmissions; specific sequelae such as prostate revisions, stricture dilations, etc.~. Given the uncertainties surrounding major areas of medical treatment such as bypass surgery and carotid enda~terectomy, it might be appropriate to con- centrate on comparing the risks associated with new and established treatments until firm data on the benefits of medical treatment are developed. Clinical Decision Making Models of the clinical decision process must present choices the way clini- cians do. These decisions may or may not require specific test results. Thus, the adequacy of administrative data for decision models depends on the condi- tion and the procedure studied. Successful modeling has been carried out for medical versus surgical treatment of infective endocarditis (11) and for watchful waiting versus surgery for prostate disease (20~. The decision tree for modeling treatment of infective endocarditis highlights the usefulness of claims data. The data base provided estimates of the probabil- ities of a number of events after two strategies: early surgery or attempted med- ical cure. Variables used in the decision tree included probabilities of operative mortality, probabilities of events (including dying and congestive heart failure) before or soon after four weeks of antibiotic therapy, probabilities of events occurring long after completion of antibiotic therapy, and life expectancies in weeks under different treatment regimes.

OCR for page 47
HEALTH INSURANCE DATA SYSTEMS 55 WEAKNESSES Structural Limitations Administrative data sets generally allow the collection of a fixed amount of information on all events for all people covered by an insurance program. These data sets are designed to answer such questions as who receives treat- ment, when was the treatment given, where was the treatment given, who gave the treatment, what was the treatment, and how much did it cost. Administrative data typically have structural limitations inherent in the record layout, available codes, and coding regulations. Such limitations can be overcome only through structural changes in either the record or the regulations. Several coding issues are of interest to the researcher. A single surgical proce- dure or hospitalization may result in a single hospital record and one or more physician bills. Linkage of the surgeon's claim to the hospital claim has been shown to be an excellent way to check on the reliability of the coding of hospital procedures. At the same time, for many procedures and diagnoses, different codes may plausibly be used to describe the same event. One physician or insurance carrier may prefer a given code, while others use different codes. Because several physicians often submit bills for treating the same condition in the same patient (surgeon, assistant, anesthetist), there is a real possibility that different codes will be used. However, multiple bills for the same event offer another way to confirm the occurrence of events or test the reliability of the initial claim. The precision of codes varies across conditions. For many conditions, the ICD-9-CM hospital codes, used in the Medicare and Manitoba data bases, and tariff codes, such as CPT, are highly precise in their specification of the proce- dure and the clinical problem. Examples include transurethral prostatectomy and carotid endarterectomy. Studying these conditions or treatments through He claims data is relatively straightforward. Sometimes the tariff codes are more precise than the ICD-9-CM codes; this seems to be the case with hip repair procedures. Other procedures may be poor- ly classified on hospital and physician claims and may be more problematic to study; vascular surgery presents difficulties in this regard. Diagnoses generally are less precise than most researchers would prefer; "congestive heart failure" and "diabetes mellitus" encompass broad ranges of severity that may mask important clinical subgroups. The detail of the coding conventions may be inadequate for some studies. The ICD-9-CM coding system does not distinguish procedures performed on the left side of the body from those done on the right side. This makes it more diffi- cult to assess the results of orthopedic surgery; a second hip or knee replacement operation on someone who has already had one may mean either a reoperation or an operation on the other extremity (21~.

OCR for page 47
56 LESLIE L. ROOS ET AL. Moreover, the data captured by administrative data systems may not be those of most interest to health outcome studies. While the data system may record the occurrence of certain events (laboratory tests, x-rays, pap smears, etc.), the results of these tests typically are not available in an administrative data system. In fact, before beginning a study with an administrative data bank, the key question is: "Can the event of interest be defined in the system and are key out- comes captured?" The answer depends on the specific recording systems used. It may be several years before a new procedure, such as angioplasty, is accu- rately recorded in this system. Finally, the timing of diagnoses during a hospitalization cannot be deter- mined from the discharge abstract alone. Consequently, conditions that develop in the course of treatment cannot be distinguished from comorbid conditions present at the time of admission an important distinction for risk-adjusted out- come analyses (22,23~. For example, Medicare patients who develop a pul- monary embolus after surgery cannot be reliably distinguished from those who had the condition before the operation. Other data systems (such as that in Manitoba) may be able to make this distinction by linking physician claims. Ongoing work is directed toward estimating the probability that such conditions will develop during surgical hospitalization for a number of procedures. Bias Due to Reporting and Coding There are several threats to the validity of claims data (24~. The data sub- mission process and coding of the data can lead to reporting and coding errors. However, financial incentives for providers to assure adequate reimbursement and for funding agencies to minimize expenditures provide some protection against lost or inaccurate data. Another source of bias is that contacts with the system generating the data have to be initiated by someone, often the patient. The probability of contact with the system may be affected by hospital and physician supply. The accuracy of procedural and diagnostic data depends upon both the physicians and the clerks involved. American Medicare data appear to record procedures performed with fair accuracy, particularly if the "order of proce- dure" is ignored. Medicare data quality may have gone up since the introduc- tion of the Prospective Payment System, but diagnostic information may not be as accurate as in the Manitoba files (25,26,27~. Medicare data also do not include outpatient information in the hospital file. In Manitoba, both surgical procedures performed in hospitals and discrete billable items (even if not major events, including tests such as pap smears) appear to be reliably captured in the claims system. The quality of diagnostic data also depends upon the source. Diagnoses on

OCR for page 47
HEALTH INSURANCE DATA SYSTEMS 57 hospital records are likely to be more accurate than diagnoses on claims gener- ated by physician's visits. In Manitoba, diagnoses are noted with a reasonable degree of accuracy and specificity in the hospital system, reflecting the profes- sional training of medical records technicians. A comparison of diagnoses recorded on hospital records with those reported in the claims showed 95 per- cent correspondence in gallbladder disease, and 89 to 92 percent correspon- dence in a study of acute myocardial infarction (28,29~. Although Medicare does not include ambulatory care diagnoses with the physician claims, other data systems may contain this diagnostic information. Such diagnoses are useful at a more general level. One fruitful approach in Manitoba has been to group diagnoses available from physician claims (for example, contacts for gynecologic problems in a study of women undergoing hysterectomy, and gallbladder disease and abdominal pain for a study of con- tacts before and after gallbladder surgery) rather than to attempt fine diagnostic distinctions (25,30~. Bias Due to Differential Contact As noted earlier, contact bias is a threat to the interpretation of claims data; the individual rather than the researcher generally initiates contact with the sys- tem generating the data. Thus, a person who is ill, but has no contact with the health care system, does not produce a record on this episode of illness or chronic condition. Such contact can be important for studying outcomes. For example, Manitoba research has used readmission to hospital in the three months after hysterectomy as an indication of post-surgical complications. The probability of an individual contacting a physician or being hospitalized varies with certain system characteristics (such as insurance coverage and sup- ply factors), individual characteristics (care-seeking behavior), and physician factors (propensity to hospitalize) (31~. Given universal insurance, relatively few ill individuals lack contact with the health care system when the measure- ment period is several years (32~. In the United States, however, co-payment is likely to accentuate contact bias. Poorly covered individuals may be precisely those who receive the poorest care; analyses thus may underestimate poor out- comes. Supply factors are important and readily studied. Assuming similar insur- ance coverage for all members of a political unit, the supply of physicians and hospital beds has been shown to affect system usage (33~. Supply variables have been shown to be statistically significant in predicting such outcomes as readmissions. Data on bed and physician supply per capita generally are fairly easy to obtain for different geographic units. By controlling for these factors on a small-area basis, analyses of readmissions and other utilization can continue in a statistically sound manner.

OCR for page 47
58 LESLIE L. ROOS ET AL. Benefits of Treatment It is difficult to identify benefits of treatment in an administrative data sys- tem. Estimates of quality of life are very indirect. Changes in the frequency of diagnoses and hospitalization provide some information, and periods of inter- vention-free survival following a key event can be calculated. These variables may be unsatisfactory as a measure of real benefit of the procedure, although some studies show substantively significant relationships between utilization and morbidity (32,341. CONTROVERSIAL AREAS Risk Adjustment Risk adjustment poses a major problem in evaluating outcomes across hospi- tals and physicians (351. If patients operated upon at Hospital A have higher mortality and complication rates than patients operated upon at Hospital B. is it because Hospital A's operating team is less skilled? Or is it because the case- mix of patients at the two institutions is different, with Hospital A treating high- er-risk patients? One issue with significant implications for studies of quality assurance and cost control is: when can claims data alone be used for these controls and when is prospective data collection necessary? What controls are good enough for testing hypotheses about the relationship between surgical volume and treat- ment outcomes, for distinguishing the better of two treatments, and for identify- ing hospitals or physicians with particularly poor or especially good outcomes? The issue of how much additional information is provided per unit of cost is vital when expensive primary data collection is being considered. Researchers have assumed that the optimal approach would incorporate primary data collec- tion, possibly combining clinical judgment with physiologic information and diagnostic testing (18,19,231. On the other hand, the ability of researchers and clinicians to predict the morbidity and mortality following medical and surgical treatment is clearly limited. Figure 5.1 illustrates our view of the utility of information. The variation explained is presented on the Y axis, while the X axis measures effort. The pre- dictive power provided by better algorithms applied to a given data type reaches a "flat of the curve" situation fairly quickly. Figure 5.1 suggests the greater pre- dictive power of the first covariates in a multivariate analysis. If primary data are collected, they may well be among the best predictors (361. But when sev- eral measures are available, they are largely substitutable for each other. One promising taxonomy for comorbidity takes into account not only the number but also the seriousness of comorbid diseases. The comorbidity index of Charlson et al. (37) explained a higher proportion of the variance in one-year

OCR for page 47
HEALTH INSURANCE DATA SYSTEMS -1 C:) IL o In In o > CD go Oh G o C) C) LIJ Q 59 Asymptote for claims, prospective data and lab tests - Asymptote for claims and prospective data /// it// Asymptote for claims data alone 7 Asymptotes will vary according to conditions and procedures involved. AMOUNT OF EFFORT INVOLVED IN PREDICTING OUTCOMES ~ FIGURE 5.1 Analytical effort involved to produce results for different types of data. Asymptotes will vary according to conditions and procedures involved. SOURCE: Roos LL, Sharp SM, Cohen MM, Wajda A. Risk adjustment in clams-based research: The search for efficient approaches. Journal of Clinical Epidemiology 1989;42:1193-1206. survival rates than a model based solely on the number of comorbid diseases. In a test population with a large set of clinical and demographic variables, age and the comorbidity index were found to be the only significant predictors of death attributable to comorbid disease. This index has been used in a number of claims-based studies (2,18,191. Computerized hospital admission/separation abstracts can be used to gener- ate covariates, such as the Charlson comorbidity index, for risk adjustment. In assessments done in Manitoba, the addition of other sorts of information (claims from physician visits, health status indices from surveys, and even some prospectively collected clinical data) generated little additional power in pre- dicting hospitalization, nursing home entry, and mortality (19,38~. Manitoba Level 3 data (from the surgical event alone) using age, sex, and limited comorbidity information have provided almost as good risk adjustment

OCR for page 47
60 LESLIE L. ROOS ET AL. in predicting mortality and post-surgical readmissions as Level 1 data (from the history of hospitalizations in the preceding six months and the surgical event). A model using only prognostic data (comorbidity inflation from the comput- erized history preceding surgery) also resulted in fairly good risk adjustment and similar overall results. Thus, Blumberg's (22) concerns about using infor- mation from the index hospitalization, rather than prognostic data, do not seem critical. Considerable progress in adjusting for risk by chart review has also been made. Daley et al. (23) have built upon the APACHE II system to develop a chart-based clinical risk adjustment system, the Medicare Mortality Predictor System, to predict hospital mortality. However, when researchers using inex- pensive nonintrusive measures such as claims must decide whether to invest scarce resources in more data collection, they must evaluate the likely yield of the additional information (391. It is difficult to find the proper point or points between "gold standard" technology assessment research that relies on exten- sive primary data collection and somewhat less accurate but cheaper and more timely approaches. We need research to compare the power of additional chart review with claims-based work. Direct comparisons of predictive power and biases would define whether widespread additional data gathering is cost effec- tive in risk adjustment. If cross-sectional data can accurately identify patients at different degrees of risk, large-scale studies of in-hospital mortality following surgery become rela- tively easy to conduct. The literature comparing outcomes across institutions is buttressed by research supporting the validity of controls generated by cross- sectional data (40,411. Claims-based research certainly suggests that useful gen- eral covariates can be produced; different covariates need not necessarily be generated for each treatment or condition studied (19,361. Outcome Measures Some outcome measures require labor-intensive data collection through patient interviews or hospital records review. On the other hand, administrative data, such as insurance claims, provide an excellent source for nonintrusive measures such as readmissions and mortality. Because many data bases are maintained and updated for administrative purposes, analyses can be done for a relatively small marginal cost. Most of our knowledge about variation in outcomes is derived from studies using nonintrusive measures. Such measures can be particularly valuable in screening large data bases "to flag events and caregivers with suspect profiles of performance" (421. Death is easily documented, usually from multiple sources such as death certificates, hospital reports, and insurance claims. However, as mortality rates decline, the number of deaths, particularly follow- ing single procedures or treatments, becomes very small. Thus, the study of

OCR for page 47
HEALTH INSURANCE DATA SYSTEMS 61 non-fatal events (morbidity) and effects on quality of life has become more important in recent years. "Intervention-free survival" has been useful for studying surgical outcomes, and claims data might also be used to measure remission-free years for chronic diseases. Other nonintrusive measures based on claims data are important here: 1. Short-term readmission to hospital, within a specified period after surgery and for post-surgical complications. Building on previous work (43), panels of specialists, meeting under the auspices of the Health Care Financing Administration (HFCA), have developed lists of reasons for readmission, which indicate possible complications after a number of common procedures; 2. Additional surgery after the initial operation; 3. Long-term problems leading to hospital readmission, such as myocardial infarction and stroke; and 4. Subsequent physician visits with diagnoses indicating continuing prob- lems. Survey measures have been widely used. Their strength is the information they provide on attitudes, feelings, and tradeoffs; their weakness has been the cost of data collection (44~. Self-perceived health, ability to perform activities of daily living, and ability to live independently in the community also are important for assessing health status. Finally, outcome studies focusing on providers generally emphasize patient satisfaction and physician performance standards. EXPANDING DATA BASES Record Linkage Record linkage the combining of separate records of the same individu- al is a powerful new research tool. Linking specialized data bases with multi- purpose claims data presents many research options, greatly increasing the amount and quality of data on individuals. Such capabilities are important because, no matter how much is recorded in any data base, specific items desired for a given study may not be available. Linkage can help make clini- cians more comfortable with using administrative data; an expanded amount of information can provide many of the details clinicians associate with the prac- tice of medicine. Record linkage helps deal with questions like: Does a given data set have enough detail to support research on efficacy and effectiveness? Are the data accurate and complete enough, and suitable for the purposes to which they are put? Additional information may be contained in other sources which permit linkage to an existing data base. In particular, administrative data bases often do not include certain tests or x-rays if they are not billable, and the results of

OCR for page 47
62 LESLIE L. ROOS ET AL. tests frequently are not included. Information on medical treatment (such as drugs used) typically is not available, making it difficult to compare medical and surgical alternatives for treatment of many conditions. Although linkages involving Medicare claims typically use Social Security number, record linkage may involve files where these numbers, as well as name and address, are not available. Record linkage depends on having a sufficient number of identifiers of adequate power. Some relevant applications of record linkage are listed; the previously mentioned prostatectomy research used the first four linkages to help the Manitoba data base reach its potential (21: 1. Linkage of enrollment files or registries with Vital Statistics files to verify deaths and provide cause-of-death information. Given appropriate confidential- ity safeguards, both Canadian and American governments cooperate with requests for death matching. These linkages underlie several longitudinal stud- ies using Canadian and/or American data (451. 2. Linkage of claims with independently collected data from cancer reg- istries to provide higher-quality information on the occurrence and date of diag- nosis of cancer, thereby facilitating better case-mix controls, validity checks, and the potential for important independent studies (461. 3. Linkage of hospital and Vital Statistics information with preoperative data collected by one hospital's Anesthesia Quality Assurance Program produced a very rich data set on preoperative status of patients and operative outcomes (471. These data can help assess the efficacy of a number of surgical procedures by providing covariates (particularly the widely used American Society of Anesthesiologists' Physical Status score) to increase the credibility of claims- based analyses. 4. Linkage of hospital claims with physician claims to verify fact and date of surgery. These methods have supported extensive quality checks in Manitoba and are also being used with American Medicare data. 5. Linkage of survey information and claims to provide a fuller picture of the relationships among functional status, self-reported health status, and surgi- cal outcomes (38~. In Manitoba, linkage of two surveys of the aged may permit incorporation of the data into studies of procedures frequently done on the elderly. Although the specific linkage keys differ in each example, the expanded files have supported a diverse set of studies. These types of linkage dramatically increase the amount and quality of individual-level data. Such an approach helps connect the perspective of the clinical epidemiologist and that of the health services researcher. Specialized data bases can be combined as appropri- ate with multipurpose claims data. Claims and detailed data from other sources can be put side by side to better understand the strengths and weaknesses of each.

OCR for page 47
HEALTH INSURANCE DATA SYSTEMS 63 Record linkage is a very valuable capability for researchers using non-exper- imental data. The mathematical concepts may be unfamiliar initially, but intro- ductory texts and user-friendly software facilitate record linkage (45,48~. A considerable amount of literature examines long-term mortality due to particular occupational health risks and provides examples of linkage studies in another context (45,49~. Primary Data Collection What role does primary data collection play in claims-based research? We can specify cases which need futher checking when individual identifiers are available in administrative data sets. One purpose of primary data collection is to add detail on diagnosis or procedure. The importance of this added detail depends on the condition and procedure studied. For example, we may want to know the number of diseased vessels for research on coronary artery disease. We need information on laterality for studies of hip fractures; one needs to know if a second operation resulted from a complication or was a new proce- dure. Primary data collection, particularly chart review, can also be used to con- firm and buttress results obtained from analysis of administrative data bases. Such work can increase the clinical credibility of studies based on claims; for example, Malenka et al. (18) have reviewed Manitoba prostatectomies from one teaching hospital, generating comorbidity indices by independent chart review. The results, comparing outcomes of transurethral versus open prostatectomies, were similar to those produced from claims analyses (2~. Studies whose primary focus is collection of new information may still depend on claims data to identify patients or providers and to trace outcomes. Thus, the monitoring of hospital mortality, as done by HCFA, can help select hospitals for primary data collection. Primary data collection within the hospi- tal can be facilitated by claims data which identify individuals, by name or number, whose charts should be pulled (18~. A fruitful way to combine methods is to use administrative data to identify individuals with a surgical treatment of interest; interviews could then examine satisfaction, subjective health status, quality of life, and so forth. Not only can claims data be used to identify specific cases but the linked data set can also generate information on outcomes (18~. Similarly, studies of the appropriate- ness of care (50,51) might find it valuable to trace outcomes using enrollment files and claims data. Combining administrative data and clinical data bases can compensate for weaknesses in claims data. For example, a proposed study of angina has isolat- ed several problems with the claims and suggested ways to deal with these diff~- culties (see table on next page):

OCR for page 47
64 Limitations of Claims Difficulty in distinguishing between stable and unstable . . . angina using coding on hospital claims (discharge abstracts). In-hospital investigations will not generally appear on discharge abstracts. Information on some risk factors (smoking) and treatments (medical therapy) not available. LESLIE L. ROOS ET AL. Ways to Handle Linkage between hospital claims and more detailed clinical data will permit sensitivity testing of the importance of the stable versus unstable distinction. Many tests are billable and will appear on physician claims. Chart review may be necessary to identify the others. This information can be obtained from clinical data base. Several valuable data bases obtained by extensive chart review are available for exploring what can and cannot be done using Medicare data. The largest linked Medicare data set seems to be that supplemented with data on Key Clinical Findings from eight Peer Review Organizations in seven states. As described elsewhere (52), the data were obtained from the medical record by a modification of the MedisGroups abstraction technique. Reviewers scan the record of the hospitalization and encode abnormalities in admission symptoms, history, the results of preadmission tests if documented in the medical record, physical examinations (including vital signs), and laboratory and specialized diagnostic tests. An extensive array of ICD-9-CM diagnostic codes (up to 30) and procedure codes (up to 36) is also recorded, as are untoward events in the course of the hospitalization. DISCUSSION Administrative data are rich in information that researchers should learn to use effectively. Research has generated questions about specific issues, such as the use of claims data to study medical treatments. Other issues are organiza- tional and technical. Because outcomes research is interdisciplinary, we must develop ways to facilitate research across centers. Because it takes consider- able cost and effort to organize administrative data for research purposes, we also need efficient information management. Other questions relate to data needs: What constitutes clinically relevant information on claims data? What auxiliary information should be collected?

OCR for page 47
HEALTH INSURANCE DATA SYSTEMS 65 Technical questions include: How good are the linkages Hat tie heals care data from different sources? How should individual records be organized? How should cleaning and checking be canned out? Current collaborations among a number of centers and researchers are posing and answering such questions. REFERENCES Jencks SF, Kay T. Do frail, disabled, poor, and very old Medicare beneficiaries have higher hospital charges? Journal of the American Medical Association 1987;257:198-202. 2. Roos NP, Wennberg JE, Malenka D, McPherson K, Anderson T. Cohen MM, Ramsey E. Mortality and reoperation following open and transurethral resection of the prostate for benign prostatic hypertrophy. New England Journal of Medicine 1989;320: 112~1124. 3. Roper WL, Winkenwerder W. Hackbarth GM, Krakauer H. Effectiveness in health care: An initiative to evaluate and improve medical practice. New England Journal of Medicine 1988;319: 1197-1202. 4. Roos LL, Roos NP. Using large data bases for research on surgery. In Rutkow IM (ed). Socioeconomics of Surgery. St. Louis: C.V. Mosby, 1989:259-275. 5. Roos NP, Ramsey E. A population-based study of prostatectomy: Long term out- comes associated with differing surgical approaches. Journal of Urology 1987;137:1184-1188. 6. Brook RH, Lohr KN. Efficacy, effectiveness, variations, and quality: Boundary- crossing research. Medical Care 1985;23:710-722. 7. Patricelli RE. Employers as managers of risk, cost, and quality. Health Affairs 1987;6:75-81. 8. Wennberg JE. Improving the medical decision-making process. Health Affairs 1988;7:99-106. Peto R. What treatments for rheumatoid arthritis can best be assessed by large, sim- ple, long-term trials? British Journal of Rheumatology 1983;22:3~. 10. Wennberg JE, Mulley AG, Hanley D, Timothy RP, Fowler FJ, Roos NP, Barry MJ, McPherson K, Greenberg ER, Soule D, Bubolz T. Fisher E, Malenka D. An assess- ment of prostatectomy for benign urinary tract obstruction: Geographic variations and the evaluation of medical care outcomes. Journal of the American Medical Association 1988;259:3027-3030. Abrams HE, Detsky AS, Roos LL, Wajda A. Is there a role for surgery in the acute management of infective endocarditis? A decision analysis and medical database approach. Medical Decision Making 1988;8:165-174. 12. Grayhack IT, Sadlowski RW. Results of surgical treatment of benign prostatic hyper- plasia. In Grayhack, Wilson, Scherbenske (eds). Benign Prostatic Hyperplasia, DHEW Publication No. NIH 76-1113, 1975:125-134. A workshop sponsored by the Kidney Disease and Urology Program of the National Institute of Arthritis, Metabolism and Digestive Diseases, National Institutes of Health. 13. Wennberg JE, Roos NP, Sola L, Schori A, Jaffe R. Use of claims data systems to evaluate health care outcomes: Mortality and reoperation following prostatectomy. Journal of the American Medical Association 1987;257:933-936.

OCR for page 47
66 LESLIE L. ROOS ET AL. 14. Cohen MM. Long-te~m risk of hysterectomy after tubal sterilization. American Journal of Epidemiology 1987;125:410-419. Ray WA, Griffin MR. Use of Medicaid data for pharmacoepidemiology. American Journal of Epidemiology 1989; 129:837-849. Kramer MS, Shapiro SH. Scientific challenges in the application of randomized tri- als. Journal of the American Medical Association 1984;252:2739-2745. Hertzman C. Morbidity studies: Are population-based data a useful benchmark for studying morbidity in special groups? Canadian Journal of Public Health 1988;79:386-387. Malenka DJ, Roos NP, Fisher ES, McLerran DF, Whaley FS, Barry MJ, Bruskewitz R. Wennberg J. Further study of the increased mortality following transurethral prostatectomy: A chart-based analysis. Journal of Urology in press. 19. Roos LL, Sharp SM, Cohen MM, Wajda A. Risk adjustment in claims-based research: The search for efficient approaches. Journal of Clinical Epidemiology 1989;42: 1193-1206. 20. Barry MI, Mulley AG, Fowler FJ, Wennberg JE. Watchful waiting vs. immediate transurethral resection for symptomatic prostatism: The importance of patients' preferences. Journal of the American Medical Association 1988;259:3010-3017. 21. Roos NP, Lyttle D. Hip arthroplasty surgery in Manitoba: 1973-1978. Clinical Orthopaedics 1985;199:248-255. 22. Blumberg MS. Risk adjusting health care outcomes: A methodologic review. Medical Care Review 1986;43:351-393. 23. Daley J. Jencks S. Draper D, Lenhart G. Thomas N. WaLker J. Predicting hospital- associated mortality for Medicare patients: A method for patients with stroke, pneumonia, acute myocardial infarction, and congestive heart failure. Journal of the American Medical Association 1988;260:3617-3624. 24. Cook TD, Campbell DT. Quasi-Experimentation. Chicago: Rand McNally, 1979. 25. Demlo LK, Campbell PM. Improving hospital discharge data: Lessons from the National Hospital Discharge Survey. Medical Care 1981;19:1030-1040. 26. Hsia DC, Krushat WM, Fagan AB, Tebbutt JA, Kusserow RP. Accuracy of diag- nostic coding for Medicare patients under the prospective-payment system. New England Journal of Medicine 1988;318:352-355. 27. Roos LL, Sharp SM, Wajda A. Assessing data quality: A computerized approach. Social Science and Medicine 1989;28:175-182. 28. Roos LL, Nicol JP, Johnson C, Roos NP. Using administrative data banks for research and evaluation: A case study. Evaluation Quarterly 1979;3:236-255. 29. Roos LL, Roos NP, Cageorge SM, Nicol JP. How good are the data? Reliability of one health care data bank. Medical Care 1982;20:266-276. 30. Davis H. Was surgery needed? The Baltimore Sun: April 6, 1986. 31. Roos NP, Flowerdew G. Wajda A, Tate RB. Variations in physicians' hospitaliza- tion practices: A population-based study in Manitoba, Canada. American Journal of Public Health 1986;76:45-51. 32. Mossey JM, Roos LL. Using insurance claims to measure health status: The illness scale. Journal of Chronic Diseases (Suppl 1) 1987;40:41S-SOS. 33. Roos NP, Wennberg JE, McPherson K. Using diagnosis-related groups for studying variations in hospital admissions. Health Care Financing Review 1988;9:53-62. 34. Diaz C, Starf~eld B. Holtzman N. Mellits ED, Hankin J. SmaLky K, Benson P. Ill 16. 17. 18.

OCR for page 47
HEALTH INSURANCE DATA SYSTEMS 67 health and use of medical care: Community-based assessment of morbidity in chil- dren. Medical Care 1986;24:848-856. 35. Sloan FA, Perrin JM, Valvona J. In-hospital mortality of surgical patients: Is there an empiric basis for standard setting? Surgery 1986;99:446~53. 36. Flood AB, Scott WR. Hospital Structure and Performance. Baltimore: Johns Hopkins University Press, 1987. 37. Charlson ME, Pompei P. Ales KL, MacKenzie CR. A new method of classifying prognostic comorbidity in longitudinal studies: Development and validation. Journal of Chronic Diseases 1987;40:373-383. 38. Roos NP, Roos LL, Mossey JM, Havens BJ. Using administrative data to predict important health outcomes: Entry to hospital, nursing home, and death. Medical Care 1988;26:221-239. 39. Harrell FE, Califf RM, Pryor DB, Lee KL, Rosati RA. Evaluating the yield of med- ical tests. Journal of the American Medical Association 1982;247:2543-2546. 40. Showstack JA, Rosenfeld KE, Garnick DW, Luft HS, Schaffarzick RW, Fowles J. Association of volume with outcome of coronary artery bypass graft surgery: Scheduled vs. nonscheduled operations. Journal of the American Medical Association 1987;257:785-789. 41 U.S. Congress, Office of Technology Assessment. The Quality of Medical Care: Information for Consumers, OTA-H-386. Washington, D.C.: Government Printing Off~ce, June 1988. 42. Berwick DM. Toward an applied technology for quality measurement in health care. Medical Decision Making 1988;8:253-258. 43. Roos LL, Cageorge SM, Austen E, Lohr KN. Using computers to identify compli- cations after surgery. American Journal of Public Health 1985;75:1288-1295. 44. Fowler FJ, Wennberg JE, Timothy RP, Barry MJ, Mulley AG, Hanley D. Symptom status and the quality of life following prostatectomy. Journal of the American Medical Association 1988;259:3018-3022. 45. Newcombe HB. Handbook of Record Linkage. New York: Oxford University Press, 1988. 46. Cohen MM, Hammarstrand KM. Papanicolaou testing without a cytology registry. American Journalof Epidemiology 1989;129:388-394. 47. Cohen MM, Duncan PG. Physical status score and trends in anesthetic complica- tions. Journal of Clinical Epidemiology 1988;41 :83-90. 48. Wajda A, Roos LL. Simplifying record linkage: Software and strategy. Computers in Biology and Medicine 1987;17:239-248. 49. Smith ME. Record linkage: Organizing the facts together. In Benneu BM, Trute B (eds). Mental Health Information Systems: Problems and Prospects. New York: Edwin Mellen Press, 1984:263-281. 50. Winslow CM, Kosecoff JB, Chassin M, Kanouse DE, Brook RH. The appropriate- ness of performing coronary artery bypass surgery. Journal of the American Medical Association 1988;260:505-509. 51. Winslow CM, Solomon DH, Chassin MR, Kosecoff J. Merrick NJ, Brook RH. The appropriateness of carotid endarterectomy. New England Journal of Medicine 1988;318:722-727. 52. Krakauer H. The use of data abstracted from medical records to assess the effec- tiveness of medical interventions. Manuscript, 1988.