Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 103
Page 103 4 Data Sources for the National Health Care Quality Report1 To be a reliable and authoritative source of information on the quality of health care in the United States, the National Health Care Quality Report (also referred to as the Quality Report) must draw from a set of data sources adequate to support measures on the components of health care quality—safety, effectiveness, “patient centeredness,” and timeliness. The set of sources must also be able to support consumer perspectives on health care needs, which include staying healthy, getting better, living with illness or disability, and coping with the end of life as they apply to each quality component. This chapter presents the major criteria that the sources for the National Health Care Quality Data Set should meet, followed by a preliminary examination of how several leading public and private data sources compare on these criteria. As discussed, the Agency for Healthcare Research and Quality (AHRQ) should pursue parallel short-term and long-term strategies in defining and using the National Health Care Quality Data Set to report on health care quality. For the next decade or so, AHRQ will have to rely mostly on current approaches to collecting data. The Medical Expenditure Panel Survey (MEPS), coupled with a Consumer Assessment of Health Plans Survey (CAHPS) component, has the potential to support measures of patient centeredness and timeliness. To support measures of effectiveness and safety, AHRQ should draw from a combination of public and private data sources such as claims and other administrative data, sur- 1 Sections of this chapter are drawn from a paper on data sources for the National Health Care Quality Report commissioned by the committee from Marsha Gold (2000).
OCR for page 104
Page 104 veys, and medical records. At the same time, AHRQ should encourage research and demonstration projects that will lead to the implementation of a robust health information infrastructure as is being assessed by the National Committee on Vital and Health Statistics (NCVHS) (U.S Department of Health and Human Services, 1999). Over the long term, fulfilling the committee's vision of a comprehensive Quality Report will be facilitated by the development of electronic clinical data systems integrated with the care process itself. In addition, the data system should permit the aggregation of individual records for the purpose of examining quality of care overall and for specific population subgroups, as well as disaggregation for the purpose of examining reasons behind potential disparities. These kind of data will be available only if significant progress is made toward development of a health information infrastructure. It is imperative that efforts be made to encourage electronic access to standardized clinical data, including patient history and diagnosis, medication and ancillary service orders and results, procedures performed, and patient outcomes, in both inpatient and outpatient settings. Means of capturing community-level information on the experience of care (most commonly through surveys) will also be necessary. RECOMMENDATIONS RECOMMENDATION 7: Potential data sources for the National Health Care Quality Data Set should be assessed according to the following criteria: credibility and validity of the data, national scope and potential to provide state-level detail, availability and consistency of the data over time and across sources, timeliness of the data, ability to support population subgroup and condition-specific analyses, and public accessibility of the data. In addition, in order to support the framework, the ensemble of data sources defined for the National Health Care Quality Data Set should be comprehensive. The data sources that are intended to support the long-term goal of a National Health Care Quality Data Set must meet certain high standards to support analysis of the state of health care quality in the United States. Although these criteria are not exhaustive, they do include the essential ideal features that should characterize data sources for the Quality Report in the future. When current data collection efforts do not fulfill these criteria, AHRQ should explore ways to enhance existent data sources and establish new data collection and reporting systems that exhibit these characteristics, in collaboration with the appropriate entities in the public and private sectors. RECOMMENDATION 8: The Agency for Healthcare Research and Quality will have to draw on a mosaic of public and private
OCR for page 105
Page 105 data sources for the National Health Care Quality Data Set. Existent data sources will have to be complemented by the development of new ones in order to address all of the aspects included in the proposed framework and resulting measure set. Over the coming decade, the evolution of a comprehensive health information infrastructure including standardized, electronic clinical data systems will greatly facilitate the definition of an integrated and comprehensive data set for the Quality Report. Elsewhere, the committee has recommended the definition of a wideranging set of measures for the National Health Care Quality Data Set based on the proposed framework and specified criteria (see Recommendations 1, 2 and 4.) To create the data set, the Agency for Healthcare Research and Quality will have to rely on a number of data sources. A formal and exhaustive review of data sources based on the suggested criteria (see Recommendation 7) will be needed. This process will be used to determine how presently available data sources can best be used and which others will have to be developed (particularly for the framework elements of safety and coping with the end of life). A preliminary and limited evaluation of several candidate data sources suggests that a combination of MEPS and CAHPS may have the best potential to supply data for measures of patient centeredness and aspects of timeliness. However, the CAHPS component presently planned for MEPS will have to include additional questions in order to meet the data requirements for these two components of quality and related consumer perspectives on health care needs. To assess effectiveness and safety and relevant health care needs, a combination of public and private data sources should be used, including MEPS, other population surveys, claims and other administrative data, medical record abstraction, and new data sources that will have to be developed. Administrative data, such as Medicare claims, represent one of the most practical and cost-effective data sources on selected components of health care quality available today. Although they may have important limitations, they can be used to identify areas that require closer study through other means such as surveys and medical record abstraction. Whenever possible, AHRQ should pursue data strategies that encourage the collection of standardized clinical data in electronic form as a part of the care process. Although there are many clinical and administrative reasons for using this type of information, in the long run it will also provide the best data on components of quality and consumer health care needs allowing for a more textured picture of quality. RECOMMENDATION 9: The data for the National Health Care Quality Report should be both nationally representative and, in the long term, reportable at the state level.
OCR for page 106
Page 106 By measuring health care quality at the national and state levels, the National Health Care Quality Report would provide benchmarks to judge how well health care delivery systems are performing at the state level in comparison to the nation as a whole. The ability to examine certain quality measures across states would substantially enhance the policy relevance, visibility, and usefulness of the report. In some cases, the available data will yield estimates that may be fairly precise for larger states, but not as precise for smaller states. In such instances, data for smaller states might be aggregated over several years before being reported. States should also be allowed to supplement the sample size called for by federal reporting requirements and fund additional data collection efforts to produce more detailed estimates. Local-level identifiers such as zip codes can be used to examine specific subpopulations when needed. Since health care is inherently a local phenomenon, further detail on the quality of care for geographic units smaller than states is usually required to address potential problems at the provider and organizational levels. However, this level of detail should generally correspond to other regional or specialized reports since the purpose of the National Health Care Quality Report is to examine the quality of care provided by the system as a whole, not by individual providers, localities, or health plans. DATA SOURCE SELECTION CRITERIA It is not surprising that national health care quality measurement demands much from potential data sources, given the fact that quality of care is a complex topic. The six criteria listed in Box 4.1 help to specify and clarify what the data source needs are. These represent the combination of characteristics of an ideal data source. It is unlikely that every potential data source will meet all of them, and a data source does not have to do so in order to qualify for the National Health Care Quality Data Set. The criteria are listed in Box 4.1 in approximate order of importance. BOX 4.1 Desirable Attributes for Sources for the National Health Care Quality Data Set 1. Credibility and validity of the data 2. National scope and potential to provide state-level detail 3. Availability and consistency of the data over time and across sources 4. Timeliness of the data 5. Ability to support subgroup- and condition-specific analyses 6. Public accessibility of the data
OCR for page 107
Page 107 Credibility and Validity of the Data The value of the Quality Report will largely reflect the credibility of the measures and the corresponding data sources. Two important determinants of credibility are the underlying validity and reliability of the data. This means that sources that meet established professional standards for data are likely to be more credible than those that do not. Factors that should be considered in evaluating the quality of potential sources of data include their prior use in research, the availability of good documentation, and review by researchers and others of their suitability for use in the Quality Report. National Scope and Potential to Provide State-Level Detail Data for the Quality Report should cover the nation and should be collected using methods that limit the bias that would otherwise exist if particular populations or geographical locales were systematically excluded or underrepresented in the sample. This means that data available nationally or from all states are more desirable than those covering only a subset of states. In addition, it is recommended that quality data be reportable at the individual state level in the long term; therefore, data sources that provide this level of detail—or have the potential to do so—should be preferred. Also, data sources (in particular, those based on population surveys) that cover all of the people in the United States are better than those that leave out subsets of the population (for example, the homeless, the uninsured, or immigrants) or particular health care settings (for example, nursing homes). Such omissions present problems for the representativeness of the data, especially when sources of comparable data for those excluded do not exist. In general, it is desirable to collect national information that is sufficiently detailed to support estimates for states and for subgroups in states that are defined by demographic variables (such as race or income) and health conditions. However, the sample sizes required to support state estimates are obviously much larger than those required to support a single national estimate or even a national estimate together with a few broad regional or subgroup comparisons. For example, a common sample size for a survey might be 1,000 respondents for each reportable domain; a measure that might feasibly be collected by a national survey could be very expensive at the state level. Even if a survey is large enough to support estimates at the state level for some general measures, it might not do so for measures that apply only to subpopulations such as those with specific health care needs. Therefore, states should be given the opportunity to supplement sample size to produce reliable state-level estimates. Although the ability to collect state-level estimates is preferable, a measure of an important aspect of health care should not be rejected because it is not available at the state level.
OCR for page 108
Page 108 Availability and Consistency of the Data Over Time and Across Sources The Quality Report must track the quality of health care and identify areas for improvement over time. Data sources that are collected on an ongoing basis are more useful for this purpose than those that are produced occasionally or at irregular intervals. Data sources based on similarly defined populations are preferable because they can be used to construct uniform denominators across measures and over time that will allow for valid comparisons. Consistency can be fostered by selecting or developing data sources where data are gathered in a standardized fashion based on uniform definitions and denominators. The National Committee on Vital and Health Statistics is addressing many of these key health data issues (U.S. Department of Health and Human Services, 1999). As a practical matter, however, the way data elements are defined may change or be improved over time, potentially enriching the content of the Quality Report but jeopardizing the ability to compare quality from one period to the next. Whenever possible, maintaining the continuity of data sources over time and across sources ought to be an important objective. One can reconcile the need for available data, the value of improved data, and the need for consistent data over time by giving preference to data sources in which changes are well documented, systematically introduced, and thoughtfully made. Although stable and consistent data sources are ideal and needed to track changes in quality over time, cross-sectional data from occasional data sources are also important. They will be useful for examining specific topics and, in the short term, may be the only way to obtain information for some of the measures. Timeliness of the Data Data available on a reasonably timely basis should be favored over data for which the lag between collection and availability is substantial. A reasonably timely basis can amount to as much as a three-year time lag, but a year or less would be better since the data would be more valuable to policy makers in assessing the effects of innovations. 2 Further, more timely data provide feedback on the current system, rather than on the system as it existed several years ago. The report should also include measures that reflect any recent systemwide interventions for quality improvement that require assessment. 2 This criterion applies to the data source as a whole and not to individual measures or indicators. Some data sources will include measures for which yearly data collection is impractical or unnecessary. However, the data source should preferably be available every year even if all the data elements are not updated yearly.
OCR for page 109
Page 109 Ability to Support Subgroup- and Condition-Specific Analyses Because equity in the quality of care for different population groups and quality of care for specific health conditions should be examined in the Quality Report, data sources must allow the definition of consistent estimates for various subgroups of the population such as children, the poor, minorities, the uninsured, and other vulnerable populations, as well as people with specific health conditions. In many cases, it will be necessary to oversample population subgroups of interest in order to obtain reliable estimates. This makes both the content and the sample design (if relevant) of data sources important to an assessment of data quality. For example, data sources that capture sociodemographic characteristics and specific health conditions and that include adequate numbers of these subpopulations should be stronger contenders for inclusion than those that do not. Similarly, individual-level or discrete data that can be used to generate diverse estimates are more suitable than sources that provide only aggregate measures. Public Accessibility of the Data As mentioned in the discussion of credibility, data for the Quality Report should be widely accepted and respected. One way to achieve this is to focus on data in the public domain, either because they are drawn from a public data source or because they are drawn from a private data source that is routinely available to public agencies. Some data from private sources may be made available to public agencies under strict procedures to ensure patient or respondent confidentiality. In some instances, organizational confidentiality may be ensured (for example, hospital reports on adverse drug events), even though state estimates may be made public. Regardless of the source, there should be a reasonable guarantee of availability to the public and of predictability in the cost of acquiring the data for the report. In some cases, part or all of the data will be made available only to researchers. This is also acceptable but public accessibility is more desirable. For example, AHRQ has recently opened the CCFS Data Center, which allows researchers access to MEPS data files not available for public use (Agency for Healthcare Research and Quality, 2001). POTENTIAL DATA SOURCES The criteria for discrete data sources and the requirement of comprehensiveness for the entire set of data sources can be used to assess candidate sources for the report. This holds true for both the short and the long term. As noted, in the short term, a mosaic of data sources will make up the National Health Care Quality Data Set. The criteria will help to rule out some sources and to clarify the strengths and weaknesses of others. In addition, the criteria indi-
OCR for page 110
Page 110 cate how the data sources used initially can be improved to provide better, more complete information. In the long term, however, the development of electronic clinical data systems will support data that more adequately meet the criteria of comprehensiveness, as well as national- and state-level coverage, availability and consistency over time, timeliness, support for subgroup- and condition-specific analyses, and public accessibility. The following is a description of potential public and private data sources that can be used in the National Health Care Quality Data Set in the next several years. It should be noted that the distinction between public and private data sources is not always a clear one. Although data sources may be produced by public entities such as federal and state governments, this does not necessarily mean that they can be accessed easily and used without restriction. For example, those that contain information on health care for individuals generate confidentiality concerns that can strictly limit their use. Private data sources run the gamut from those that have minor restrictions to those that are proprietary. Generally, proprietary sources can be used only for a fee or by meeting other requirements such as organizational membership. Public Data Sources There are several kinds of public data sources, including population-based health surveys and payer and provider data. Most of these data sources can provide only national or regional estimates. Some, however, include state-level detail. This section contains a brief overview of some of the public data sources that could be used in the National Health Care Quality Data Set. This is followed by a discussion of how these sources fulfill the criteria proposed by the committee and how they cover specific components of quality. They are listed alphabetically. Behavioral Risk Factor Surveillance Survey The Behavioral Risk Factor Surveillance Survey (BRFSS) is a state-administered survey. It is designed for telephone administration, and it has core sections and optional modules. Topics for the core section include health status, health care access, demographics, particular diseases, and risk factors. Topics for the optional module include health care coverage and utilization, health care satisfaction, preventive behavior and practices, and other diseases and risk factors. In addition, states can add their own questions on matters of particular local interest, as Oklahoma did when it examined stress and other health issues following the 1995 bombing in Oklahoma City. Sample size varies across states and ranges from 1,800 to 7,500 people per year depending on the state (Centers for Disease Control and Prevention, 2000a; Powell-Griner, 2000).
OCR for page 111
Page 111 Consumer Assessment of Health Plans Survey The Consumer Assessment of Health Plans Survey is a survey and reporting tool. CAHPS is actually a family of surveys and consists of several core questionnaires that can be combined with supplements on special topics (Agency for Healthcare Research and Quality, 2000a). The core questionnaires include the adult core questionnaire, the Medicaid managed care questionnaire, the child core questionnaire, the child Medicaid managed care questionnaire, and the Medicare managed care questionnaire. Supplements include those on mental health care, prescription medicine, and communication with providers. CAHPS is used by more than 20 states, 10 employer groups, the National Committee on Quality Assurance (NCQA), the Agency for Healthcare Research and Quality, the Health Care Financing Administration (HCFA) for Medicare, the Federal Employees Health Benefits Program (FEHBP), the Ford Motor Company, and a number of health plans (Agency for Healthcare Research and Quality, 2000a). Only recently have selected items in CAHPS been administered by AHRQ within MEPS. The National CAHPS Benchmarking Database, funded by AHRQ, publishes CAHPS data yearly in each of the three major sectors (commercial, Medicare, and Medicaid) (Agency for Healthcare Research and Quality, 2000b). CAHPS data are also released annually by NCQA, by HCFA for Medicare, and by state Medicaid programs. Healthcare Cost and Utilization Project The Healthcare Cost and Utilization Project (HCUP) is a family of databases created from data in the Nationwide Inpatient Sample (NIS) and from the State Inpatient Databases (SID). The Nationwide Inpatient Sample is based on a national sample of more than 1,000 hospitals. State Inpatient Databases consist of inpatient data collected voluntarily by community hospitals for use in HCUP and now cover 31 states (Agency for Healthcare Research and Quality, 2000c). HCUP quality indicators (QIs) are a set of 33 clinical performance measures drawn from HCUP databases. The measures concern the quality of inpatient care and access to primary care. The measure set is currently being revised and expanded (Agency for Healthcare Research and Quality, 2000c). HCUP QIs involve three dimensions of care. First, there are adverse hospital outcomes, which include inpatient mortality rates among low-risk patients who have common elective procedures and complication rates related to events that occur during hospitalization. The second dimension is potentially inappropriate use of hospital procedures, which include utilization rates for procedures identified as overused or underused. Third, there are potentially avoidable hospital admissions, which are indirect measures of access to—and appropriateness of—primary care (Agency for Healthcare Research and Quality, 2000c).
OCR for page 112
Page 112 Medical Expenditure Panel Survey The Medical Expenditure Panel Survey is a nationally representative survey of health care utilization, spending, insurance coverage, and other data elements. It applies mainly to noninstitutionalized civilians, although there is also a MEPS survey of nursing home residents. Since 1997, MEPS has been conducted annually, with the National Health Interview Survey (NHIS) national core sample as the sampling frame. It contains cross-sectional and longitudinal data. The main component of MEPS is a household survey of noninstitutionalized civilians (roughly 10,000 families and 24,000 individuals) that yields data at the household and individual levels. It can be used to produce estimates at the national and regional levels, but not at the state level. It asks respondents about their health conditions, status, access to care, use of various care settings, prescribed and over-the-counter medicines, and medical expenses for the prior two years (Agency for Healthcare Research and Quality, 2000d). Other core components are followback surveys of insurers and providers, including physicians, hospitals, and pharmacies. Followback surveys are used to validate and supplement information provided in the household component and to support analyses of individual behavior and choices (Agency for Healthcare Research and Quality, 2000e). The Healthcare Research and Quality Act of 1999 calls for MEPS to be expanded in several ways to improve its capacity as a major data source on the quality of care. These include the collection of data needed “to study the relationships between health care quality, outcomes, access, use, and cost, measure changes over time, and monitor the overall national impact of Federal and State policy changes on health care,” as well as “the quality of care and patient outcomes for frequently occurring clinical conditions for a nationally representative sample of the population including rural residents” (Healthcare Research and Quality Act, 1999). In response to the Healthcare Research and Quality Act, AHRQ is planning changes for MEPS that may include expanding the survey's coverage of such topics as preventive care and the treatment of particular medical conditions. AHRQ is also planning to incorporate some measures of patient experience with care borrowed from CAHPS, including consumer satisfaction, patient centeredness, and timeliness (Lefkowitz, 2000). Medicare Claims or Payer Data HCFA has developed extensive databases that feature claims-based information from intermediaries and carriers on inpatient and outpatient services for which it has paid under coverage provided by Medicare Part A (called hospital insurance) and Part B (called medical insurance) (Health Care Financing Administration, 2000b). As of 1998, about 39 million people, or 95 percent of those
OCR for page 113
Page 113 65 years of age or older were covered by Medicare Part A, Medicare Part B, or both (Health Care Financing Administration, 1999). Claims data can be linked to enrollment data to provide information on providers, up to nine patient diagnoses during an inpatient stay, medical treatment, some medical services, demographics, and mortality (Eggers, 2000; Health Care Financing Administration, 2000a). Such data could be used to target surveys of patients with specific conditions or treatments to complete the picture of their care. They have been used to make risk-adjusted outcome comparisons among states (Peterson et al., 1998) and to estimate the rate of complications following procedures (Lu-Yao et al., 1994). Claims data, however, omit care that is not covered by Medicare Parts A or B. Claims data do not reflect care that is paid by supplemental medical coverage, nor do they reflect care in the form of laboratory services, even if covered by Medicare. These data provide information on insurance claims for the approximately 85 percent of beneficiaries who are enrolled in fee-for-service Medicare, but not for the remaining beneficiaries who are enrolled in Medicare managed care (Eggers, 2000; Hannan et al., 1997). Medicare Current Beneficiary Survey The Medicare Current Beneficiary Survey (MCBS) is an ongoing longitudinal panel survey of 12,000 Medicare beneficiaries, drawn from Medicare enrollment files. It includes the institutionalized and noninstitutionalized Medicare population. Data collection covers a three-year period. The survey includes information on health care utilization, expenditures, insurance coverage, and health status, resulting in measures similar in some ways to those in MEPS. Medicare links survey data with claims data from Medicare files, but to date, the linked data have included only beneficiaries in fee-for-service arrangements. This design has complicated efforts to develop common measures of utilization by type of plan (Health Care Financing Administration, 2000a). Medicare Quality Data HCFA collects a range of other data that specifically support quality measurement, such as the Medicare Health Plan Employer Data and Information Set/Consumer Assessment of Health Plans Survey (HEDIS/CAHPS). As of 2000, most Medicare managed care organizations must use performance measures based on HEDIS, which includes the Medicare Health Outcomes Survey (HOS). Medicare CAHPS also provides information on consumer experience with Medicare managed care plans. Examples of CAHPS measures include those on effectiveness of care for specific conditions, access to or availability of care, and utilization of services. Beginning in 2001, health plans will report data in different ways. As in previous years, most will continue to report on the area
OCR for page 128
Page 128 to the system of care; (2) timeliness in obtaining care for a particular problem; and (3) timeliness within and across episodes of care. Access to the system of care coincides with one of the chief data collection objectives of the MEPS survey. The household component of MEPS contains several sections that directly or indirectly examine issues of access. For example, there is a series of questions on the regular place to which household members go for health care including specific questions that probe reasons for not having a regular place of care or for preferring to use places such as emergency rooms for care. Additional MEPS questions that are being proposed would also focus on access to care (Agency for Healthcare Research and Quality, 2000f). MEPS also contains data on the ability of household members to obtain care for a particular problem—another aspect of timeliness. For example, in the section on access to care there are questions on transportation, convenience of office hours, ease of getting appointments, and telephone access. Several questions drawn from CAHPS are also being proposed (Lefkowitz, 2000). Aside from two questions related to diabetes, they are general and not tied to specific problems or conditions. Instead, they involve individual visits and solicit responses on the ease of obtaining care as soon as it was wanted as soon as respondents or their doctors thought it necessary (Lefkowitz, 2000). It should be noted that not all condition-specific data require condition-specific questions. For example, one could assess responses to questions on timeliness from patients who are seen for chronic illnesses versus those who are seen just for preventive care or acute problems. Currently, MEPS does not have questions that specifically address timeliness within and across episodes of care. The sixth contract cycle for the HCFA PROs contains data related to timeliness within and across episodes of care that could be used as a supplement. For example, for pneumonia, there are data indicating whether and when Medicare patients received antibiotics, blood cultures, and other appropriate treatments. The same holds true for timeliness in the episode of care with respect to AMI (Jencks, 2000). However, because contract cycles target particular conditions and aspects of quality of care that can vary from cycle to cycle, relying on supplemental data from PROs can be only a very short-term strategy for supplementation. Medicare claims data could also be used to examine timeliness within and across episodes of care for Medicare beneficiaries. Although claims data have well-documented drawbacks (Fowles et al., 1995; Jollis et al., 1993; Lohr, 1990; Romano et al., 1994; Weintraub et al., 1999), they could be used to examine such dimensions as dates of diagnoses and treatments billed to Medicare, as well as some patient and provider characteristics. DATA SOURCES FOR THE NATIONAL HEALTH CARE QUALITY REPORT In the short term, a mosaic of existent data sources will be used to create the National Health Care Quality Data Set, which in turn will be used to examine a
OCR for page 129
Page 129 number of areas in health care quality as discussed in Chapter 2 and Chapter 3. However, given the many gaps in the presently available data sources on effectiveness and timeliness, AHRQ should supplement these sources with targeted medical record abstraction. To more adequately address these gaps, AHRQ should work to further the development and implementation of electronic data systems, including electronic medical records. In the long term, standardized electronic clinical information holds the key to providing data that will meet the criteria already presented in this chapter. Data Sources in the Short Term The Dartmouth Atlas of Health Care in the United States is an example of an annual report that uses the variety of existent public and private data sources to examine aspects of health care quality, including effectiveness (Dartmouth Medical School, Center for the Evaluative Clinical Sciences, 1998). In particular, the atlas makes extensive use of Medicare claims data to analyze whether beneficiaries received treatments, services, or drugs that have been proven effective or are believed to be so. For example, for the populations measured, the evidence suggests national underuse of immunization for pneumonia, certain tests and drugs for diabetes, and certain treatments for heart attacks. The atlas also presents benchmarks generated by the use of private data sources such as Kaiser-Permanente to make quality performance comparisons across the nation on a range of treatments, including those for heart attack and diabetes. AHRQ could make similar use of public and private data sources to produce useful findings on timeliness and effectiveness in the National Health Care Quality Report. In addition, it will have to use different types of data in order to cover all aspects of the framework. For example, patient surveys are usually needed to examine patient centeredness and aspects of timeliness, while claims data for billing purposes have greater potential to capture information regarding safety and effectiveness. However, data drawn from administrative records present problems as well. For example, in addition to the limits on Medicare claims data previously described, certain kinds of conditions and services tend to be underreported. Other limits include the difficulty of performing risk adjustment on claims data because necessary information is not available (Iezzoni, 1997; Institute of Medicine and National Research Council, 1999; Malin et al., 2000). The development of standardized electronic information systems to capture clinical data will be necessary to efficiently obtain detailed data on effectiveness and safety on a wide scale. As explained above, current administrative sources also provide inadequate data on safety, for different reasons. At present, safety reporting by providers and health care organizations is limited, which lessens the amount of data on safety that is available for analysis. Current reporting tends to be voluntary, confidential, and nonstandardized (Institute of Medicine, 2000). AHRQ, as part of the Quality Interagency Coordination Task Force (QuIC), is working to address
OCR for page 130
Page 130 issues related to patient safety, including issues that limit reporting (Quality Interagency Coordination Task Force, 2000). Until these issues are resolved and new data sources are developed, data on safety will be relatively sparse and incapable of supporting useful measurement. After existent data sources have been used to identify areas of effectiveness and timeliness in which closer study is needed, AHRQ could turn to targeted medical record abstraction, at least in the short term, given that electronic clinical data are not available. Information supplied by medical records includes medical history; diagnostic data such as information from the physical examination performed; presence of other diseases or comorbidities; clinical information such as the results of laboratory tests; and description of the treatment plan (Institute of Medicine and National Research Council, 1999). Compared to other administrative sources such as claims data, data from medical records tend to have greater clinical detail. For example, according to one study, medical records more clearly differentiate complications and comorbidities than do administrative data (Hannan et al., 1995). They may also include information on outcomes. In addition, they can supply information that is often needed to perform risk adjustment. For inpatient care, they differentiate between a person's condition before hospitalization and a new condition that might have arisen during hospitalization, which claims data often do not (Iezzoni, 1997; Institute of Medicine and National Research Council, 2000). RAND's QA Tools system is a data source in development that provides an example of medical record abstraction as a means of supplementing administrative data. Medical records abstracted for QA Tools supply data that can be used primarily to support measures of effectiveness, although they can also support some measures of safety, patient centeredness, and timeliness. They can also support the health care needs—staying healthy, getting better, living with illness or disability, and coping with the end of life (see Appendix B) (McGlynn, 2000). Collecting data from paper medical records can also be problematical (Institute of Medicine, 1997; Institute of Medicine, 2001; Institute of Medicine and National Research Council, 2000; McDonald et al., 1997; Palmer, 1997). Locating and abstracting physical records take time and labor, and abstracted data will likely contain some errors. However, the kind of information collected from medical records makes it easier to measure health care quality comprehensively, across components of health care quality and consumer perspectives on health care needs. It also facilitates examination of the quality of care for specific health conditions. For example, to measure effectiveness, inpatient medical records make it possible to assess whether people received prescriptions for appropriate medications given their medical profiles. To measure timeliness, inpatient medical records make it possible to identify the time at which a particular drug was administered, which a patient may not be able to recall accurately. Medical records are useful to examine certain safety problems. However, because patient injuries occur relatively infrequently, administrative files are usually necessary
OCR for page 131
Page 131 to target the search. The long-term solution to the need for accessible data on these aspects of health care quality lies in electronic clinical data systems that span health care settings. This would greatly facilitate access to information currently registered in paper medical records and should be part of a new health information infrastructure that contributes to quality reporting and improvement. Another short-term challenge with data sources should be acknowledged: it is especially difficult to obtain population-based measures for processes of care that affect relatively small populations. Examples would include those that refer to a particular chronic condition that is not very common, or to a specific procedure. A survey such as the expanded MEPS or a population-based medical record review will usually not have large enough samples of patients from such subpopulations to measure the quality of their care adequately. Typically, such measures are targeted to the appropriate subpopulation by using administrative record systems (such as claims databases) to find eligible patients and then conducting medical record reviews for those patients. Although this approach may be adequate for measures applied to members of specific health plans or being treated at specific hospitals, at this time there is no corresponding universal database that can serve as a sampling frame for collecting such data for measures of an entire population. Encouraging the Long-Term Development of Electronic Clinical Data Systems Compared to paper records, electronic clinical data systems would offer several clear advantages in promoting health care quality. For example, they could provide linkages to clinical knowledge bases needed to support health care decision making. Electronic clinical data systems would facilitate quality reporting by making it more feasible to collect more comprehensive information. Depending on how standardized they become across health care settings, they could also make it easier to produce the kind of universal database needed to support a sampling frame for measures of processes that affect small populations. Currently, the availability of medical records is a significant issue. According to a study by the General Accounting Office, one hospital it examined was not able to find the proper records 30 percent of the time (1991). According to other studies, lost, misplaced, and inaccessible paper records are not uncommon (Institute of Medicine, 1997). For the Agency for Healthcare Research and Quality to take an active role in fostering the development and implementation of electronic data systems is consistent with the Healthcare Research and Quality Act of 1999. The act calls for the agency to promote a range of innovations in health information, including the “use of computer-based health records in all settings for the development of personal health records for individual health assessment and maintenance, and for monitoring public health and outcomes of care within populations” (Healthcare Re-
OCR for page 132
Page 132 search and Quality Act, 1999:Sec. 914). Experts in the field of information technology have advocated a diverse set of solutions to encourage the development of electronic clinical data systems (Institute of Medicine, 1997; Stead, 1998; Stead et al., 2000). These include the definition of uniform data standards, the development of standard software architectures, and the use of emerging e-commerce technologies to support patient (as contrasted to facility) ownership of the record. NCVHS recently issued a report to the Secretary of Health and Human Services with recommendations for the definition of uniform data standards for the electronic exchange of patient medical record information (National Committee on Vital and Health Statistics, 2000). The implementation of these standards should facilitate the development of a health information infrastructure that could support the type of Quality Report recommended by this committee. The development and implementation of electronic clinical data systems in health care will also require several long-term strategies. These include support for medical informatics research, support for demonstration projects, and incentives for the use of electronic clinical data systems in medical practices. Incentives could be linked to billing requirements, for example, or to evidence of quality improvement and implementation of best practices. It is important to note that electronic data systems should be designed primarily to assist in patient care so that they can be used effectively and the data can be coded accurately. However, the design of such systems, especially with respect to confidentiality and consistency in terminology and coding, should reflect the need to pool data across organizations (Stead, 1998). New regulations to protect the privacy of health information could limit access to data in patient records that may be needed for the National Health Care Quality Data Set. On December 20, 2000, DHHS announced the final rule to protect the confidentiality of patients' medical records, formulated in response to one of the provisions of the Health Insurance Portability and Accountability Act (HIPAA) of 1996 (Health Insurance Portability and Accountability Act, 1996; U.S. Department of Health and Humans Services, 2000a). Under this final rule, patients have considerable control over how their health information is used. Health plans, health care clearinghouses, and health care providers who conduct certain transactions electronically must obtain patient consent to release their medical records. However, certain exceptions are allowed when the need for access to information for the public good outweighs the need to protect individual privacy. The Quality Report may be one of these cases, given that information may be disclosed without individual authorization for the “oversight of the health care system, including health assurance activities” (U.S. Department of Health and Human Services, 2000b).
OCR for page 133
Page 133 INCREASING ACCESS TO THE NATIONAL HEALTH CARE QUALITY DATA SET The data sources required to support a comprehensive set of measures of health should be made available as the National Health Care Quality Data Set. However, all information in the data set cannot be included in the Quality Report. As explained in Chapter 5, both the print and the web versions of the report should be selective in what they contain. To capture the attention and interest of consumers, the media, policy makers, and other audiences, neither version should be overly long or detailed. The Agency for Healthcare Research and Quality should make it easy for researchers and other policy specialists to use the data to explore trends, developments, and patterns in health care quality. The data set may be too large to place on the Web in its entirety. However, the agency should develop comprehensive public use data sets, along with all appropriate documentation, to the extent feasible. Where possible, researchers should be able to download data for analysis with statistical software. It would also be helpful if researchers could readily generate summary statistics, along with additional simple analyses such as cross-tabulations and other kinds of tables. Data that cannot be placed on the Web should be made as readily available as possible for use by researchers and other specialists. SUMMARY The focus of this chapter has been on data sources for the National Health Care Quality Report. The chapter has presented selection criteria to help guide the choice of data sources for the National Health Care Quality Data Set, along with a preliminary evaluation of how well several public and private data sources meet the criteria. As explained in the chapter, the Agency for Healthcare Research and Quality should pursue both short- and long-term strategies in choosing data sources. In the short term, the realization of the committee's vision for the Quality Report will be restricted by the limitations of existing data sources in terms of content, data format, and representativeness of the data with respect to the entire population. According to the committee's preliminary evaluation, MEPS combined with CAHPS has the potential to serve as an important data source for the Quality Report in the areas of patient centeredness and timeliness. However, this will have to be supplemented with other public and private data sources to adequately measure safety and effectiveness. At the same time, AHRQ should pursue a long-term strategy. Although population surveys will remain the best way of examining patient centeredness and some aspects of timeliness, AHRQ should encourage the development and broad-scale implementation of electronic clinical data systems that will provide the best data to evaluate effectiveness and particular aspects of the timeliness of care (for example, the time elapsed between diagnosis and the start of treat-
OCR for page 134
Page 134 ment). New data sources will also have to be developed to examine health care safety. Ultimately, a new health information infrastructure based on existent and new data sources (including computerized clinical data systems, population surveys, and specialized data systems) will be essential to generate an adequate database for the production of the Quality Report. This new health information infrastructure should also include data on specific population subgroups and should closely articulate local-, state-, and national level data systems. The committee is aware that considerable obstacles must be overcome in order to achieve this vision. HIPAA and regulations concerning the patient's right to confidentiality can potentially restrict access to medical records. The lack of uniform data standards impedes the aggregation of data from local to national levels as advocated by the committee. In addition, the very nature of the subject—quality of care—requires access to a wide range of information that cannot be found in any single data source or combination of existent data sources. The Quality Report should be an instrument for driving change in federal data policy such that needed data that are not currently available are collected. Although considerable, these barriers are not insurmountable, and only by making headway in this direction will it be possible to adequately assess and track the quality of health care delivery in the United States. REFERENCES Agency for Healthcare Research and Quality. 2000a. From the Pipeline of Health Services Research—CAHPS: The Story of the Consumer Assessment of Health Plans. [on-line] Available at: http://www.ahrq.gov/research/cahptrip.htm [Nov. 17, 2000]. Agency for Healthcare Research and Quality. 2000b. Consumer Assessment of Health Plans (CAHPS): Fact Sheet . [on-line] Available at: http://www.ahrq.gov/qual/cahpfact.htm [Feb. 18, 2001]. Agency for Healthcare Research and Quality. 2000c. Overview: Healthcare Cost & Utilization Project (HCUP), 1988–97. [on-line] Available at: http://www.ahrq.gov/data/hcup/hcup-pkt.htm [Nov. 17, 2000]. Agency for Healthcare Research and Quality. 2000d. Household Component. [on-line] Available at http://www.meps.ahrq.gov/Data_Pub/HC_TOC.htm [Jan. 3, 2001]. Agency for Healthcare Research and Quality. 2000e. Overview of MEPS [on-line]. Available at: http://www.meps.ahrq.gov/WhatIsMEPS/Overview.htm [Jan. 3, 2001]. Agency for Healthcare Research and Quality. 2000f. What Is MEPS? [on-line]. Available at: http://www.meps.ahrq.gov/whatis/htm [Jan. 3, 2001]. Agency for Healthcare Research and Quality. 2001. CCFS Data Center (CCFS-DC) [on-line]. Available at : http://www.meps.ahrq.gov/datacenter.htm [Jan. 24, 2001]. Arispe, Irma. 2000. Federal Data Sources for a National Quality Report. Presentation at the Institute of Medicine Workshop, “Envisioning a National Quality Report on Health Care,” May 23. Arispe, Irma, 2000a. Personal communication , November 27. National Center for Health Statistics.
OCR for page 135
Page 135 Balas, E. Andrew, Farah Jaffrey, Gilad J. Kuperman, Suzanne A. Boren, Gordon D. Brown, Francesco Pinciroli, and Joyce A. Mitchell. 1997. Electronic communication with patients. Evaluation of distance medicine technology. Journal of the American Medical Association 278(2): 152–159. Blumberg, Steven, 2000. Personal communication, December 4. Centers for Disease Control and Prevention (CDC). Centers for Disease Control and Prevention. 2000a. Tracking Major Health Risks in America: The Behavioral Risk Factor Surveillance System [on-line]. Available at: http://www.cdc.gov/nccdphp/brfss/at-a-gl.htm [Dec. 13, 2000]. Centers for Disease Control and Prevention. 2000b. The National Immunization Survey, [Online]. Available at: http://www.cdc.gov/nis/default.htm [Dec. 22, 2000]. Centers for Disease Control and Prevention. 2000c. HIP: National Nosocomial Infections Surveillance (NNIS) System, [Online]. Available at: http://www.cdc.gov/ncidid/hip/NNISfirstname.lastname@example.org [Nov. 17, 2000]. Cohen, Steven B. 2000. Existing Data Sources to Support the DHHS National Health Quality Monitoring Initiative (draft). Unpublished. Cynamon, Marcie L., 2000. Personal communication, November 30. CDC. Darby, Charles, 2001. Personal communication, February 3. Agency for Healthcare Research and Quality (AHRQ). Dartmouth Medical School, Center for the Evaluative Clinical Sciences. 1998. The Dartmouth Atlas of Health Care, 1999. Chicago: American Hospital Publishing. Dickey, Wayne, 2000. Personal communication, December 1. CDC. Economic and Statistics Administration and National Telecommunications and Information Administration. 2000. Falling Through the Net: Toward Digital Inclusion: A Report on Americans' Access to Technology Tools. Washington, D.C.: U.S. Department of Commerce. Available at: http://www.esa.doc.gov/fttn00.htm. Eggers, Paul W., 2000. Personal communication, November 30. National Institute of Diabetes and Digestive and Kidney Diseases. Elixhauser, Anne, 2000. Personal communication, November 20. AHRQ. Elstein, Paul, 2000. Personal communication, December 1. Health Care Financing Administration (HCFA). Eppig, Frank, 2000. Personal communication, December 1. HCFA. Fowles, Jinnet B., Anne G. Lawthers, Jonathan P. Weiner, Deborah W. Garnick, Doris S. Petrie, and R. Heather Palmer. 1995. Agreement between physicians' office records and Medicare Part B claims data. Health Care Financing Review 16(4): 189–199. General Accounting Office. 1991. Medical ADP Systems: Automated Medical Records Hold Promise to Improve Patient Care. Washington, D.C.: U.S. Government Printing Office. Gold, Marsha. 2000. Data Sources and Potential Indicators for a National Quality Report. Commissioned paper for the Institute of Medicine Committee on the National Quality Report on Health Care Delivery. Hankey, Benjamin F., Lynn A. Ries, and Brenda K. Edwards. 1991. The Surveillance, Epidemiology, and End Results Program: A national resource. Cancer Epidemiology, Biomarkers and Prevention 8(12): 1117–1121. Hannan, Edward L., Albert L. Siu, Dinesh Kumar, Harold Kilburn, and Mark R. Chassin. 1995. The decline in coronary artery bypass graft surgery mortality in New York State. Journal of the American Medical Association 273(3): 209–213.
OCR for page 136
Page 136 Hannan, Edward L., Michael J. Racz, James G. Jollis, and Eric D. Peterson. 1997. Using Medicare claims data to assess provider quality for CABG surgery: Does it work well enough? Health Services Research 31(6): 659–678. Health Care Financing Administration. 1999. Health Care Financing Review: Medicare and Medicaid Statistical Supplement, 1999. Baltimore, Md.: U.S. Department of Health and Human Services. Health Care Financing Administration. 2000a. HCFA: Medicare, Medicaid, and the State Children's Health Insurance Program (SCHIP) [on-line]. Available at: http://www.hcfa.gov [Dec. 4, 2000a]. Health Care Financing Administration. 2000b. Medicare Basics Overview [on-line]. Available at: http://www.medicare.gov/basics/overview.htm [Nov. 17, 2000b]. Health Insurance Portability and Accountability Act of 1996. 1996. Statutes at Large. Vol.110, Sec. 1936. Healthcare Research and Quality Act. 1999. Statutes at Large. Vol. 113, Sec. 1653. Iezzoni, Lisa I. 1997. Assessing quality using administrative data. Annals of Internal Medicine 127(8): 666–674 . Institute of Medicine 1997. The Computer-Based Patient Record. An Essential Technology for Health Care (revised edition). eds. Dick Richard S., Elaine B. Steen, and Don E. Detmer. Washington, D.C.: National Academy Press. Institute of Medicine. 2000. To Err Is Human: Building a Safer Health System. eds. Linda T. Kohn, Janet M. Corrigan, and Molla S. Donaldson. Washington, D.C.: National Academy Press. Institute of Medicine. 2001. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, D.C.: National Academy Press. Institute of Medicine and National Research Council. 2000. Enhancing Data Systems to Improve the Quality of Cancer Care. eds. Maria Hewitt and Joseph V. Simone. Washington, D.C.: National Academy Press. Jencks, Stephen F. 2000. Clinical performance measurement—A hard sell. Journal of the American Medical Association 283(15): 2015–2016. Jencks, Stephen F., Timothy Cuerdon, Dale R. Burwen, Barbara Fleming, Peter M. Houck, Annette E. Kussmaul, David S. Nilasena, Diana L. Ordin, and David R. Arday. 2000. Quality of medical care delivered to Medicare beneficiaries. Journal of the American Medical Association 284(13): 1670–1676. Johnson, Clifford L., 2000. Personal communication, December 19. Centers for Disease Control and Prevention. Joint Commission on Accreditation of Healthcare Organizations. 2000a. Welcome Page: Who We Are [on-line]. Available at: http://www.jcaho. org/who_we_are.html [Nov. 17, 2000]. Joint Commission on Accreditation of Healthcare Organizations. 2000b. Core Measures Implementation Plan [on-line]. Available at: http://www.jcaho.org/perfmeas/coremeas/implement.html [Nov. 17, 2000]. Joint Commission on the Accreditation of Healthcare Organizations. 2000c. Sentinel Event Policy and Procedures [on-line]. Available at: http://www.jcaho.org/sentinel/se_pp.html [Dec. 4, 2000]. Jollis, James, Marek Ancukiewicz, Elizabeth DeLong, David B. Pryor, Lawrence H. Muhlbaier, and Daniel Mark. 1993. Discordance of databases designed for claims payment versus clinical information systems. Annals of Internal Medicine 119(8): 844–850.
OCR for page 137
Page 137 Kinsella, Audrey. 1998. Home telecare in the United States. Journal of Telemedicine and Telecare 4(4): 195–200. Lohr, Kathleen. 1990. Use of insurance claims data in measuring quality of care. International Journal of Technology Assessment in Health Care 6(2): 263–271. Lefkowitz, Doris. 2000. Memo to Thomas Reilly, October 10. AHRQ. Lu-Yao, Grace L., Michael J. Barry, C. H. Chang, John H. Wasson, and John E. Wennberg. 1994. Transurethral resection of the prostate among Medicare beneficiaries in the United States: Time trends and outcomes. Prostate Patient Outcomes Research Team (PORT). Urology 44(5): 698–699. Malin, Jennifer L., Steven M. Asch, Eve A. Kerr, and Elizabeth A. McGlynn. 2000. Evaluating the quality of cancer care: Development of cancer quality indicators for a global quality assessment tool. Cancer 8(3): 701–707. McDonald, Clement J., J. Marc Overhage, Paul. Dexter, Blaine Y. Takesue, and Diane Dwyer. 1997. A framework for capturing clinical data from computerized sources. Annals of Internal Medicine 127(Part 2): 675–682. McGlynn, Elizabeth A. 2000. QA Tools and the National Quality Report. Presentation to the Institute of Medicine Committee on the National Quality Report on Health Care Delivery, August 21. National Center for Health Statistics. 1995. NCHS Instructions for Classifying Underlying Causes of Death, 1995. Hyattsville, Md.: U.S. Department of Health and Human Services. Available at: http://www.cdc.gov/nchs/data/2amanual.pdf. National Center for Health Statistics. 1999. Programs and Activities, DHHS Publication No. 99-1200. Hyattsville, Md.: U.S. Department of Health and Human Services. National Center for Health Statistics. 2000a. NAMCS/NHAMCS Frequently Asked Questions (FAQ's) [on-line]. Available at: www.cdc.gov/nchs/about/major/ahcd/faq.htm [Dec. 5, 2000]. National Center for Health Statistics. 2000b. NHANES-National Health and Examination Survey—Homepage [on-line]. Available at: http://www.cdc.gov/nchs/nhanes.htm [Nov. 17, 2000c]. National Center for Health Statistics. 2000c. National Health Interview Survey [on-line]. Available at: http://www.cdc.gov/nchs/nhis.htm [Nov. 17, 2000a]. National Center for Health Statistics. 2000d. State and Loca Area Integrated Telephone Survey [on-line]. Available at: http://www.cdc.gov/nchs/slaits.htm [Nov. 17, 2000]. National Center for Health Statistics. 2000e. National Vital Statistics System [on-line]. Available at: http://www.cdc.gov/nchs/nvss.htm [Jan. 24, 2001b]. National Committee for Quality Assurance. NCQA Overview [on-line]. Available at: http://www.ncqa.org/Pages/about/overview3/htm [Sept. 25, 2000]. National Committee on Vital and Health Statistics. 2000. Report on Uniform Data Standards for Patient Medical Record Information . Available at: http://ncvhs.hhs.gov/hipaa000706.pdf Palmer, R. Heather. 1997. Quality of care. Journal of the American Medical Association 277(23): 1896–1897. Peterson, Eric D., Elizabeth R. DeLong, James G. Jollis, Lawrence H. Muhlbaier, and Daniel B. Mark. 1998. The effects of New York's bypass surgery provider profiling on access to care and patient outcomes in the elderly. Journal of the American College of Cardiology 32: 993. Powell-Griner, Eve., 2000. Personal communication, November 31. CDC.
OCR for page 138
Page 138 Quality Interagency Coordination Task Force. 2000. Doing What Counts for Patient Safety: Federal Actions to Reduce Medical Errors and Their Impact Rockville, Md. Romano, Patrick S., Leslie L. Roos, Harold S. Luft, James G. Jollis, and Katherine Doliszny. 1994. A comparison of administrative versus clinical data: coronary artery bypass graft surgery as an example. Ischemic Heart Disease Patient Outcomes Research Team Journal of Clinical Epidemiology 47(3): 249–260. . Schappert, Susan M., 2000. Personal communication, December 19. Centers for Disease Control and Prevention. Schilling, Brian, 2000 . Personal communication, December 4. National Committee for Quality Assurance. Schyve, Paul, 2000. Personal Communication. Joint Commission on Accreditation of Healthcare Organizations. Sekscenski, Edward, 2000. Personal communication, December 1. Health Care Financing Administration. Stead, William W. 1998. Issues Related to Patient Medical Record Information: Testimony to National Committee on Vital and Health Statistics, Subcommittee on Standards and Security [on-line]. Available at: [Nov. 9, 2000]. Stead, William W., Randolph A. Miller, Mark A. Musen, and William R. Hersh. 2000. Integration and beyond: linking information from disparate sources and into workflow. Journal of the American Medical Informatics Association 7(2): 135–145. U.S. Department of Health and Human Services. 1999. NCVHS Charter 1999 [on-line]. Available at: http://www.mc.vanderbilt.edu/infocntr/stead/med-rec.html [Dec. 6, 2000]. U.S. Department of Health and Human Services. 2000a. Press Briefing on Federal Privacy Regulations [on-line]. Available at: http://aspe.os.dhhs.gov/admnsimp/final/brief2.htm [Feb. 6, 2001]. U.S. Department of Health and Human Services. 2000b. HHS Fact Sheet: Protecting the Privacy of Patients' Health Information, Summary of the Final Regulation [on-line]. Available at: http://aspe.os.dhhs.gov/admnsimp/pvcfact1.htm [Feb. 6, 2001]. Waidmann, Timothy A., and Shruti Rajan. 2000. Race and ethnic disparities in health care access and utilization: An examination of state variation. Medical Care Research and Review 57(1 Supplement): 181–217. Ware, John E., Mary K. Snyder W. Russell Wright, , and Allyson R. Davies. 1983. Defining and measuring patient satisfaction with medical care. Evaluation and Program Planning 6(3–4): 247–263. Weintraub, William S., Christi Deaton, Leslee Shaw, Elizabeth Mahoney, Douglas C. Morris, Candice Saunders, Debbie Canup, Stephanie Connolly, Steven Culler, Edmund R. Becker, Andrzej Kosinski, and Stephen J. Boccuzzi. 1999. Can cardiovascular clinical characteristics be identified and outcome models be developed from an in-patient claims database? American Journal of Cardiology 84(2): 166–169. Yang, Wei-Chung, 2000. Personal Communication, December 29. Research Data Assistance Center.
Representative terms from entire chapter: