Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
7 Knowing What Is Happening and What is Needed: Planning, Evaluation, and Research The goal of emergency medical services for children (EMS-C) is to achieve the best possible outcome for all acutely ill and injured children by rapidly and accurately assessing their medical needs and then providing access to the appropriate care. Those services have to be effective from both a medical and a cost perspective to ensure that the available resources can support the maximum amount of care. The three previous chapters have identified steps that the committee believes must be taken regarding education and training, organization and delivery of care, and communica- tion to ensure that children across the country have access to optimal emer- gency care. Those steps are, however, only part of what needs to be done. There is widespread agreement that more and better EMS-C data and data systems are needed. The information resources that are currently available are too limited, scattered, and unconnected to support the planning and evaluation that EMS-C needs. Without a broad and reliable base of infor- mation, it is hard for anyone emergency care providers, administrators, parents, policymakers to determine in any systematic way how successful EMS-C systems are in providing appropriate, timely care or what they ought to do to improve performance and patient outcomes. This committee believes that not all children are getting the emergency care that they need, but that the full extent and nature of the problem is not known. Assembling descriptive data that will make it possible to answer basic questions about EMS-C systems, patients, and care and to provide accountability for EMS-C system functions must, therefore, be a high prior- ity. In addition, research is needed to establish sound clinical and organiza 224
PLANNING, EVALUATION, AND RESEARCH 225 tional principles for the care that children are given and the ways it is provided. Since routine information gathering appropriate for planning and evaluation activities often is not sufficient to meet the needs of research activities, special data collection efforts are needed to add to the knowledge base for EMS-C. This chapter addresses issues of the eollect~' analysts' and use of data for planning and evaluation. It discusses actions recommended by the com- mittee to improve the quality and comprehensiveness of EMS-C data and gives special attention to creation of a national uniform data set for EMS-C. The chapter closes with recommendations for key elements of a broad re- search agenda in this field. UNDERSTANDING THE INFORMATION GAP information gathering for planning and evaluation purposes ideally would be aimed at EMS-C systems, but the initial step is to compile data on the particular role that individual system components play in emergency care. When it is possible to take the next step of linking those data, a much more comprehensive picture of emergency care will emerge. The following ques- tions reflect the committee's central concerns: · What is the structure of the system? Data on the numbers and characteristics of the facilities, emergency care providers, and services available in the system establish the context in which a system provides emergency care. · Who uses the system? Data on the demographic characteristics of the patient population such as age, sex, and residence (or location of emer- gency) are essential for understanding who needs care.) · For what is the system used? Data on the illnesses and injuries that bring children into the emergency medical services (EMS) system and on their condition (e.g., level of consciousness) establish the epidemiology of . . . pet latrlc emergencies. · What services or procedures are provided to a patient? These data are the basis for describing the process of care in an EMS system. When are services provided? In the EMS context, time intervals in various stages of care can be significant elements of the process of care. · What are the outcomes of using the system? Clinical outcomes based on functioning, patient well-being, morbidity, and mortality are of interest; so are process outcomes, such as hospital admission or referral to a tertiary care facility. · What are the global costs of the system? Both the direct and indirect economic costs of operating EMS-C systems, as well as the monetary sav- ings over time that may be realized by successful expansion of EMS-C and integration into EMS systems, are of interest. .
226 EMERGENCY MEDICAL SERVICES FOR CHILDREN With good descriptive data, planners, evaluators, and researchers can begin to answer the more significant question: How well does the system perform? Performance can be judged on the basis of outcomes or processes of care for individual cases and for populations. Clinical outcomes can be assessed across a broad spectrum (from basic physiologic and biologic mea- sures through health-related quality of lifer- ideally with severity or acuity adj-~stn~ents using appropriate tools such as those discussed In Appendix A at the end of this chapter. Various elements of the process of care such as appropriateness of triage, timeliness of treatment, or completeness of docu- mentation can be evaluated. Compliance with structural guidelines, such as provider qualifications or equipment available, is another evaluation crite- rion that can be used. Cost-effectiveness must be a consideration as well. The assumption in evaluating system process and structure is that deficien- cies in those areas are likely to have an adverse impact on the ultimate results of the care that children (or adults) receive. Even this assumption can be tested only when data exist. The need for more and better data on the volume, nature, and outcomes of emergency care and the operation of emergency care services has been widely recognized for many years (NAS/NRC, 1970a, 1978a, 1980; MacKenzie, 1989; Eisenberg et al., 1990; Lescohier et al., 1990; IOM, l991b; Seidel and Henderson, 1991; CDC, 1992b). Progress in improving data resources has been limited, however. As was noted in Chapter 2, much of what has been learned about EMS-C, including the work discussed in this report, is based on one-time studies or studies in a single institution or community, which raises problems of external validity and generalizability of the results. Nevertheless, conclusions and policies are based on these studies simply be- cause more broadly based information on the structure and operation of EMS systems, and especially on outcomes of care, is not widely available. Some individual hospitals, state and local EMS agencies, and emer- gency dispatch centers that are parts of EMS systems do have sophisticated data collection and analysis programs that generate valuable information. EMS systems, however, depend on successful coordination of services from many separate components; only rarely are these individual components able to link their data together to learn more about the complete course of a patient's emergency care. By contrast, trauma registries are able to compile detailed information on all phases of patient care but only for a small por- tion of the patient population cared for by EMS systems; they too may lack certain elements such as linkages with autopsy reports or rehabilitation fa- cilities. Even comparisons among similar kinds of system components are difficult because little or no consensus exists on how important concepts are defined or on what data are needed to operationalize those concepts. Fur- thermore, data that might already be deemed appropriate are not routinely compiled or published.
PLANNING, EVALUATION, AND RESEARCH 227 Efforts to ensure the best possible outcomes for children are also hin- dered by limitations in the clinical and health services research base. In EMS, much research is still needed to understand the clinical merits of various current practices arid to develop better ways to apply what is known. Much of the needed research demands specialized data, but some work might be done with better EMS and EMS-(: data on system structures opera- t~onal processes, and patient outcomes. Assessing the costs and cost-effectiveness of erne-rgency care and EMS systems must be a priority in times of fiscal constraint, but accurate, com- prehensive, and meaningful data are difficult to obtain. Billing information, for example, can be obtained from hospitals and insurers, but the charges assessed for services are not equivalent to the cost of providing that service. Furthermore, system costs extend far beyond the immediate care of indi- vidual patients to the overall operation of EMS agencies and hospitals. With services provided by widely varying combinations of public and pri- vate sector organizations, ways are needed to aggregate highly diverse forms of cost data. Four aspects of this information gap can be singled out for special emphasis. First, to use a conceptual framework from the health care quality assurance field, information on "structure, process, and outcomes" for EMS- C systems is critical.2 Second, ways to use information about EMS to highlight prevention needs and target related activities efficiently must be devised. Third, questions about individual components of an EMS system and about the system as a whole must be addressed. This in turn implies that information on individual patients must be available across settings and providers; thus, being able to link records is a significant requisite. Fourth, EMS-C data are needed at the local, state, and national level; the particular kinds of data needed at each level may vary, but in all cases, data collected for one level (e.g., national) should be useful at every level below that (e.g., states and localities). These points should be kept in mind in reviewing the strengths and weaknesses of current data systems discussed later in this chapter. To answer the questions and fill the information gaps posed above, three activities must receive attention. PLANNING, EVALUATION, AND RESEARCH This committee regarded planning and evaluation as lying at the heart of effective implementation: planning is required to determine how the health care system can best meet children's needs for emergency care, and evaluation is needed to assess how well that care is being delivered. It also regarded research as essential to validate the clinical merit of care that is given, to identify better kinds of care, to devise better ways to deliver that care, and to learn where best to direct prevention activities.
228 EMERGENCY MEDICAL SERVICES FOR CHILDREN This chapter discusses the importance of planning and evaluation with a particular emphasis on matters related to data and data systems. The com- mittee is not aiming to address here the details of specific data collection technologies. Rather, it is looking more broadly at the need to assemble a core of nationally comparable data on pediatric emergencies and emergency care and at special concerns about the Chit? reliability' and validity of Norma derived Mom data collection and analysis efforts; as a part of this topic, the committee advances ideas for a uniform national EMS-C data set. It then turns to research, outlining priority issues for a comprehensive research agenda. The underlying theme is that all three tasks planning, evaluation, and research require access to data, analytical resources to transform those data into meaningful information, and ways to use and disseminate the information to improve the care that children receive. Planning Planning is a crucial step and cannot be completely divorced from ei- tner research or evaluation. Planners need up-to-date descriptive informa- tion about the current state of affairs (together with a sound idea of where they want to be in the future) in a number of areas. Among these areas are epidemiology of injury and illness in the geographic area and population for which the EMS system is responsible; facilities, agencies, services, and related equipment and personnel; financial and other resources to support and maintain those providers and programs; and EMS training and retrain- ing needs. Planners need information that will help them improve the use of available resources (hence the link to evaluation), often in fairly short time frames; taking a longer perspective, they must develop sound argu- ments to justify requests for increased resources. Evaluation Evaluation is concerned with understanding whether value has been received for the resources expended on an enterprise; more formally, evalu- ation determines "what outcomes desired and undesired, anticipated and unanticipated have occurred as a result of a policy or program i" (IOM, 1990a, p. 91~. This encompasses assessments of quality of care, efficacy and effectiveness, efficiency, and new program directions and practices. EMS-C evaluation must address at least three concerns. First, it should seek to know whether the system is "doing things right": for example, whether the existing full range of services, from prehospital care through definitive inpatient and outpatient care through rehabilitation and counsel- ing, as well as prevention, has been of high quality and whether certain practices or interventions could be improved. This essentially involves a
PLANNING, EVALUATION, AND RESEARCH 229 series of quality-of-care questions (IOM, 1990b): Have all appropriate and necessary services been provided, and inappropriate and unnecessary ser- vices not provided? Have the technical and interpersonal aspects of care been adequate? These questions can be addressed through assessments of both the process and the outcomes of care, and problems can be tackled through form al programs of equality assurance and improve-men/.3 A related question is whether the system is *`doing the right things. This in turn calls for directly assessing the effectiveness Elf not the efficacy) of services and interventions and using sound practice guidelines, protocols, and criteria to determine the appropriateness and necessity of care. It also may involve eliminating unnecessary processes and practices. Effective- ness must be assessed from a cost standpoint as well as from a clinical one. Second, evaluation should be concerned with how efficiently services are being provided. Here questions might focus on whether various opera- tions dispatch, transport, patient transfer, communications and medical con- trol, and so forth are functioning optimally, without undue duplication or wasted effort and without significant gaps or lapses. A third area of evaluation underscores the link to planning as well as to the quality and efficiency questions just posed. Specifically, evaluators need to address systematically a variety of questions about the effects of changing practice and doing new things. These matters often call for longi- tudinal analyses. The importance of cross-sectional analyses should not be lost, however, insofar as regional or facility comparisons need to be made and differences understood; serial cross-sectional data can show patterns over time. Research Planning and (especially) evaluation clearly overlap with research, but important distinctions should be noted. Perhaps the most obvious is that research is often intended to answer relatively specific, detailed, or basic questions that are separate from day-to-day system operations of a program or service. EMS-C research questions cover a wide spectrum: for instance, elucidating biologic factors that contribute to the success or failure of basic or advanced life support (BLS, ALS) interventions in children, developing improved injury and illness acuity measures for children, investigating the etiology and epidemiology of childhood trauma or illnesses that account for the major part of EMS calls, predicting the emergency care needs of chil- dren with chronic or disabling illnesses, assessing outcomes of emergency care over time, or investigating factors that make public education cam- paigns about safety and healthy lifestyles successful. Thus, compared with planning and evaluation for EMS and EMS-C system operations, research asks new and different questions and requires rigorous and specialized study
230 EMERGENCY MEDICAL SERVICES FOR CHILDREN design, data collection, and analysis. Costs must not be overlooked; con- ducting studies that are clinically and statistically meaningful can be expen- sive. For purposes of this chapter, the common thread through these activi- ties is data and information. EMS-C data are now available from an assort- ment of sources? but each source provides only a particular type of data; coord~aticT~ among sources ~s usually minimal. There is also no consis- tency across EMS systems in whether data on similar activities will be collected. The next sections briefly review selected sources and limitations of data related to EMS-C and present several committee recommendations concerning coding, data collection, and a national uniform data set. The chapter then considers a research agenda for EMS-C. UNDERSTANDING CURRENT AND EMERGING SOURCES OF DATA AND DATA SYSTEMS Strengths and Weaknesses In principle, data on emergency medical care for children and the sys- tems through which it is provided are available from an assortment of sources. Those different data systems have different advantages and disadvantages, and in no case can one source provide the full range of information needed for the planning, evaluation, and research activities discussed earlier. Prehospital Services Various approaches are used in collecting information on prehospital services. Dispatch centers, which are the first point of contact for most prehospital care, generally maintain information on the nature and timing of the calls they receive and on the response to those calls. The use of en- hanced 9-1-1 and computer-assisted dispatch systems provides dispatch ser- vices with increased data capture capabilities. EMS agencies often have extensive data collection systems that capture information about the vehicles and the personnel that respond, patients and their conditions, treatments used, time intervals in stages of prehospital care, and where patients are taken for further care. Where they are re- ported, these data are a resource for assessing patient care and system op- erations. More than 40 states have developed ambulance reporting forms (although the information on them is not always forwarded to central or state offices); 29 states have some form of statewide data gathering system (chiefly but not exclusively on ambulance runs); a few states more or less systematically acquire regional or county data, do periodic surveys, or have statewide data collection programs under development or revision (Emer
PLANNING, EVALUATION, AND RESEARCH 231 gency Medical Services, 1992~. Thus, a substantial number of states have essentially no reporting requirements at all. Local or regional data collec- tion systems often exist, with or without statewide data collection. Because there is no widely accepted "standard" EMS data set, it is difficult to inte- grate data from separate systems. Cutting across all these points are the questions, often hard to answer? as to whether data are retrievable' reliable, and -valid. Hospitals Emergency Departments EDs are an especially important source of information about emergency medical care for children. They have the potential to produce the broadest array of data on emergency care because many patients receive no prehospital or inpatient services. In addition, learning more about ED care provided to children who are not seriously ill or injured is important, in part to understand better the demands on the EMS system and in part to clarify the barriers to access to primary care that form a crucial part of the health care reform debate in this country. National data on ED services will become available for the first time through the National Hospital Ambulatory Medical Care Survey (NHAMCS), a new annual survey being conducted by the National Center for Health Statistics (NCHS) of the Centers for Disease Control and Prevention (CDC), U.S. Department of Health and Human Services (DHHS). Results for 1992 should be available in late 1993 (although the sample size is too small to provide estimates for individual states or local areas). Currently, systems such as the National Electronic Injury Surveillance System (NEISS) and the Drug Abuse Warning Network collect data only on specific kinds of ED visits. Without more comprehensive ED reporting, it is difficult to track an individual, chief complaint, or type of call through the system; to assess EMS system performance; or to make comparisons among EDs. (Even if complete ED data were available, those emergency cases in which patients were admitted immediately on arrival at the hospital would not be identi- fied.) The usefulness of ED records depends not only on their accessibility but also on the adequacy of the clinical information they contain. Com- pared to prehospital data systems, the content and management of ED records are rarely addressed in the emergency medicine literature. An unpublished study conducted for the Wisconsin EMS-C demonstration project (one of those supported by the Health Resources and Services Administration tHRSA] in DHHS) found that some hospital information systems that recorded in- jury cause and ambulance information for ED registration did not include the information in the electronic medical record (Karlson and Eisenberg, 1990~. The investigators also determined that hospital billing records for
232 EMERGENCY MEDICAL SERVICES FOR CHILDREN ED patients may not include information routinely reported for inpatients (e.g., discharge status) because that information is not required by third- party payers. The patient's hospital record usually includes the missing information, but no systematic use is made of it. The investigators quote one hospital source to the effect that discharge status "would be 'easy to capture if there were some use for it"' (KarIson and Fisenl'erg' 1990- p. 10~ Inpatient Services Full inpatient records capture (in principle) more of the essential clinical information. The current state of paper-based medical records is quite lamentable, but greater use of computer-based record sys- tems (IOM, 1991a) will make much of that information more accessible. Apart from information in the hospital records of individual patients, useful information may be retrieved through various databases created from sample surveys and discharge abstract reporting systems. The National Hospital Discharge Survey (NHDS) of the NCHS, for example, collects information from a sample of acute care hospitals across the country on patient charac- teristics, diagnosis, treatment, and disposition at discharge. As was noted regarding the NHAMCS, the NHDS is not designed to provide state or local estimates. Thirty states, however, maintain their own discharge data sets; billing data are the basis of discharge reporting for 23 of these 30 states (CDC, 1992a). Discharge data are especially useful in studying injury because the di- agnostic categories describing the nature of an injury (e.g., head injury, burn, fracture) are readily identifiable. Six states also require that discharge reports with injury diagnoses include separate external cause-of-injury codes (so-called E-codes from the ICD-9-CM tInternational Classification of Dis- eases, ninth revision, clinical modification]), such as for falls, motor ve- hicle crashes, and assaults (CDC, 1992a). Adding E-codes to discharge data enables researchers, policymakers, and others to use the data in planning and assessing injury prevention efforts. Provisions for reporting E-codes are also being made in the newly revised standard hospital billing form- the "UB-92," which should be available for use in 1993 (CDC, 1992a). Discharge data tend to be more useful in studies of injury than in studies of illness because reliable methods have not been developed to identify, through ICD-9-CM diagnostic codes or other means, children hospitalized for emer- gency treatment of illness. For all patients, however, mortality reflected in discharge data is lim- ited to deaths occurring in the hospital. Children who are declared dead outside the hospital and are not transported to the hospital will not be included in discharge statistics and analyses a fact that could seriously distort conclusions and policy actions taken on the basis of such findings. One study, for instance, determined that 24 percent of pediatric injury deaths in an urban area were not seen in hospitals (Cooper et al., 1992~. Such
PLANNING, EVALUATION, AND RESEARCH 233 deaths must be identified by other means if the aim is to estimate overall mortality from injury or illness severe enough to require emergency care. Analysts also must be cautious about using discharge data to estimate the incidence of injury or illness requiring hospitalization, lest children who are transferred after admission or who are readmitted for follow-up care be counted more than once. Efforts by the loins Commission on Accreditation of Healthcare Organi- zations (JCAHO) to incorporate ongoing monitoring of hospital performance into its approach to accreditation is likely to lead to new, targeted data collection in order to report specific performance indicators (JCAHO, 1987, 1988; Jurkiewicz, 1988; O'Leary, 1991~. Trauma care is one of the areas for which indicators are now being tested. For example, an indicator of the appropriateness of airway management of comatose trauma patients is the proportion of such patients who are discharged from the ED before a me- chanical airway is established (JCAHO, 19911. A set of data elements needed to determine performance on the indicators has been identified. How efficiently and effectively data amassed through this mechanism might be used in EMS and EMS-C planning, evaluation, or research on a broad scale remains to be seen. Trauma Registries Many hospitals maintain trauma registries that capture detailed infor- mation on the care and outcomes of patients admitted with traumatic inju- ries. Where trauma systems have been established, a registry may cover all the hospitals participating in the system. A few statewide registries also exist. Unlike many other emergency care data systems, trauma registries typically include data on prehospital and hospital care. The CDC advises inclusion of core data elements in seven categories (Pollock and McClain, 1989~: demographic and identifying data; incident description; prehospital care; ED care; surgical care; anatomic diagnosis; and outcome data. Software packages are now available that permit hospi- tals to manage their registries on personal computers.4 Two long-term efforts have been made to collect data on an even broader basis to help evaluate trauma care. The Major Trauma Outcome Study (MTOS) uses data from 139 hospitals to establish broad characteristics of trauma and to develop national norms for severity-adjusted survival (Cham- pion et al., 1990b). By 1987, it had demographic, etiologic, severity, and outcome information on more than 80,000 patients of all ages; of these, just over 10 percent were pediatric patients. MTOS data have been used to calculate severity indexes and outcome norms (e.g., for probability of sur- vival) for adults and for children and to analyze topics such as causes of injury, death rates, and lengths of hospital stays.
234 EMERGENCY MEDICAL SERVICES FOR CHILDREN The National Pediatric Trauma Registry (NPTR) has focused specifi- cally on the trauma experience of children (Tepas et al., 1989~. A multi- institutional shared database designed to compile and evaluate information on all aspects of pediatric trauma care, it began in 1985. By mid-1992 it nap co~tectect Information on more than 36,000 children from 61 participat- ing centers. These data are available to any participating investigator (with ce-na~n provisos concerning publication). For example, studies on use of helicopter transport, incidence of discharge disabilities and impairments, and referral patterns for trauma centers have all been conducted using this database, and pediatric-specific trauma score norms have been generated. As valuable as trauma registries are in studying the nature and quality of trauma care, they capture data on only a limited portion of the children who suffer serious injury. As with discharge data, information on children who die outside the hospital is not incorporated into these files. Those children with traumatic injuries who are cared for in hospitals that do not maintain a trauma registry or contribute to a trauma system registry are also missed as are children treated only in the ED. The absence of these cases from the data set makes comparing outcomes of care in a trauma system with outcomes in other hospitals impossible. Similarly, investigators can- not determine from the registry whether the children who were treated out- side the trauma system were appropriately triaged. Findings from work based on the data in the two trauma registries mentioned above have not resolved a key issue in this field namely, whether pediatric-specific injury or severity scales are necessary or whether mea- sures for adults (perhaps reweighted or recalibrated in certain ways) will suffice. The discussion in Appendix 7A also touches on this controversy, and the material below about data validity reflects a similar debate about severity-of-illness measures. Generally, the committee believes that continued support for national or comprehensive regional databases of these sorts might facilitate progress in these areas, for instance by fostering the development and validation of objective outcome assessment measures (for death, disability, and quality- of-life domains) or the implementation of prospective clinical trials (e.g., on different methods for initial resuscitative care). Expansion and refine- ment of demographic and epidemiologic data sets also may help provide ongoing statistical support for development and evaluation of regionalized systems of pediatric trauma care. Ideally, given the comparative lack of information on childhood illness, such databases might even be expanded to include pediatric medical emergencies. , . .. . ~ . ~ .. Health Insurance Claims and Administrative Data Sets Claims data are increasingly used in research on variations in the use of
PLANNING, EVALUATION, AND RESEARCH 235 health care and on the effectiveness and outcomes of medical interventions (Lohr, 1990~. Insurance claims data on children come from two main sources:s Medicaid and the private-sector health insurance industry. In principle, insurance claims files constitute a large pool of data on inpatient and outpa- tient services and have several attractive features (e.g., the theoretical possi- bility of traclcin.g the care for ate individual "episode" through dime and across settings). In practice, claims data are often difficult to obtain, lack complete or reliable information, or suffer from other drawbacks that make them a relatively poor source of anything but the most rudimentary informa- tion in the EMS context. Furthermore, Medicaid data may not be represen- tative of the same populations from state to state, because of the variation in eligible and enrolled populations across states and over time. Finally, man- aged care plans are not likely to maintain insurance-claim-like records, so data on such patients would not be available through these types of data sets. Nevertheless, as claims data become better and more complete (e.g., by virtue of linking payment to accurate coding of diagnoses and procedures) and as population-based files containing both inpatient and outpatient data are developed (e.g., as is being done for the Medicare program for the elderly), use of claims data becomes more appealing and feasible. In par- ticular, the ability to track follow-up care after hospital treatment for an emergency would provide valuable insights into postdischarge mortality, residual morbidity, and similar outcomes of interest to researchers and policymakers. Death Records Although hospital records can be used to identify a large proportion of the deaths from serious illness or injury, they will not include deaths that occur before any emergency care can be initiated in a hospital setting. They are also poorly suited to large retrospective studies of national or statewide mortality. Hospital records also will not necessarily include the official cause of death for patients who died while still inpatients. Other sources such as vital statistics systems or coroners' reports may be more useful for some purposes (e.g., simple epidemiologic studies, draw- ing samples for studies that will rely on other data sources). Mandatory E- coding in vital statistics records for injury-related deaths may make them a valuable resource for assessing the contribution of injury to mortality and for injury prevention studies (although it should be recognized that "trau- matic injury" is no longer acceptable as a cause of death). Nevertheless, death certificates have considerable drawbacks limited amount of data, inaccuracy and incompleteness of data that are supposed to be recorded, lack of standardization in assigning cause of death (the most critical data), and lack of information on EMS systems use. These limitations make them
236 EMERGENCY MEDICAL SERVICES FOR CHILDREN a poor stand-alone source of data; vital statistics records must be regarded as only one component of a comprehensive data system. Other Data Sets More specialized data Qua pediatric eme-rgencies~ part~1~1y injury' can also be found In various other data systems. A few of these are described here, although less is known about their strengths and weaknesses for EMS- C planning, evaluation, or research purposes than about the data sources discussed above. A more extensive list of national, state, and local data sources on injury has been assembled by the National Committee for Injury Prevention and Control (1989~. Poison Control Centers These centers (which are also discussed in Chapter 6) are a source of information on the nature, circumstances, and outcome of a large volume of cases of ingestion and other forms of expo- sure to potential toxins. The National Data Collection System of the Ameri- can Association of Poison Control Centers is based on voluntary reporting by centers across the country (Litovitz et al., 1992~. These data capture many cases of suspected and actual poisoning that can be successfully man- aged by the caller without other medical assistance; such cases would, of course, not appear in any prehospital or hospital data. NEISS Operated by the Consumer Product Safety Commission of the U.S. Department of Commerce, NEISS collects data from a sample of EDs on injuries and deaths related to consumer products (CPSC, 19864. Esti- mates of injuries are produced for each calendar year on about 820 prod- ucts, which cover an immense range of consumer goods (e.g., bicycles and other sports equipment or activity; beds, chairs, and other items of furni- ture; nails and screws; knives and other kitchen utensils; electronic appli- ances of all sorts; cleaners and caustics; prescription drugs; and fireworks). Automobiles and guns are not included, although gasoline and toy guns are. These data, which are reported for standard age groups (0-4, 5-14, 15-24, 25-64, and 65 and older), are especially valuable for identifying new injury risks (NRC/TOM, 1985~. Motor Vehicle Crashes The National Highway Traffic Safety Admin- istration (NHTSAJ of the U.S. Department of Transportation (DOT) main- tains two data systems with reports on motor vehicle crashes. The Fatal Accident Reporting System assembles data from states on police reports of fatal motor vehicle crashes. The National Accident Sampling System, also based on police reports, captures information on nonfatal motor vehicle collisions, including those involving pedestrians and bicycles (National Committee for Injury Prevention and Control, 1989~. The detail that these data systems
PLANNING, EVALUATION, AND RESEARCH 237 provide about the circumstances of crashes is useful in identifying injury risks such as those associated with driver or passenger behavior and vehicle or roadway condition or design. These data sets are good models of the kinds of information that might be obtained when such files are linked at the national level. Uniform Crime Report The Federal Bureau of Investigation maintains information on violent crime, notably homicides, in its Uniform Crime Re- port (UCR); the data are collected from police reports nationwide. Reports on annual rates of crime and weapons involved are published through the UCR. These data are potentially useful in monitoring patterns of weapon availability and use and for helping to target prevention programs. Infor- mation on victims and violent crime are, obviously, more complete than data on perpetrators. EMS-C Demonstration Projects Many of the EMS-C demonstration projects assembled databases for special studies on pediatric emergency care, and some have sought to establish ongoing data collection systems. The Arkansas project, for example, put in place at Arkansas Children's Hospital a program to collect data on admissions to pediatric intensive care units and critical care transports. A more extensive program was scaled back because of funding constraints. The New York program established an upstate regional registry for all seriously ill or injured children seen in the region's EDs. The project in Utah is working with nine other states in the intermountain region in an effort to establish a regional EMS-C database. Software has been developed to facilitate reporting by prehospital provider agencies and hospital EDs. The state of Maine has computerized its prehospital run sheets, and it has also established electronic linkages with computerized police accident report data and hospital discharge summaries, allowing offi- cials to monitor data from the scene of injury through hospital discharge. Special Concerns Although the various data sources described above can provide infor- mation by which to study pediatric emergency care, they all have important shortcomings beyond the source-specific ones just noted. These problems tend to cut across information management goals; some critical areas identi- fied by one EMS-C project, for instance, included the following: (1) defin- ing clinical and epidemiologic data needed to identify problems to be ad- dressed by EMS-C subsystems; (2) examining options for accurate information management methods; (3) determining who should collect data and the in- terface between the different entities that collect data; and (4) promoting use of data for evaluating and improving the effectiveness of EMS-C pro- grams (California EMS Authority, 1992~.
238 EMERGENCY MEDICAL SERVICES FOR CHILDREN Issues of particular concern to this committee are introduced below; the literature on problems with health data sources generically is too vast to review here.6 In the short term, these constraints must be understood and, insofar as possible, accommodated, or they will seriously limit what can be learned. In the longer run, the committee believes that its proposals and recommendations in this and other chapters will help the EMS community to address and overcome some of these problems In a more permanent way. Lack of Uniformity and Consensus About Data Elements A fundamental difficulty with data on EMS and EMS-C today is lack of agreement about what data elements are important and about how (even agreed-upon) data elements ought to be defined. These conceptual prob- lems also make it difficult to address technical issues, such as defining appropriate standards by which such information could be collected, stored, or transmitted electronically, that arise with the increasing use of computer- based information systems. Recognition of the need for standard data ele- ments and definitions is not new, but only limited progress has been made in developing a common framework. In the prehospital setting, for example, efforts to develop a nationally recognized prehospital data set date back to guidelines for a 20-item mini- mum data set published in 1974 by the Health Services Administration of the Department of Health, Education and Welfare (now DHHS) (Steele, 1974~. More recent proposals have attempted to identify a core of widely used data elements (Joyce and Brown, 1991) or have focused on specific emergency conditions (e.g., cardiac arrest) (Eisenberg et al., 19901. Guide- lines for a complete EMS management information system (MIS) are being formulated through the American Society for Testing and Materials (ASTM, especially Committee F-30) (Robinson, 1992; Ryan, 1992~. Planning is also under way by NHTSA for a national consensus devel- opment conference on EMS data elements (as part of an EMS-MIS), which is expected to be convened by fall 1993 (NHTSA, 1991a; Ryan, 1992~. The ASTM guidelines may serve as a starting point for conference discussions. Organizations with a pediatric perspective are being included in these con ~ . . . terence activities. Other efforts to establish uniform definitions and data elements are also related to EMS-C data needs. In particular, a 1989 conference convened by the National Institute of Child Health and Human Development (NICHD, 1992) identified significant data elements in child injury research and for- mulated definitions. The conference report also addressed measurement and data collection procedures. Broad-based data collection programs such as the MTOS (Champion et al., 1990b) and the NPTR (Tepas et al., 1989) also encourage wider development of more uniform data.
PLANNING, EVALUATION, AND RESEARCH 239 The committee believes that efforts such as these are not adequate by themselves to ensure that critical conceptual and information needs in EMS- C will be met. Rather, a national uniform data set for EMS-C must be developed. The committee has taken preliminary steps to establish a data set by identifying some of the data elements that it sees as essential (see below and Appendix 78~. (Certain of tile elements proposed should help address some of the other significant limitations to existing data and data sources discussed below, especially linking and aggregating data over pa- tients, settings, and time. Linking and Aggregating Data Linking Information on Individuals for Episodes of Care Being able to link information on patients, settings, and providers within and across systems is vital to a meaningful data collection and analysis effort, and hence to planning, evaluation, and research. An especially troubling prob- lem for EMS is the difficulty of linking data collected by separate system components. Requirements on the part of JCAHO or others for hospitals to incorporate prehospital reports into patient records could facilitate links in one direction, especially if prehospital data can be incorporated into com- puterized record systems. (One effort along these lines is being mounted in Santa Cruz, California, where ED personnel will enter information into a computer-based clinical data system on patients who arrive via prehospital transport. This task, however, is not the same as linking existing data sets.) Prehospital providers have an especially hard time gaining access to hospital data on patients whom they have treated (Joyce and Brown, 1991~. Hospitals themselves may have no routine way to learn about subsequent care in outpatient settings or other facilities such as tertiary care hospitals or rehabilitation programs. Without information about the care that patients receive in other parts of the EMS system and the outcomes of that care, planners and evaluators will find it hard to determine what care is contribut- ing to good or bad outcomes. Various obstacles block the linking of system data. For example, major technical difficulties can arise in trying to match records across data sys- tems. In some ways, linking insurance claims data may be the easiest task technically, because patient identifiers will, in theory, be available. Linking medical record information across prehospital, hospital, and ambulatory set- tings to span primary care, acute care, and rehabilitation is likely to prove much more difficult. Common patient identifiers beyond name may not exist, and available identifiers may be protected by privacy and confidenti- ality regulations. Linking information from, for example, police reports on motor vehicle crashes or autopsy reports is more problematic yet. The
240 EMERGENCY MEDICAL SERVICES FOR CHILDREN cooperation of providers, facilities, and office-based physicians (among others) is critical. The committee discussed the need for unique individual identifiers and strongly endorsed the concept that individuals should be assigned such an identifier at birth or as close after birth as possible. One prominent option Is the social security numbers It has been raised in proposals for revisions to the Uniform Hospital Discharge Data Set (CHDDS) (INCUS, 1992b) and in recommendations of the Workgroup on Electronic Data Interchange established by the Secretary of DHHS (WEDI, 1992~. Modifications neces- sary to establish truly universal and unique identifiers based on social secu- rity numbers appear more feasible than creation of any new system. Use of social security numbers remains a perennially knotty problem with many political, legislative, technical, and cost ramifications; psychological and attitudinal factors, such as people's views about privacy and confidentiality of sensitive medical, financial, and other data, also are considerations. The committee reached no conclusion on the specific merits of using social security numbers as the universal patient identifier, but it does believe that this is one logical option that should be explored and that, in any case, some . . ^. . tic enterer Is essential. Aggregating Data Across Systems Without a common framework for defining, collecting, or reporting EMS-C data, assembling data across dif- ferent systems and comparing their experiences are very difficult tasks. Furthermore, the higher the level of aggregation (e.g., across counties, states, DHHS regions, or the nation), the worse the problem will be. As reiterated in Chapter 8, some attention to this issue should be high on the agendas of the proposed federal center and state EMS-C agencies. Moreover, the desir- ability of aggregating information across systems justifies, in part, the use of certain variables in the core data set proposed later in this chapter, and it also highlights the need for additional resources for this complex activity. Diagnostic Coding for Mechanism of Injury Information on the mechanism of injury is valuable in assessing prob- able type of injury. In fact, mechanism-of-injury criteria are often part of prehospital triage guidelines. Information on the factors that precipitate injuries is even more valuable in identifying specific risk factors, setting targets for injury prevention programs, monitoring the effectiveness of pre- vention efforts, and assessing the cost of care for specific kinds of injuries. The ICD-9-CM E-codes (external cause-of-injury codes, which are equiva- lent conceptually to mechanism of injury) are a convenient way of includ- ing mechanism of injury in reporting on diagnosis or cause of death that is, in contrast to many of the problems identified in this section, this one
PLANNING, EVALUATION, AND RESEARCH 241 has a ready solution. For these reasons, the committee recommends that states and other relevant bodies adopt requirements that ICD-9-CM E- codes be reported for all injury diagnoses for hospital and emergency department discharges. As noted earlier in this chapter, only six states currently require that hospital discharge data include Encodes for injury diagnoses (CDC, 1992a). A studier using discharge data from Maryland found that without mandatory reporting the completeness of E-coding varied over time, by patient age, among hospitals, and by nature and severity of injury (Marganitt et al., 1990~. Limited space on record abstracts or billing forms (e.g., the UB-82) for reporting diagnoses can leave no room to report an E-code. Even if space is available on the forms, providers have little incentive to report E- codes if they are not required, in part because the codes do not affect reimbursement (Sniezek et al., 1989~. The federal government, through the Health Care Financing Administration, could, however, create a strong in- centive by requiring E-codes for Medicare or Medicaid reimbursement for hospitalizations or ED visits. Other factors that appear to deter E-code reporting are concerns over the cost of the additional coding responsibilities, inadequate guidelines for using E-codes, and insufficient information in the medical record to deter- mine an appropriate code (Sniezek et al., 1989~. Studies suggest, however, that costs are minimal. Coding of discharge data was estimated to impose an average of $600 in one-time implementation costs and $600 per year in operating costs (Rivara et al., 1990~. One hospital ED is successfully using a checklist completed by the triage nurse to record E-code information in the patient's record (Ribbeck et al., 19921. The upcoming addition of an E- code field in the revised standard hospital billing form (the UB-92) and the recommendation that E-coding be included in the UHDDS (CDC, 1992a) should spur more extensive use of E-codes. The committee's recommendation for use of E-codes in hospital dis- charge data joins that of other prominent groups such as the American Public Health Association (APHA, 1992), the CDC (1992b), the National Committee on Vital and Health Statistics (NCVHS, 1992a), and numerous state health departments, academic research centers, and nonprofit groups. In the committee's view, efforts to incorporate E-codes in patient records must extend to the hospital ED as well as to inpatient care. Validity of Data and Performance Indicators The problem of how valid information is for answering questions of access, costs, and quality is always paramount in health care assessment.7 In the committee's view, special attention must be given to matters relating to diagnosis and patient acuity (e.g., severity of injury or illness). Patient
242 EMERGENCY MEDICAL SERVICES FOR CHILDREN outcomes also must occupy a special place in planning and evaluation, as well as research. Diagnosis Having reliable and comprehensive data readily available lies at the heart of validity of analyses. In the health arena, accurate and complete information on diagnosis and treatment is critical for assessment of mortality and morbidity. There are, however, continuing questions about those aspects of diagnosis and diagnostic coding. These problems are being addressed by many in the health sector who are engaged in survey work and research, and the EMS community will want to track developments accord- ingly. A review by Iezzoni (1990) identifies several concerns about diagnosis that apply to all uses of data. At the heart of many difficulties is the inconsistent nature of the coding categories offered by the ICD-9-CM; they include clinical diagnoses, pathological processes, symptoms, physical find- ings, test findings, severity indicators, and potential quality indicators (Iezzoni, 1990~. Because the specific clinical basis for determining the "correct" codes is not always clear, coding for similar conditions may vary among providers, institutions, or regions. This inconsistency undermines the face value of diagnostic data in comparative studies. Errors in coding or changes in coding practices (e.g., "unbundling" to maximize reimbursement) can further confound the meaning of diagnostic data and attenuate the reliability and validity of data collected. Coding guidelines that, in the absence of a definitive diagnosis, call for using conditions identified as questionable (e.g., hospital admission to "rule out" a diagnosis) contribute another element of confusion. This may pose a special problem for EDs when physicians must make judgments quickly and on the basis of relatively little information. Clearer criteria for assignment of diagnoses and better training for physicians and coders might reduce some of the problems. The availability of ICD-9-CM diagnosis codes for the nature of illness or injury (so-called N-codes) and the external cause of injury (E-codes) makes injury diagnoses a valuable tool for retrospectively identifying chil- dren who received emergency care and for identifying injury risks. (The value of E-codes for injury prevention was discussed above.) Retrospective identification of inpatients who received emergency care for illness-related conditions is much more problematic, and the obstacles to doing so deserve more attention than they have received to date. Evidence of admission through the ED might be helpful, but it is not always available in discharge data sets; moreover, not all patients requiring emergency care come to the hospital through the ED, so they might be missed altogether. In sum, this committee takes a strong position that better ways need to be developed to identify patients with serious illnesses who received emergency care.
PLANNING, EVALUATION, AND RESEARCH 243 Acuity Objective indicators of acuity of illness and injury are needed to guide clinical decisionmaking and to assess quality and effectiveness of care, but their successful development has proved to be difficult, especially for pediatric patients. Even when certain acuity or severity measures ap- pear to be reliable and valid for use in adults, the question of scoring them for pediatric cases remains. Appendix 7A briefly explores issues relating to measures of acuity and severity of trauma and illness. Several problems can be singled out. Observation at time of care is unreliable for instance, it is inconsistent across bystanders, caregivers, or other involved parties and this is especially true when the patient is a child. Appearance can be a poor indicator of the state of pathophysiologic derangement, and children's vital signs may be misleading. A mix of ana- tomic, physiologic, necrologic, and psychological factors may need to be assessed. The need to know about time since onset of the problem is often crucial, but it is easier to determine for trauma-related than for illness- related emergencies. Furthermore, the issue can be muddied when the "emer- gency" is actually the result of insidious deterioration of a chronic illness that suddenly reaches a crisis stage or when baseline status is not normal for age and sex. In actual emergencies, acuity measures are especially important for decisionmaking about triage, transport, and transfer. Yet time may be of the essence, meaning that some physiologic measures may not be feasible to take and use on the spot. Thus, more research is needed on a reliable, valid, fast, and parsimonious set of measures that will accurately reflect severity of injury or illness as the emergency, and the EMS response to it, is actually unfolding. In broader planning and evaluation efforts, acuity measures are particularly important for comparative assessments of quality of care, be- cause valid analyses require controlling for differences in the distribution of cases by severity. As reflected in the discussion in Appendix 7A, however, questions remain about how well existing measures perform in this capacity. The difficulties in developing scoring systems for pediatric illness in particular have led to a suggestion to establish a medical registry, along the lines of the trauma registries discussed above, which would make it pos- sible to accumulate the large volumes of data needed to support develop- ment of a broadly valid assessment measure (Buchert and Yeh, 1992~. A different kind of response has emerged from observations of the inability of adult scoring systems to predict outcome accurately: Civetta (1991), citing seminal work by Feinstein (1967), argues that a better path may lie in closer observation of, and reflection on, the clinical characteristics of illness rather than in attempts to develop new, or yet more complex, scoring systems based on yet more sophisticated laboratory tests and physiologic param- eters.
244 Patient Outcomes EMERGENCY MEDICAL SERVICES FOR CHILDREN Ultimately, the question of highest priority in planning and evaluation of EMS-C must center on patient outcomes. Asserting that, however, does not clarify several key points: What Outcomes to Measure (outcomes of interest will differ markedly according to several factors, such as characteristics of patients and their emergencies, kinds of services rendered, and settings in which the interven- tions were provided. In some cases, alive or dead is the outcome of con- cern. Mortality is the most readily available and widely used outcome indicator in analyses of emergency care, but relatively few of the children needing or receiving emergency care die. (On frequency grounds, death is actually not an especially good indicator of outcome for many conditions or health care interventions across the entire age range.) Other kinds of out- come indicators are needed to assess residual morbidity from emergencies and the impact of differences in care. That is, in most other cases, a wide range of health status outcomes is pertinent: 8 presence or absence of disease, of various types of impair- ments, of functional limitations, and of "disabilities" that interfere with age-appropriate activities (such as those for activities of daily living fADLs] or, less likely for children, instrumental ADLs).9 Of special interest may be the core domains of health status measurement physical mobility and func- tioning, social and role functioning, and emotional and mental well-being- although measuring these reliably in children is an extremely difficult chal- lenge. EMS-C outcomes assessment can also be based on comparisons be tween "expected" and actual outcomes. Information on initial severity and acuity of patients' conditions can be used to estimate probabilities of par- ticular outcomes. Death is frequently used as the outcome of interest, but other conditions such as permanent impairment or other forms of morbidity might also be used. Discrepancies between expected and actual outcomes may point to systems that are doing very well or very poorly.~° When to Measure Outcomes Leaving aside the complex challenge of obtaining a "baseline" measure of health status, analysts still face the diffi- cult decision of when (and how many times) to assess outcomes. Is the appropriate time to evaluate outcomes of prehospital care upon arrival at the ED or at some later point? For children suffering significant trauma requiring intensive inpatient and outpatient care and rehabilitation, when in the sequence of events should outcomes be assessed? Studies of rehabilita- tion for traumatic brain injury, for example, have found improvements from treatment as long as three years after the injury (Haffey and Lewis, 1989~. Even the (comparatively easier) question of when to measure hospital deaths
PLANNING, EVALUATION, AND RESEARCH 245 (e.g., inpatient deaths only, inpatient deaths plus deaths occurring up to 30 days after discharge, or deaths occurring in or out of the hospital up to 30 days after admission) has been the subject of considerable debate (in this case for evaluations of hospitalization for Medicare patients, as discussed in, for example, Jencks et al., 1988; Kahn et al., 1988; Chassin et al., 1989; lDnte and Lloyd 1989~. How much more complicated then' is the task of deciding when and how often to assess the broad set of health status mea- sures that arguably will tell much more about the end results of emergency care. The ideal would be to measure outcomes at various points in an episode of care in order to judge the variability in care provided, but the costs and practical considerations of taking that approach are extremely high and complex. How to Measure Outcomes Two key issues relate to assessment tech- niques. One pertains to the assessor, that is, health care professional, par- ent, or child. Clearly, for infants and younger children as well as for pro- foundly impaired individuals, outcomes must be reported by adults. For most older children, adolescents, and teenagers, outcomes can be reported by the patients themselves, and in certain circumstances or for certain mea- sures, this may be more desirable and more valid than relying solely on reports by either providers or parents. The other issue pertains to the mea- sure and whether it and its scoring rules are appropriate for children (or for children of different age groups and developmental levels). Privacy and Confidentiality Equally complex and perhaps more contentious are issues of privacy and confidentiality and the related potential for legal liability or serious harm to individuals in the case of breaches of those legal or ethical rights. As noted in the discussion above on linking data, statutory or administrative constraints can restrict access to demographic and other identifiers needed to link data on individual patients, chiefly out of concern for privacy and confidentiality of sensitive medical information on specific, and potentially identifiable, patients and for the possible harm that might come from inad- vertent disclosure of such information. This committee encourages ef- forts to develop and implement data collection programs and policies that will protect patient privacy and confidentiality while making it possible to conduct needed studies for system evaluation. In another vein, facilities and agencies collecting routine information may be reluctant to share data with institutional, physician, or other pro- vider identifiers intact; they may fear potential malpractice liability or pub- lic disclosure of unflattering information about performance and patient outcomes. Here, too, an understanding of the need for evaluation and con
246 EMERGENCY MEDICAL SERVICES FOR CHILDREN sumer information must temper anxieties about release of valid and useful provider-specific information. Many states have legislation that protects the confidentiality of quality assurance information for at least some types of institutions and providers (particularly hospital medical staffs); Califor- nia, for example, protects committees appointed by agencies of local gov- ernments to "monitors evaluate and report on the necessity quality and level of specialty health services, including, but not limited to, trauma care ser- vices" (State of California, Evidence Code, § 1157.74. Typically, however, no such protection is afforded to other entities such as fire departments or ambulance companies responsible for emergency care that engage in quality assurance activities on their own. IMPROVING INFORMATION RESOURCES The preceding discussion has shown that developing better and more extensive data collection and analysis programs must be a high priority for EMS-C systems across the country. If common elements can be established in the data of separate systems, then planners and evaluators can assess EMS-C needs not only from a local perspective but also at the national level. A common core of basic descriptive data needs to be adopted. National Uniform Data Elements for EMS-C Exactly what data need to be collected to develop a basic understanding of EMS-C systems? Although each individual system has unique character- istics, the committee concluded that a core of nationally accepted uniform data elements that encompass all phases of emergency care must be defined. The arguments are several: to see that essential EMS-C data are identified, to foster development of a body of nationally comparable data, to provide the EMS-C community with good data and analyses, and to convey to policymakers the national importance of EMS-C issues. Thus, the committee recommends that states implement a program to collect, analyze, and report data on emergency medical services; those data should include all the elements of a national uniform data set and describe the nature of emergency medical services provided to children. Further, the committee recommends that mechanisms be developed to link all data on a specific case, where those data are generated by sepa- rate parts of the emergency medical services system. The remainder of this section discusses key aspects of such a national uniform data set. For purposes of this report, "national" means that the core applies to data collected in both the public sector (by or for federal, state, or community governments) and the private sector (by or for all relevant pro- viders in the EMS system). "Uniform" conveys the notion of commonly agreed-upon data elements and definitions for those elements. Ultimately,
PLANNING, EVALUATION, AND RESEARCH 247 that uniformity might also extend to standards for electronic acquisition, transmission, and storage of data. Moreover, a wide range of sources must be involved, extending beyond traditional EMS and health care providers even to first responders (such as police officers) and coroners or medical examiners. The committee has sorted the process of defining a data set by propos- ~ng core data elements for prehospital and ED care. A comprehensive data set must go beyond this beginning to include data from the many other settings in which emergency-related services are provided. Even for prehospital and ED care, the proposed data elements constitute only a starting point, because neither the committee nor other experts can say with certainty what the utility of these data items may turn out to be in practice. Furthermore, a considerable amount of testing of inter-rater reliability and validity remains to be done. Nevertheless, the committee believed strongly that efforts must begin to define and develop such a core data set, if the field of EMS-C is to realize the advances envisioned for it throughout this report. Criteria for Inclusion in a National Uniform Data Set The committee identified seven criteria that should guide the selection of data elements for a national uniform data set for EMS. First, data ele- ments should be included if and only if they serve a specific, identifiable purpose. Second, the data should be useful and used; a core data set must be relevant at all levels at which it is collected. Third, data should be obtained routinely from all providers and settings in an EMS system, where an EMS system is defined broadly to span prehospital, ED, inpatient, inten- sive care, and rehabilitation services. Fourth, the information should be sufficiently easy to collect that a high degree of reliability can be expected for data from different sources regardless of level of training. Fifth, the cost of data collection should not be excessive. Sixth, because of the im- portance of linkage, some way of tracking individuals through an entire episode of care must exist. Seventh, the data elements themselves should be "nonjudgmental"; using them in system evaluation is a separate step. Priority Data Elements Appendix 7B outlines the data elements on prehospital and ED care that this committee has determined should have highest priority for inclusion in a national uniform data set for EMS-C. The rationale for including each item is given. Appendix 7B also lists and comments on other data elements that the committee considered but chose not to include in its priority list. The committee classified some of these as desirable but not currently appro- priate for inclusion owing to factors such as limited current use. Other
248 EMERGENCY MEDICAL SERVICES FOR CHILDREN items were rejected because of difficulties in obtaining accurate information or in using them at the national level. The committee's decision to focus its attention on specific data ele- ments for prehospital and ED settings (see Table 7-1) should not be taken to mean that data are not needed from the many other components of emer- gency care systems (such as dispatch operations~ critical care transport. pediatric Tntens~xre care, or rehabilitation). Rather, it reflects the committeets judgment that the prehospital and ED sectors, where the largest numbers of children receive emergency care, are the appropriate starting point for de- veloping what should become a comprehensive EMS-C data set. The committee lacked the time and the expertise to specify data elements for all of the other aspects of emergency care; that work must be part of subse- quent efforts by the EMS-C community. An Information Core, Not an Information Constraint These proposed "core" data elements must be seen as only one facet of a much broader program of data collection for EMS-C across the country. The committee emphasizes that it is not proposing that EMS-C data collec- tion efforts be limited to only those data elements especially valuable for national assessment. Interest at the national level in EMS-C data should spur states and localities and even individual health care providers to assess what data they need to support the planning, evaluation, research, or pre- vention activities that ought to be a part of their EMS-C programs. At these levels, in contrast with national-level analysis, it becomes possible to make meaningful use of a much broader range of data captured in greater detail to assess and guide improvements in local performance or to identify specific circumstances that may require special study. In short, the committee's proposed data elements should be thought of as a beginning and not an end a core and not a constraint. The committee focused most of its attention on the patient-based data generated by individual episodes of care. Those clinical data, along with other data on costs, are critical in answering a great many of the questions about the efficiency and effectiveness of emergency medical care for chil- dren. To answer other questions, however, additional "structural" data must be available: for example, on qualifications of individual providers, re- sources of specific hospitals, and numbers of ground or air ambulances available in a region. Data of these sorts can usually be maintained with only periodic (perhaps annual) surveys. Further Work The committee cannot emphasize enough how essential a strong data collection and analysis program is for understanding and improving EMS-C
PLANNING, EVALUATION, AND RESEARCH 249 TABLE 7-1 Priority Data Elements for a National Uniform Data Set for Emergency Medical Services for Children: Prehospital Services and Emergency Departments Prehospital Services Date of birth (or age) Gender Date and time of: call for assistance (or EMS dispatch), arrival on scene, departure from scene, and ED arrival Prehospital assessment (nature of patient's condition) Spontaneous breathing (Yes/No), on arrival at scene and at hospital Spontaneous pulse (Yes/No), on arrival at scene and at hospital Level of consciousness on arrival at scene (assessed as Alert, Response to Voice, Response to Pain, or Unresponsive) Disposition: Left at scene or transported Transported to: Hospital identification number, possibly hospital zip code Zip code for site of EMS encounter Prehospital provider identification number Prehospital run report number Type of responder (e.g., ALS, BLS) Emergency Departments Date of birth Gender Race/ethnicity Mode of transport (e.g., self, EMS, interfacility, air, ground) Date and time of arrival at ED Date and time of discharge from ED Disposition (e.g., dead on arrival, died in ED, discharged home, admitted, transferred) Diagnostic codes (multiple ICD-9-CM N-codes) Procedure codes (multiple CPT-4 codes) External cause-of-injury codes (multiple ICD-9-CM E-codes, including site of injury) Glasgow Coma Scale (separate scores on each component eye opening, verbal response, and best motor response) Vital signs (initial readings for pulse, respiratory rate, systolic blood pressure, and temperature) Insurance/payer codes (e.g., self-pay, Medicaid, public assistance, CHAMPUS, private insurance) Prehospital provider identification number Prehospital run report number Hospital identification number Hospital zip code Home zip code (or country if not a U.S. resident) NOTE: ICD-9-CM, International Classification of Diseases, ninth edition, clinical modifica- tion includes the widely used N- (or nosology) codes and a separate, more specialized set of E- (or external cause-of-injury) codes; CPT-4, Current Procedural Terminology, fourth edition; CHAMPUS, Civilian Health and Medical Program of the Uniformed Services.
250 EMERGENCY MEDICAL SERVICES FOR CHILDREN and seeing best how to integrate EMS-C into existing EMS programs. The proposed data elements reflect the judgment of the committee as to the kinds of data that are needed to develop a national view of the nature and operation of emergency medical services for children, but the committee also recognizes that they are just a first step. To make progress in this arena' the committee believes that federal action will be needed. To that end, therefore, the committee recommends that the federal center re sponsible for emergency medical services for children (proposed in Chapter 8) develop guidelines for a national uniform data set on emergency medical services for children. Much additional work must be done on many fronts to achieve the committee's goal. For example, researchers, health care providers, and administrators must help develop the kinds of common definitions and re- porting guidelines that are essential for aggregating data from separate sources and for making meaningful comparisons among separate systems. Efforts such as those described above on maintaining national trauma registries, generating standard variables and definitions for childhood injury research (NICHD, 1992), and developing consensus on EMS data elements (NHTSA, 1991a) need to be encouraged and coordinated to achieve as much consis- tency as possible. Organizations responsible for collecting data must determine which tech- nologies and methods are best suited to their needs and resources. Among the choices that can be considered are abstracting data from written records, completing machine-readable forms, or entering data directly into a com- puter-based data system. Centralized data processing (e.g., at the state level) may be appropriate for some systems, whereas others may prefer to delegate the task to local areas. As discussed earlier in this chapter, linking data may require that sepa- rate data systems adopt common identifiers for patients or resolve technical questions about combining data created in differing formats. Administra- tive or statutory regulations that restrict access to personal records to pre- serve confidentiality may pose even more complicated problems. Data collection must not be treated as an end in itself. Agencies, insti- tutions, and systems must commit specific resources to analyzing and re- porting data as well as collecting them. When their analytical capabilities are limited, they should seek technical assistance and additional training, perhaps through programs of the federal center and state agencies proposed in the next chapter. Furthermore, reporting plans must ensure that informa- tion flows back to those individuals and organizations whose activities are the source of the raw data. In addition to conveying useful information derived from those data, this feedback helps establish the value of specific data elements and promotes continuing interaction between those who gen- erate the data and those who use them.
PLANNING, EVALUATION, AND RESEARCH 251 When data are available, using them for evaluation and quality assur- ance purposes requires defining the standards against which performance will be judged. Where quality assurance programs do not yet exist, for individual organizations or larger systems of emergency care, they should be established. Research also benefits from the availability of routinely reported data even though specialized data collection Is often necessary as wets. Standard data can sometimes yield valuable findings on their own, and they can often guide researchers to questions that require more detailed study. Federal and State Guidance Successfully establishing the state and national programs for data col- lection and analysis that the committee believes are needed will require the sustained cooperation and coordination of many parties. Few public or private groups, perhaps none, currently have the influence or the resources to serve as convener and moderator for the work that is needed. The com mittee is proposing, therefore, that a federal center and state agencies on EMS-C assume responsibility for determining how such activities should be organized and supported. The role that the committee envisions for these entities is discussed more extensively in Chapter 8. Lessons and Remaining Challenges Several lessons and ideas for the future emerge from the experiences to date in collecting and analyzing EMS data. One is the need for continuing attention to training personnel in data collection and analysis skills. One- time training exercises or educational courses are not sufficient because of factors such as staff turnover, lack of understanding of the uses to which data will be put, changes in the system itself, or changes in requirements by external groups (e.g., regulatory agencies or professional accreditation bod- ies) that mandate new ways to record or report data. Furthermore, in all settings the direct impact of data collection efforts on care delivery and patient-provider interaction must be minimized, or all parties are likely to become frustrated or anxious (or both) about the health encounter itself. Computerization raises a different set of challenges. Among the ob- stacles to greater use of computer-based data collection are general antipa- thy to direct data entry on the part of physicians and other health care professionals; the sheer enormity of the number of persons on whom data might be collected (especially for EDs that are very busy and handle a large number of true emergencies and for prehospital services in large metropoli- tan areas); the time needed to enter interpretation of events into computer language; the need for better software; and costs.~3
252 EMERGENCY MEDICAL SERVICES FOR CHILDREN For various reasons relating to costs, practicality, or community atti- tudes, EMS systems may remain reluctant to engage in broad data collec- tion or analysis efforts. Costs, in particular, must be a concern in develop- ing data sets and data collection programs. These efforts must be accepted as a high priority in EMS systems but will be difficult to sustain if their costs appear to compromise adequate funding for delivery of services In- centives introduced at the national, state, or local level might help to over- come some of these obstacles. The most obvious incentives are financial. They might take various forms: providing equipment and software, or training, free of charge; giv- ing cash grants up front to acquire equipment and software, to underwrite the costs of training courses, or to pay outside contractors; and reimbursing for data collection and analysis activities (e.g., salary support for data col- lectors). Payer reimbursement practices also may play a role; better and more complete data may have sufficient utility to third-party payers that they would be willing to underwrite some of the costs of getting them. Less direct incentives might be found in contingencies for funding approvals that require EMS systems to maintain some minimum level of effort and to report data periodically to a central office. Compliance with data collection requirements can also be established as a performance criterion for individual providers, facilities, or even EMS systems. Other incentives might relate to liability protections that sound documentation can provide. Finally, the satisfactions of participating in research projects may be an incentive for EMS personnel to increase their participation in data collection and analysis efforts. IMPLEMENTING A RESEARCH AGENDA Answering many of the questions posed at the outset of this chapter, as well as in earlier chapters, will require focused, well-conducted research programs. Although the data set described above might be used for re- search, that is not its primary application. Thus, all parties interested in advancing EMS-C will need to put forth a manageable research agenda, recognizing that it will entail, among other things, some investment in pri- mary data collection beyond what has been proposed in this chapter. Proposals for a "national institute of emergency medicine" (Mickel, 1990), which could support research related to EMS-C and EMS more gen- erally, were not considered by the committee. Instead, the focus was on a proposed federal center for EMS-C (see Chapter 8) that would be charged with supporting an appropriate research agenda. Nevertheless, some of the subjects on which a national institute might conduct or support research are directly relevant to EMS-C. Cases in point (see Mickel, 1990) include basic research in physiologic dysfunction, physiologic reserves, and resus
PLANNING, EVALUATION, AND RESEARCH 253 citation; toxicologic research, especially with respect to illicit drugs; medi- cal, surgical, and psychological problems associated with disasters; improved triage criteria; effective management and coordination of EMS in urban and rural environments, with a view to understanding interrelationships across local, state, and national levels; and assessment of resources (personnel, e~llipmeIlt~ and final needed to provide adequate emergency care ~n such senings. MacKenzie (1991) has also offered a set of key topics for EMS-C re- search. Although oriented more toward trauma care, her questions are quite pertinent for EMS-C more generally: (1) Are the right patients getting to the right level of care from time of initial injury or illness requiring emer- gency interventions through post-acute-care services? (2) Are objective criteria available for classifying where patients should have been treated? (3) Can data across components of EMS systems (e.g., prehospital run re- ports, hospital discharge abstracts) be acquired through computerized meth- ods, and can such data be linked? (4) Are ways available to set standards of performance for EMS-C systems and to change and improve performance according to those standards? (5) Do reliable and valid measures of out- comes physical, mental, psychological, and functional exist for children? (6) Do ways exist to take external factors, such as the physical, social, and economic environment, into account in assessing outcomes of care? (7) Can the costs and benefits of EMS-C be documented? She emphasizes three key elements needed to build a scientific base for EMS-C: popula- tion-based data, longitudinal research focused on outcomes other than sur- vival, and well-designed projects to document "what works best and at what cost." High-Priority Topics Space does not permit a detailed presentation of a comprehensive re- search agenda in pediatric EMS (let alone EMS generally). Eight generic questions posed at the outset of this chapter identify issues of special con- cern: (1) What is the structure of the EMS-C system? (2) Who uses the system? (3) For what is the system used? (4) What services or procedures are provided to a patient? (5) When are services provided? (6) What are the outcomes of using the system? (7) What are the global costs of the system? (8) How well does the system perform? Because each of these topics could generate a long list of research projects, particularly important issues must be identified. Therefore, the committee recommends that research in emergency medical services for children be expanded and that priority attention be given to seven areas: clinical aspects of emer- gencies and emergency care; indices of severity of injury and, espe- cially, severity of illness; patient outcomes and outcome measures; costs;
254 EMERGENCY MEDICAL SERVICES FOR CHILDREN system organization, configuration, and operation; effective approaches to education and training, including retraining and skill retention; and prevention. These points are outlined below, together with some examples of specific studies and investigations that might usefully be carried out. In the committee's view, support for EMS-C research should be avail- able through a research grant program under the auspices of the proposed federal center for EMS-C. In the short run, the current EMS-C demonstra- tion grant program might use its "targeted issues" grants to address some of these research topics. Furthermore, the committee believes that this re- search agenda is appropriate for consideration and attention by an appre- ciable number of other Public Health Service agencies (certainly some insti- tutes of the National Institutes of Health, the Agency for Health Care Policy and Research, the Indian Health Service, and HRSA), other parts of DHHS (such as the Administration on Children and Families), and other executive- branch departments (particularly DOT). Moreover, many different private- sector foundations with interests in children, emergency care, the organiza- tion, financing, and delivery of health care services, and the production of health care personnel might well see different parts of this research agenda as quite pertinent to their long-term research plans. Therefore, the topics proposed here are meant for a wide array of supporters of research and demonstrations, not just one (comparatively small) federal office. It would be desirable if the data elements in the proposed core data set discussed above could be used in research efforts. The committee recog- nizes, however, that primary data collection will likely be needed to address many of the issues presented below and that broader secondary analyses of existing databases may prove helpful. The primary aims of the minimum national data set lie more in the areas of program planning and evaluation than in research, and the committee cautions against expanding or tailoring the proposed data set to meet more comprehensive research needs. The seven areas briefly discussed below (and outlined in Table 7-2) are those the committee regarded as especially significant for early and rigor- ous attention, but no ranking among them is implied. (Similarly, no priori- ties are implied by the order in which particular topics are discussed within each of the seven sections.) Clinical issues are discussed first because, for many physicians, nurses, and EMS personnel, the challenges presented by the child in front of them will be those with the most immediacy. Research issues involving severity measures and risk adjusters, outcomes measures and costs are, in some sense, all precursors to being able to mount compre- hensive, credible investigations of system configuration and operation, so they are discussed in that order. Research into more effective approaches to education and training, and to prevention of illness and injury, will require long-term efforts and arguably will cut across many of the preceding issues, so they are addressed last in this section.
PLANNING, EVALUATION, AND RESEARCH 255 TABLE 7-2 Summary of High-Priority Topics for a Research Agenda in Emergency Medical Services for Children Clinical Aspects of Emergencies and Emergency Care Posttraumatic shock, septic shock, hemodynamic instability, and hypotension Pediatric resuscitation Appropriate uses of and techniques for incubation, bag-valve-mask ventilation, vascular access and intraosseous infusion, and pneumatic antishock garments in children Physiologic responses of injured or sick children to certain interventions and when parents are present or absent Operative versus nonoperative management of blunt hepatic, splenic, and pancreatic . . . 1nJurles Management of injured children using blood products and synthetic blood products Evaluation of minor head trauma including indications for CT scans, observation, and . . admission Treatment of acute asthma attacks, including initial assessment and management and determining whether inpatient or outpatient follow-up care is needed Psychosocial support for families of injured or sick children Appropriate techniques for prehospital management of ill or injured children, including decisions about on-scene stabilization or immediate transport Severity and Acuity Measures for Injury and Illness Development and validation of injury and illness scoring scales for children, with special attention to applications in the field (for triage) and for research purposes Ways of discriminating between life-threatening and non-life-threatening illness and injury Survivability by level of severity of injury or illness Scales for severity of abdominal and chest trauma injuries Methodologies to triage patients more accurately in the field and in EDs, with particular attention to young children with possible serious illness Patient Outcomes and Outcome Measures Development and validation of practical functional and other outcome measures Disability assessment and scoring systems for children Innovative approaches to longitudinal follow-up Costs Determination of true direct and indirect costs associated with EMS-C for all major types and settings of emergency care Assessment of the marginal (i.e., incremental) costs of improving EMS systems sufficiently to be able to handle the pediatric age group adequately Economic consequences of pediatric trauma or severe illness to families and to taxpayers Evaluation of the cost-effectiveness of different EMS-C program configurations, with particular attention to a broad set of program benefits (i.e., outcomes) System Organization, Configuration, and Operation Effective and practical ways to upgrade EMS system components Effectiveness, efficiency, and other outcomes of various EMSIEMS-C arrangements for different populations and settings Extent to which children who need emergency medical services receive them, with particular attention to care received (or not received) in hospital EDs continued
256 TABLE 7-2 Continued EMERGENCY MEDICAL SERVICES FOR CHILDREN System Organization, Configuration, and Operation~ontinued Population-based estimates of need for ED and PICU beds for assessment of emergency care resources appropriate for local and regional areas Pediatric emergency care in special circumstances and locales Effective ways to measure and improve the quality of EMS caret for different; types of EMS providers, systems, and settings Education and Training Effective ways to provide professional education and training and retraining to maximize skill acquisition and retention and to improve practice patterns and patient outcomes Public education in prevention, basic emergency care skills, use of the EMS system, and similar topics, with attention to level of resulting behavior change Prevention Effective strategies for injury prevention, including comparative effectiveness of laws, regulations, and educational programs Outcomes, costs, and cost-effectiveness of prevention programs NOTE: CT, computed tomography; ED, emergency department; EMS, emergency medical services; EMS-C, emergency medical services for children; PICU, pediatric intensive care unit. Clinical Aspects of Emergencies and Emergency Care Posttraumatic shock, septic shock, and hemodynamic instability, including the definition, identification, and treatment of pediatric hypotension, pose special problems for clinicians; even the adequacy of commonly used mea- sures, such as capillary refill, blood pressure, and respiratory rate, is not well established. Pediatric resuscitation (e.g., adequate airway manage- ment, appropriate fluid volume, correct measures for monitoring central nervous system metabolic function, and stress response) also occupy a cen- tral place in management of severely injured or sick children. Appropriate uses of and techniques for intubation, bag-valve-mask ventilation, vascular access and intraosseous infusion, and pneumatic antishock garments in chil- dren must be established. Questions relating to the physiologic responses of injured or sick children often arise, including use of conscious sedation and traditional pain management and differences in responses when parents are present or not present for interventions. For injured children, the merits of operative versus nonoperative man- agement of blunt hepatic, splenic, and pancreatic injuries and use of blood products and synthetic blood products remain significant concerns. Critical elements for evaluating minor head trauma need to be established, including indications for CT (computed tomography) scans, observation, and admis- sion. Several aspects of the treatment of acute asthma attacks require fur- ther investigation, including initial assessment and management, predictors
PLANNING, EVALUATION, AND RESEARCH 257 of successful outpatient treatment, the role of observation units, and optimal strategies for preventing repeat emergency visits. Psychosocial support for families of injured or sick children is a special, but not well understood, aspect of EMS-C. Finally, the appropriate field management of ill and injured children (i.e., by prehospital personnel) must be determined, espe- cially establishing the biologic and physiologic aspects of on~sc.ene stabilize zation versus immediate transport. Severity and Acuity Measures for Injury and Illness As highlighted in Appendix 7A, research to develop and validate injury and illness scoring scales for children, with special attention to applications in the field (for triage) and in research, is greatly needed. Among the topics deserving particular attention are ways of discriminating between life-threat- ening and non-life-threatening illness and injury, survivability by level of severity of injury or illness, and scales for severity of abdominal and chest injuries. In keeping with the clinical aspects of field management, method- ologies to triage patients more accurately in the field and in EDs, with particular attention to young children with possible serious illness, will be especially important areas of investigation. Patient Outcomes and Outcome Measures In keeping with the comments about severity and acuity measures, de- velopment and validation of patient outcome measures is a critical need. Special attention must be given to functional outcomes and to practical measures that can be used for system evaluation and for later research (e.g., in the area of system organization and operation). Disability assessment and scoring systems for children should be given particular notice as well. Finally, better approaches to longitudinal follow-up need to be developed. Costs Throughout this chapter (and the report) concerns about costs and cost- effectiveness of EMS-C have been raised. These include, in particular, determination of the true direct and indirect costs associated with EMS-C for all major types and settings of emergency care; assessment of the mar- ginal (i.e., incremental) costs of improving EMS systems sufficiently to be able to handle the pediatric age group adequately; investigation of the eco- nomic consequences of pediatric trauma or severe illness for families and for taxpayers; and, finally, evaluation of the cost-effectiveness of different EMS-C program configurations, with particular attention to a broad set of program benefits (i.e., patient and system outcomes).
258 EMERGENCY MEDICAL SERVICES FOR CHILDREN System Organization, Configuration, and Operation Assuming attention to researchable questions in the areas already noted, some research funding also should be directed to determining effective and practical ways to upgrade EMS system components, including data linkages and communication technologies. This would include investigations into the effectiveness, efficiency anct other outcomes of various EMS or EMS-C arrangements for different populations and settings (e.g., poor/nonpoor, mi- norities, those with chronic illness or disability, urban/suburban/rural geo- graphic areas). Determining the extent to which children who need emer- gency care receive it, particularly in hospital EDs, will be critical. Related to this is developing population-based estimates of the need for ED and PICU beds that can be used to assess the emergency care resources appro- priate for local and regional areas. Some efforts to study pediatric emer- gency care in special circumstances (e.g., on Indian reservations or U.S. military bases, or children in homeless families or those with fragmented support systems) are also warranted. Finally, effective ways to measure and improve the quality of EMS care available from different types of EMS providers, systems, and settings must be developed and tested. ^~ ~ err · ~ . ~ . Education and Training In keeping with the distinctions raised in Chapter 4, the committee identified research needs relating to professional education and training as well as public education. As to the former, effective ways, in all settings and at all levels, to educate and train EMS professionals initially and through- out careers, were regarded as especially important. Particular attention to improved practice patterns, patient outcomes, and skill acquisition and re- tention is warranted. With regard to public education, the committee under- scored the need for studies in effective ways to educate the public in pre- vention, basic emergency care skills, use of the EMS system, and similar topics, with attention to level of resulting behavioral change. Prevention In the eyes of many experts, sustained efforts at prevention have the potential to be the most effective EMS "intervention," but much research is still needed in this area. Effective strategies are needed for injury preven- tion (e.g., ways to promote use of bicycle helmets, car safety seats, or devices to prevent scalding, techniques to prevent drowning). The com- parative effectiveness of laws, regulations, and educational programs in injury prevention should be assessed. Research on the outcomes, costs, and cost-effectiveness of prevention programs is also needed. Because children often can do little to control risks to which they are exposed, improving the
PLANNING, EVALUATION, AND RESEARCH 259 effectiveness of prevention will require a substantial focus on adults. The CDC's National Center for Injury Prevention and Control is an important resource in these efforts. Other Areas for Research Apart from the seven h~gh-priority areas just discussed, committee members identified three other areas warranting targeted research efforts. Some of these pertain to epidemiology and biostatistics, and others to applied re- search in certain segments of the spectrum of care in EMS-C. Epidemiology The epidemiology of emergency illness and injury should receive further attention, including the overlap of illness and injury. Spe- cific concerns regarding illness include incidence and natural history of specific conditions. Research is also needed on use of services, particularly among chronically ill, immunosuppressed, and technology-dependent chil- dren. Concerns regarding injury include the risk of injury in general and by socioeconomic status. A more specific concern is patterns of musculoskel- etal injury from motor vehicle crashes and the biomechanics of injury in restrained and unrestrained children. Attention also should be given to characteristics of firearm injuries. Finally, a better understanding of the epidemiology of transport is needed, including types of children transported according to sociodemographic characteristics, diagnoses, reasons for trans- port, and dispositions. Prehospital Care Studies should be done to identify the types of skills and procedures most often needed in prehospital care. Whether care ren- dered by BLS and ALS responders leads to different outcomes should also be examined. Rehabilitation EMS-C research on rehabilitation should determine the types of follow-up or rehabilitation services needed by children with sig- . i. . . ~ . . . . . . Scant comp .~cat~ons or consequences of serious Illness or Injury that ~n~- tially required emergency interventions. Also needed are studies to develop successful methods of returning severely ill or injured children to school. The research plan developed for the National Center for Medical Rehabili- tation Research offers many useful examples in areas such as mobility; behavioral adaptation; whole-body system responses; assistive devices and technologies; approaches to measurement, assessment, and epidemiology; treatment effectiveness; and education and training (NCMRR, no date). SUMMARY Currently, inadequate EMS-C data and data systems make it difficult to conduct the planning, evaluation, and research that are needed to determine
260 EMERGENCY MEDICAL SERVICES FOR CHILDREN whether children are getting the emergency care they need, when and where they need it. Most of this chapter addresses data collection issues concern- ing planning and evaluation at the institutional, system, local, state, and national levels. It also highlights special data needs for research and exam- ines specific areas where research is needed. All of these tasl<s require access to data' analytical resources to trans- form those data into meanings] information, and ways to use and dissem~- nate the information to improve the care that children receive and to target prevention efforts. Four central points guided the committee's thinking about EMS-C data needs: (1) information is needed on structural aspects of care, processes of care, and outcomes of care; (2) individual components of an EMS system as well as the system as a whole must be examined; (3) EMS-C data are needed for analyses at the local, state, and national level; and (4) routine information gathering is needed for planning and evaluation purposes supplemented by additional primary, targeted data collection and analysis for research. The committee addressed several concerns regarding data and data col- lection Lack of uniformity and consensus about data elements must be overcome. The ability to link and aggregate data, in particular to link information from separate EMS-C system components on individual pa- tients and to aggregate data across systems, is essential. Use of ICD-9-CM diagnostic codes identifying external causes of injury (E-codes) in ED and hospital discharge records must be expanded. Valid data and performance indicators, particularly for diagnosis and measures of acuity or severity of illness and injury, need to be developed. Information on patient outcomes is needed, but three technical matters must be addressed: what outcomes to measure, when to assess them, and how. The strength of the committee's positions on linking data and on use of E-codes is reflected in formal rec- ommendations (see Box 7-1~. Developing better and more extensive data collection and analysis pro- grams, with a common core of basic descriptive data, must be a high prior- ity for EMS-C systems across the country. To promote progress in this area, the committee recommends that states implement a program to collect, analyze, and report EMS-C data that are consistent with a national EMS-C data set. It also recommends that the federal office assigned responsibility for EMS-C (discussed more fully in Chapter 8) develop guidelines for a national uniform data set. As a first step toward this data set, the committee proposes data elements for prehospital and ED care. Research is needed to validate the clinical merit of emergency care procedures, to identify better kinds of care, to devise better ways to deliver that care, and to understand the costs and benefits of the EMS and EMS-C systems now in place and toward which the nation should move. The committee recommends that priority be given to seven areas clinical as
PLANNING, EVALUATION, AND RESEARCH I..... 261 pects of care; indices of severity of injury and of illness; patient outcomes and outcome measures; costs; system organization, configuration, and op- eration; education and training (including skill retention); and prevention. Other areas warranting targeted research efforts include the epidemiology of illness and injury, skills needed in prehospital care, and rehabilitation services. As with the earlier data-related recommendations, the committee believes that the proposed federal EMS-C center must take a strong leader- ship position in supporting a comprehensive research agenda for EMS-C. NOTES 1. For a complete picture of EMS and EMS-C that would be helpful for planning and evaluation purposes, one would also want to know about who has not used the EMS system in circumstances that apparently would warrant such use. Learning about "nonevents" of this sort is more difficult than studying who has used a system. The committee advises that planners and researchers give thought to the survey or other techniques that might be implemented to obtain such information, as was attempted by the National Center for Health Statistics in the Child Health Supplement of the National Health Interview Survey in 1988.
262 EMERGENCY MEDICAL SERVICES FOR CHILDREN 2. The structure, process, and outcome framework for quality assurance in health care was first proposed by Avedis Donabedian (1966, 1980, 1982, 1985) in the context of assessing and ensuring the quality of medical care. Structural measures are the characteristics of the re- sources in the health care delivery system and serve essentially as indicators of the presumed capacity of the practitioner or facility to delivery quality health care. Process of care refers to what is done to and for the patient as he or she moves through a health care delivery system, and can encompass interventions ranging from primary and secondary prevention through diagnosis and definitive c e-= through whab~litat~n, -palliation, and counseling. Outcomes are the end results of care that is, the impact that the care process has had on the health and well- being of patients and populations. These concepts have a clear application to planning and evaluation of EMS-C programs. 3. This committee did not examine programs of quality assurance (QA) or quality im- provement for specific facilities or providers, since its concern was more with systemwide questions. It notes, however, that some advice about assessing the quality of emergency medicine, at least in hospitals, is available. For example, the American College of Emergency Physicians has produced a manual that gives considerable detail on implementing an effective QA program in the hospital ED (ACEP, 1987b), and the California EMS Authority, through the Sierra-Sacramento Valley EMS Agency, has developed a detailed set of components and standards for a quality assurance program for a pediatric critical care system (Sierra-Sacra- mento Valley, 1992). An important portion of the literature on trauma also relates to program evaluation and related activities such as evaluating prehospital services and in-depth auditing of trauma deaths for quality control purposes (see, e.g., Sacco et al., 1988; Wesson et al., 1988; Nakayama et al., 1989; Pories et al., 1989; ACEP, forthcoming). Outcome-based, quantitative quality assurance and efficiency assessment, as it relates to pediatric ICU care has been re- cently reviewed (Pollack, 1993). McArdle and Cooper (1993) discuss aspects of system qual- ity assessment, surveillance, and improvement (chiefly relating to pediatric trauma care). The Joint Commission for Accreditation of Healthcare Organizations has long-standing standards for emergency services; these involve a written plan, organization, direction, staffing, integra- tion, training and education, policies and procedures, facility design and equipment, medical records, quality control mechanisms, and monitoring and evaluation (JCAHO, 1990). The National Highway Traffic Safety Administration's program for assessment of state EMS sys- tems is discussed in Chapter 8. 4. A case in point concerns software for managing trauma registries: To assist in data collection for purposes of developing norms for measuring outcomes and assessing quality of care, one national group (the American College of Surgeons) has developed a software pack- age to facilitate hospital-based collection and analysis of trauma registry data using personal computers (ACS, 1992). This multi-institutional network is intended to facilitate ongoing investigation of the efficacy of therapeutic interventions and to help define the socioeconomic effect of injury. 5. Insurance claims data on the elderly and certain other populations such as patients with end-stage renal disease are available through the Medicare program, and arguably the work being done to combine inpatient and outpatient data for the elderly promises to produce the best working files on health care utilization of any insured population. 6. The literature on limitations of various kinds of data sources amassed over the past two decades alone is quite large. The reliability of hospital discharge data, for example, was examined in the late 1970s by study committees of the Institute of Medicine (IOM, 1977, 1980b); the situation has improved some since the introduction of Medicare's Diagnosis- Related Groups Prospective Payment System for hospitals, but the problems are by no means solved. An authoritative review of proposed changes in major NCHS surveys (NRC/TOM, 1992b) documents many of the challenges that NCHS will need to address concerning these data collection efforts. With regard to so-called administrative databases, a 1990 conference
PLANNING, EVALUATION, AND RESEARCH 263 outlined many of the problems, but also the advantages, of these types of files (Hanft et al., 1990). 7. Discussions about data for planning, evaluation, and research are typically framed in terms of reliability and validity. Reliability is the extent to which data are accurate and reproducible. For instance, high reliability requires that the same data collected at two differ- ent times, by two different observers, or by two different measures be essentially the same; it also requires that the rate of missing (or deliberately false) data be low. Validity is the extent -lo which data actually measure what they pu~u to measure. Or ~e, ~ measure of physical mobility should not be confounded by elements of psychological well-being (or vice versa), and a measure of hospital charges may well not correctly indicate hospital costs. Technically, the level of reliability of a data collection instrument or method sets the level of achievable validity, and a measure can never be valid if it is not reliable. Reliability and validity as defined here are critical, especially for data being gathered in many locations, by many types of providers, for several different uses. The concepts are particularly significant in evaluating outcomes; for example, death may be a reliable indicator of the outcome of care in that it is usually correctly recorded, but it may not be a particularly valid indicator of the quality of the care rendered. 8. The literature on functional outcomes, health status, and health-related quality of life is quite large certainly too comprehensive to review here. For seminal articles and publications in this area over the past 30 years, the reader is referred to Katz et al. (1963); Patrick et al. (1973); Kaplan and Bush (1982); Wenger et al. (1984); Bergner (1985); Feinstein (1987); Katz (1987); Lohr and Ware (1987); McDowell and Newell (1987); Stein et al. (1987); Lohr (1989, 1992); Patrick and Bergner (1990); Spilker et al. (1990); Greenfield and Nelson (1992); and Stewart and Ware (1992). Of significance to EMS-C is the fact that the field of health status assessment for children is underdeveloped compared with the situation for adults. 9. One IOM report, Disability in America, makes a firm distinction between disability and impairments (even permanent ones) and functional limitations. Disability arises only when the demands of the environment exceed a person's capabilities to meet them or to maintain capac- ity for age-appropriate tasks (or activities of daily living) (IOM, l991b). ADLs include tasks such as eating, dressing, and getting in and out of bed. IADLs (instrumental ADLs) include more complex activities such as shopping, use of transportation, and use of telephones. Apply- ing these concepts, especially the latter, to children requires identifying the right age-specific . . . actlvltles. 10. This "expected versus observed" approach to assessing performance is essentially the tactic used by the Health Care Financing Administration in calculating and publishing hospi- tal-specific mortality rates for the Medicare program. Attractive conceptually, the analyses are extremely hard to do because of the need to control or adjust for many case-mix (patient, diagnosis, and other) variables. Such an approach might be somewhat simpler when applied in the EMS context only, for instance in analyses of trauma deaths in a hospital setting using the so-called TRISS methodology (Champion et al., 1981; Boyd et al., 1987) (see Appendix 7A). However, the enduring controversies about acuity and severity adjustors, coupled with the range of settings in which emergency care can be rendered, makes any broad application of this approach debatable. 11. Privacy and confidentiality concerns have become increasingly visible in the past de- cade or so. Some issues are particularly significant in caring for children. Families may be anxious to restrict access to information about children's conditions such as HIV infection. Adolescents may wish to receive care without the knowledge of their families (e.g., for preg- nancy or abortion). Cases of suspected child abuse involve not only the medical system but legal and social services as well. A considerable literature exists about the extent to which sensitive, private information about individuals is (or is not) protected in this country. For seminal publications over the past
264 EMERGENCY MEDICAL SERVICES FOR CHILDREN 20 years, see generally Westin and Baker (1972); Privacy Protection Study Commission (1977); Rule et al. (1980); OTA (1986); Flaherty (1989); NRC (1991a); Powers (1991). The issues extend well beyond medical information to confidential financial and other personal data and to inadvertent as well as deliberate, and overt as well as covert, invasions of privacy (Rothfeder, 1992). With the advent of new abilities of scientists to map the human genome and increasing use of health-related information by employers and insurers, however, anxiety about misuse of such information has become more acute. The issue was recently examined by ~ IOM committee concerned with the compot-~-~d p~t =~M (IOM:, I991~- and by- the DHHS Secretary's Workgroup on Electronic Data Interchange (WEDI, 1992). In 1992, privacy and confidentiality of personal health data was also to one degree or another the subject of debate by three IOM committees considering, respectively, employment-based health benefits (IOM, 1993c), the assessment of genetic risks (IOM, forthcoming), and questions of regional health databases (with a report due out in early 1994). 12. Concerning the significance of ED care in the hospital sector, Matson (1991) cites a prediction from the American Hospital Association that, by the mid-199Os, more than 50 percent of all hospital admissions will come through EDs; he notes that, today, perhaps as much as half of a hospital's total revenue stems from the ED and that within a few years the ED may emerge as the "single largest and most important department of many hospitals" (p. 18). This is already happening in many areas of the country. 13. For a thorough discussion of the difficulties, as well as the opportunities, presented by the computer-based patient record, see The Computer-Based Patient Record: An Essential Technology for Health Care (IOM, 1991 a).
PLANNING, EVALUATION, AND RESEARCH Appendix 7A Methods for Scoring Acuity and Severity of :Injury and IlIness 265 A large array of measures has been developed over the past 20 years or so to enable clinicians, hospital staff, and prehospital personnel to judge how severely injured or ill an individual is. The main purposes of these measures are threefold: (1) to enable EMS personnel to triage patients appropriately; (2) to permit clinicians, researchers, and others to adjust for the severity or acuity of patients' problems or to predict outcome in carry- ing out various types of research projects or comparative analyses; and (3) to conduct evaluations of institutions and systems. The literature on these measures is quite large, in part because of controversies in three areas: the reliability and validity (or sensitivity and specificity) of the measures, their utility for pediatric (as contrasted with adult) patients, and their practicality and feasibility in institutional settings or in the field. This appendix briefly introduces the main measures of injury and ill- ness severity and the controversies about them, identifying the issues that led the committee to give such priority in its research agenda to this topic. The appendix is selective and illustrative, not exhaustive, in the measures discussed. It focuses more on the extensive work on trauma and injury measures than on the less well-developed illness instruments. It does not deal with the many generic or diagnosis-specific measures of health status or health-related quality of life that are used for health services, health policy, or biomedical research purposes. (Readers wishing to pursue these topics further are directed to the citations in Note 8 in the main text of this chapter.) Neither does it catalog the myriad of instruments available to track childhood physical and mental development, which typically could not be used as indicators of severity of illness or, especially, of injury. TRAUMA AND INJURY MEASURES Many measures and scoring systems have been developed for trauma and injury. Some are intended especially for field or later triage; this in- volves identifying injured patients who should be transported by, for in- stance, aeromedical services, transported directly to and treated in trauma centers, transported from one hospital to another that has more sophisti- cated facilities, or in other ways managed with services that imply consider- able resource use because of increased medical need. Other measures are calculated only after complete diagnostic testing and evaluation; they are
266 EMERGENCY MEDICAL SERVICES FOR CHILDREN intended more to predict mortality or for purposes related to quality assur- ance, planning, and evaluation. Most of these measures, when validated at all, have been validated against death. Relying on death as an indication of validity is, however, regarded as an incomplete approach to psychometric evaluation. Some measures have been validated against various x~nmortal outcomes or d~s- abil~y status, but generally, rel~abiI~ty and validity studies need to be con- ducted more frequently against benchmarks other than mortality. Selected Measures Among the better known measures focused on injury and trauma are (alphabetically) the following (with only key citations noted): · the Abbreviated Injury Score (AIS) (Committee on Medical Aspects of Automotive Safety, 1971; AAAM, 1980; 1985; Sacco et al., 1993~; · CRAMS (the Circulation, Respirations, Abdomen, Motor, Speech scale) (Gormican, 1982; Clemmer et al., 1985~; . the Glasgow Coma Scale (GCS) (Teasdale and Jennett, 1974~; · the Injury Severity Score (ISS) (Baker et al., 1974; Baker and O'Neill, 1976; for a critique, see Copes et al., 1988), which is derived from the AIS; · the Modified Injury Severity Score (MISS) (Mayer et al., 1980; Walker et al., 1984~; · the Trauma Score (TS) (Champion et al., 1981, 1983) and the Re- vised Trauma Score (RTS) (Boyd et al., 1987; Champion et al., 1989~; and · the Trauma Triage Rule (Baxt et al., 1990; Emerman et al., 1992~. TRISS Analysis The TRISS method of analysis (Champion et al., 1981, 1983; Boyd et al., 1987; Sacco et al., 1993) generates a probability of survival from a mathematical formula that combines a patient's Trauma Score (or Revised Trauma Score), Injury Severity Score, and age. Although TRISS is subject to certain methodologic problems of the original component measures (par- ticularly the ISS) in underestimating survival probabilities for some types of patients, it has two major advantages. First, it links physiologic and anatomic information in a highly accurate and statistically powerful way. Second, it permits comparisons of survival outcomes to a national norm, thereby enabling hospitals and trauma centers to judge their outcomes against national data, adjusting for severity mix. When Champion and colleagues (Champion et al., 1990a; Sacco et al., 1993) attempted in a methodology called ASCOT (A Severity Characteriza- tion of Trauma) to modify TRISS to address certain of its limitations, they
PLANNING, EVALUATION, AND RESEARCH 267 amassed information indicating that ASCOT was a significant improvement over TRISS (in a study involving nearly 80,000 patients). Markle and colleagues (1992), however, in a study involving about 3,000 patients, did not find that ASCOT produced quantitative gains sufficient to offset the greater complexity and need for computer processing. Thus, more research with different patient data sets will be needed before a definitive judgment Abbott ASCOT can be made. TRISS, like the other listed measures, was developed on and for adults, leaving (at least in some users' minds) a significant gap insofar as EMS-C is concerned. The reasons for expecting children to respond to injury dif- ferently than adults are summed up in the aphorism "children are not minia- ture adults." Anatomy (e.g., surface area, airway structure), physiology and physiologic reserve, ability to verbalize, and other factors are different for children than for adults. To the extent (which is great) that trauma scoring tools are based on those variables as calibrated for adults, they risk being inappropriate for children. That is, when used for triage (or severity or case-mix adjustments), they may not be very sensitive or specific for some or all of the pediatric age range (as discussed below). Not all experts in this field agree with this assessment. They argue that the AIS, ISS, and TRISS have all been validated or at least used success fully in pediatric populations and studies and that the reliability of some adult-based measures is sufficiently high in pediatric populations to permit their use (Eichelberger et al., 1988a,b, 1989a,b, 1993; Nakayama et al., 1989; Kaufmann et al., 1990, 1991; Fortune et al., 1992; Reynolds, 1992~. Furthermore, updates to the AIS in the early 1990s include definitions of injury that accommodate differences in severity assessment between adults and children; because this change will improve its potential for scoring pediatric injury, it will also improve the applicability of the ISS (and hence TRISS) for children. Pediatric Trauma Score To address the perceived void, however, Tepas and his colleagues (1987, 1988) developed the Pediatric Trauma Score (PTS) specifically for children. PTS uses scores on six parameters: size of patient, airway status and ad- equacy of breathing, blood pressure (as an indicator of perfusion status), central nervous system status (e.g., from awake to comatose), skeletal status (e.g., closed or open fractures), and cutaneous status (e.g., minor contusions and abrasions through penetrating injury and soft tissue loss). These com- ponents are intended "to quantify injury by the numeric gradation of ana- tomic, physiologic, or functional derangement" (Tepas, 1989, p. 65; Reynolds, 1992~. These variables reflect what clinicians regard as the most predictive indicators of how severely injured a patient might be, based on initial pre
268 EMERGENCY MEDICAL SERVICES FOR CHILDREN sensation or encounter, and they are specifically intended to reflect both physiologic derangement and anatomic dysfunction. Various other researchers have evaluated the PTS in both prehospital and hospital settings (Ford et al., 1988; Ramenofsky et al., 1988; Aprahamian et al., 1990~. The PTS is intended also to be a scoring protocol or short checklist that will assist in decisions regarding the pubertal use of considerable- resources (e.g., when patients need to be transported from one hospital to another, higher-capability facility). Because its components are objective indicators of clinical status, without elements that might reflect socioeco- nomic or other considerations, its proponents argue that the PTS provides a useful mechanism for improving the operations of regionalized trauma systems and for tracking the appropriateness of transports and transfers (Tepas, 1992~. Other Pediatric-Specific Measures Problems with applying the CRAMS score to children led one hospital to develop, as an alternative, a criteria-based Children's Trauma Tool (Kitchen end Haubner, 1989J. Thetriage-revisedTS (Eichelbergeret al., 1989b) takes account of physiologically higher respiratory rates in children younger than 2 years but is otherwise equivalent to the RTS. Efforts also are being made to develop pediatric norms (based on children 14 and younger) that might replace adult norms in TRISS pediatric survival predictions. Applications Trlage The ISS is often regarded as a "gold standard" measure of severity of injury (mainly for adults but arguably for children as well), and it has been used to validate pediatric applications of such measures as the TS, RTS, and the PTS. The final ISS score cannot be calculated, however, until patients have been fully evaluated in a hospital setting; hence, it is not itself a triage tool. By contrast, the TS (or RTS) is intended to be (and is widely used as) a triage tool in the field, and some advocate that EMS systems routinely collect its component data elements (Morris et al., 1986~. A related tool, the Triage Decision Scheme, is a flow-chart instrument designed to help emergency medical technicians (EMTs) make prehospital evaluations and decisions (chiefly as to whether to take the patient to a trauma center or evaluate and proceed under medical control) (NAEMT, 19901. Many of the trauma measures, including the TS, have been criticized as being insufficiently sensitive or specific to be useful as prehospital trauma prediction rules. Using it only on an adult population, Morris and col
PLANNING, EVALUATION, AND RESEARCH 269 leagues (1986) showed, for instance, that the sensitivity and specificity of the TS (for correctly identifying patients needing or not needing to be trans- ported to a trauma center) are highly influenced by the specific TS score selected to indicate "severe" injury. They present data to suggest that the TS may well identify patients likely to have severe injury, but it may also have some [~1~-negatives arid miss patients with major thoracic injury; in addition, problems of undertriage or overtriage may differ depending on whether the injury reflects penetrating or blunt trauma. Baxt et al. (1989) determined that the TS, RTS, CRAMS, and Prehospital Index (PHI) could accurately predict which trauma victims would die but that they could not accurately pinpoint which patients of those who initially appeared physi- ologically normal (in the prehospital setting) were in fact victims of major trauma (i.e., could not distinguish major from minor trauma injury). Kane et al. (1985) evaluated several existing measures (e.g., the TS) and some new instruments for triage and concluded that "the most striking finding . . . is the disappointing performance of all [emphasis in original] the triage techniques studied" (p. 4881. They question whether methods based on rigidly defined variables can achieve high sensitivity, high specificity, and positive predictive validity, and they propose that the reliability of para- medic observation itself might deserve further exploration. By contrast, Eichelberger and various colleagues (Eichelberger et al., 1989b) have developed evidence from a pediatric database (0 to 14 years) that the TS, PTS, and unweighted RTS have an acceptable sensitivity for triage decisions (e.g., whether to transport a child to a pediatric trauma center), although the last-named may have a less acceptable specificity. As noted above, the PTS has been shown to have good predictive validity when used for trauma triage in the field, and Tepas (1992) argues that a tool such as the PTS may have its "greatest applicability in . . . [improving] care for the rural population" (p. 1769. Not all experts in the field argue so strenuously for the use of formal trauma or triage tools. Emerman and colleagues (1991), for example, re- ported that, in one metropolitan area, EMTs' judgments about overall sever- ity of trauma and predictions of mortality and need for immediate operative intervention were as accurate as scores from the triage-revised TS, the PHI, and CRAMS. (This point would seem to accord with the sentiments ex- pressed by Kane et al., 1985.) With respect to triage-transport decisions, Orr and colleagues (1992) comment that the PTS (as well as the GCS and MISS) "may prove useful for certain categories of patients, such as those with multiple trauma or necrologic disease" (p. 1573~. Overall, therefore, the evidence about the application of trauma severity measures in triage seems to be mixed; relatively poor performance in predicting major trauma for adults will likely be worse for pediatric patients.
270 Mortality Predictors EMERGENCY MEDICAL SERVICES FOR CHILDREN Some measures, such as the Pediatric Risk of Mortality (PRISM) score (Pollack et al., 1988b) and TRISS (Eichelberger et al., 1988b), can help identify unexpected (i.e., potentially preventable) deaths as well as unsurvivable injuries (i.e., expected deaths). At least one study, however, suggests that -trauma scoring systems based on AlS and ISS, when used for pediatric patients, may overestimate the "non-salvageable" population in that age group, perhaps because of overemphasis on head injury or limitations of the original evaluation; some children so designated do survive, and at least some of those who do have a high probability of good long-term functional outcomes (Jaimovich et al., 1991~. Generally, the state of the art of mortal- ity prediction among injured children, which relates directly to ongoing debates about the adequacy of trauma scoring systems for pediatric patients, remains in flux. Questions about mortality measures are pertinent to efforts to evaluate pediatric trauma programs. One issue centers on whether "preventable death rate" (PDR, the proportion of all deaths that could have been avoided had care been optimal) or"effectiveness" (E, the proportion of severely injured patients who were salvageable and survived, which excludes those who would be expected to die despite optimal treatment) is the better evaluative measure. Wesson and colleagues (1988), for instance, have argued that the PDR is sensitive to case mix which in turn would be sensitive to the injury severity index used and that E would be a superior way to assess how well a pediatric trauma program was functioning, but commentary on the work suggests that the issue is not resolved. ILLNESS MEASURES Scoring systems to assess the severity of pediatric illness, particularly in the prehospital setting for triage purposes, or to predict the outcome from illness, have proven even more difficult to develop than those available for trauma. Indeed, the committee recognizes that no reliable or valid illness severity scores exist that could be universally applicable for illness assess- ment of all pediatric patients. The following discussion provides a brief overview of some of the major illness measures presently available, but it is not all-inclusive. For example, disease-specific scores that are used in pediatric emergency care, such as for asthma, croup, or meningococcemia, are not discussed. As with trauma and injury scores or other physiologic measures that, in theory at least, reflect illness acuity, most illness severity measures have been developed and used for adult patients in a general hospital population. They tend to be "global" measures (in much the same way that trauma or
PLANNING, EVALUATION, AND RESEARCH 271 injury can be considered a "global" or "generic" condition), rather than disease- or condition-specific. Not surprisingly, they may often not be appropriate for pediatric patients. Selected Measures General Illness-Related Measures Iezzoni (1991) provides a definitive review of measures to standardize (i.e., adjust for) severity of illness, especially in their role in research, quality assessment, and health policy work. Among the better known sever- ity-adjustment tools are the following: · Acute Physiology and Chronic Health Evaluation (APACHE and APACHE II) (Knaus et al., 1981, 1985~; · the Computerized Severity Index (Horn and Horn, 1986; Averill et al., 1989~; · the disease staging methodology (Gonnella et al., 1976,1984; SysteMetrics, 1988~; · MedisGroups (Brewster et al., 1985; Iezzoni and Moskowitz, 1988~; and · Patient Management Categories (Young et al., 1982; Young, 1984~. Measures that are well known for or used for trauma and injury but are also regarded as applicable to illness include the PHI (Koehler et al., 1986), the Physiologic Stability Index (PSI) (Yeh et al., 1984), and the Therapeutic Intervention Scoring System (TISS) (sullen et al., 1974; Yeh et al., 1982~. Pediatric Measures Certain pediatric instruments can also reflect both illness and injury severity. For example, PRISM (Pollack et al., 1988b; Pollack, 1993) is a simplification of the PSI; one variant (Dynamic Objective Risk Assessment, or DORA) uses multiple PRISM scores to assess worsening or improving levels of severity of illness (Ruttimann et al., 1986; Ruttimann and Pollack, 1991~. A pediatric version of the Glasgow Coma Scale has been proposed (Reilly et al., 1988~. The Maryland EMS-C Project (1992) developed a preliminary version of a Pediatric Severity Assessment Tool (PSAT) that is designed to be ap- propriate for prehospital providers as well as clinicians in community hos- pital EDs and physicians' offices. Its designers intend that it quickly iden- tify patients who need various levels of emergency care without having to resort to much in the way of laboratory data (except perhaps for an electro
272 EMERGENCY MEDICAL SERVICES FOR CHILDREN cardiogram or pulse oximetry reading). The PSAT is still undergoing field testing. For children, the greatest success has occurred in formulating measures that assess mortality risk or the "intensity" of the care needed by the most critically ill (or injured) children; these include PRISM, PSI, and TISS. (BuGhert and Yeh. 1992' review some of the prehospital illness scoring measures.) For severity of zIIness measures, a variety of physiologic and behavioral indicators respiratory rate, fever, lethargy, irritability contrib- ute to assessments of severity of illness. Children, especially younger ones, may not manifest ordinarily well-known signs and symptoms of the ill- nesses they have, so a measure that would indicate which children require laboratory evaluation to rule out serious or life-threatening illnesses and which children have only minor ailments will be useful. One such measure what has come to be called the Yale Observation Score was developed by McCarthy et al. (1982) in an attempt to identify items that could be used to help clinicians recognize, reliably, validly, and quickly, serious illnesses in children with fever. The model includes six observation items (quality of cry, reaction to parent stimulation, variation in awake-asleep status, color, hydration, and response to social overtures), each scored on an item-specific three-point scale (normal, moderate impair- ment, severe impairment). One test in the hospital ED setting raises ques- tions, however, as to whether this scale, even in the hands of experienced physicians, will provide sufficient information to identify serious febrile illness in very young infants (Baker et al., 1990~. The "SAVE A CHILD" triage approach, based on a simple mnemonic (skin, activity, ventilation, eye contact - abuse - cry heat immune system level of consciousness and _ _ , _ , , , dehydration) is another example of a protocol for EDs intended to provide markers of possibly serious illness (Wiebe and Rosen, 1991~. The "SAVE" part of the triage guide involves simple observations, and the "A CHILD" portion involves observations and simple history questions; in total, it takes less than one minute to complete, and its developers thus claim that it would be particularly valuable for busy EDs. Comment Generally, it can be said that generic or global measures of acuity or severity of illness in children lag those for trauma and injury by a consider- able degree. In view of the evidence presented in Chapter 2 about the extent to which illness is the reason that children (especially younger chil- dren) require emergency care, this is a major gap in the tools available to EMS-C personnel, researchers, and evaluators. In addition, measures other- wise considered reasonably applicable to the pediatric age group tend to be unreliable in infants under 2 months.
PLANNING, EVALUATION, AND RESEARCH 273 Typically, instruments are developed first for research purposes (which call for retrospective scoring of the severity of illness), and they may be quite sophisticated. Those intended for prospective clinical use come later and may not, at the outset, be as complex, reliable, or valid. Few if any global illness assessment scores can be used to predict outcomes reliably across the entire pediatric age span; even among the di$ea$e-specific m.ea- sures, predictability may not be high. Far less has been done to try to apply existing measures in any program evaluation or quality assurance effort. Thus, an appreciable amount of investigation and validation lies ahead to provide the EMS-C field with reliable and valid instruments related to illnesses across the pediatric age range that can be employed in research and evaluation activities. As with the trauma and injury measures, then, the committee concluded that this was an area warranting high priority in the research agenda proposed in the main part of this chapter.
274 EMERGENCY MEDICAL SERVICES FOR CHILDREN Appendix 7B Emergency Medical Services for Children Proposed National Uniform Data Elements As is clear in the main text of Chapter 7, the Institute of Medicine study committee firmly believes that high priority must be given to collection and analysis of nationally comparable data at local, state, and national levels on emergency medical services for children (EMS-C). Data must be obtained from all phases of care for emergency conditions, including prehospital, emergency department (ED), intensive care and other inpatient services, and rehabilitation. The committee recognizes that, for data collection and analysis to succeed, a wide range of interested parties (e.g., health care providers, public agencies, hospitals, researchers, and administrators) will need to reach consensus regarding definitions and formats for specific data to be col- lected. Substantial work also remains to resolve many administrative and technical issues, such as determining which federal and state agencies will have responsibility for assembling and analyzing data or what electronic format will be used to store the data. The committee agreed that it was not the appropriate body to resolve these various issues. Instead, the committee sees its role as having three parts: to make clear the importance of data collection and analysis in the further development of EMS-C, to emphasize the need to establish an es- sential core of data to support important analyses, and to encourage indi- viduals and institutions to take action. The committee felt very strongly that getting started on defining a core data set and collecting data was critical. To that end, it has proposed an initial set of data elements for EMS-C, which is described in detail in this appendix. These data elements must be seen as a starting point for a more comprehensive data resource. In its discussions of these issues, the committee focused particularly on prehospital and ED care, believing that systematic collection of nationally comparable EMS-C data should begin with those settings. It then identified data elements that it believes ought to be part of a national data set. Several criteria guided the selection of specific elements, including utility of a data element for national and local analyses, adaptability to changing informa- tion requirements, ability and willingness of providers to capture accurate data, and availability of reliable and valid measures. The committee also considered whether evidence was available that data can be collected and used successfully without undue cost. Finally, it focused on the utility of this core data set for planning and evaluation purposes, rather than for research.
PLANNING, EVALUATION, AND RESEARCH 275 The "priority" or "core" data elements are listed below along with com- ments reflecting the committee's reasons for selecting them or other obser- vations about the data element. In the case of E-codes, the formal recom- mendation made in the full report is noted. Also listed are two other categories of data elements "desirable" and "rejected" that the committee discussed but chose not to select' and annotations concerning the basis for consider at~on or rejection (respectively) are also given. General statements agreed to by the committee regarding the collection of these data are presented just below. SPECIAL CONSIDERATIONS Prehospital and Transport Services The committee advanced three broad principles to guide collection of these data in the prehospital setting. First, data should be collected on every call to which a prehospital provider responds, even if no patient is transported. Second, data should be collected from each ambulance (or other EMS unit) that responds to a request for assistance; coding schemes should be developed to make it possible to link separate reports on the same event. Third, data should be collected on interfacility transport provided by ambulance units; data on these transports should be distinguishable from prehospital transports. (The data elements relevant for interfacility trans- ports may differ from those presented below for prehospital reporting; the committee did not attempt to develop a list of data elements specifically for interfacility transports.) Emergency Departments The committee noted that several parameters that are markers for the physiologic status of the child should be collected routinely in EDs (even if they are not now so collected); they may then be helpful in comparing institutions, systems, regions, and states. These measures include the ele- ments of the Glasgow Coma Scale (GCS) and vital signs. The committee recognizes that problems exist at present in collecting these data reliably; for example, clerical personnel who routinely abstract data may not be familiar with the particular elements (especially of the GCS). A somewhat similar case is made for collecting certain kinds of zip code information as part of the prehospital data set. That is, even though such information is not now routinely collected by EMS agencies, it will be valuable for system evaluation, planning, and descriptive work, and the committee therefore advises that these data elements be included in routine data collection.
276 EMERGENCY MEDICAL SERVICES FOR CHILDREN TABLE 7B-1 Data Elements Considered for a National Uniform Data Set for Emergency Medical Services for Children: Prehospital Services and Emergency Departments Prehospital Data Elements Data Element Basis ~r Se-lechon, Considerations or Rejection Priority Data Elements Date of Birth/Age Gender (M/F) Date and Time: · Call for assistance (or EMS dispatch) · Arrival on scene · Departure from scene · ED arrival Prehospital Assessment Spontaneous Breathing (YIN): · On arrival at scene · On arrival at hospital Spontaneous Pulse (YIN): · On arrival at scene · On arrival at hospital Level of Consciousness on Arrival at Scene (A/V/P/U) Alert Response to Voice Response to Pain Unresponsive Disposition: Left at scene Transported Transported to: Hospital identification number (Hospital zip code) Zip Code for Site of EMS Encounter Relationship of age to number and nature of emergency events and to outcomes; facilitate linkage between ED and EMS records; may not be routine part of current Prehospital data collection; the committee encourages use of date of birth, which is a better tool for record matching. Distinguish differences in types of emergencies (and therefore service needs), especially among adolescents. Calculate elapsed time in stages of Prehospital care; travel time can be an indicator of distances; ED arrival time can facilitate linkage between EMS and ED records. Time reporting is often retrospective; need to be aware of limits on accuracy. Nature of patient's condition (e.g., injury, illness, uncertain). State of breathing and pulse give an indication of acuity; deterioration or improvement on arrival at hospital. Indication of acuity, salvageability; use to assess destination choices and triage practices. Distribution of cases by action taken; assess relationship of acuity to disposition. Nature of destination ED (community hospital versus referral center); possible development of denominator data; facilitate linkage between EMS and ED records. A consistent national system; location of emergency relative to location of initial hospital care; analysis of transport and triage practices; location of events for prevention analysis; possible development of denominator data; can aid in matching EMS and ED records when location zip code is also home zip code.
PLANNING, EVALUATION, AND RESEARCH TABLE 7B-1 Continued 277 Prehospital Data Elements continued Data Element Basis for Selection, Consideration, or Rejection Prehospital Services: · Pros An humor · Prehospital run report number Type of Responder (e.g., ALS, BLS) Desirable Data Elements Mechanism of Injury EMS Interventions Used Rejected Data Elements Vital Signs Loss of Consciousness or Conscious (Y/N/Uncertain) Dead on Scene/ Alive on Arrival at ED Resuscitation Needed Medical/Surgical/Other Facilitate linkage between ED and EMS records. Qualifications of responders. Good correlation with outcome; already part of trauma system and much EMS reporting; valuable for analysis of determinants of injury; appropriate response categories need to be defined before inclusion in a national data set. Appropriate response categories need to be defined before inclusion in a national data set; may not be possible to determine appropriateness of intervention use or nonuse; very important for local quality assurance/quality improvement activities. Not useful for aggregated analysis; may not be reliably collected in the field. Replaced as acuity indicator by level of consciousness on arrival at scene; potential for uncertainty in determination; requires careful specification of time of assessment. Acuity indicator; complications in determination of death (e.g., state before and after resuscitation); restrictions on EMT authority to declare death; significant regional differences in practices regarding declaration of death. Acuity indicator; based on judgment rather than objectively observable or measurable state. Nature of case; replaced by condition categories more appropriate for determination by EMS personnel. continued
278 TABLE 7B-1 Continued EMERGENCY MEDICAL SERVICES FOR CHILDREN Hospital Emergency Department Data Elements Data Element Priority Data Elements Date of Birth Gender (M/F) Race/Ethnicity Mode of Transport (e.g., self, EMS, interfacility, air, ground) Date and Time of ED Arrival Date and Time of Discharge Disposition (e.g., dead on arrival; died in ED; discharged home; admitted; transferred) Diagnostic Codes (allow for reporting multiple ICD-9-CM nosology [or N] codes for each patient) Procedure Codes (allow for reporting multiple CPT-4 codes for each patient) External-Cause-of-Injury (E) Codes (allow for reporting multiple ICD-9-CM including site of injury) Glasgow Coma Scale (GCS) Components Verbal response ~ . Bye opening Best motor response Vital Signs (initial readings) Pulse Respiratory rate Systolic blood pressure Temperature Basis for Selection, Consideration, or Rejection Relationship of age to number and nature of emergency events and to outcomes; facilitate linkage between ED and EMS records. Distinguish differences in types of emergencies (and therefore service needs), especially among adolescents. Assess relationship to incidence and acuity (implications for access to other care). Distribution of cases by transport; relationship between transport and acuity; relationship between transport and outcome; appropriate response categories and criteria need to be established. Calculate length of time to discharge; facilitate linkage between ED and EMS records. Calculate length of time from arrival to discharge; relationship between time, acuity, and nature of a~spos~t~on. Distribution of cases by disposition; assess referral patterns; indication of acuity; identify cases to track for need/use of further services. Distribution of conditions presenting to the ED; indication of acuity. Hospitals may not currently be coding diagnoses and procedures if they do not bill for ED services; ICD-9-CM coding is often time consuming and imprecise; alternatives should be explored. Indication of acuity; appropriateness of care; resources and training used; accuracy of coding may be poor. Correlation with outcomes; assessing prevention needs and impact of prevention programs; committee strongly recommends universal adoption of E-codes. Acuity indicator; valuable research tool; application to children may need further refinement. Necessary element for TRISS analysis, so required for aggregated comparative analyses when adjustments for differences in patient injury severity are desirable; also useful for adjustments for illness severity (TRISS analysis is based on Revised Trauma Score and Injury Severity Score).
PLANNING, EVALUATION, AND RESEARCH TABLE 7B-1 Continued 279 Hospital Emergency Department Data Elements continued Data Element Basis for Selection, Consideration, or Rejection IrtsurancelPayer Codes (e g^' self-pay, Med~caid,-publ~e assistance, CHAMPUS, private insurance) Prehospital services: · Provider identification number · Run report number Hospital Identification Number Hospital Zip Code Home Zip Code/Country Desirable Data Elements Date and Time of Physician Encounter Date and Time of Disposition Usual Source of Care (e.g., yes, no, not known) Communication Barriers Between Provider and Patient (or Family) Prior Health Status Rejected Data Elements Loss of Consciousness (history at time of arrival) Clarify impact A insurance status on ED a~ EMS use (actual and perceived costs; relationship to acuity, diagnosis, transfer, access to other care). Facilitate linkage between ED and EMS records. Characterize level of care available from ED (community hospital versus referral center); facilitate linkage between ED and EMS records. Assess area resources through other sources of data; possible development of denominator data. Facilitate other analyses: correlate area resources for family of patient; describe population characteristics of area; possible development of denominator data; relation of residence to location of initial emergency care; referral patterns; regional concerns regarding use of services by residents of other areas (counties, states, countries). Assess timeliness of care or transfer to more appropriate level of care in relation to acuity. Identify procedural delays between ED assessment or treatment and discharge or transfer to most appropriate care. Indicate access to routine care (i.e., a medical home); timing and acuity of visits when source of routine care is available; needs further study before being added to the recommended data elements. Clarify relationship with acuity, outcome; appropriate indicators need to be determined. Identify children with chronic illness or other special health care needs; appropriate indicators need to be determined. Acuity indicator; replaced by diagnostic and procedure codes.