Medical Informatics for Detecting Adverse Events
GENEVIEVE B. MELTON
University of Minnesota
The detection and prevention of adverse events (AEs) in medicine represents a national priority. AEs, defined as injuries that occur in the course of medical management, have important consequences, including higher costs, higher morbidity, and higher mortality. Large-scale focus by the Institute of Medicine (Kohn, 2000) has emphasized most prominently the importance of detecting and preventing AEs to improve patient outcomes. By identifying the AE and analyzing, the context in which it occurred, AE detection can improve system factors and cognitive processes surrounding possible future events and direct resources into targeted efforts to prevent AEs in healthcare.
Unfortunately, with traditional voluntary reporting methods, many AEs are not reported. Manual reviews of charts, although effective, are too costly for routine use. Information technology and informatics tools that use data from electronic health records (EHRs) can potentially improve AE detection and have been identified as important tools for creating a “culture of safety.”
Several classes of automated AE detection systems have been described, most of which use numeric or coded data from EHRs, such as codes for diagnoses and procedures, records of medication administration, laboratory values, and vital signs. Substantial progress has been made in detecting and preventing adverse drug events (ADEs), particularly with the introduction of electronic prescriptions in both inpatient and outpatient settings. Natural language processing (NLP), a set of automated techniques for converting narrative text into a format appropri-
ate for computer-based analysis, can be used alone or in combination with other automated methods to improve AE detection.
CHALLENGES AND CONSIDERATIONS FOR AUTOMATED SYSTEMS
AE detection can be challenging because of the complexity and lack of standardization of health care EHR systems and associated electronic data, particularly the lack of standards for the quality and formatting of data, standard definitions of AEs, the variable performance of heuristic rule-based systems, and sparse data sets on low-incidence events. Although most automated AE systems provide feedback retrospectively, “active surveillance” systems that alert providers or administrators of events as they occur can potentially identify and investigate AEs much more quickly.
Many robust EHR systems code data for administrative and billing processes (International Disease Classification version 9 codes and Current Procedural Terminology codes), demographics, laboratory results, admission and discharge registrations, medication administration, and computerized physician order entry (CPOE). However, with the exception of coding for administrative processes, even these structured data are often formatted differently for different EHR and hospital systems.
Although some AEs can be found using coded data, a large number of them require supplementary methods and data sources. One reason for this is that administrative and billing data can be incomplete or inaccurate, and they often do not include AEs explicitly. In addition, more sophisticated data necessary for AE detection, such as clinical reasoning, signs and symptoms, clinical summaries, and physical findings are typically not included as structured data.
Standardized definitions of AEs are a fundamental prerequisite for accurate measurement and analysis. However, centralized nomenclatures, or taxonomies, have not been agreed upon for each health care setting. National initiatives will be necessary to reach agreement and bring about consensus. Several promising AE classification systems have been proposed according to setting or discipline, including the JCAHO Patient Safety Event Taxonomy and the Clavien-Dindo Classification of Surgical Complications.
Up to now, rule-based heuristic systems based on data from a variety of sources have been primarily used for AE detection. Although such systems perform well for certain tasks, they rely heavily on “triggers,” such as abnormal laboratory values or low blood pressure, to indicate a possible AE. Machine-learning techniques are a promising set of approaches that can help to detect events within datasets. Supervised and semi-supervised techniques are particularly helpful in cases where the AE is not obvious or intuitive, particularly when a large, robust, and well-defined dataset exists to allow for adequate system training and optimized performance.
However, classification systems using machine learning can also perform poorly, particularly for AEs that have a low incidence (< 1 percent), for which data sets may be sparse and unbalanced. Several techniques have been proposed for balancing such data sets, and some have been using sampling techniques with variable success.
Developers of AE detection systems must also be aware of the cost of false negatives and false positives. An important trade-off must be made between the clinical indication and relative cost of screening extra patients to find AEs and the cost of missing AEs. Most AE detection systems are designed to minimize false negatives so as to maximize the overall detection rate; adjunct manual screening is usually used to confirm identified AEs.
ADVERSE DRUG EVENTS: AN EXAMPLE OF IMPROVED DETECTION
Most ADEs occur when drugs are ordered (55 percent), administered (35 percent), transcribed (5 percent), or dispensed (5 percent). In hospitals that use CPOEs, orders for medications and other clinical care are entered directly into the EHR system. CPOE has been the most successful example of an information technology that helps to detect ADEs. Because many CPOE systems now include alerts and reminders about drug prescriptions, they can also prevent many ADEs.
Additional “triggers” in coded data about medication administration or abnormal laboratory values (e.g., supra-therapeutic or sub-therapeutic drug levels, low hemoglobin, or poor renal function) can improve detection or even prevent many ADEs. This has been demonstrated in several clinical trials in both inpatient and outpatient settings.
NATURAL LANGUAGE PROCESSING: A TOOL TO IMPROVE DETECTION
Clinical documents in EHRs are promising data sources for AE detection systems because they often include clinical reasoning, signs and symptoms, clinician’s summaries, and physical findings, all of which may be helpful for AE detection. Although such narratives are rich in content, they present significant challenges to automated systems in the medical domain. Several investigators have tried using “trigger words,” such as “perforation,” “iatrogenic,” or “error,” for AE detection. However, this technique has limited utility because it does not distinguish between a real, potential, or past event or condition.
Serious challenges to using NLP, or medical text-mining, will have to be overcome. Clinical documents are variably formatted with section headers, tabs or other spatial formatting, and transcription errors (i.e., misspellings and grammatical errors). Meaning in medical texts is not straightforward; there are often
uncertainties, negations, and questions about timing. In addition, medical terms include synonymy (related or synonymous terms), abbreviations (often redundant), and context-specific meanings.
Several automated text-mining tools have been developed, including opensource tools available through the National Library of Medicine. One widely used proprietary medical NLP application, MedLEE, uses a vocabulary and grammar to extract data from text. Although MedLEE was initially used to extract information from radiographic reports, it has been expanded for application to a wide range of medical texts. When applied to discharge summaries, it has demonstrated a significant improvement in AE detection compared to traditional reporting alone. Thus NLP techniques can potentially improve AE detection systems.
Automated AE detection systems with automated informatics techniques have shown promising results for improving the detection and ultimately the prevention of AEs. National initiatives for the adoption of universal EHR systems and advances in informatics techniques for AE detection are likely to increase the use of these systems, which are now widely used only for ADE systems in health care. Addressing technical challenges related to AE nomenclature, machine-learning methods, sampling techniques, and NLP will improve system performance and, ultimately, improve patient safety.
Chang, A., P.M. Schyve, R.J. Croteau, D.S. O’Leary, and J.M. Loeb. 2005. The JCAHO patient safety event taxonomy: a standardized terminology and classification schema for near misses and adverse events. International Journal for Quality in Health Care 11(2): 95–105.
Clavien, P.A., J. Barkun, M.L. de Oliveira, J.N. Vauthey, D. Dindo, R.D. Schulick, E. de Santibañes, J. Pekolj, K. Slankamenac, C. Bassi, R. Graf, R. Vonlanthen, R. Padbury, J.L. Cameron, and M. Makuuchi. 2009. The Clavien-Dindo classification of surgical complications: five-year experience. Annals of Surgery 250(2): 187–196.
Handler, S.M., R.L. Altman, S. Perera, J.T. Hanlon, S.A. Studenski, J.E. Bost, M.I. Saul, and D.B. Fridsma. 2007. A systematic review of the performance characteristics of clinical event monitor signals used to detect adverse drug events in the hospital setting. Journal of the American Medical Informatics Association 14(4): 451–458.
Kilbridge, P.M., and D.C. Classen. 2008. The informatics opportunities at the intersection of patient safety and clinical informatics. Journal of the American Medical Informatics Association 15(4): 397–407.
Kohn, L.T., Corrigan, J.M., and Donaldson, M.S., editors. 2000. To Err is Human: Building a Safer Health System. Washington, D.C.: National Academy Press.
Melton, G.B., and G. Hripcsak. 2005. Automated detection of adverse events using natural language processing of discharge summaries. Journal of the American Medical Informatics Association 12(4): 448–457.