appropriate and timely decisions regarding the withdrawal or termination of care in critically ill ICU patients. Although it is attractive to use these models to make these difficult decisions on a more rational basis informed by patient risk, we should be cautious in embracing these models for this purpose. This essay will briefly touch on several problems related to this use of these models.

  • 1.  

    There are statistical limitations.

    The models produce probability estimates. They are developed and validated on large patient populations. The data and models must be valid and reliable for aggregated groups of patients to satisfy statistical and methodological requirements. They have not been validated for individual patient decisions. For example, a model that predicts a 50 percent mortality for 100 ICU patients may prove to be 100 percent accurate when applied to the next 100 ICU patients viewed as a group, but if the model is used to identify the 50 individual patients who will live and the 50 who will die, it is theoretically possible to misclassify all of the patients and still be 100 percent accurate in aggregate. Among patients with a very high probability (>90 percent) of death, different problems exist. The calibration of the models is most suspect at the extremes of probabilities. The models invariably perform best in the midrange of probabilities and are most useful in that range when used on aggregated patients. Models lack statistical power among very high-risk patients because of the low number of cases in the very high-risk strata. The confidence limits around predictions of very high risk of death are likely to include probabilities that might make both clinicians and family members cautious about discontinuing therapy. In other words, our ability to predict death with sufficient certainty is unlikely to be an achievable goal on statistical grounds alone. The size of the database that would be required for statistical certainty among very high-risk patients is unlikely to ever be achieved within realistic cost and time constraints.

  • 2.  

    The models are inherently imperfect.

    The models are in need of constant revision, hence APACHE III. This should give us pause about viewing the most current version of a model as definitive enough to make life or death decisions based on the model alone. These models have a number of potential limitations. Data elements used in the model may be missing or inaccurate. The time at which the variables are captured and their relationships to interventions are also potential sources of bias (see below). The model must be proven to give the same predictive accuracy when applied in different settings and over time (it must be "ro-



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement