Click for next page ( 46


The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 45
3 Risk Assessment A woman's eligibility to remain in a particular birth setting through delivery is contingent on safety considerations, partly defined by an assessment of obstetric risk. Some birth centers automatically eliai- nate a prospective mother if a certain characteristic, such as high blood pressure, is present. Others rely on a score derived from a ~ bination of characteristics. It is important that investigators under- stand risk assessment, because it deteraines the population eligible to deliver in a given setting and thus the populations available for study. The different ways in which such assessaents are applied aeans that com- parisons among settings aust be made with care to ensure aatched groups. Obstetric risk assessaent instruaents are described at some detail in this chapter and in Appendix B. Much of the controversy surrounding childbirth settings would no longer exist if it were possible to predict with certainty that a low-risk woman or her fetus would experience no complications when delivering in a nonhospital setting. Research is needed to develop more accurate risk assessment instrumentsa While most instruments are useful for predicting neonatal mortality, they are less useful for predicting neonatal morbidity or maternal complications. OBS'l'BTRIC RISK ASSBSSMBNT A typical risk assessment scoring systea is based on variables associ- ated with the occurrence of ca.plications or adverse outcomes of preg- nancy or childbirth. Nineteen risk assessaent instruments are reviewed in Appendix B. They use different aethodologies and scoring systems. Three instruments used most frequently to assess risks in women and their neonates area 1. The Maternal-child Health care Index of Nesbitt and Aubry (1969) with the Labor Index of Aubry and Pennington (1973), which uses 50 fac- tors for assessing risks in women. 2. The Antepartua Petal Risk Score of Goodwin et al. (1969), which uses 21 factors for assessing risks in women. 3. The Problem Oriented Perinatal Risk Assessaent System (POPRAS) of Robel, et al. (1973) or its modification (Sokol, et al., 1977), which uses 91 factors to assess risks in the woman and 35 factors to assess risks in the neonate. 45

OCR for page 45
46 Measurement of Risk Measurement of risk provides a probability statement with respect to a future event. The choice of an appropriate risk assessment instrument fro. among the many available instruments should be baaed on how accu- rately it identifies the level of risk for particular subjects to be enrolled in a study (see Table 1). This accuracy can be evaluated by determining the instrument's level of validity (defined as the ability of a teat to measure a condition truly present). The sensitivity and specificity of the instrument are two indicators of ita validity. Sensitivity is an indication of a screening method's ability to identify correctly those patients with a given disease or condition. Among those with the disease, a very high proportion will be scored as •positive• on the risk instrument if the instrument is sensitive. !bus, the proportion of predicted perinatal deaths actually occuring to mothers who were classified as •high risk• is the sensitivity of the risk assessment method for the outca.e: perinatal death. !hose cases of perinatal death in which the mothers were labeled •1ow risk• are false negatives. !he false negative rate is one measure of predictive inaccuracy and the insensitivity of the risk assessment method. Specificity is an indication of the screening method's ability to identify correctly those patients without the condition. Among those free of disease, if a high proportion are labeled as low risk by the risk assessment instrument, then the teat is highly specific. !bus, the proportion of live births whose mothers were labeled as low risk is a measure of specificity. !hose live-born infanta whose mothers were labeled as high risk are false positives. Thus, the false positive rate is a measure of the nonspecificity of the test. ~e ability of the risk assessment instrument accurately to predict the eventual outcome is called the predictive value. Women are assessed as being at high risk with no certain foreknowledge of the actual out- coat, so that accuracy must be estimated from previous use of the in- strument. !he predictive value of high-risk assignment measures the percentage of those subjects in the high-risk group who experience complications. In assessing risk for alternative birth sites, the percentage of women assigned a low-risk score who subsequently experience ca.plica- tions is important. Most existing risk assessment instruments show good ability to predict that a low-risk pregnancy will net result in a peri- natal deatha More than 98 percent of woaen classified as low risk will have live infanta at the end of the perinatal period. (See references and the last column of Table 2 in Appendix B for percentages of low- risk women associated with subsequent perinatal deaths.) ~e risk assessment instrument must clearly differentiate between high- and low-risk groups and should achieve a sensitivity of 80 per- cent, an acceptable minimum according to Richards and Roberta (1967). Assuming that 5 percent of a population is at high risk, the occurrence of an outoo.e must be 16 times higher in the high-risk group than in the low-risk group before the sensitivity of the high-risk assessment can reach 80 percent. As an example, the Apgar score (Apgar, 1953) indicated that 6 percent of newborns were at high risk. ~e death rate

OCR for page 45
47 TABLE 1 Validity Measures of Screening Tests True Disease Occurrence Risk Assessment Persons Without Result Diseased Persons Disease Totals High risk With disease and Without disease All persons with high risk but with high labelled as assignment risk assignment •high risk• (true positives) (false positives) A B A+B Low risk With disease but Without disease All persons with low risk and with low risk labelled as assignment assignment •1ow risk• (false negatives) (true negatives) c D C+D Totals Total number of Total number of diseased persons persons without disease A+C B+D SOURCE: Adopted from Wilson and Junger, 1968. for the high-risk group was 4 percent, versus 0.1 percent for the low- risk group. Bence, the death rate was 40 times higher for the high- risk group. The sensitivity of the score was 92 percent. Many current risk assessment methods are not as successful as the Apgar score in differentiating high- and low-risk groups for some specific neonatal outcomes. Selection of Variables for Obstetric Risk Assessment Variables common to most risk assessment instruments include demo- graphic and socioeconomic data, data from past pregnancies, past medi- cal history, and present pregnancy. In some of the more recent studies, fetal heart rate and uterine contraction data from electronic monitoring are included (see Appendix E, Table 1). Some variables are good predic- tors of more than one adverse outcome, e.g., some of the same factors that predict low birth weight also predict neonatal mortality. Because decisions on the type of care to be provided during preg- nancy and delivery usually are made prior to labor, most risk assessment instruments include variables that apply only to the prepartum period.

OCR for page 45
48 Only a few instruments contain sections to assess risk during labor and childbirth (Aubry and Pennington, 1973; Bobel et al., 1973). Nonethe- less, the information collected before delivery can be used to evaluate the need for transfer of patients to settings that can provide care for severe complications. In designing or using a risk assessment instru- ment, it is important to know when selected data will be obtained (i.e., when during pregnancy, labor, or postpartum) because the prediction of risk can be altered according to the time the measurement is taken. The decision about comprehensiveness of a risk assessment instrument cannot be isolated from the total study design nor from the place and practice in which the instrument will be used. The number of items in- cluded in a risk instrument is frequently of concern to researchers because .are items to measure may mean more time spent on each subject. Furthermore, the total number of variables included is not always in- dicative of accuracy. Probably the minimum set of factors are those used by Goodwin et al. (1969). The sensitivity for neonatal mortality achieved by this measure has ranged from 46 percent to 86 percent de- pending on the study reporting itr the specificity of the measure has an even broader range (15 percent to 82 percent). The percentage of low-risk women who experience a neonatal death is very low (0.2 percent). The Bobel et al. (1973) method includes the greatest number of factors to measurer it has a specificity of 48 percent and sensitivity of 59 percent. The percentage of low risk women experiencing a neonatal death is also very low (0.3 percent). It should be noted that Babel's method can be used as the patient's medical record. Each of the variables included in a risk assessment instrument has some association with the outcome of interest, the strength of the association differing among variables. Low birth weight, for example, is more strongly associated with neonatal mortality than is maternal education. In an aggregation of variables to predict the occurrence of neonatal mortality, the birth weight variable should therefore be given more emphasis (weight) than should maternal education. A synopsis of methods used for weighting risk factors is in Appendix E, Table 1. Weighting the Variables Once women are assessed for risk, several options are available for weighting risk variables. The weights of all characteristics can be summed (Apgar, 1953; Goodwin et al., 1969; Hobel et al., 1973; Nesbitt and Aubry, 1969). The sum may indicate the level of risk or it may be subtracted from a perfect score, perhaps 100 as used by Nesbitt and Aubry (1969). Second, one can use a multivariate technique for scoring (Butler and Alberman, 1969; Chik et al., 1979; Hobel et al., 1979; Larks and Larks, 1968; Rantakallio, 1969; Stembera et al., 1975). Last, one might calculate odds ratios and then multiply them (Fedrick, 1976).

OCR for page 45
49 Aaaiga.ant of Risks TWo aetboda are frequently reported to assign risks• (1) predeterained cut-points, and aasiga.ant of the ..an as •yea or no• bigb risk, or (2) a continuous score with a probability statement attached to tbe score. It is t.portant to choose the score that will designate points at wbicb each risk level begins or ends, i.e., tbe •cut-points,• although risk is a probability statement and, therefore, a continuous variable. lOr ex.-ple, if a soore of 10 is chosen as a cut-point, wc.en classified above that point are at risk while those below it are not at risk. !he predictability, sensitivity, and specificity of tbe instrument are regulated by the cut-points used to classify a woman as bigb risk, i.e., tbe score at wbicb risk levels shift from low to bigb. The cut-points for declaring risk level are iaportant and, ideally, should be derived for each population to be studied. TWo populations with different d..ograpbic characteristics aay re- quire different cut-points for accurate declaration of bigb risk. An illustration of this is the black and Hispanic population studied by Winters et al. (1979) using tbe Robel instrument. When the cut-point for bigb risk is a score of 10 or more, wbicb is what Bobel et al. (1973) eaployed, 95 percent of the group was labeled bigb risk. A cut-point of 40 or more points yielded 41 percent as being bigb risk. The sensitivity for a cut-point of 10 was 100 percent, while tbe sen- sitivity for a cut-point of 40 in this population was 52 percent. 'lbe original sensitivity ascertained by Robel et al. (1973) at a cut-point of 10 in a California population was 37 percent. Researchers using siailar instruments should be aware that shifting cut-points from study to study decreases the co.parability of studies of different groups. 'lbere are several ex.-ples of this in Appendix Br Table 2. Nesbitt and Aubry (1969) classified scores of 0-70 as bigb risk and achieved 43 percent sensitivity for perinatal death. Wilson and Sill (1973) used the ••e instrument but changed the scoring to 0-40 for bigb risk and showed only 6 percent sensitivity. Bebb et al. (1980) used the Goodwin et al. (1969) instrument with a score of 4+ indicating bigb risk and found 86 percent sensitivity, while Morrison and Olsen (1979) bad a sensitivity of 70 percent, using a score of 3+ to designate bigb risk. Bxaaplea of predeterained cut-points abound (see Appendix Br Table 1). Bebb et al. (1980) eaployed a sUIIIIIed score and decided that a woman with a score of 4 or aore was at bigb risk of perinatal aortal- ityJ those with lower scores were not at bigb risk. Variables aaking up tbe score were weighted fro. 0 to 10. The disadvantage of this •tbocl is that two waaen, each with a cc.posite score of s, •ay have very different profiles in teras of the variable values affecting their scores. One woman aay have a single characteristic weighted by 5 (wbicb is a .oderately severe weighting) while tbe other aay bave five charac- teristics each weighted by 1 (the score for a relatively unblportant factor) • !he attending physician aay respond to the condition of each ..an in a different way, but the composite score identifies both va.en as being at equal risk. In practice, a woman with a score of 20 aay be regarded by a physician as being at the s_. risk as a ..an with a score of s.

OCR for page 45
so Examples of use of continuous scores are less frequently found. Several authors (Donahue and Wan, l973J HObel et al., 1979J Larks and Larks, 1968J Rantakallio, 1969) used multivariate scoring, but some, like Hobel et al. (1979), revert to using preestablished cut-points when classifying the women. Others maintain the continuous scores and establish percentage level cut-points: Rantakallio (1969) classified those scoring SO percent or more as being at high risk for perinatal deathJ Donahue and Wan (1973) used the upper 25 percent of the distri- bution of multivariate scores to designate high risk. Multivariate techniques tend to standardize weighting and scoring for each popula- tion to which an instrument is applied. Cut-points on the distribution can be similar, e.g., using the upper 25 percent of the distribution. Preassigned weighting systems such as that of Nesbitt and Aubry (1969) can be employed to assign probabilities to each scoreJ a more customized score for each woman may be achieved. Bobel (1979) discusses the use of such a scoring method and presents an example that can be calculated on a hand-held calculator. Collecting Information When conducting studies using obstetric risk assessment, the instrument for assigning risk should be a standard one that is applied uniformly at predesignated periods during pregnancy. It should be used by trained observers and tested before use so that results are similar among dif- ferent observers and among multiple observations by the same observer. The procedures for conducting these analyses should be detailed by the investigator. Most risk instruments necessitate direct observation of the woman and a few require that she be interviewed. Both observation and inter- view need to be conducted in a standardized manner. These methodologies are well developed and can be profitably used by researchers. Decision making about the presence or absence of a characteristic also must be standardized. The level of risk assigned is critical in separating the women at high risk of maternal difficulties from the women at low risk. The lack of reliability resulting from nonuniformity in the collection of data at different times or across different cases can seriously compromise the findings of a study. Because these instruments are currently used in many freestanding birth centers to admit prospective parents into the program, investigators will have to ascertain which instrument is being used. A large number of false negatives occuring in groups assigned a low-risk score could lead to incorrect conclusions about the childbirth setting under study. There should be provisions in the research design for handling changes in risk status during pregnancy. Some criteria should exist for changing the birth setting when risk factors are detected after original assignment. A study proposal should state how women will be followed through changing risk status and from one institution to another to assure complete collection of information.

OCR for page 45
51 Selection of a Risk Assessment Instrument Decisions about the appropriate instrument and how to use it will depend on the purposes and design of research. 'Dle factors said to be measured by the chosen instrument should have a demonstrative association with the outcoae being investigated. !he Bobel record (1976) is by far the most comprehensive obstetric risk assessment instrument. It also differs from many others in that it serves as the medical (obstetric) record for the patient. The infor- aation on weights and scoring is integrated into the patient record sys- tem, not collected separately. Moat of the other risk assessment in- struments are independent data collection ayateaa. '!hey are usually added to existing record systems for special purposes. Occasionally biochemical teats and fetal monitoring are used in con- junction with risk aaaeaa..nt instruments. Appropriate use of these additional indices as successful predictors requires detailed knowledge and a thorough understanding of the implications of the results. Petal aonitoring, together with risk aaaesa..nt methods, appears to increase specificity but not sensitivity. Petal monitoring does not appear to decrease the false negative rate, which is one of the aajor concerns in risk assessment. The outcome variables of interest should be considered when select- ing an instrument (see Chapter 4). Although moat reported instruments have a scoring system baaed on the occurrence of neonatal mortality, aaternal complications are frequently considered as outcome variables in research on childbirth settings. Exaaplea of maternal complications might include infection, hypertension, multiple pregnancy, abnormal presentation, failure to progress in labor, second state arrest, post- dates, and meconium staining. Examples of neonatal complications might include respiratory distress, infection, low birth weight, birth injury, or prolapsed cord. (Por a more complete listing, see Table 1 in Chey et al., 1976.) The risk instrument that contains factors and weights derived from data and literature and that is designed to focus on neo- natal (or perinatal) mortality may have lower sensitivity when predicting morbidity (complications). In general, false negative rates produced by risk assessment instru- ments are high. In Table 2 of Appendix E, for example, columna labeled •fal-• provide percentages of false negatives in studies reporting neo- natal complications, low birth weight, or perinatal mortality outcomes. More than 20 percent of women or their infanta experiencing undesirable outcomes typically have been assigned to a low-risk group. (See columna labeled •t low risk with problem• in Table 2, Appendix E.) Pew exist- ing risk assessment instruments achieve 80 percent sensitivity in pre- dicting perinatal mortality. (See the beading •Perinatal Death• in Table 2, Appendix E, and coapare, for these studies, the percentages under the column •aena,• the sensitivity of the screening tool.) Those that do achieve this level of sensitivity are the Goodwin et al. inatru- •ent as used by &ebb et al. (1980) and RObel's instrument as used by Sokol et al. (1977). Because moat risk assessments will probably be performed in the prenatal period, provisions must be made for identify- ing women whose risk level aay change during pregnancy or woaen who may

OCR for page 45
52 develop unanticipated complications in labor and delivery. A procedure will be needed for documenting such occurrences in a study and for handling thea in the research analysis. False positive rates present another research problem. It baa been shown that 14 percent of low-risk women transferred from alternative birth facilities to other facilities because of predicted complications did not experience any complications (Bennetts, 1981). It is t.portant to document the occurrence of falsely assigning women to the high-risk category. LIMITATIONS OF CURRBRT INSTRUMENTS Current risk assessment instruments employ one set of weights for all mothers, regardless of the major d..agraphic factors of age, ethnic group, and socioeconomic group. Yet there probably are group differ- ences in responses to problems identified by risk factors. Separate weighting and scoring systems for each age group, ethnic group, and socioeconomic group might improve predictability but may not be feas- ible to put into practice. More research is needed in this area. ~e same scoring system is used for prt.agravida and for multi- gravida wo.en. A large part of moat scoring systems depends on past pregnancy history to predict untoward events in the pregnancy. !here- fore, the risk scores are not as good at predicting some probleaa among prt.agravida women (Fedrick, 1976). !here is evidence that the weights assigned to risk variables aay require changes over time, even within the same population. Robel (1979) reports detecting a change in the strength of the association of some variables with mortality between 1973 and 1979. !his is another topic on which research is needed. Weighting of factors is probleaatic because there is no •pure• ...- sure of risk. We can never know the true rate of occurrence of disease related to a particular factor because, once a problem is detected, treat.ent occurs that may lessen the association between factor and disease. ~is means that all scores and cut-pointe are baaed on imper- fect knowledge. ~e risk inherent in the group may not apply to an individual be- cause risk factors themselves (and weights for the•) are derived from population or grouped data. !hue, clinical jud~nt is appropriate in determining treatment for an individual woman, and the risk assessment approach can be a useful adjunct. For research purposes, however, a risk assessment instrument is a more standardized ..thod than clinical judgaent for selecting groups of women with similar risks. Apgar, V. A. 1953. A Proposal for a new •thod of evaluation of the newborn infant. Current Researches in Anesthesia and Analgesia 32a260-267.

OCR for page 45
53 Aubry, R. B., and J. c. Pennington. 1973. Identification and evaluation of bigb-riak pregnancy1 'lbe perinatal concept. Clinical Obstetrics and Gynecology 16:3-27. Bennetts, B. 1981. Out-of-hospital childbearing centers in the United Statea1 A descriptive study of the demographic and medical- obstetric characteristics of women beginning labor therein, 1972-1979. Ph.D. thesis. University of Texas Health Science Center at Houston. Butler, H. R., and E. D. Alberman, eda. 1969. Perinatal Problema I 'l'he Second Report of the British Perinatal Mortality Survey. London1 Livingstone. Cbey, R. A., D. Haire, E. J. Quilligan, and M. B. Wingate. 1976. High risk pregnanciea1 Obstetrical and perinatal factors. In Prevention of Embryonic, Petal, and Perinatal Disease, Robert L. Brent and Maureen Barris, eda. DREW Publication No. (NIB) 76-853. Bethesda, Md.1 National Institutes of Health. Chik, L., R. J. Sokol, M.G. Rosen, s. K. Pillay, and s. E. Jarrell. 1979. Trend analysis on intrapartum monitoring dataa A basis for a computerized fetal monitor. Clinical Obstetrics and Gynecology 221665-679. Donahue, c. L., Jr., and T. B. Wan. 1973. Measuring obstetric risks of prematurity1 A preliminary analysis of neonatal death. American Journal of Obstetrics and Gynecology 116a911-915. Pedrick, J. 1976. Antenatal identification of women at high risk of spontaneous pre-term birth. British Journal of Obstetrics and Gynecology 83a351-354. Goodwin, J. w., J. T. Dunne, and B. w. 'l'haaaa. 1969. Antepartum identification of the fetus at risk. canadian Medical Association Journal 101(8)157ff. Bebb, M., I. MacPherson, D. Cudmore, K. Scott, L. Weldon, M. Smart, and E. Ley. 1980. Nova Scotia fetal risk project. canadian Paaily Physician 2611664 ff. Robel, c. J. 1976. Recognition of the high-risk pregnant woman. In Management of the High Risk pregnancy, William H. Spellacy, ed'. Baltimore1 University Park Preas. Robel, c. J. 1979. Assessment of the high risk fetus. Clinics in Obstetrics and Gynecology 61367-377. Robel, c. J., M. A. Byvarinen, D. M. Okada, and w. Oh. 1973. Prenatal and intrapartum high-risk screening: 1. Prediction of the high- risk neonate. American Journal of Obstetrics and Gynecology 117: 1-9. Robel, c. J., L. Youkelea, L., and A. Forsythe. 1979. Prenatal and intrapartum high-risk screening. II. Risk factors reassessed. American Journal of Obstetrics and Gynecology 135:1051, 1056. Larks, s. D., and G. G. Larks. 1968. Prenatal prediction of birth process problema1 Biaaathematical approaches. Mathematical Bioscience& 3al35-139. Morrison, I., and J. Olsen. 1979. Perinatal mortality and antepartum risk scoring. Obstetrics and Gynecology 53:362-366. Nesbitt, R. E. L., and R. B. Aubry. 1969. High-risk obstetrical II.

OCR for page 45
54 Value of semi-objective grading system in identifying the vulnerable group. American Journal of Obstetrics and Gynecology 103:972-985. Rantakallio, P. 1969. Groups at risk in low birthweight infants and perinatal mortality. Acta Pediatrica Scandinavica (supplement No. 193). Richards, I. D. G., and c. J. Roberts. 1967. The •at risk• infant. Lancet 2:711-713. Sokol, R. J., M.G. Rosen, J. Stojkov, and L. Chik. 1977. Clinical application of high-risk scoring on an obstetric service. American Journal of Obstetrics and Gynecology 128:652-661. Stembera, z. K., J. Zezulakova, J. Dittrichova, and K. Znamenavcek. 1975. Identification and quantification of high-risk factors affecting fetus and newborn. In Perinatal Medicine, 4th European Congress of Perinatal Medicine, z. K. Stembera, K. Polacek, and v. Sabata, eds. Acton, Mass.: Publishing Services Group. Wilson, E. w., and B. K. Sill. 1973. Identification of the high risk pregnancy by a scoring system. New Zealand Medical Journal 78: 437-440. Wilson, J. M. G., and G. Junger. 1968. Principles and practice of screening for disease. World Health Organization Public Health Papers No. 34. Winters, s., s. Itzkowitz, and K. Johnson. 1979. Prenatal risk assessment: An evaluation of the Hebel record in a Mount Sinai clinic population. Mount Sinai Journal of Medicine (New York) 46: 424-427.