6

Data Systems and Measurement

Amajor overarching theme of the workshop discussion was the need for more data to help move the birth setting research agenda forward and to help inform decision making and ways that those data could be collected or, in some cases, are already being collected. Much of this discussion occurred during Panel 5. Moderated by Diane Rowley, M.D., M.P.H., University of North Carolina at Chapel Hill, Panel 5 speakers elaborated on specific examples of how birth data are being collected, analyzed, interpreted, and used to inform decision making, and the challenges and limitations of birth setting data. This chapter summarizes their presentations and the discussion that followed. Box 6-1 summarizes key points made by individual speakers.

THE USE OF DATA FOR DECISION MAKING: BIRTH SETTING1

William Barth’s presentation was focused on one professional organization’s use of data for decision making, that is, the American Congress of Obstetricians and Gynecologists’ (ACOG’s) use of birth setting data to articulate its planned home birth committee opinion (ACOG, 2011). He also discussed the limitations of existing datasets and described what he thought constituted the “ideal” dataset. Before Barth spoke, he listed some disclosures: He is an obstetrician whose salary is supported by the number

______________________________________

1This section summarizes information presented by William H. Barth, Jr., M.D., Massachusetts General Hospital, Boston, Massachusetts.



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 93
6 Data Systems and Measurement A major overarching theme of the workshop discussion was the need for more data to help move the birth setting research agenda for- ward and to help inform decision making and ways that those data could be collected or, in some cases, are already being collected. Much of this discussion occurred during Panel 5. Moderated by Diane Rowley, M.D., M.P.H., University of North Carolina at Chapel Hill, Panel 5 speak- ers elaborated on specific examples of how birth data are being collected, analyzed, interpreted, and used to inform decision making, and the chal- lenges and limitations of birth setting data. This chapter summarizes their presentations and the discussion that followed. Box 6-1 summarizes key points made by individual speakers. THE USE OF DATA FOR DECISION MAKING: BIRTH SETTING1 William Barth’s presentation was focused on one professional organi- zation’s use of data for decision making, that is, the American Congress of Obstetricians and Gynecologists’ (ACOG’s) use of birth setting data to articulate its planned home birth committee opinion (ACOG, 2011). He also discussed the limitations of existing datasets and described what he thought constituted the “ideal” dataset. Before Barth spoke, he listed some disclosures: He is an obstetrician whose salary is supported by the number 1  his T section summarizes information presented by William H. Barth, Jr., M.D., Massachu- setts General Hospital, Boston, Massachusetts. 93

OCR for page 93
94 RESEARCH ISSUES IN THE ASSESSMENT OF BIRTH SETTINGS BOX 6-1 Data Systems and Measurement Key Points Made by Individual Speakers •  illiam Barth observed most data informing the birth setting dialogue have W been observational and retrospective, with no randomized controlled trials of sufficient size on home births. •  ow available data are used to inform decision making depends on who is H making the decision, with different people valuing outcomes differently and using the same data in different ways. Barth described how the American Congress of Obstetricians and Gynecologists (ACOG) used data from the controversial Wax et al. (2010) meta-analysis to articulate its planned home birth committee opinion (ACOG, 2011). •  aitlin Cross-Barnet described how Strong Start, a Center for Medicare and C Medicaid Innovation (CMMI) initiative, is collecting data on three different mod- els of prenatal care. While the focus is on preterm births, CMMI is collecting data on a range of other outcomes as well. Many birth setting researchers face challenges such as variation in data availability, state-level variation in Medic- aid coverage, and inconsistent recording of data on U.S. birth certificates. •  lliott Main and William Barth each elaborated on several additional data E and measurement challenges, such as statistical issues resulting from small sample sizes. For example, given that the total number of home births in the United States per year is 27,000, having sufficient statistical power to detect differences in an outcome like perinatal mortality, which is typically 1 to 2 per 1,000, is very difficult. of in-hospital deliveries he attends;2 he is a fellow of ACOG, past Chair of the ACOG Committee on Obstetric Practice and primary author of the planned home birth committee opinion (ACOG, 2011), and medical direc- tor for a large hospital-based midwifery service in Boston. ACOG Use of Data for Decision Making Barth described ACOG’s use of data in its home birth setting committee opinion decision making as a “dance,” one similar to the “dance of legisla- tion” described by Eric Redman in his now classic 1973 book on legislation that led to creation of the National Health Service Corps (Redman, 1973). The ACOG dance started with a policy statement on home births in the United States issued by the ACOG executive board. The statement contained a couple of what Barth called “lightning rods,” namely, “ACOG strongly opposes home births” and “ACOG does not support programs or 2  t A the time of the workshop.

OCR for page 93
DATA SYSTEMS AND MEASUREMENT 95 individuals that advocate for or who provide home births.” The reasoning behind its positioning on home birth was the lack of available data to in- form the issue, according to Barth. That policy statement fueled the 6th edi- tion of the American Academy of Pediatrics (AAP) and ACOG Guidelines for Perinatal Care (AAP and ACOG, 2007), which stated, “The hospital, including a birthing center within a hospital complex, or freestanding birth- ing centers that meet the standards of the [AAAHC, JC, AABC]3 provide the safest setting for labor, delivery, and the postpartum period.” The 6th edition guidelines also stated, “Until such data are available, home births are not encouraged.” Also setting the stage for the dance was a controversial study by Pang and colleagues (2002), a retrospective cohort study conducted in Washing- ton State that relied on birth certificate data as its sole data source. With respect to where births were being delivered when the ACOG dance began, Barth said 24,970 home births were reported in 2006. As- suming that about two-thirds of those home births were planned, about 1 of every 263 births delivered that year was a planned home birth. As more data were collected, Barth said, “the tides began to change.” Several studies appeared showing that neonatal deaths and other newborn outcomes associated with planned home births are no different than those associated with hospital births. These include a retrospective cohort study conducted in Sweden and based on data collected from the Swedish medi- cal birth register and from follow-up phone calls (Lindgren et al., 2008); a retrospective cohort study conducted in the Netherlands and based on three different linked national perinatal databases (de Jonge et al., 2009); a retrospective cohort study conducted in British Columbia and based on provincial perinatal database registry data (Janssen et al., 2009); and a retrospective cohort study conducted in Ontario, Canada, and based on the Ministry of Health midwifery care database (Hutton et al., 2009). In addition to the relatively similar newborn outcomes in planned home birth versus hospital settings, another common theme of these studies was a decreased rate of interventions among planned home births compared to hospital births. Also contributing to the landscape of the ACOG dance were the differ- ent views on home births being advocated by different organizations. Some organizations, like the Royal College of Midwives and the Royal College of Obstetricians and Gynaecologists, supported home births, while others, like the Royal Australian and New Zealand College of Obstetricians and Gynaecologists, did not endorse home births. The process for formulating an ACOG committee opinion is long. It 3  AAHC, A Accreditation Association for Ambulatory Health Care; JC, Joint Commission; AABC, American Association of Birth Centers.

OCR for page 93
96 RESEARCH ISSUES IN THE ASSESSMENT OF BIRTH SETTINGS begins when a subject is proposed. Subjects can be proposed by individual members of the committee or may result from correspondence received by the College from concerned members of the public, public representa- tives or elected officials, or others. When proposed, the committee decides whether the subject is worth pursuing. If so, a primary author is assigned and a professional literature search conducted. Under the leadership of the primary author, a first draft of the committee opinion is drafted within 6 to 12 months. The first draft is discussed by the committee, which is composed of ACOG Fellows and staff, as well as representatives of the AAP, American Academy of Family Physicians, American College of Nurse- Midwives, Eunice Kennedy Shriver National Institute of Child Health and Human Development, Centers for Disease Control and Prevention, and Society for Maternal-Fetal Medicine. Committee comments are assembled by ACOG staff and presented back to the primary author, who revises the draft. The revised draft is reviewed by the committee 6 months later. If it clears the committee, the draft is sent to a clinical document review panel that examines the draft for internal consistency (i.e., consistency with other ACOG policy statements). Once cleared by the clinical document review panel, the draft is sent to the ACOG executive committee. The final draft is published in Obstetrics and Gynecology and a press release issued online. Barth noted that, very importantly, unlike an ACOG policy statement, an ACOG committee opinion has a lifetime. That is, it is regularly reviewed and changed if new data and science warrant a change. Barth described it as a “living document.” It was during this long process that the Wax et al. (2010) meta-analysis was published. Drawing the most attention in Wax et al. (2010) was the twofold increase in neonatal deaths and almost threefold increase in non- anomalous deaths among planned home births compared to hospital births. Barth remarked that even his standing there saying that it is “threefold,” when the actual odds ratio is 2.87 triggers an emotional reaction in many people. “It’s such an emotionally charged subject,” he said. “That’s an understatement.” Barth pointed out that what sometimes “gets lost” in dis- cussions of the meta-analysis are its findings indicating dramatic reductions in interventions among planned home births compared to hospital births. The response was dramatic in what Barth called the “wake of the meta-analysis,” with many letters to editors in various journals, not just in the American Journal of Obstetrics and Gynecology (where the article was published), but also in The Lancet and BMJ, as well as in popular blogs and in many non-peer-reviewed but well-read websites. The editors of the American Journal of Obstetrics and Gynecology published a number of the letters they received. They also took the very unusual step of reconvening a panel of experts in meta-analysis, all of whom were maternal-fetal medicine specialists. The independent review panel derived slightly different results

OCR for page 93
DATA SYSTEMS AND MEASUREMENT 97 but concluded that there was no difference in the direction of the point estimate of the pooled odds ratio or in the overall statistical significance of the results. The panel recommended that the journal publish online full summary graphs for each outcome that was assessed in the study and that there was no need for retraction of the article. Thus, the ACOG committee on obstetric practice went forward with its committee opinion on planned home births and included in its opinion a statement on the twofold to threefold increased risk of neonatal death among planned home births when compared with planned hospital births. Barth emphasized that the committee opinion is the opinion of an orga- nization, not the opinion of an individual, and that the emphasis on the increased risk of neonatal death among planned home births is an organi- zational opinion. Still, it is a “lightning rod.” The new committee opinion, including its remarks on the neonatal death risk associated with planned home births, fed into the seventh edi- tion of the ACOG Guidelines for Perinatal Care (AAP and ACOG, 2012). Barth offered some personal observations on the ACOG Committee Opinion 476: Planned Home Birth (ACOG, 2011). First, he noted the rigor- ous review process. It is much more rigorous than standard peer review, in his opinion. Second, it is written from a U.S. perspective. Data from outside the United States were used cautiously. Third, the opinion was carefully worded to minimize ambiguity and avoid overstatement. Fourth, he reiter- ated that it is an opinion only. Finally, there is great regional variation in health care infrastructure, with driving times to hospitals with maternity centers varying from less than 15 minutes to over an hour. The opinion may not apply in some regions. Barth mentioned that the course of events left him “a little bit whip- sawed.” The words of his friend, Jeffrey Ecker, M.D., calmed him: “No one can force someone to have a hospital birth. . . . No one can force providers to support home birth or interpret data differently than they do.” Data That Have Been Used to Inform the Literature on Birth Setting Data informing the literature on birth setting have been almost exclu- sively observational and mostly retrospective. In any observational study, comparison groups are inevitably different. Some of those differences are known, but others are not. There has been only one randomized controlled trial on home births, and it accrued only 11 patients. Available data include state-reported birth certificate data (i.e., the 2003 U.S. standard certificate of live birth), registry data (e.g., National Birth Center Study data [American Association of Birth Centers], Midwives Alliance’s Statistics Project [MANAstats]), datasets compiled for individual reports, and payer data.

OCR for page 93
98 RESEARCH ISSUES IN THE ASSESSMENT OF BIRTH SETTINGS From Barth’s perspective, features of an ideal dataset, that is, the type of dataset he would want to have in hand if he were to attend a meeting to resolve the issue of home birth, include ascertainment (i.e., intended place of delivery); selection criteria (i.e., appropriateness of candidacy for home birth); type of attendant (i.e., education, certification, licensure); integra- tion of the health system (i.e., whether transport agreements were in place, geography of the health system, and indication for transport); standardized definitions for outcomes; single electronic records per person; mandatory, audited, and enforced reporting of data; and public availability for down- loading and analysis. Currently, 36 states, plus the District of Columbia, Puerto Rico, and Northern Marianas, use the U.S. standard certificate of live birth. In addi- tion, 32 states use the U.S. standard report of fetal death (Personal com- munication, Marion MacDorman, National Center for Health Statistics [NCHS]). ACOG has been pushing for adoption of the U.S. standard certificate of live birth for about 10 years and wrote model legislation in 2009 that was distributed to all states for public comment. The college has been pushing for it in every issue of Guidelines for Perinatal Care. There is reason for optimism, with the National Association for Public Health Statistics and Information Systems and NCHS agreement meaning that all states should be using the certificate by January 2014. Importantly, in Barth’s opinion, there are some things that the 2003 U.S. standard certificate of live birth does not do. It does not capture planned home births transferred to hospitals. So for women whose deliv- eries occur in hospitals, there is no indication whether the delivery was planned as such, nor does it capture reason for transfer or distinguish among different routes to midwifery (i.e., certified professional midwife [CPM] versus licensed midwife versus other). Likewise with the 2003 U.S. standard report of fetal death: there is important information that it does not capture, including planned home births transferred to hospitals, type of midwife provider, and location of intrapartum fetal death (i.e., home or hospital). All of these missing items are “within our range to tweak,” Barth said. MANAstats provides another example of the limitations of avail- able data being used to inform the birth setting dialogue. Enrollment in MANAstats is voluntary, with participation rates among providers at only about 20 to 30 percent for CPMs and 17 percent for certified nurse midwives (CNMs) and certified midwives (CMs). Efforts are under way to encourage or mandate reporting by providers, as are efforts to ensure data quality (through the use of a “data doula”). Individual patients must consent to participate, with fewer than 3 percent declining, yet about 8 percent of participants withdraw from reporting before finishing their reg- istered event. Despite these participation limitations, outcomes based on

OCR for page 93
DATA SYSTEMS AND MEASUREMENT 99 MANAstats data, such as those reported by Johnson and Daviss (2005), are similar to outcomes being reported in other types of studies. Provider participation is also a challenge for birth center data (e.g., Stapleton et al., 2013), with only 41 percent of birth centers being members of the AABC and only 78 percent of AABC members participating in its online registry. Other data sources include various national perinatal data collection efforts (e.g., efforts by the University Health Consortium and National Perinatal Information Center), states’ perinatal reporting beyond birth cer- tificates (e.g., the California Maternal Quality Care Collaborative), payer or system datasets (e.g., Kaiser, Department of Defense), and fledgling efforts by professional organizations such as the Women’s Health Registry Alliance. Use of Data for Decision Making Unfortunately, there have been no randomized controlled trials of suf- ficient size to inform the birth setting dialogue. The only science at our disposal right now is imperfect case series and cohort studies. Available data are limited by ascertainment problems (i.e., ascertainment of intended birth setting); lack of knowledge about provider education, training, certification, and licensure; nonstandard selection criteria; nonuniform definitions of out- comes; and tremendous regional variation in health system infrastructure. Also, ultimately, the data are limited by the lack of a uniform platform for adequately comparing birth settings. For home births, the MANAstats plat- form is probably the leading platform. For birth center births, it is probably the AABC. But for spanning across all birth settings, the 2003 U.S. standard certificate of live birth is the “best shot,” in Barth’s opinion. He encouraged all states to adopt the certificate and encouraged slight modifications to help inform the discussion on birth settings (e.g., address intention, etc.). Mean- while, how data are used for decision making depends on who is making the decision, with use of the same data varying and outcomes being valued differently. Patients, providers, payers, government agencies, and other interested parties each have their own perspective and values. STRONG START: APPROACHES TO DATA COLLECTION AND EVALUATION4 Strong Start, a Center for Medicare and Medicaid Innovation (CMMI) initiative, has two components. The first, Strong Start I, is a nationwide public awareness effort to improve the health of moms and babies by en- 4  his T section summarizes information presented by Caitlin Cross-Barnet, Ph.D., Center for Medicare and Medicaid Innovation, Baltimore, Maryland.

OCR for page 93
100 RESEARCH ISSUES IN THE ASSESSMENT OF BIRTH SETTINGS couraging mothers and practitioners to let labor begin on its own. Strong Start I is campaigning in partnership with the March of Dimes and ACOG to reduce early elective deliveries. According to Caitlin Cross-Barnet, many women are confused by the emphasis on remaining pregnant for 39 weeks. That is, for many women, when 39 weeks hit, they think, “Now I can have my induction.” While the primary goal is to reduce the incidence of early scheduled inductions and other elective deliveries (i.e., Cesareans), especially those that occur before 39 weeks, Strong Start I is pushing the idea that, for pregnancies with no medical indication, labor should begin on its own. Strong Start II The goal of the second program component, Strong Start II, is to reduce the incidence of preterm birth among high-risk Medicaid benefi- ciaries. Merely being on Medicaid is not enough to be considered high risk, even though poverty is a risk factor for preterm birth. The focus of the program is on women at highest risk for preterm birth based on geographic, demographic, physical, and psychosocial risk factors. Specifi- cally, the program uses Institute of Medicine (IOM) criteria for high risk (IOM, 2007). Four different approaches to enhanced prenatal care are being evalu- ated. One of the approaches is being evaluated through the Maternal, In- fant, and Early Childhood Home Visiting program, a Health Resources and Services Administration project with a mandate from the Affordable Care Act to measure home visiting. Strong Start is looking at a component of the Mother and Infant Home Visiting Program Evaluation (MIHOPE). Spe- cifically, MIHOPE-Strong Start (MIHOPE-SS) measures home visiting as- sociated with the Nurse Family Partnership and Healthy Families America programs. Strong Start II provides funds for MIHOPE-SS and consults with the program regularly but is not managing the program. The other three ap- proaches are being evaluated through CMMI. They include (1) care through birth centers, (2) group prenatal care (e.g., CenteringPregnancy™), and (3) maternity care homes (i.e., medical care homes for pregnant women). At the time of award, Strong Start II had a total of 27 awardees serving more than 80,000 women at 182 sites in 32 states. The program serves many geographic regions, both urban and rural, with sites ranging from feder- ally qualified health centers in extremely poor rural areas to sites in the middle of Washington, DC. The level and type of risk for preterm birth varies among and within states, as well as among and within practices. The demographic composition of intervention participants also varies among states and regions. Cross-Barnet described the different types of providers across the three

OCR for page 93
DATA SYSTEMS AND MEASUREMENT 101 different care models. (1) Women delivering in maternity care homes may see a number of different service providers through care coordination. These may include social workers, lactation consultants, nutritionists, ob- stetricians, midwives, and nurses. The primary prenatal care provider may or may not be the one who attends the delivery. (2) Facilitators of centering/ group care programs have varying qualifications. They include obstetri- cians, registered nurses, CNMs, and nurse practitioners. The programs are often facilitated by a medical provider and a more lay-oriented individual, with the facilitators (and the peer group) staying consistent throughout prenatal care. However, the facilitators may or may not attend the actual deliveries. Often a woman meets her delivery practitioner for the first time only when she enters the birth setting. (3) In birth centers, prenatal care providers are usually midwives, and the prenatal care providers usually at- tend the delivery. However, the midwife in attendance may not necessarily be one that a woman has seen much throughout prenatal care. As with providers, birth settings vary among the three different care models. (1) Maternity care homes have no requirements for birth setting, although the birth setting is almost always a hospital. The hospital setting may or may not be affiliated with the care provider setting, so some women may receive care in a maternity care home and then delivery in a facility not directly affiliated with that home. (2) Likewise with centering/group care: there is no requirement for birth setting. But again, it is usually a hospital. And again, the care provided prior to delivery may or may not be provided in a setting affiliated with the actual birth setting. A woman may make her own birth arrangements. (3) All birth center awardees are freestanding birth centers where women are almost always receiving care in the same facility where they give birth and with familiar practitioners. The focus of CMMI is on value-based medicine, that is, medicine that produces better care and better health at lower cost. With respect to lower- ing cost, while preventing preterm births obviously reduces neonatal costs, Strong Start II is also evaluating cost beyond the early postpartum period by following women and their babies for 1 full year. With respect to better health, the program is examining both maternal and infant health, again through the first year of the baby’s life. So even though the initiative is fo- cused on preterm birth (i.e., reducing the incidence of preterm birth), it is also examining longer-term outcomes. Preterm birth is being measured by gestational age and birth weight. Cross-Barnet noted that gestational age can be a “very fuzzy” measure- ment, which is why birth weight is also being measured. Currently, ACOG’s preferred mode of measurement of gestational age is a first-trimester ultra- sound (before 20 weeks). But for women who enter prenatal care later dur- ing their pregnancy, gestational age is often estimated based on the woman’s last menstrual period (LMP). In addition to preterm birth, care costs are

OCR for page 93
102 RESEARCH ISSUES IN THE ASSESSMENT OF BIRTH SETTINGS being evaluated for pregnancy, delivery, and the postpartum period through 60 days (and up to a year if Medicaid eligibility continues). The expecta- tion is that, with the Affordable Care Act, more mothers will be eligible for Medicaid through that first full year of the baby’s life and, thus, more mothers will be followed through Strong Start II for longer. Other outcomes being measured include length of stay for delivery; neo- natal intensive care unit (NICU) admission and length of stay; unplanned maternal ICU admission; frequency of ongoing prenatal care; timing of prenatal care (i.e., when a woman enters care), with differences expected based on the type of care (e.g., centering/group care standards are that a woman enters care before 18 weeks, while the birth center standard of care is 20 weeks); appropriate use of antenatal steroids; whether the delivery is vaginal or Cesarean and, if vaginal, whether it is operative or not (i.e., involves use of forceps, vacuum extraction, etc.); elective deliveries prior to 39 weeks (as well as medically indicated deliveries prior to 39 weeks); ap- propriately timed postpartum care for the mother (e.g., care for postpartum depression, breastfeeding success, future planning); and patient experience of care (i.e., at intake, at the third trimester, and at the postpartum visit). Evaluation of Awardees Variation in data availability and program design complicate the evalu- ation process. For example, there is a large amount of variation in ma- ternity care home measures, with some groups focused on patient care coordination, but others not. Also, care enhancements offered vary (or are similar) both across and within care models. For example, peer counseling might be offered by both birth centers and maternity care homes. The same is true of birth centers and group prenatal care. In order to capture as much of this variation as possible, Strong Start II is using a multipronged evalu- ation approach. Cross-Barnet described approaches under consideration. First, the evaluators may conduct a baseline comparison using a con- temporaneous comparison group and based onsite visits, interviews, and state Medicaid and vital records data. An issue with this approach is that baseline is not necessarily standard obstetrical care (e.g., a woman visits her obstetrician for a 10-minute visit, etc.), with that care suddenly replaced by another type of care (e.g., birth center or centering/group care) when the Strong Start intervention begins. The notion of a standard level of care is complicated by variation in state Medicaid coverage. Cross-Barnet em- phasized that even though Medicaid has a large federal component, it also has a large state component, with states making many individual decisions about what Medicaid will cover. Strong Start pays only for enhanced care provisions not covered by state Medicaid. For example, CPMs are covered by Medicaid in some states, but not in other states. In addition to this

OCR for page 93
DATA SYSTEMS AND MEASUREMENT 103 state-level variation in “standard” of care, while birth centers are already operating as birth centers and while many centering/group care and mater- nity care home sites are already operating as such, some sites that are just starting up have no baseline of any kind. Without a baseline, a baseline comparison cannot be made. In addition to baseline comparisons, another evaluation approach be- ing considered is analysis of state-linked data. State-linked data are vital records data (e.g., birth and death certificate data) that some states link to Medicaid beneficiaries. Strong Start evaluators compare the vital record data to information provided on Medicaid claims. However, there is tre- mendous state-level variation in the type of data being linked. Some, but not all, states link Medicaid records of both mothers and their infants, whereas other states link only portions of records. States link records for varying reasons. Some do it to set insurance rates, others to screen for eligibility (i.e., they link them only when someone requests Medicaid and their birth certificate needs to be verified for eligibility), and still others link data to study a particular issue and only for a certain period of time. Some states have no data links at all. Cross-Barnet mentioned data linking in Washington State as an exemplary model of state-linked data. State-linked data might actually be enough to evaluate Strong Start II sites, if the medical portion of the U.S. certificate of live birth was al- ways filled out completely. However, it is often not complete. Although gestational age and birth weight are usually recorded relatively accurately, many other fields are not consistently recorded. These include risk factors of relevance to preterm births (e.g., having a prior preterm birth), place of birth, and birth attendant. About half of all states do not routinely record place of birth, and 11 states do not routinely record the name and title of the birth attendant. Also, “other midwife” has multiple meanings. For example, CPMs are licensed only in some states. Another challenge with state-linked data is that, again, Medicaid cover- age is largely determined by states and therefore varies among states. The federal government mandates that Medicaid cover all pregnant women up through 133 percent of the poverty line, but states have the option of covering women who have higher incomes than that. Some states cover women with significantly higher incomes, while others do not. Thus, the range or depth of poverty that women on Medicaid are experiencing varies from state to state. Medicaid coverage also varies by immigration status, with some states covering prenatal care for immigrants who do not have legal status while other states do not (although those other states cover the birth, because technically the birth is for the baby). There is no way to know from Medicaid claims if women received prenatal care through non-Medicaid means. Additionally, Medicaid coverage of services varies from state to state, with some states covering centering/group care and

OCR for page 93
104 RESEARCH ISSUES IN THE ASSESSMENT OF BIRTH SETTINGS others not. Similarly, there is significant variation in provider coverage, as Medicaid covers only providers who are licensed in their state and not all types of providers are licensed in all states (e.g., CPMs). Even in states that do license a particular type of provider, the liability insurance requirements might be so astronomical that the providers are unable to practice. Additional challenges to state-linked data include the lag time for state submission of claims data to the Centers for Medicare & Medicaid Services; inaccuracy of claims codes (e.g., in states where vaginal and Cesarean de- liveries receive the same reimbursement, there may not be much attention directed to which code is being used on a claims form); and global billing (i.e., a set fee is paid for all prenatal care and birth services), which makes it difficult to know how many prenatal visits there were and to associate prenatal visits with outcomes. A third approach that is being considered to evaluate the different Strong Start service models is via comparison groups. Some awardees offer Strong Start service models only to some, not all, patients, because of the large number of eligible patients. In those cases, people who are eligible but do not receive services can be compared with people who are eligible and who do receive services. Or, some communities may be large enough that they have other sites that are serving high-risk Medicaid women but not through Strong Start; those sites could be compared to Strong Start sites. Regardless of the approach used to evaluate the different Strong Start service models, care model bias poses a challenge. That is, do women en- roll in certain Strong Start sites because they are seeking a particular type of care? For example, is there a particular type of woman that chooses a birth center as opposed to a group care facility? If so, does seeking a par- ticular type of care compromise valid comparisons among care models? Can nontraditional care serve women with the same risk profiles as those in traditional care? For example, is it acceptable to care for a woman with preeclampsia, gestational diabetes, and a prior preterm birth at a birth center as opposed to a maternity care home? Are the same risk profiles distributed equally among the three interventions? Finally, Cross-Barnet asked, is seeking traditional care a bias? Many people think of seeking birth center care as being a bias. Conversely, she said there are plenty of women who would never give birth outside a hospital. It is unclear how much “true choice” really exists. People seek particular types of care for multiple reasons, including insurance coverage, availability, transportation or child care concerns, and other issues. In addition to the combination of evaluation strategies being used to evaluate sites, Strong Start II is considering how to use state data judi- ciously, given its limitations; relying on standardized measurement tools (e.g., tools that are consistent across all sites); and conducting considerable qualitative inquiry into the patient and caregiver experience.

OCR for page 93
DATA SYSTEMS AND MEASUREMENT 105 DATA SYSTEMS AND MEASUREMENT: FORMAL DISCUSSION5 At the end of the Session 5 panel, Elliott Main was invited to reflect on data systems and measurement issues. Main commented on the “big data” handled by the California Maternal Quality Care Collaborative, with Cali- fornia home to more than 500,000 births annually, some years as many as 550,000 births, and with the collaborative responsible for evaluating the quality of maternity care in more than 280 hospitals and other settings. He and his colleagues deal with both administrative data and merged clini- cal data­ ets. In addition to his work with the collaborative, Main directly s supervises quality for 20 Sutter Health birthing facilities, including 2 for which the majority of births are delivered by midwives. He also provides outpatient consultations for several hundred northern California maternity providers, including midwives at freestanding birthing facilities. He stated that he had no financial disclosures. Challenges to Evaluating Birth Setting Data Main discussed several challenges to evaluating birth setting data that earlier presenters had mentioned: limitations of vital records, denominator and numerator size issues, power limitations, comparison issues, and iden- tification of high risk factors. Limitations of Vital Records The U.S. standard certificate of live birth is limited by its lack of infor- mation on intended place of birth. Without that information, it does not capture planned births transferred to hospitals. This is a critical issue as the “transferred” group has a much higher risk of serious morbidity, at least based on what Main and colleagues have observed in northern California. Not only does the birth certificate not capture transfer from home, on- screen instructions for filling it out indicate that information being collected on transfers is for intrafacility transfers (e.g., hospital to hospital, birthing facility to hospital) and not for transfers from home to hospital. In a sense, Main opined, it is a difficult question to ask because mothers come to a hospital from home regardless of whether they intended to deliver at home or not. In Main’s opinion, even the suggested revised U.S. birth certificate has a number of potentials for error and will take several years for new or added fields to be accurately completed on a widespread basis. “Minor” 5  his T section summarizes information presented by Elliott Main, M.D., California Maternal Quality Care Collaborative, Stanford, California.

OCR for page 93
106 RESEARCH ISSUES IN THE ASSESSMENT OF BIRTH SETTINGS fields on the birth certificate are least likely to be completely accurate, with many birth certificate clerks not very adept at asking or completing those questions. Generally, there has been little attention directed toward birth certificate quality in the United States. Main noted that a large birth certifi- cate data quality project is being started in California. Denominator and Numerator Issues With respect to denominator issues, many outcomes are reported in small numbers. For example, perinatal mortality is typically 1 to 2 per 1,000. To identify a difference between 1 per 1,000 and 2 per 1,000, the recommended sample size is 23,500 per arm. Given that the total number of home births in the United States is 27,000 per year, having sufficient statistical power to detect differences is very difficult. The same small numbers (e.g., 1 to 2 neonatal deaths per 1,000) create numerator issues as well. For example, there would be only 10 to 20 cases in a sample size of 10,000, with misattribution or nonreporting of even a few cases leading to significant differences in calculated rates. Ensuring a highly accurate numerator requires ongoing extensive focus on the data elements, which in turn requires extensive money and time. Even then, accuracy is not guaranteed. Based on a number of published studies that have analyzed the accuracy of U.S. birth certificate data, some fields of data are highly accurate: birth date and time, birth weight, parity, plurality, maternal demographics (e.g., race and ethnicity), and method of delivery. Other fields of data are acceptable, but not perfect, for example clinical estimate of gestational age (i.e., not the LMP estimate). Unfortunately, in Main’s opinion, many of the fields of data used for risk adjustment are those that are known to be poorly collected and represented: pregnancy complications, labor and delivery complications, neonatal complications, and NICU admission. There is a place on the birth certificate for informa- tion about neonatal complications and NICU admission to be recorded, but most of that information is not known when the certificate is filled out (usually within hours of birth). Recording pregnancy or labor and delivery complications is sometimes beyond what a birth certificate clerk is best at doing. However, when information on these various poorly collected fields is recorded, it is usually accurate. The poor collection of these data makes it difficult to risk-adjust for medical factors based solely on birth certificate data. Main used a study by Snowden et al. (2013) comparing three different ways of analyzing hospital versus planned home birth data from Oregon to illustrate denominator and numerator data limitation issues. The research- ers compared 3 years of data, from 2008 to 2010. One of the comparisons was between planned home birth data and “typical” hospital birth data,

OCR for page 93
DATA SYSTEMS AND MEASUREMENT 107 which included all near-term births; the second comparison was between planned home birth data and hospital data excluding facility transfers (with the intention of excluding intended home births, although they likely did not); and the third comparison was between planned home birth data and hospital birth data, excluding births that did not meet Oregon eligibility cri- teria for home births. The state of Oregon’s eligibility criteria are based on a series of exclusionary factors (e.g., under 35 weeks, no preeclampsia, etc.). The Snowden et al. (2013) analysis involved only 2,736 home births, with 7 neonatal deaths, which amounts to a 0.26 percent neonatal death rate, similar to that for total hospital births. However, when compared to hospital births that met the Oregon criteria for home births, the actual home birth neonatal death rate was three times higher than the hospital rate. A different numerator, even just one or two fewer or more deaths, could have changed the rate significantly. Another approach would be to increase the denominator, and many have recommended combining data from multiple states (or even national data), but that can be misleading because of variation in home birth attendant certifications, guidelines, and cultures and traditions. But, as Snowden et al. (2013) illustrate, single-state data collected even over several years, in a state with one of the highest rates of home births, still provides an inadequate denominator (even several years of data collection yielded a sample size smaller than 3,000) and un- stable numerators (again, just a couple fewer or more deaths would change percentages significantly). Snowden et al.’s (2013) study also serves as a good example of the limitations of birth certificate coding. It is unclear whether home births, hospital births, or both were undercoded for maternal complications. Like- wise, it is unclear whether birth certificate data really capture intended home births that end up in the hospital. Although Oregon changed its birth certificate question about intended home birth in 2012, it remains to be seen how that change is going to roll out and how the question will be completed in the hospital. Birth certificates can provide large denominator numbers; all births in the United States are recorded, although more robust sources of data, like linked datasets that contain information on medical conditions, are needed for risk-adjustment denominators. Main expressed uncertainty about how to manage the numerator issue. “I don’t have a good answer for that,” he said. Comparison Issues With respect to comparison issues, Main asked, “If we cannot random- ize, how can we make the groups comparable?” The question is relevant to all birth settings. There needs to be some control for medical and de-

OCR for page 93
108 RESEARCH ISSUES IN THE ASSESSMENT OF BIRTH SETTINGS mographic factors, as well as control for commitment to the program (i.e., commitment to home versus hospital versus birth center) and commitment to one’s end goal (e.g., value of vaginal birth). Identifying High Risk Main described the IOM list of risk factors, upon which Strong Start is based, as a “grab bag” (IOM, 2007). According to Main, the list includes everything that has ever been reported as being associated with preterm birth, but most of these do not put the fetus at risk for late stillbirth or neonatal mortality. Some of the factors listed are higher risk than others, for example having had a prior preterm birth and race/ethnicity; likewise with multiple gestations, which in Main’s opinion should be considered separately and not even included in the risk analysis (see last paragraph on this page). With respect to placental abnormalities, some are high risk, others not, according to Main. With respect to the use of marijuana and other illicit drugs, the drug of concern with respect to preterm birth is methamphetamine, not marijuana. In sum, some risk factors are more important than others. Also, for some risk factors, the issue is one of gradations, not a dichoto- mous yes or no. For example, transient hypertension poses a different risk than a history of hypertension, as does mild versus severe hypertension (i.e., women with mild hypertension not being medicated versus women with severe hypertension taking one or two medications). Obesity is another factor that needs to be considered in terms of gradations. Obesity is defined as a body mass index (BMI) greater than 30. Half of U.S. women have a BMI of at least 30, but few have a BMI greater than 50, which Main said is probably where the risk is. The same problem exists with anemia, which can range from a hematocrit of 20 to 34 percent, with very different risk profiles for women at different points along that spectrum. Likewise with maternal age, the risk associated with a maternal age of 30 to 35 years is very different than the risk associated with a maternal age of 45. In sum, Main said, “The plea here is that risk adjustment needs a lot of work.” He suggested simplifying it through the use of fewer factors, and simultane- ously complicating it by considering gradations and interactions. The dominant factor for successful labor outcome in any risk adjust- ment is parity. This is true across most typically analyzed labor outcomes (e.g., Cesarean birth rates, labor length, labor pain, physiological birth rates, and successful birthing center or home births). Nulliparous women have much higher rates of all adverse outcomes, regardless of birth set- ting. It is much harder to find increased risks among low-risk multiparous women. Again, this is true across settings.

OCR for page 93
DATA SYSTEMS AND MEASUREMENT 109 Limitations of Data Sources Main identified two required data sources: vital records and patient dis- charge diagnosis (PDD) datasets. Challenges with vital records data include continuing issues with attribution and accuracy. Also, vital records are not a good source for data on comorbidities and complications. PDD datasets, on the other hand, are actually a pretty good source for data on comorbidities and complications and are easily linkable to vital records. Main remarked that California routinely links PDD datasets and vital records. However, while PDDs are submitted by every hospital to a central state agency, they are not collected for home births or freestanding birth center births. Voluntary data sources include registries and research datasets. Reg- istries are not universal and are nonstandard. Plus, their voluntary nature raises questions about missing cases. Main applauded those who are con- ducting quality assessments of registry datasets, which he said “has to be done.” The challenge with research datasets is that they are expensive. Because of the expense and time involved, it is difficult to collect sufficiently large numbers (for the denominator). Understanding Small Risks Main explained that many patients or families have a difficult time understanding small risks, and, importantly, different people interpret them differently. For example, in prenatal diagnosis, where this has been studied extensively, some families are unwilling to take a 1-in-10,000 risk for a baby with Down syndrome, while others are very happy taking a 1-in-150 or even 1-in-50 risk. These varying assessments of risk affect how risks among the different birth settings are interpreted. There is no objective or external standard. It is a personal choice. Related to the issue of varying risk perceptions, Main offered what he described as his “editorial” on what is driving the increase in home births in the United States. Based on his talks with women in northern California, he thinks the increase in home births is being driven by a fear of overmedical- ization of birth with too many interventions. Main said Cesarean delivery rates have increased by 50 percent in the past decade and vaginal birth after Cesarean (VBAC) rates have markedly decreased. He mentioned the “near disappearance of VBACs in many hospitals,” saying, “women have fewer choices in hospitals, and so they are looking for alternatives.” Variation in Outcomes Among Hospitals With respect to the 98 percent of births occurring in hospitals, Main emphasized that not all hospitals are the same. For example, California

OCR for page 93
110 RESEARCH ISSUES IN THE ASSESSMENT OF BIRTH SETTINGS hospitals show significant geographic variation in median hospital Cesarean rates (both nulliparous term singleton vertex Cesarean [NTSV CS] rates and total term Cesarean rates). The median NTSV CS rate in his region (in northern California) is down around 21 percent, but the median rate in Los Angeles is over 30 percent. The national target for NTSV CS rates is 23.9 percent. As another example, the California Maternal Data Center is a statewide data center that links birth certificate data provided by the state (every 45 days) and hospital-supplied PDD or International Classifica- tion of Diseases, 9th revision (ICD-9) codes. The data are linked “on the fly,” with 99.8 percent completion, and used to calculate a series of data quality measures (e.g., missing or inconsistent delivery method) and clinical quality measures (e.g., elective delivery under 39 weeks). Facilities can use the results to compare themselves to the state, a region, or other hospitals. NTSV CS rates range from 15 percent in some hospitals to 40 percent or more in others. Main observed that, just as there has been much discussion (during the workshop) about not lumping midwives together (in analyses), hospitals should not be lumped either. Main concluded by asking what he said was a rhetorical question: Given the variation that exists, is there an opportunity to pay based on outcomes? Is maternity care an opportunity for value-based purchasing? DISCUSSION WITH THE AUDIENCE6 Workshop attendees addressed several issues during the discussion with Session 5 panelists, including how patients perceive risk and how providers discuss risk with their patients, numerator and denominator issues, issues related to the lack of data on intended place of birth, other miscellaneous data issues, language used to discuss birth setting research, and a woman’s choice of birth setting. Patient Perspective of Risk and Provider-Patient Risk Communication An audience member remarked that much of the workshop discussion on risk was “highly categorical,” but that risk is a continuous variable for patients. He said that patients are not evaluating whether they are at low or high risk. Rather, they are evaluating the likelihood that certain things will happen to them. That is, patients are evaluating “whether [they] are a numerator.” They are also evaluating the impact of those events on their families. 6  his T section summarizes the panel discussion with the audience that occurred at the end of Session 5.

OCR for page 93
DATA SYSTEMS AND MEASUREMENT 111 There was a question about how data are being discussed in provider- patient relationships. William Barth replied that the issue is, at least partly, a numeracy issue. As a provider, he counsels patients very differently based on what he perceives as their appreciation of numeracy. For example, he would counsel a software engineer differently than he would counsel a patient from Somalia who has been in the United States for only 4 months and is frightened during the visit. For many people, describing the risk of neonatal death as less than 1 percent is not alarming; but describing it as threefold higher at home than in a hospital dramatically changes the con- text of the conversation. As a provider, he has to not only read the patient without necessarily knowing anything about that patient, but also check his own personal biases. He remarked that, although nondirective counseling is “the big issue today,” it is hard not to have an opinion about a risk and to not advise a person based on that opinion. Numerator and Denominator Issues The discussion of risk led into some further discussion of the Wax et al. (2010) study, concerns about which had been addressed in an earlier question-and-answer period. Marian MacDorman commented on the focus on neonatal mortality risks reported in that study. In MacDorman’s opin- ion, perinatal mortality risk is a better measure of risk. Measures of perina- tal mortality risk combine both late fetal deaths and early infant (neonatal) deaths. In contrast, neonatal mortality only measures the risk of death from live birth through 27 days of age. The denominator of the perinatal mortal- ity risk estimate reported by Wax et al. (2010) was on the order of 331,000, compared to only 12,000 for the neonatal mortality risk. She described the neonatal mortality risk estimation as a “very underpowered analysis.” That study reported no increase in perinatal mortality among home births. MacDorman said, “So the whole [controversy] about neonatal mortality was sort of misguided.” She also commented on the fact that, while some studies included in the Wax et al. (2010) meta-analysis reported a slightly higher relative risk in neonatal mortality among planned home births, none reported high absolute risks. She expressed concern that the notion of ab- solute risk is “underutilized.” William Barth explained that the difference in the denominators resulted from a decision to include neonatal mortality data only for those studies that extended out to 28 days. He emphasized that the authors of the ACOG committee opinion on planned home births tried to be very careful with their words by differentiating between relative and absolute risk. That is, the opinion reads along the lines of, “Although the absolute risks are low, there may be an increase in the risk of neonatal death” Barth quoted a colleague (Michael F. Greene, M.D.), who said, “Risk, like beauty, is in the eye of the beholder.” He explained that dif-

OCR for page 93
112 RESEARCH ISSUES IN THE ASSESSMENT OF BIRTH SETTINGS ferent people will value different outcomes and risks differently and “our charge is to convey what we do know based on the imperfect information that’s out there.” Lack of Data on Intended Place of Birth An audience member lauded efforts to add intended place of birth to all birth certificates. However, if the goal of doing so is to better understand the factors involved in out-of-hospital births that need to be moved into the medical system, it is not enough to know the intended place of birth. One also needs to know whether a care provider was involved in the transfer decision making. According to the commenter, there has been an increase in unassisted out-of-hospital births. Some women choose that. Others do not have any licensed provider in their area to help with a home birth. It is important to know whether it was a friend, neighbor, or qualified provider who made the risk assessment that led to the transfer of care. She urged those who are making efforts to add intended place of birth to all birth certificates to also consider adding that information as well. MacDorman responded that, while there may be some minor tweaking with the U.S. certificate of live birth in another 5 years or so to improve data on specific items, there is no plan in place for a global revision. She suggested writing a letter to the NCHS. However, she cautioned, “It’s pretty hard to change the birth certificate.” She described Oregon’s question on their birth certificate about whether hospital births were planned to be hospital births or not as a “big improvement.” Other Miscellaneous Data Issues There were a couple of remarks made on various other miscellaneous data issues. First, Cross-Barnet clarified that Strong Start collects data only for birth centers, maternity care homes, and centering/group care. The pro- gram does not collect data for home births, even though Medicaid pays for home births in some states. Second, the issue of voluntary versus mandatory data reporting was brought up. The North American Registry of Midwives is currently taking steps with the Midwives Alliance of North America to combine efforts toward mandatory data collection from all CPMs in 2015. Language Used to Discuss Birth Setting Research An audience member observed that much of the workshop discussion seemed to pit the promotion of women’s voices and evidence-based health care against each other. She said, “Those two things are not mutually ex- clusive or even contradictory.” She also observed that referring to hospital

OCR for page 93
DATA SYSTEMS AND MEASUREMENT 113 births as “traditional” and home births as “nontraditional” does not help the discussion. Up until very recently, hospital births were not a traditional way to give birth. Nor does using the word “versus,” as in “hospital births versus home births” help the discussion. Another workshop participant suggested that “standard” be used instead of “traditional.” A Woman’s Choice of Birth Setting A comment was made in response to one of William Barth’s quotes. Specifically, he had quoted his friend Jeffrey Ecker, M.D.: “No one can force someone to have a hospital birth.” The commenter said, “Someone can and someone has.” She referred to the state, often acting at the behest of obstetricians, with women who prefer to give birth at home being forced to deliver in hospitals. She pointed to the Laura Pemberton case in Florida as an example. The commenter asserted that women have also been forced to have Cesarean deliveries that they did not consent to. Barth clarified that he was quoting his friend. He said, “What we should say is, ‘no one should be forced to have a hospital birth.’”

OCR for page 93