4

Imputation and Estimation

This chapter begins with the questions that panelists were asked to address on imputation and estimation provided in advance to workshop participants in Kwan et al. (2013, p. 8). Unlike previous chapters, no additional background information was provided. The second section of the chapter summarizes the discussion among panel members on specific issues related to the questions, followed by open discussion with the audience.

QUESTIONS ON IMPUTATION AND ESTIMATION

Given the study design proposal1 above, and using the example cohort proportions proposed in the Session 2 questions, what enhancements can be made to address the following estimation and imputation challenges:

1. How can the data from the two cohorts be combined to increase the effective sample size?

a. What should the parameters for the sampling procedure, for example, using the same PSUs, be in order to enhance data combination?

b. What sampling protocol deviances could impact the ability to combine data?

c. What considerations (if any) for sample weights need to be taken into account in the sample design? Specifically when certain groups

_________

1The study design proposal was described at the beginning of Chapter 3 and was presented in Kwan et al. (2013).



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 45
4 Imputation and Estimation T his chapter begins with the questions that panelists were asked to address on imputation and estimation provided in advance to workshop participants in Kwan et al. (2013, p. 8). Unlike previ- ous chapters, no additional background information was provided. The second section of the chapter summarizes the discussion among panel members on specific issues related to the questions, followed by open discussion with the audience. QUESTIONS ON IMPUTATION AND ESTIMATION Given the study design proposal1 above, and using the example cohort proportions proposed in the Session 2 questions, what enhancements can be made to address the following estimation and imputation challenges: 1. How can the data from the two cohorts be combined to increase the effective sample size? a. What should the parameters for the sampling procedure, for exam- ple, using the same PSUs, be in order to enhance data combination? b. What sampling protocol deviances could impact the ability to com- bine data? c. What considerations (if any) for sample weights need to be taken into account in the sample design? Specifically when certain groups 1The study design proposal was described at the beginning of Chapter 3 and was pre- sented in Kwan et al. (2013). 45

OCR for page 45
46 DESIGN OF THE NATIONAL CHILDREN’S STUDY may be oversampled in one cohort (such as women receiving no pre- natal care who would only be present in the birth cohort), should any special considerations be made for the sampling probability in order to construct appropriate weights? 2. How can data imputation be used effectively, particularly for prenatal exposure? a. What threshold level of imputation of prenatal exposure data is acceptable? b. What should the proposed trigger for the more expensive com- prehensive sampling look like—should this be a random sampling, event-based trigger, or a validation subset or some combination? c. How should the sample be recalibrated in the future to account for attrition? KEY POINTS OF THE DISCUSSION Steven Cohen (Agency for Healthcare Research and Quality) moder- ated this session, and the panelists were Graham Kalton (Westat, Inc.), Colm O’Muircheartaigh (Harris School of Public Policy and NORC at the University of Chicago), and Richard Valliant (Joint Program in Survey Methodology at the University of Maryland and University of Michi- gan, Ann Arbor). The panel discussion below is organized by the topics covered during the session, which included the desirability of a unified design, allocation, weighting, missing data, and the special population sample. Unified Design In responding to the questions above on combining data from dif- ferent cohorts and imputation, Kalton, O’Muircheartaigh, and Valliant agreed that a unified design with a clearly defined population of infer- ence, as proposed by Kalton, has many advantages over an approach with separate prenatal and birth cohorts. In the unified design outlined by Kalton, the population of inference is defined as all births in a specified enrollment period. In that approach, a sample of women who would give birth in the enrollment period would be selected from a list of prenatal care providers that includes hospitals and birthing centers. Women who do not receive prenatal care, or receive it from a provider that is not on the list, would be sampled at the hospital or birthing center (at the birth). The integrated design has significant analytic advantages over a design with separate prenatal and birth cohorts, as well as leading to important simplifications in weighting, point estimation, and variance estimation. Some of the details and complications of implementing a unified design are described below.

OCR for page 45
IMPUTATION AND ESTIMATION 47 O’Muircheartaigh noted the first task in a sampling problem is to think about what one is trying to represent in a study and what makes the study different from a convenience study. The special characteristic of the National Children’s Study is that it could allow for a population-based inference that is substantively much deeper and more intricate than most studies. Kalton proposed defining the target NCS population as all births in the United States during a given enrollment period. The enrollment period may be two years (the shorter the better, he noted), but it should be multiples of years to cover seasonality. With a two-year enrollment period, the sample should be representative of all the births during that two-year period. This implies that to collect prenatal data, some pregnant women would have to be recruited and enrolled prior to the beginning of that period, and births associated with these women would be included in the sample only if the births occurred during the two-year period. The same issue arises at the end of the period—pregnant women are included in the sample only if the birth will occur before the end of the two-year period. He noted births may also be picked up at hospitals and birthing centers, but all would be births that occur during the two-year period. This design has the advantages that it is a single integrated design with a clearly defined temporal definition of the population of inference, and benchmark data from birth certificates can be used to support assessment and adjustment. A disadvantage of this approach is the operational com- plexity introduced by the specified time frame. Kalton then described two potential variations. The first is the provider-based sampling approach he described earlier (see Chapter 3) that is now in the field at the Vanguard sites. In this design, he said, a sample is selected from a frame of prenatal care providers (including hospitals), with hospitals the provider of last resort. With this approach, women who receive prenatal care but deliver at home are included in the survey population, which provides additional coverage for the use of a hospital frame alone (although this is not of great significance since about 98 percent of births occur in hospitals). The downside to this design is that a list of prenatal care providers has to be compiled in the sampled areas, with the sample of providers then being drawn from that list. Another downside is that the biospecimen data collection at birth may be spread across a large number of hospitals. This is an important practical issue. He said an alternative approach selects a sample of hospitals at the first stage of sample selection. For each selected hospital, the prenatal providers associated with that hospital are identified and a sample of these providers is selected. This approach concentrates the data collec- tion at birth in just the sampled hospitals. Two issues to be addressed are that some prenatal care providers are not linked to just one hospital, and some may not have any hospital linkage (depending on how “linkage”

OCR for page 45
48 DESIGN OF THE NATIONAL CHILDREN’S STUDY is defined and operationalized). Both of these approaches can be viewed in the unified approach framework once the target population of interest has been determined. Issues of practical implementation in the recruit- ment of prenatal providers and hospitals and in the enrollment of women are important factors to consider in making comparisons between them. O’Muircheartaigh further explained the unified approach, suggest- ing that it be viewed as covering all parts of the population and that it is possible to cover a fairly large part of the population through a provider- based sample. However, some providers may not be included on the frame, and some mothers do not seek prenatal care. He acknowledged some non-response by providers, but the remainder would be captured through sampled providers. The rest of the population that has not been covered is due to failures of non-coverage and non-response. He said this would suggest supplementing the sample with coverage for the cases that are missed, and birthing centers or hospitals are the right places to go. If the hospital is considered as a part of the sample design rather than as a separate venture unrelated to the prenatal care providers, then there is a unified, stratified approach to the sample design, and no problem in accumulating data across the two. O’Muircheartaigh stated his default option would be to have equal probabilities of selection for each birth in the defined inferential popula- tion. This design takes advantage of structured hierarchies, and the size of the strata (cohorts) would be determined by the empirical reality of data collection. In regions with many births to mothers who do not receive prenatal care, a large number of providers who refuse, or many pro­ iders v are missing from the frame, the hospital/birthing center sample would be larger because there would be more eligible births at the point of delivery. He said it is not necessary to decide now how big the cohorts/strata are. Rather, they will define themselves because they are strata in the popula- tion rather than a predetermination about relative sizes. He said he does not view a conflict between the idea of cohorts and the idea of a unified design. Instead, he said it is viewing the cohorts as non- overlapping strata rather than as possibly overlapping units with joint probabilities that is difficult to estimate. In the single design approach, each birth will be classified into one stratum. When patients arrive at a hospital from a prenatal care provider who refused to participate, they would be eligible for selection in the hospital. Births to a woman who did not receive prenatal care would also be eligible for selection in the hospital. The combination provides essentially a probability sample of all births. O’Muircheartaigh said the big problem with the cohorts is not with the concept of different ways of collecting data. Instead, if done in an unorganized way, without advanced consideration of how to put the

OCR for page 45
IMPUTATION AND ESTIMATION 49 pieces together, there will not be a good analytical product at the end of the process and specialized methods would be needed to combine cohorts for estimating descriptive statistics, such as variances and fitting statisti- cal models. In contrast, with a single unified sample design, there is no problem with dual estimation, multiple frame estimation, or figuring out joint probabilities between two cohorts because it really is only one design with multiple components. In a unified design, variance estimation becomes straightforward. Valliant said the idea of the hospital being the last-resort selection is important in the sample selection in this unified design because sampling with probably proportional to size (number of births) would tend to select really big hospitals with high probability. But if the hospital is the last resort for picking up women who did not receive prenatal care, an adjusted measure of size will reflect the number of births per year to women who did not get prenatal care. He termed this a sticky technical detail, but said it also avoids the issue brought up in an earlier session (see Chapter 3) about potential refusals by large hospitals and the loss to the sample. Kalton agreed that determining a measure of size for the PBS selection of hospitals is not straightforward. It is not the total number of births in that hospital, but rather the total number of births that would not have had a chance of selection from a listed prenatal care provider. Women may not have had a chance of selection from a prenatal care provider because the woman had no prenatal care or because the provider was not on the provider list frame. The measure of size is difficult to estimate but to con- trol the sample size selected from the hospital, it is important to estimate it as well as possible. He said another alternative for the sample plan would be to develop a sampling frame of hospitals based on a list from the American Hospital Association of 6,000 hospitals and the number of births in nearly all of them. It would be straightforward to sample from that list (with some likely need for clustering for undersized hospitals that would not support the required sample size). The sample would still be a clustered sample design, with the hospital as the cluster as distinct from the geographical area. Allocation Valliant noted in terms of optimal design, even a unified design approach has allocation issues, including questions about the number of geographic primary sampling units (PSUs), the number of providers per PSU, and the sample size in each provider. The usual solution is to estimate variance components associated with each of these steps, which would require identifying one or more important statistics. They could be descriptive statistics: How many women had underweight babies? How

OCR for page 45
50 DESIGN OF THE NATIONAL CHILDREN’S STUDY many women were exposed prenatally to something? What is the relative risk of a certain condition? He explained if the estimator is complicated it could be linearized and written in such a way that variance components for PSUs, providers, and women are calculated. He said it is likely that there are insufficient data directly related to the variables that the NCS is going to collect, and it will be necessary to piece together available information to make somewhat informed decisions about allocations. He pointed to the somewhat related datasets from the Vanguard sites, with about 4,000 or 5,000 births, and that the NHANES data are health-related with many physical measurements. He also noted the American Hospital Association publishes hospital data. Valliant reminded the audience that the problem in deciding how many providers to sample per PSU boils down to thinking about how much alike the women are with a particular provider. He suggested another way to think about it is how much the providers differ in size. The way the math works out, the variance between providers depends on how many women they serve. Hospitals can serve hundreds or thousands of women in a year’s time, while individual doctors’ offices are much smaller. This built-in disparity in size will push toward sampling more providers rather than more women per provider, and there are cost impli- cations to going to many more providers. Probability-proportional-to-size sampling of providers is efficient if the measure of size is the number of women served. However, selecting an efficient sample is complicated by the fact that the counts of women served by each provider may be inaccurate. He said with enough data, it would be possible to follow Duan’s advice (see Chapter 3), resulting in a mathematical programming problem. The general idea is to determine an allocation of the sample to optimize an objective function subject to a set of constraints. With a set of statistics of interest, variances can be weighted according to their impor- tance to the survey to form an objective function to be minimized. A fixed budget may be a constraint, and other additional constraints may also be needed, such as a minimum number of providers and women per PSU. To do this allocation properly requires a lot of data. Weighting Kalton noted that with the design he proposed, the study has bench- mark data from birth certificates that can be used to adjust the sam- pling weights to account for some births that are missing due to non-­ participation or non-response. As in other standard panel survey designs, weighting adjustments are typically used to account for attrition (chil- dren who drop out of the study and cannot be followed) as the study moves forward. An initial weighting adjustment based on vital records is

OCR for page 45
IMPUTATION AND ESTIMATION 51 intended to make sure that at the outset, the sample is representative of the defined population of inference. Kalton noted the sample design could include oversampling of cer- tain groups. Oversampling is usually done for groups that may be dif- ferent in some way, such as low socioeconomic status, race, or ethnicity group. Valliant observed the fact that groups are oversampled is prima facie evidence that the survey manager thinks that they are different, which leads to weights that are different for the oversampled groups. He said the question about which vital records to use to create calibrated weights may be a modeling problem. There are at least two reasons to use calibrated estimators. First, if the study undercovers or mis-covers different parts of the population—for example, if it is known that too few lower socioeconomic status women are being recruited, which he said is typical in U.S. household surveys—calibrating can rebalance the sample to make sure it better reflects the population. He noted calibration only works if the women in the sample are a good representation of the non- sample women. However, if there is a skewed representation of lower socioeconomic women, then calibrating will not help. The second reason he gave to calibrate is to reduce variances. This process requires covariates that are related to coverage and to the key variables being estimated. Birth certificates contain a lot of information: birth weight, APGAR score, whether the infant required assisted venti- lation, and whether he or she was admitted to the intensive care unit. Many other potential variables, in addition to the mother’s characteristics, could be tabulated and used for calibration or control totals. He called all of these variables fair game for a research project to determine the most appropriate control totals. Missing Data Kalton noted the problem of missing prenatal data for some births is similar to the attrition problem, except looking at time in the reverse direction. He suggested the data can be considered to be geared around the birth with incomplete responses in both time directions, and missing data might occur before birth (the prenatal data) or after (attrition). While complex to analyze, he said, conceptually, it provides a framework in which to think about approaches to the problem of missing data. When the study is viewed as two separate cohorts, one can perform analysis on each cohort separately, but it is very unclear how the cohorts can be combined. The unified approach provides a framework that supports joint analysis. O’Muircheartaigh noted the issue of replenishment is difficult. He agreed with Kalton that the strength of the NCS as a longitudinal study

OCR for page 45
52 DESIGN OF THE NATIONAL CHILDREN’S STUDY is that people are measured very early on and over time. A replenish- ment using Kalton’s approach could be conceptualized, making infer- ences backward from a sample boosted by replenishment. It could be thought of as a parallel cohort, for example, as a sample of adolescents added to the sample. They would be followed forward, and some of their characteristics would be tracked in relation to the earlier panel. This might be a possibility in 10 or 15 years if a scientific question about this age group becomes apparent and the original NCS panel has become too small, although this approach would not provide any birth or early child- hood data for the new sample. Valliant noted that even in the unified design, some women will be recruited only at the hospital. They will not have prenatal covariates except to the extent that they can be collected by consulting medical records or auxiliary measurements (e.g., dust, EPA databases). One option would be to use the sampled women for whom prenatal covariates, expo- sures of different kinds, are available and use them as donors to impute for women who are missing the prenatal covariates. Multiple imputation is one approach. A valid imputation model would be as correct as possible and could be evaluated, possibly using simulation.2 A statistic that could be used to assess the impact of imputation is the fraction of missing data. If there are too much missing data, he said imputation may do more harm than good. This fraction of miss- ing information has a “between” and a “within” component for a mul- tiple imputation variant and is a measure of the variability being injected by imputation. One question is whether imputation represents too large a p ­ roportion of the total variance. This, too, could be measured in a simula- tion study. He said if it were possible to put together a pseudo-population based on the Vanguard data or NHANES and then divide that popula- tion into women with and without the prenatal covariates, for example, a simulation might inform the study about the impact of missing data. Valliant noted the University of Michigan conducts longitudinal sur- veys, including the Health and Retirement Study and the Panel Survey of Income Dynamics (PSID), with a number of cohorts, recruiting a new panel every five years or so. In practice, the big losses occur immediately at the first interview with people who do not want to cooperate, while the people who cooperate on the first interview are likely to continue. The Health and Retirement Study collects information about older people, and he said he thinks respondents like to have somebody to talk to periodi- cally. Attrition is very low after the first few interviews. He posited that 2During the open discussion later in this chapter, Roderick Little (University of Michigan, Ann Arbor) noted that this type of imputation has its limitations when the data that are missing (e.g., the prenatal data) are to be used in causal analysis.

OCR for page 45
IMPUTATION AND ESTIMATION 53 once people are convinced the NCS is important and sign up for it, they will continue to participate. O’Muircheartaigh added that continuation may be particularly dif- ficult for the NCS. He agreed almost all longitudinal surveys have most of their non-response in the first wave, and conditional response rates to later waves are quite high, often from 95 to 99 percent. This argues for minimal intrusion at the earliest stage to maximize the initial response rate. Unfortunately, in the NCS Vanguard sites, the opposite has been the practice, with collection of as much data as possible at the first visit. He said not all the data are necessary at the earliest point, and collecting only the necessary data at early points would maximize the initial response rate. He noted after 20 years, the overall attrition from the PSID was about 50 percent, with about half in the first wave. Planning carefully how to maximize the initial response to the NCS will be important. Special-Population Sample of 10,000 Cohen asked the panel to comment on the set-aside special-population sample of about 10,000 and the impact of attrition over time. He noted Kalton talked about weighting back to the original sample, adding that after 15 years with all the different levels of non-response, overall repre- sentation might be fairly low. Cohen asked if the 10,000 might be used as a replenishment sample. Kalton suggested that the special-population sample could be reserved for studies of rare populations, such as births that came about from assisted technologies. He said if the study were designed in this way, that group would not contribute to the national estimates in any way, because it would involve oversampling a miniscule population at a very high rate. But if there are special interests, this methodology could be used for a benchmark comparison. O’Muircheartaigh said the special sample would not contribute to the overall NCS, but it would be possible to do some linkage to the NCS because it would be contemporaneous and share some characteristics. But if it is not linked to the design, then it does not give any strength in terms of inferences to be made about the national sample. He questioned the desirability of setting money aside to tackle a specific problem that cannot be tackled within the framework of the NCS. Open Discussion About Imputation and Estimation Irwin Garfinkel (Columbia University) asked how the sample can be weighted without data on the proportion of births served by each of the prenatal clinics. Kalton said in the current PBS, the prenatal care pro­ iders v

OCR for page 45
54 DESIGN OF THE NATIONAL CHILDREN’S STUDY are sampled with probability proportional to estimated size, where the measure of size is the estimated number of first prenatal visits in the past year. If the sampling scheme within a sampled location were to select one week in four and take all eligible women during the selected weeks, the selection probability for a sampled woman would be four times the location selection probability. He noted that the efficiency of the sample design and the spread of the workload across the sampled locations depend on the quality of the estimated measures of size. To O’Muircheartaigh, the options are to fix the probabilities or fix the sample size, both of which can be fixed only with a lot of information. Fixing the probabilities is not difficult, but fixing them while simultane- ously controlling the sample size requires information. If there is good prior information and the provider data are accurate, then the sample size will be more or less as planned. If they are completely wrong, the sample size will fluctuate. Garfinkel noted both O’Muircheartaigh and Kalton are proposing to use the hospital only as the last resort and asked whether it might be simpler with better data and fewer assumptions to use the information on the number of hospitals and number of births. Kalton said that, in the context of a sample of pregnant women, the hospital sample is designed to provide coverage for women who had no prenatal care or who had pre- natal care only from a prenatal care provider that was not included in the provider sampling frame. The number of these women has to be guessed, although sometimes the number can be based on birth certificate data for the hospital from the past year. Although a problem, it could be dealt with, referring to O’Muircheartaigh’s observation that if a probability is misestimated or underestimated, then there will be a sampling fraction that will result in the sample including more or fewer births than planned. O’Muircheartaigh added it is unfortunate to use the term “last resort” when referring to the hospital, as it is the appropriate place to select certain women. It is saying that if the birth has not been covered by the sample of providers, the hospital will be the stratum that generates the birth. This group has several categories, and estimates are needed for the numbers of women in each category. Some of these numbers are available only through field activities in the location. One category is births to women who have no provider, and a second is births to women who use prenatal care providers that choose not to cooperate with the study. The operation will have uncertainty, no matter how sampling is done, and the exact number of births in the sample will not be known in advance. It is some- thing that is empirically determined by the population and not by some presupposition. Valliant noted not having complete control over the sample size in the survey is fairly standard. Another unknown besides provider cooperation

OCR for page 45
IMPUTATION AND ESTIMATION 55 is cooperation of women, who are at a stressful time in life and may not want to participate. Even well-founded advanced estimates will result in some inaccuracies in the number of people who agree to participate. It might be possible to do what household surveys typically do by creating replicates of sample units. If the sample is smaller than expected after recruiting for six months, a replicate of the provider sample could be released. Roderick Little (University of Michigan, Ann Arbor) made clarifying comments about imputation. He said in a regression, there is a Y, some Z variables that are observed, and an X variable that is the early pregnancy variable that is missing for some cases. If the values of the X’s are imputed purely based on Y and Z, the imputed values provide no information about the association between X and Y given Z, the topic of interest. The only way to get information on the association between X and Y given Z is by having auxiliary data to help with the imputation. Those auxiliary data could be from a questionnaire, from a dust measurement, or in an auxiliary database. It is important to realize that multiple imputation only helps if there is additional information to be recovered in the data that are used for imputation. If the relationship between Y and X is the topic of interest, some other variables are needed. Kalton noted that birth certificates might also provide valuable information to use in imputation. Nigel Paneth (Michigan State University) thanked the panel for clarifying that one cohort perhaps with different strata is the sensible approach and raised a question about statistical prioritization. He asked whether it depends on the questions being asked. He stated the NCS as currently described is a study about every childhood outcome and every potential exposure. With that as a framework, Paneth said it is impossible to decide whether prenatal data are more important than delivery data, or other important tradeoff questions. The struggle over sampling strat- egy and design reflects the absence of any closure concerning prioritiza- tion of public health relevant outcomes, key exposures to be investigated, and their relationships. Some of these considerations were subsumed by the hypotheses the NCS once had, he noted, but now the NCS does not have hypotheses. He said answers to important sampling questions cannot be answered until relevant health outcomes and exposures are determined. Kalton noted the integrated design comprises 100,000 births to follow from birth forward. Prior to birth, there is the critical issue of how many women will be sampled from prenatal care providers and how early they are enrolled. As has been noted, there will likely be some subgroups of women, such as the socially disadvantaged, who are underrepresented in the prenatal sample and will be covered mainly through the birth s ­tratum. He said issues about the effectiveness of this strategy remain

OCR for page 45
56 DESIGN OF THE NATIONAL CHILDREN’S STUDY to be examined, but the unified approach seems to be the best provider- based approach. O’Muircheartaigh also argued that if the problem is defined as obtain- ing a representative sample of 100,000 births, then the unified design with equal probabilities of selection would be the best design. The result would be a representative sample of births with as much information, includ- ing prenatal data, as possible. If pre-pregnancy data for later siblings are also collected for as many cases as possible, the result will maximize the amount of information contained in a representative sample of births, which he termed a noble ambition and a fine achievement whatever the outcome. Before the sample is selected, it is possible to debate about whether urban areas, inner urban areas, poor rural areas, or areas with high environmental risk should be oversampled, which is possible within the structured design. He noted that by taking a population representa- tion approach and maximizing the information available on as represen- tative a sample as possible, a platform is created on which many studies of different kinds can be based, including currently unspecified studies. Greg Duncan (University of California, Irvine) asked O’Muircheartaigh and Kalton for clarification about subsequent births. He referred to the comments in the first panel (see Chapter 2) about the importance of exposures very early in pregnancy and the potential importance of expo- sures preconception. Unless subsequent births are included in the design, Duncan said there would not be representative samples of births with preconception and very early pregnancy exposure information. Kalton replied he had been describing the basic design. Within a two- year enrollment period, there may be good grounds for including any subsequent births in a selected family with certainty. Then to retain an equal probability sample, it will be important to ensure that prenatal care providers and hospitals do not independently allow the sample to include subsequent pregnancies or births to mothers with a previous birth within the NCS enrollment period. He noted that there may be some potential advantages to having sibling data. The design has some operational efficiency, and the data can be used for sibling comparisons. He views the inclusion of siblings over a more extended time period to be an adjunct study that warrants careful assessment of its operational feasibility and associated costs, as well as a full examination of how it can be applied to yield data for the very early period of pregnancy. O’Muircheartaigh followed up saying that an ear- lier, widely advocated design was household-based probability sampling of women to be interviewed if they were of childbearing age. Clearly, a preconception sample is possible, and the previous approach would be a good solution if there were no costs or practical considerations. However, some evidence shows that the approach is impractical. Any

OCR for page 45
IMPUTATION AND ESTIMATION 57 design that does not involve recruitment of women in those age ranges regardless of pregnancy status is not going to collect a representative pre- conception sample. However, recruiting subsequent births to a mother recruited into the NCS has strong advantages in terms of providing some preconception and prenatal data. Duncan said he agreed that a two-year recruitment period would cap- ture some second births, but it would be an unusual sample with a fairly short time interval between births. A longer interval, perhaps five years rather than two, would include more births and be more representative. He suggested amending Kalton’s proposal to include a longer interval with oversampling of births early in the period, which would include some subsequent births and some first births later on. Although difficult for samplers, it may be another way of potentially providing a single integrated sample over a five-year period that would include both initial and subsequent births. Kalton responded that Duncan’s suggested design might be very expensive. He suggested another way to describe it might be to extend the two-year enrollment period to five years and then follow on the model that he had put forward. He said he would argue in the other direction, a one-year enrollment period, for a variety of reasons related to efficiency of data collection. It would avoid such problems as field workers going to one household to conduct a fifth interview while going to another to conduct a second interview. These types of mixtures, he said, make data collection much more difficult. Additionally, providers change over time, coming in and out of business. He said Duncan’s suggestion is possible, but it is not clear whether or not it is feasible to tie that design into the basic child-related data collections. If the design were to provide pre- conception information, all the women would have to be followed and go through questionnaires, although only some of them (perhaps 20 percent) would become pregnant. Kalton noted the unified design will collect child data on a schedule of every three months initially and then every six months, and he asked how that would work with the desire to know, almost immediately, when a woman becomes pregnant. He said he had heard the suggestion of preg- nancy tests by mail. To operationalize the sibling data collection, a method for capturing data from a woman at point of pregnancy should fit in with the ongoing collection of child information. O’Muircheartaigh agreed that it is a complex and difficult question. To address a simpler question, he suggested the number of initial recruitments might be reduced if the sample were supplemented with siblings over a two-year or five-year period, which, he said, would not affect the principle of the design. Duan followed up on Duncan’s point about the duration of the recruitment window, noting a longer duration would enhance the repre-

OCR for page 45
58 DESIGN OF THE NATIONAL CHILDREN’S STUDY sentativeness of the sibling cohort and might have its own merits. If the sample were entirely within one year, it would reflect the idiosyncrasies of that year, while a longer duration gives a better representation over time. The population of interest is not just children born in 2014, so a longer duration could help capture variations in economic and environmental events and in the weather. He said he appreciated Kalton’s point that it might be more costly for the same sample size, but potentially it might yield more useful scientific results. Duan agreed with Kalton’s proposal to look at the likely missing early pregnancy data in this unified approach. Many of the women who can be recruited through providers will not be in the sampling frame until after the first trimester. He said Kalton’s point about using statistical methods like weighting and imputation and applying them backward is an inter- esting idea, but there is a difference between time forward and time back- ward. Looking at time forward, a study with a strong field operation usu- ally can maintain the sample over time, and the conditional response rate after recruitment is usually very good. Going backward for imputation, the missing data are not under the study’s control because they are miss- ing from a time before participants were recruited. With the missing data rate going backward much higher than going forward, this sort of missing data methodology can be very sensitive to underlying assumptions. For that reason, Duan said it is important to supplement the data with either a sibling cohort or with alternative ways to get to early pregnancy data. Kalton noted one of the key issues is the proportion of women who can be recruited during pregnancy and in the first trimester. If the study could pick up 70 percent of participants in the first trimester, it would be similar to the PSID experience of 75 percent response to the first wave. Garfinkel asked O’Muircheartaigh about the possibility of high early attrition. If very expensive prenatal data are collected and attrition is high after that time, then expensive data have been wasted. Because of this, he said he disagreed with the statement that it does not matter when the money is spent. If siblings are important, then it matters greatly when the money is spent. As an example, if it costs $18,000 to collect data on a prenatal birth, $2,000 to enroll them, and $2,000 to enroll the m ­ others at the hospital, 10 times more women could be enrolled in a birth cohort than would be possible in a prenatal cohort. Thus, he said, if many women in the prenatal cohort are lost to attrition, it matters greatly when the money is spent. O’Muircheartaigh countered that if it cost $18,000 to recruit one way and $2,000 the other, it does not mean the sample can have nine times as many one way as the other, because these people will be maintained in the sample throughout the 21 years. He explained that is why consideration of both long-term costs and short-term costs are important in determin-

OCR for page 45
IMPUTATION AND ESTIMATION 59 ing the optimum allocation. If it costs $200,000 to cover each child for 21 years, then the comparison is between $218,000 and $202,000 in terms of the cost of a case in the NCS. He said it is only if the decision has to be made based on the money spent this year that one would make that deci- sion, but, in his opinion, that is entirely the wrong decision. He reminded the audience that it is critical to remember that short-term recruitment costs are only a small fraction of the costs of a case in the NCS. The appropriate basis of comparison is the total cost of the case under each of the scenarios. He said there is no reason to believe the later costs are any different depending on the method of recruitment, and therefore the imbalance is not 9 to 1 but perhaps 1.05 to 1. Kalton agreed with O’Muircheartaigh and Duan, who made the same point in the previous session, noting the cost of investing in a good sam- ple is paid over the life of the study. He noted he did not fully understand Garfinkel’s point about costing of the cohorts. Jennifer Madans (National Center for Health Statistics) asked for clarification about how women who have had no prenatal care or had a provider not on the frame would be identified at the hospital. Kalton responded that, as currently conceived, the data collectors at the hospital have a list of all the prenatal care providers on the sampling frame, and they are instructed to exclude from hospital recruitment all the women who attended any of these providers. The exclusion can be determined either prior to data collection (based on hospital records) or as part of the screening interview. Kalton said a woman has different potential routes of getting into the sample, but each woman is eligible for the sample through only one of these routes. In one route, they come in for their first prenatal visit to a sampled provider who has agreed to participate. Only those women who have had no prior visits for that pregnancy at that provider location are potentially eligible. When a woman is interviewed, she is asked if she has had any earlier prenatal care visits for that pregnancy with this or another provider. If she has visited this provider before, she is ineligible for the sample. If she has visited another provider, a check is made to see whether that provider was on the provider sampling frame. If so, the woman is again ineligible. Since eligibility is based on whether it is the first visit, there is only one route for sample selection. The same approach applies for the hospital cases: if the woman has had a prenatal care visit at a provider that was on the sampling frame, she is ineligible. The eligibility screener also includes questions on age, and whether the woman lives in the sampled county or not. Women can often be prescreened by the hospital as not being eligible based on hospital records.

OCR for page 45