National Academies Press: OpenBook
« Previous: 4 Imputation and Estimation
Suggested Citation:"5 Moving Forward." National Research Council and Institute of Medicine. 2013. Design of the National Children's Study: A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18386.
×

5

Moving Forward

This chapter begins with the question that panelists were asked to address in the final wrap-up session on moving forward. These were provided in advance to workshop participants via Kwan et al. (2013, p. 9). The chapter continues with key points made by the panelists, followed by a summary by the panel moderator and highlights of the open discussion.

QUESTION ABOUT MOVING FORWARD

Panelists were asked to synthesize the tradeoffs among factors, issues, and values that need to be balanced and considered by NCS leadership.

KEY POINTS OF THE DISCUSSION

The moderator for the final session of the workshop was Greg Duncan (School of Education, University of California, Irvine). The panelists were Ana Diez Roux (Department of Epidemiology, School of Public Health, University of Michigan, Ann Arbor), Roderick Little (Department of Biostatistics, University of Michigan, Ann Arbor), and Edward Sondik (National Center for Health Statistics, Centers for Disease Control and Prevention).

Suggested Citation:"5 Moving Forward." National Research Council and Institute of Medicine. 2013. Design of the National Children's Study: A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18386.
×

Remarks by Edward Sondik

Sondik said he would try to identify recurring themes, issues where there does not seem to be agreement, tradeoffs, and finally offer a comment and suggestion. He said he tried to look at his remarks from the standpoint of the study and what kinds of information might move it forward.

He listed five recurring themes. First, the population of interest is children born during a fixed time period, although there was not agreement on the exact time period. Second, no one raised the issue that the proposed design includes only a very small preconception sample, although collection of preconception data in the NCS has been part of the dialogue from the beginning. He said prioritization of the collection of preconception data might rely on examination of the hypotheses that drive collection, which he called a significant decision. Third, examination of total costs over the lifetime of the study in addition to up-front costs might lead to different conclusions, referring to the point that although recruitment costs are high, they can be viewed in the context of study results and the total cost of the study. Fourth, mobility of sampled families is an important issue and may result in potential loss to the sample if families cannot be followed; further, it will be important to be able to link environmental measurements to (possibly changing) family residence. Fifth, he noted discussion about the tradeoffs in the number of primary sampling units (PSUs), which he said is a complex and important issue that has not been discussed completely. Allied with that is the question of how to handle the geographic environmental variables, and whether or not they are clustered. If environmental exposures are uniformly distributed geographically, a good probability sample is sufficient. If exposures are not uniform, then it is not clear how to include geography in the sample design. He said he called this an area of agreement because he said he thought that people would agree that, in general, more is better.

Under important issues with little or no agreement, he noted very little discussion during the workshop about study objectives. In his opinion, a clear articulation of the ability of the design to produce information important to public health would be invaluable to justify the study. He observed that the first panel (see Chapter 2) specifically addressed the proposed measurements in light of their ability to learn about asthma, endocrine disruptors, and neurological problems, but this was the only discussion about public health impacts and how well the design supports understanding them. He said this raises the question of whether or not the design will support learning about the impact of specific environmental exposures on health.

Suggested Citation:"5 Moving Forward." National Research Council and Institute of Medicine. 2013. Design of the National Children's Study: A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18386.
×

As tradeoffs, Sondik noted one of the open questions is the extent to which the design will include prenatal measures. Referring to Graham Kalton’s earlier estimate that 70 percent of women see a doctor during the first trimester, he pointed out that another participant had said a measurement at six weeks is more important. Sondik asked about the importance of collecting information at six weeks and how to do so.

The second tradeoff has to do with the sample composition, noting the agenda started off posing that the design will include a 50-50 split between the birth and the prenatal cohorts, and invited the panel to consider other compositions. Sondik proposed the optimal composition of the sample is related to the science, noting that he was concerned about the challenges associated with recruiting women in labor or just post-labor. The question of coverage of first-born versus siblings is important and brings up many other tradeoffs to consider. He praised the elegance of Kalton’s sample design.

Sondik’s final suggestion also concerned the relationship of the science to the design. He expressed concern about the power of the study to determine relationships. He stated that he understands the reluctance to identify a single specific set of hypotheses, but one way to evaluate a sample design might be in terms of an exemplar set of hypotheses and consideration of the power that exists within the design to evaluate them. As an example, he asked about the power of the study to determine relationships among the variables of first-born, the income level or poverty level of the family, race, and asthma. Given the short time before the Main Study is to begin, he suggested commissioning a panel to look at and prioritize the set of reasonable science questions the study might address. Given those priorities, perhaps a second panel (or NCS staff) could look at the set of priorities in terms of the design and assess the power of the design to identify relationships. He said this would provide NCS with a very powerful argument to give to decision makers about the study’s focus.

He noted in closing that, in the past, the Framingham Heart Study was raised as an example of data collection that did not start with a set of hypotheses. Clearly, a broad database could provide possibilities for exploring relationships, but it is important to be able to give examples of the kinds of analytic power the study is expected to have. Sondik said keeping track of those estimates early on, as the study progresses and as the sample develops, would be a very important management tool.

Remarks by Roderick Little

Little suggested the NCS needs to articulate specifically what it is adding over existing studies, pointing to other new studies, many in

Suggested Citation:"5 Moving Forward." National Research Council and Institute of Medicine. 2013. Design of the National Children's Study: A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18386.
×

other countries. He observed that when he was on the NCS Advisory Committee, a huge effort was made to develop hundreds of hypotheses. While he said he was somewhat critical of the scope of that effort, it was laudable in some ways. He said now with no hypotheses, the NCS is proposed to be a data platform that somehow will address many different things and suggested searching for a happy medium between these two extremes. He said he does not see an obvious way to make decisions about optimal design without some specific objectives articulated through hypotheses. He suggested seeking a relatively small number of “sentinel” hypotheses that could be viewed as being the current burning issues and providing the power calculations for those hypotheses.

Little noted the workshop focused on the role of prenatal exposures, particularly environmental exposures. Though clearly important, they are not the only component of the NCS and pointed to work to be done with exposures after birth.

He praised the supplemental sample of 10,000 in the draft sample design and agreed with O’Muircheartaigh and Kalton that, for most of the sample, an equal probability sample design makes sense, particularly without clearly articulated hypotheses. On the other hand, he said getting good variability in exposures is important and suggested developing an index of environmental risk and oversampling areas that have high risk. This might increase the power of the sample for looking at some of these associations.

He observed the progress made in sample design, with the debate now about which particular kind of probability sample to use. He said he supports probability sampling, or something as close as possible to a probability sample with the possibility of some missing data. He said it may not be possible to collect early trimester information on everyone. Even if there is only partial information, it is still a probability sample. He called the NCS a very useful study because there are many things that will be analyzed that do not necessarily use first-trimester information.

He noted three overarching issues in terms of specific choices of a design: (1) choice of the frame and whether to use a provider frame or a hospital frame, (2) point of contact, and (3) timing of the initial visit. He said very detailed specifications for these alternatives are important. He suggested that costs will illuminate the argument about the birth cohort versus siblings versus prenatal provider cohort, but there is divergence of opinion and confusion about the relative costs. Given the utility of the prenatal information for some hypotheses, he said he favors trying to get direct information for as many people as possible during early pregnancy to help ensure the representativeness and utility of the sample.

He noted even though there has been conflicting information about cost and practicality, he is somewhat more inclined toward the provider

Suggested Citation:"5 Moving Forward." National Research Council and Institute of Medicine. 2013. Design of the National Children's Study: A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18386.
×

approach if it can be operationalized satisfactorily. He said he defers to those who have been in the field doing work on these issues, but the provider sampling approach looks promising, as does the unified way of thinking about the design described by Kalton and O’Muircheartaigh.

Little reminded the audience about his comments earlier in the workshop about the imputation of missing early pregnancy data. Good auxiliary data are needed for imputing early pregnancy exposure (or other) data. Auxiliary data may come from proxy interviews or some other source (such as dust measurements or linkages to EPA exposure databases, as discussed in Chapter 2). The validity of an imputation method relies on its ability to estimate the relationship between exposure variables and outcomes.

Little’s final comment concerned the original 105 PSUs versus a smaller number of PSUs. He suggested that there would be added variance in going to a more highly clustered design and doubts that it would be worth the cost savings if amortized over the life of the study.

Remarks by Ana Diez Roux

Diez Roux began her comments by acknowledging that the NCS is trying to address a very complex and broad-ranging set of issues that require engagement of many different disciplines. She addressed several big-picture items that she said could be done relatively quickly. She commented on the criteria for making design decisions, the process, and other points that came up during the workshop.

Diez Roux said the study will continue to grapple with prioritizing study objectives in order to make design decisions, and there are tradeoffs in that the Main Study will not be able to properly address certain items. She said this is acceptable, but it is important to acknowledge what the study will and will not do. When people are collecting data, having a sense that specific objectives are being targeted helps to keep the group focused on a common goal.

She noted two big sets of objectives. One potentially important objective, not discussed during the workshop, is estimating incidents and prevalence of different conditions among U.S. children. She was part of a National Research Council/Institute of Medicine panel (National Research Council and Institute of Medicine, 2013) that looked at the U.S. health disadvantage compared to other high-income countries. She reported that health under age 50 and specifically among children and adolescents features prominently as an area in which the United States does substantially worse than other high-income countries. One of the panel’s findings was insufficiency of data on the prevalence and incidence of many health conditions among U.S. children that can be compared

Suggested Citation:"5 Moving Forward." National Research Council and Institute of Medicine. 2013. Design of the National Children's Study: A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18386.
×

to other high-income countries. Filling this information void may be an objective for the study to consider in order to demonstrate its value.

She said the other big study objective has to do with etiological investigation, which can be driven in two ways. First, it can be driven by very specific research questions, although the disadvantage, as the NCS experienced in its early years, is that too many hypotheses make the process unmanageable. Conversely, the other extreme approach is to be completely agnostic and say data will be collected to support unknown analyses in the future. She expressed her belief that a purely agnostic approach is virtually impossible because decisions have to be made, and in making those decisions one needs to know what questions are to be answered and what their priorities are.

She noted, however, that some aspects of the design may be applicable to many different questions, so the hypothesis-driven and agnostic approaches are not totally incompatible. However, a number of other decisions may require thinking about priority objectives. She agreed with Little that a middle ground between these two extremes exists and suggested thinking about a typology of the priority questions that the study might answer. For example, she asked whether there is one set of important questions about prenatal exposures. If so, that would indicate the importance of collecting prenatal information. She asked whether there are sets of environmental factors that are important, and if they are common factors, important factors, factors that potentially have very adverse impacts, or factors that may vary geographically. The latter, she noted, would indicate a geographically distributed sample is more appropriate. Deciding what kinds of environmental factors are of primary interest will help the NCS make decisions.

She argued less for a specific list of questions than a typology. She noted the typology might include rare outcomes, outcomes that have a public health impact, or common outcomes that are causing the United States to have much worse health than other high-income countries for unclear reasons. She suggested this kind of thinking might also help in some of the design decisions. She said a third kind of question has to do with investigation of disparities and asked whether disparities are key to this study. This kind of typology has implications for the core design, sample size, and other issues, as well as for the core measures and key exposures of interest.

She said she thinks that developing this typology would not take much time. A group would have to develop consensus and not everyone would agree. She observed the differences in opinion expressed during the workshop’s discussion of design and measurement reflect underlying differences of views about what the study should address. Making these things explicit, she said, will assist in decision making.

Suggested Citation:"5 Moving Forward." National Research Council and Institute of Medicine. 2013. Design of the National Children's Study: A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18386.
×

Diez Roux noted a first step is to establish a set of core typology questions that prioritize the study without being overly specific or detailed. Collecting as much additional data as possible is a good idea because so much is unknown. Many new questions will emerge in the future, including exposures and outcomes of predisease markers and support for epigenetics. She said some criteria have to do with the expected utility based on what is known. While incomplete because there are many things that are unknown, it is one starting point. She noted some data are easy to collect, such as global positioning system locations of residences to allow linkage to a wealth of information down the line.

She noted storage is a second criterion. If samples can be stored and may be useful, collecting and storing them (within logistical constraints) is a good idea.

Another aspect of the study that needs balance is complexity versus simplicity in design data collection decisions. She said she weighs toward simplicity, which has advantages in terms of running a study on site and of analyzing the data later. She reminded the audience more complicated designs make it more difficult to use these data later. She noted simpler is better, but not so simple that it defeats the purpose. To her, the unified design approach proposed by the previous panel (see Chapter 4) is very appealing because it is a simpler approach than having multiple cohorts that have to be weighted differently and combined.

Drawing on her experience in multisite studies, she said it is important for the study to find the right balance between centralized and decentralized activities. The NCS can learn from other multi-site studies and the Vanguard sites, including the investigators involved in these studies. She said when she served on the NCS Advisory Committee she had the sense that a wealth of information was not being used as much as it could be.

She concluded with two comments on ideas that came up during the workshop. Given differences in the biology of first pregnancies versus subsequent pregnancies, and also birth order effects on a number of social and health outcomes, she stated that it is very plausible that prenatal factors interact with birth order.

She stated that the tradeoffs concerning the sibling sample are important to consider. The advantages might be the potential ability to get preconception information, with perhaps some cost implications, and within family sibling comparisons that could be informative. There are also logistical issues to work out. However, depending on what the priority questions are, adding siblings to the sample will reduce sample variation in exposures that are invariant within families and reduce statistical power if outcomes are clustered within families.

Suggested Citation:"5 Moving Forward." National Research Council and Institute of Medicine. 2013. Design of the National Children's Study: A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18386.
×

Closing Comments by Greg Duncan

Duncan stated that he detected agreement among Sondik, Little, and Diez Roux on some issues. They all strongly endorsed probability samples, and the two presenters who discussed PSUs (Sondik and Little) endorsed more versus fewer of them. All three panelists expressed the opinion that there ought to be some version of hypotheses or, as Diez Roux proposed, typologies.

Duncan said he would push to first consider the kind of objectives that have important bearing on the design. He said if it is true that preconception exposures and exposures very early in pregnancy are not very important, then a sibling sample may not be worth it. The question that remains is whether preconception exposures and very early exposures in pregnancy are important to address with this study.

His second point concerned geographically varying environmental exposures. If environmental measures are either not geographically varying or are not important to the study, then geography is not important to the sample design. However, if measures vary by geography and are important, considering geography in the design may be important.

Open Discussion about Moving Forward

Michael Bracken (Yale University) noted he found it encouraging that the moderator and three panelists agreed that hypotheses are important, at least selected sentinel hypotheses. He also agreed that hypotheses that reflect what this study could do and that demonstrate its value are important, noting a study’s targeted health effects help communities decide to support a project, rather than just serve as a data platform. He agreed with Diez Roux in urging some speed in developing sentinel hypotheses and the value of reviewing previous work to determine whether hypotheses of current interest are included. He noted experts in various disciplines spent countless hours developing these hypotheses, and the documentation is available.

Nigel Paneth (Michigan State University) thanked the panel for emphasizing the need for prioritization, for systematically developing a schema that would allow NCS to prioritize properly and to find the balance between enormous numbers of hypotheses and the current state of no hypotheses. He went on to support Diez Roux’s suggestion for developing a typology of hypotheses that would help the study to make important decisions.

He noted the NCS has already specified a sample size of 100,000, which he termed the wrong approach. Because the sample size has been specified to be 100,000, many hypotheses cannot be considered while

Suggested Citation:"5 Moving Forward." National Research Council and Institute of Medicine. 2013. Design of the National Children's Study: A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18386.
×

others are easily accommodated. He urged NCS leadership to pay attention to the views expressed during the workshop.

Naihua Duan (Columbia University) asked about the plan not to proceed with the household screening sample to collect preconception measures. He noted his purpose in raising the issue is for brainstorming. One message he heard from the first panel was the importance of early pregnancy or maybe even preconception data. Some of those data could be captured in the unified sampling approach but might not be captured. He said he wondered what could be obtained from the prenatal sample relative to what potentially could be accomplished with a household sample.

Even though the household sample has been found to be too expensive, Duan noted that many at the workshop commented on the importance of early pregnancy and preconception measures. This suggests that recruitment costs are not the only costs to consider; total lifetime costs for the study are important as well. He noted several other panelists commented that the difference in recruitment costs might not be that large when combined with follow-up costs. Part of the high cost of recruitment of preconception women is following women who never become pregnant. To reduce these costs, one solution might be to limit the preconception sample to women who are actively trying to get pregnant. He said there may be a benefit to retaining some household sample (perhaps in hot spots, or other geographic areas of interest) in order to answer important scientific questions.

Duan agreed with Irwin Garfinkel and the other panelists that the sibling sample has much merit but also some limitations. Questions about the first-born and what might not be covered in the sibling sample are not trivial. One advantage of the household sample, if it could be retained in some affordable way, would be to fill that gap.

Sondik agreed, saying a comparison of the characteristics of household sampling versus provider-based sampling in terms of data that could be collected might provide a useful basis for evaluating each method and understanding in what circumstances it would be useful.

Kalton commented that he perceived that the participants understood the value or potential value of the NCS. In terms of costs, he said there is also an issue of political will. He said political will helps with this study because both political parties understand the study well enough to support it. He urged making sure the NCS is a good strong study that is properly implemented.

Steven Hirschfeld (National Children’s Study director) thanked participants for a very informative and stimulating discussion, saying that he and others will continue the evolution of the design of the National

Suggested Citation:"5 Moving Forward." National Research Council and Institute of Medicine. 2013. Design of the National Children's Study: A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18386.
×

Children’s Study. He said its potential is well appreciated, and the goal of the NCS is to have that potential not only realized to meet current expectations but also to have a platform that would exceed expectations in the future so it becomes an ongoing resource for informing about the health and development and growth of children.

Suggested Citation:"5 Moving Forward." National Research Council and Institute of Medicine. 2013. Design of the National Children's Study: A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18386.
×
Page 61
Suggested Citation:"5 Moving Forward." National Research Council and Institute of Medicine. 2013. Design of the National Children's Study: A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18386.
×
Page 62
Suggested Citation:"5 Moving Forward." National Research Council and Institute of Medicine. 2013. Design of the National Children's Study: A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18386.
×
Page 63
Suggested Citation:"5 Moving Forward." National Research Council and Institute of Medicine. 2013. Design of the National Children's Study: A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18386.
×
Page 64
Suggested Citation:"5 Moving Forward." National Research Council and Institute of Medicine. 2013. Design of the National Children's Study: A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18386.
×
Page 65
Suggested Citation:"5 Moving Forward." National Research Council and Institute of Medicine. 2013. Design of the National Children's Study: A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18386.
×
Page 66
Suggested Citation:"5 Moving Forward." National Research Council and Institute of Medicine. 2013. Design of the National Children's Study: A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18386.
×
Page 67
Suggested Citation:"5 Moving Forward." National Research Council and Institute of Medicine. 2013. Design of the National Children's Study: A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18386.
×
Page 68
Suggested Citation:"5 Moving Forward." National Research Council and Institute of Medicine. 2013. Design of the National Children's Study: A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18386.
×
Page 69
Suggested Citation:"5 Moving Forward." National Research Council and Institute of Medicine. 2013. Design of the National Children's Study: A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18386.
×
Page 70
Next: References »
Design of the National Children's Study: A Workshop Summary Get This Book
×
Buy Paperback | $38.00 Buy Ebook | $30.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

The Children's Health Act mandated the National Children's Study (NCS) in 2000 with one of its purposes being to authorize the National Institute of Child Health and Human Development (NICHD) to study the environmental influences (including physical, chemical, biological, and psychosocial) on children's health and development. The NCS examines all aspects of the environment including air, water, diet, noise, family dynamics, and genetics, on the growth, development, and health of children across the United States, for a period of 21 years. The purpose of NCS is to improve the health and well-being of children and to contribute to understanding the role of these factors on health and disease.

The research plan for the NCS was developed from 2005 to 2007 in collaboration among the Interagency Coordinating Committee, the NCS Advisory Committee, the NCS Program Office, Westat, the Vanguard Center principal investigators, and federal scientists. The current design of the study, however, uses a separate pilot to assess quality of scientific output, logistics, and operations and a "Main Study" to examine exposure-outcome relationships. The NCS proposed the use of a multilayered cohort approach for the Main Study, which was one of the topics for discussion at the workshop that is the subject of this publication.

In the fall of 2012, NICHD requested that the Committee on National Statistics (CNSTAT) of the NRC and the IOM convene a joint workshop, to be led by CNSTAT. The workshop was to focus on issues related to the overall design (including the framework for implementation) of the NCS. The committee was provided a background paper which it used to select the challenges that were discussed at the workshop. Design of the National Children's Study: A Workshop Summary presents an overview of the workshop held on January 11, 2013. The publication includes summaries of the four sessions of the workshop, a list of participants, and the agenda.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!