5

Moving Forward

This chapter begins with the question that panelists were asked to address in the final wrap-up session on moving forward. These were provided in advance to workshop participants via Kwan et al. (2013, p. 9). The chapter continues with key points made by the panelists, followed by a summary by the panel moderator and highlights of the open discussion.

QUESTION ABOUT MOVING FORWARD

Panelists were asked to synthesize the tradeoffs among factors, issues, and values that need to be balanced and considered by NCS leadership.

KEY POINTS OF THE DISCUSSION

The moderator for the final session of the workshop was Greg Duncan (School of Education, University of California, Irvine). The panelists were Ana Diez Roux (Department of Epidemiology, School of Public Health, University of Michigan, Ann Arbor), Roderick Little (Department of Biostatistics, University of Michigan, Ann Arbor), and Edward Sondik (National Center for Health Statistics, Centers for Disease Control and Prevention).



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 61
5 Moving Forward T his chapter begins with the question that panelists were asked to address in the final wrap-up session on moving forward. These were provided in advance to workshop participants via Kwan et al. (2013, p. 9). The chapter continues with key points made by the panel- ists, followed by a summary by the panel moderator and highlights of the open discussion. QUESTION ABOUT MOVING FORWARD Panelists were asked to synthesize the tradeoffs among factors, issues, and values that need to be balanced and considered by NCS leadership. KEY POINTS OF THE DISCUSSION The moderator for the final session of the workshop was Greg Dun- can (School of Education, University of California, Irvine). The panelists were Ana Diez Roux (Department of Epidemiology, School of Public Health, University of Michigan, Ann Arbor), Roderick Little (Department of Biostatistics, University of Michigan, Ann Arbor), and Edward Sondik (National Center for Health Statistics, Centers for Disease Control and Prevention). 61

OCR for page 61
62 DESIGN OF THE NATIONAL CHILDREN’S STUDY Remarks by Edward Sondik Sondik said he would try to identify recurring themes, issues where there does not seem to be agreement, tradeoffs, and finally offer a com- ment and suggestion. He said he tried to look at his remarks from the standpoint of the study and what kinds of information might move it forward. He listed five recurring themes. First, the population of interest is children born during a fixed time period, although there was not agree- ment on the exact time period. Second, no one raised the issue that the proposed design includes only a very small preconception sample, although collection of preconception data in the NCS has been part of the dialogue from the beginning. He said prioritization of the collection of preconception data might rely on examination of the hypotheses that drive collection, which he called a significant decision. Third, examina- tion of total costs over the lifetime of the study in addition to up-front costs might lead to different conclusions, referring to the point that although recruitment costs are high, they can be viewed in the context of study results and the total cost of the study. Fourth, mobility of sampled families is an important issue and may result in potential loss to the sample if families cannot be followed; further, it will be important to be able to link environmental measurements to (possibly changing) family residence. Fifth, he noted discussion about the tradeoffs in the number of primary sampling units (PSUs), which he said is a complex and impor- tant issue that has not been discussed completely. Allied with that is the question of how to handle the geographic environmental variables, and whether or not they are clustered. If environmental exposures are uniformly distributed geographically, a good probability sample is suf- ficient. If exposures are not uniform, then it is not clear how to include geography in the sample design. He said he called this an area of agree- ment because he said he thought that people would agree that, in gen- eral, more is better. Under important issues with little or no agreement, he noted very ­ ittle l discussion during the workshop about study objectives. In his ­ pinion, o a clear articulation of the ability of the design to produce information important to public health would be invaluable to justify the study. He observed that the first panel (see Chapter 2) specifically addressed the proposed measurements in light of their ability to learn about asthma, endocrine disruptors, and neurological problems, but this was the only discussion about public health impacts and how well the design supports understanding them. He said this raises the question of whether or not the design will support learning about the impact of specific environmental exposures on health.

OCR for page 61
MOVING FORWARD 63 As tradeoffs, Sondik noted one of the open questions is the extent to which the design will include prenatal measures. Referring to Graham Kalton’s earlier estimate that 70 percent of women see a doctor during the first trimester, he pointed out that another participant had said a measure- ment at six weeks is more important. Sondik asked about the importance of collecting information at six weeks and how to do so. The second tradeoff has to do with the sample composition, noting the agenda started off posing that the design will include a 50-50 split between the birth and the prenatal cohorts, and invited the panel to consider other compositions. Sondik proposed the optimal composition of the sample is related to the science, noting that he was concerned about the challenges associated with recruiting women in labor or just post- labor. The question of coverage of first-born versus siblings is important and brings up many other tradeoffs to consider. He praised the elegance of Kalton’s sample design. Sondik’s final suggestion also concerned the relationship of the sci- ence to the design. He expressed concern about the power of the study to determine relationships. He stated that he understands the reluctance to identify a single specific set of hypotheses, but one way to evaluate a sample design might be in terms of an exemplar set of hypotheses and consideration of the power that exists within the design to evaluate them. As an example, he asked about the power of the study to determine rela- tionships among the variables of first-born, the income level or poverty level of the family, race, and asthma. Given the short time before the Main Study is to begin, he suggested commissioning a panel to look at and prioritize the set of reasonable science questions the study might address. Given those priorities, perhaps a second panel (or NCS staff) could look at the set of priorities in terms of the design and assess the power of the design to identify relationships. He said this would provide NCS with a very powerful argument to give to decision makers about the study’s focus. He noted in closing that, in the past, the Framingham Heart Study was raised as an example of data collection that did not start with a set of hypotheses. Clearly, a broad database could provide possibilities for exploring relationships, but it is important to be able to give examples of the kinds of analytic power the study is expected to have. Sondik said keeping track of those estimates early on, as the study progresses and as the sample develops, would be a very important management tool. Remarks by Roderick Little Little suggested the NCS needs to articulate specifically what it is adding over existing studies, pointing to other new studies, many in

OCR for page 61
64 DESIGN OF THE NATIONAL CHILDREN’S STUDY other countries. He observed that when he was on the NCS Advisory Committee, a huge effort was made to develop hundreds of hypotheses. While he said he was somewhat critical of the scope of that effort, it was laudable in some ways. He said now with no hypotheses, the NCS is proposed to be a data platform that somehow will address many differ- ent things and suggested searching for a happy medium between these two extremes. He said he does not see an obvious way to make deci- sions about optimal design without some specific objectives articulated through hypotheses. He suggested seeking a relatively small number of “sentinel” hypotheses that could be viewed as being the current burn- ing issues and providing the power calculations for those hypotheses. Little noted the workshop focused on the role of prenatal exposures, particularly environmental exposures. Though clearly important, they are not the only component of the NCS and pointed to work to be done with exposures after birth. He praised the supplemental sample of 10,000 in the draft sample design and agreed with O’Muircheartaigh and Kalton that, for most of the sample, an equal probability sample design makes sense, particularly without clearly articulated hypotheses. On the other hand, he said getting good variability in exposures is important and suggested developing an index of environmental risk and oversampling areas that have high risk. This might increase the power of the sample for looking at some of these associations. He observed the progress made in sample design, with the debate now about which particular kind of probability sample to use. He said he supports probability sampling, or something as close as possible to a prob- ability sample with the possibility of some missing data. He said it may not be possible to collect early trimester information on everyone. Even if there is only partial information, it is still a probability sample. He called the NCS a very useful study because there are many things that will be analyzed that do not necessarily use first-trimester information. He noted three overarching issues in terms of specific choices of a design: (1) choice of the frame and whether to use a provider frame or a hospital frame, (2) point of contact, and (3) timing of the initial visit. He said very detailed specifications for these alternatives are impor- tant. He suggested that costs will illuminate the argument about the birth cohort versus siblings versus prenatal provider cohort, but there is diver- gence of opinion and confusion about the relative costs. Given the utility of the prenatal information for some hypotheses, he said he favors trying to get direct information for as many people as possible during early pregnancy to help ensure the representativeness and utility of the sample. He noted even though there has been conflicting information about cost and practicality, he is somewhat more inclined toward the provider

OCR for page 61
MOVING FORWARD 65 approach if it can be operationalized satisfactorily. He said he defers to those who have been in the field doing work on these issues, but the provider sampling approach looks promising, as does the unified way of thinking about the design described by Kalton and O’Muircheartaigh. Little reminded the audience about his comments earlier in the work- shop about the imputation of missing early pregnancy data. Good aux- iliary data are needed for imputing early pregnancy exposure (or other) data. Auxiliary data may come from proxy interviews or some other source (such as dust measurements or linkages to EPA exposure data- bases, as discussed in Chapter 2). The validity of an imputation method relies on its ability to estimate the relationship between exposure variables and outcomes. Little’s final comment concerned the original 105 PSUs versus a smaller number of PSUs. He suggested that there would be added vari- ance in going to a more highly clustered design and doubts that it would be worth the cost savings if amortized over the life of the study. Remarks by Ana Diez Roux Diez Roux began her comments by acknowledging that the NCS is trying to address a very complex and broad-ranging set of issues that require engagement of many different disciplines. She addressed several big-picture items that she said could be done relatively quickly. She com- mented on the criteria for making design decisions, the process, and other points that came up during the workshop. Diez Roux said the study will continue to grapple with prioritizing study objectives in order to make design decisions, and there are tradeoffs in that the Main Study will not be able to properly address certain items. She said this is acceptable, but it is important to acknowledge what the study will and will not do. When people are collecting data, having a sense that specific objectives are being targeted helps to keep the group focused on a common goal. She noted two big sets of objectives. One potentially important objec- tive, not discussed during the workshop, is estimating incidents and prevalence of different conditions among U.S. children. She was part of a National Research Council/Institute of Medicine panel (National Research Council and Institute of Medicine, 2013) that looked at the U.S. health disadvantage compared to other high-income countries. She reported that health under age 50 and specifically among children and adolescents features prominently as an area in which the United States does substantially worse than other high-income countries. One of the panel’s findings was insufficiency of data on the prevalence and incidence of many health conditions among U.S. children that can be compared

OCR for page 61
66 DESIGN OF THE NATIONAL CHILDREN’S STUDY to other high-income countries. Filling this information void may be an objective for the study to consider in order to demonstrate its value. She said the other big study objective has to do with etiological inves- tigation, which can be driven in two ways. First, it can be driven by very specific research questions, although the disadvantage, as the NCS experienced in its early years, is that too many hypotheses make the process unmanageable. Conversely, the other extreme approach is to be completely agnostic and say data will be collected to support unknown analyses in the future. She expressed her belief that a purely agnostic approach is virtually impossible because decisions have to be made, and in making those decisions one needs to know what questions are to be answered and what their priorities are. She noted, however, that some aspects of the design may be appli- cable to many different questions, so the hypothesis-driven and agnostic approaches are not totally incompatible. However, a number of other decisions may require thinking about priority objectives. She agreed with Little that a middle ground between these two extremes exists and sug- gested thinking about a typology of the priority questions that the study might answer. For example, she asked whether there is one set of impor- tant questions about prenatal exposures. If so, that would indicate the importance of collecting prenatal information. She asked whether there are sets of environmental factors that are important, and if they are com- mon factors, important factors, factors that potentially have very adverse impacts, or factors that may vary geographically. The latter, she noted, would indicate a geographically distributed sample is more appropriate. Deciding what kinds of environmental factors are of primary interest will help the NCS make decisions. She argued less for a specific list of questions than a typology. She noted the typology might include rare outcomes, outcomes that have a public health impact, or common outcomes that are causing the United States to have much worse health than other high-income countries for unclear reasons. She suggested this kind of thinking might also help in some of the design decisions. She said a third kind of question has to do with investigation of disparities and asked whether disparities are key to this study. This kind of typology has implications for the core design, sample size, and other issues, as well as for the core measures and key exposures of interest. She said she thinks that developing this typology would not take much time. A group would have to develop consensus and not everyone would agree. She observed the differences in opinion expressed during the workshop’s discussion of design and measurement reflect underlying differences of views about what the study should address. Making these things explicit, she said, will assist in decision making.

OCR for page 61
MOVING FORWARD 67 Diez Roux noted a first step is to establish a set of core typology ques- tions that prioritize the study without being overly specific or detailed. Collecting as much additional data as possible is a good idea because so much is unknown. Many new questions will emerge in the future, including exposures and outcomes of predisease markers and support for epigenetics. She said some criteria have to do with the expected utility based on what is known. While incomplete because there are many things that are unknown, it is one starting point. She noted some data are easy to collect, such as global positioning system locations of residences to allow linkage to a wealth of information down the line. She noted storage is a second criterion. If samples can be stored and may be useful, collecting and storing them (within logistical constraints) is a good idea. Another aspect of the study that needs balance is complexity versus simplicity in design data collection decisions. She said she weighs toward simplicity, which has advantages in terms of running a study on site and of analyzing the data later. She reminded the audience more complicated designs make it more difficult to use these data later. She noted simpler is better, but not so simple that it defeats the purpose. To her, the unified design approach proposed by the previous panel (see Chapter 4) is very appealing because it is a simpler approach than having multiple cohorts that have to be weighted differently and combined. Drawing on her experience in multisite studies, she said it is impor- tant for the study to find the right balance between centralized and decen- tralized activities. The NCS can learn from other multi-site studies and the Vanguard sites, including the investigators involved in these studies. She said when she served on the NCS Advisory Committee she had the sense that a wealth of information was not being used as much as it could be. She concluded with two comments on ideas that came up during the workshop. Given differences in the biology of first pregnancies versus subsequent pregnancies, and also birth order effects on a number of social and health outcomes, she stated that it is very plausible that prenatal fac- tors interact with birth order. She stated that the tradeoffs concerning the sibling sample are impor- tant to consider. The advantages might be the potential ability to get preconception information, with perhaps some cost implications, and within family sibling comparisons that could be informative. There are also logistical issues to work out. However, depending on what the prior- ity questions are, adding siblings to the sample will reduce sample varia- tion in exposures that are invariant within families and reduce statistical power if outcomes are clustered within families.

OCR for page 61
68 DESIGN OF THE NATIONAL CHILDREN’S STUDY Closing Comments by Greg Duncan Duncan stated that he detected agreement among Sondik, Little, and Diez Roux on some issues. They all strongly endorsed probability samples, and the two presenters who discussed PSUs (Sondik and Little) endorsed more versus fewer of them. All three panelists expressed the opinion that there ought to be some version of hypotheses or, as Diez Roux proposed, typologies. Duncan said he would push to first consider the kind of objectives that have important bearing on the design. He said if it is true that pre- conception exposures and exposures very early in pregnancy are not very important, then a sibling sample may not be worth it. The question that remains is whether preconception exposures and very early exposures in pregnancy are important to address with this study. His second point concerned geographically varying environmental exposures. If environmental measures are either not geographically vary- ing or are not important to the study, then geography is not important to the sample design. However, if measures vary by geography and are important, considering geography in the design may be important. Open Discussion about Moving Forward Michael Bracken (Yale University) noted he found it encouraging that the moderator and three panelists agreed that hypotheses are important, at least selected sentinel hypotheses. He also agreed that hypotheses that reflect what this study could do and that demonstrate its value are impor- tant, noting a study’s targeted health effects help communities decide to support a project, rather than just serve as a data platform. He agreed with Diez Roux in urging some speed in developing sentinel hypotheses and the value of reviewing previous work to determine whether hypotheses of current interest are included. He noted experts in various disciplines spent countless hours developing these hypotheses, and the documenta- tion is available. Nigel Paneth (Michigan State University) thanked the panel for emphasizing the need for prioritization, for systematically developing a schema that would allow NCS to prioritize properly and to find the balance between enormous numbers of hypotheses and the current state of no hypotheses. He went on to support Diez Roux’s suggestion for developing a typology of hypotheses that would help the study to make important decisions. He noted the NCS has already specified a sample size of 100,000, which he termed the wrong approach. Because the sample size has been specified to be 100,000, many hypotheses cannot be considered while

OCR for page 61
MOVING FORWARD 69 o ­ thers are easily accommodated. He urged NCS leadership to pay atten- tion to the views expressed during the workshop. Naihua Duan (Columbia University) asked about the plan not to proceed with the household screening sample to collect preconception measures. He noted his purpose in raising the issue is for brainstorming. One message he heard from the first panel was the importance of early pregnancy or maybe even preconception data. Some of those data could be captured in the unified sampling approach but might not be captured. He said he wondered what could be obtained from the prenatal sample relative to what potentially could be accomplished with a household sample. Even though the household sample has been found to be too expen- sive, Duan noted that many at the workshop commented on the impor- tance of early pregnancy and preconception measures. This suggests that recruitment costs are not the only costs to consider; total lifetime costs for the study are important as well. He noted several other panelists com- mented that the difference in recruitment costs might not be that large when combined with follow-up costs. Part of the high cost of recruitment of preconception women is following women who never become preg- nant. To reduce these costs, one solution might be to limit the preconcep- tion sample to women who are actively trying to get pregnant. He said there may be a benefit to retaining some household sample (perhaps in hot spots, or other geographic areas of interest) in order to answer impor- tant scientific questions. Duan agreed with Irwin Garfinkel and the other panelists that the sibling sample has much merit but also some limitations. Questions about the first-born and what might not be covered in the sibling sample are not trivial. One advantage of the household sample, if it could be retained in some affordable way, would be to fill that gap. Sondik agreed, saying a comparison of the characteristics of house- hold sampling versus provider-based sampling in terms of data that could be collected might provide a useful basis for evaluating each method and understanding in what circumstances it would be useful. Kalton commented that he perceived that the participants understood the value or potential value of the NCS. In terms of costs, he said there is also an issue of political will. He said political will helps with this study because both political parties understand the study well enough to sup- port it. He urged making sure the NCS is a good strong study that is properly implemented. Steven Hirschfeld (National Children’s Study director) thanked par- ticipants for a very informative and stimulating discussion, saying that he and others will continue the evolution of the design of the National

OCR for page 61
70 DESIGN OF THE NATIONAL CHILDREN’S STUDY Children’s Study. He said its potential is well appreciated, and the goal of the NCS is to have that potential not only realized to meet current expec- tations but also to have a platform that would exceed expectations in the future so it becomes an ongoing resource for informing about the health and development and growth of children.