The National Children's Study Research Plan: A Review (2008)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Executive Summary O n October 17, 2000, Congress enacted the Children’s Health Act (Public Law 106-310). Section 1004 of the act “authorize(s) the National Institute of Child Health and Human Development to conduct a national longitudinal study of environmental influences (includ- ing physical, chemical, biological, and psychosocial) on children’s health and development.” In response to this act, the National Institute of Child Health and Human Development (NICHD), in cooperation with the Cen- ters for Disease Control and Prevention, the U.S. Environmental Protection Agency, and the National Institute of Environmental Health Sciences, is planning to conduct the National Children’s Study (NCS). This report is a review and assessment of the research plan for the NCS (National Institute of Child Health and Human Development, 2007). The NCS is planned to be the largest long-term study of environmental and genetic effects on children’s health ever conducted in the United States. It proposes to examine the effects of environmental influences on the health and development of approximately 100,000 children across the United States, following them from before birth until age 21. It defines environ- ment broadly by including a wide array of measures of biological, chemical, physical, genetic, social, psychological, cultural, geographical, and other factors in a child’s environment that can affect health and development. It proposes to examine the relationship between many different exposures and many different outcomes. By archiving all of the data collected, the NCS is intended to provide a valuable resource for analyses conducted many years into the future.  

 THE NATIONAL CHILDREN’S STUDY RESEARCH PLAN NICHD requested the Committee on National Statistics of the National Research Council (NRC), in collaboration with the Board on Children, Youth, and Families of the NRC and the Institute of Medicine (IOM) and the IOM Board on Population Health and Public Health Practice, to con- duct a review of the research plan for the NCS. The purpose of the review is to assess the scientific rigor of the NCS and the extent to which it is being carried out with methods, measures, and collection of data and specimens to maximize the scientific yield of the study. The panel concludes that the NCS offers an excellent opportunity to examine the effects of environmental influences on child health and devel- opment, as well as to explore the complex interactions between genes and environments. If the NCS is conducted as proposed, the database derived from the study should be valuable for investigating hypotheses described in the research plan as well as additional hypotheses that will evolve. Nev- ertheless, there are important weaknesses and shortcomings in the research plan that diminish the study’s expected value below what it might be. This Executive Summary provides a brief overview of our assessment of the study’s strengths and weaknesses; the box at the end of this summary lists the panel’s recommendations for improvements to the study. Although we recognize that their implementation may raise issues of added cost and response burden, we urge that they receive serious consideration. STRENGTHS If the NCS is conducted as proposed, its strengths would include: 1. Responsiveness to the Children’s Health Act of 2000 The stated goals for the NCS, and the design of the NCS for achieving those goals, broadly reflect the stipulations of the Children’s Health Act. 2. The large number of births to be included 100,000 births would provide enough statistical power to examine many hypothesized relations that cannot be investigated with smaller samples. 3. The longitudinal design stretching from before birth until age 21 A data set that contains data gathered prospectively over the entire course of pregnancy, childhood, adolescence, and early adulthood will enable many new life-cycle relations between exposures and outcomes to be investigated. Data gathered prospectively (or with relatively short retrospective periods) should be more precise than data that are based on long periods of recall. A particularly attractive feature of the study is the effort to recruit births before conception and during very early periods of gestation, when certain environmental exposures may prove to be critically important. 4. The many variables to be generated on both outcomes and expo- sures The enormous array of social, psychological, biological, chemical, and 

EXECUTIVE SUMMARY  physical measures that will be generated under present plans will permit investigation of relationships that have not previously been studied. Some of these relationships are included among the study’s hypotheses, but fortu- itous and unanticipated findings can also be expected. At the same time, the large number of variables increases the risk of establishing “false positives,” that is, relationships that appear to be statistically significant but have in fact been generated by chance. In addition to the variables generated by the study itself, many ancillary variables on characteristics of participants’ communities from readily available sources, such as the decennial census, surveys, and administrative records, can be appended to the data set to augment the analytical usefulness of the study. 5. The well-designed national probability sample The births selected for the NCS will be identified from a probability sample of households cho- sen with standard and well-justified sampling techniques. The use of estab- lished random selection methods at each sampling stage will ensure that the NCS samples of households, eligible women of childbearing age, and births are national probability samples. We endorse the study’s decision to use probability sampling without oversampling any groups. Statistical power could have been increased for any particular investigation by implementing a different sampling design. But modifying the sampling scheme to better address any particular relationship would often have reduced the power for investigating other relationships. The possibility of adding more precisely targeted studies—when resources and respondent burden permit—has been appropriately built into the research design. Nevertheless, it is important to note that the sample size and sampling scheme of the study represent a compromise and are not designed to address any single hypothesis. WEAKNESSES AND SHORTCOMINGS 1. Absence of an adequate pilot phase A principal shortcoming of the NCS, as planned, is the absence of a pilot phase. The study design is extremely complex in terms of identifying subjects, enlisting their enroll- ment and continued participation, administering the very large number of survey and clinical instruments, and managing huge databases generated by disparate organizations. In addition, we raise many questions about the instruments that have been chosen and about the timing of their applica- tion. We think that, if the study is to achieve its promise, experimentation is needed with respect to methods to increase response rates and data instru- ments. Many of the concerns that we raise about the research plan could be addressed in a pilot phase. Data-gathering will begin at the seven centers designated by NICHD as Vanguard Centers a year earlier than elsewhere. The data that will be generated in Vanguard Center sites are expected to become part of the final 

 THE NATIONAL CHILDREN’S STUDY RESEARCH PLAN data set, meaning that experimentation is likely to be kept to a minimum. Moreover, the lead time between the start of data collection at the sites in the Vanguard Centers and elsewhere appears too short to take full advan- tage of the lessons that will be learned. The absence of a pilot phase is a serious shortcoming that could be at least partially addressed by increasing the delay between data-gathering in the Vanguard sites and elsewhere and by treating the Vanguard sites in a more experimental fashion. A delay for the enrollment phase of the study would also allow time to consider more fully the appropriate conceptual framework and specification of hypotheses and measures for the study. 2. Decentralization of data collection Data collection will be the di- rect responsibility of 35-40 different study centers. While the study centers will be supervised by government officials and the coordinating center, they will individually contract with outside agencies for data collection. This unusually decentralized data collection strategy reduces the chances that data will be of uniformly high quality over the life of the study and sharply increases the burden of supervision. More centralized and conventional models, such as that employed by the large National Longitudinal Study of Adolescent Health, appear more likely to produce high-quality data. Given that the decentralized approach could not readily be altered at this stage because contracts already have been signed, it will be incumbent upon the government to ensure that staff and other resources are sufficient to closely monitor data collection activities and take prompt remedial steps as neces- sary. A detailed and specific contractual plan is essential for this purpose. 3. Inadequacy of plans to maximize response rates and retention rates The success of the study will depend critically on the initial survey response rates and the subsequent rates of sample attrition. The NCS re- search plan does not explicitly address the best methods and procedures for achieving the ambitious baseline response rates that are targeted. The importance of efforts to increase initial response rates by dealing rapidly with underperforming sites cannot be overstated. Maintaining the represen- tativeness of the sample over time is key to the quality of the results. Little is said in the research plan about how the study expects to maximize retention of sample cases. The risk of sample attrition is especially great in a study such as the NCS that targets children and young adults, the most mobile segments of the population. Typically, the largest loss to follow-up occurs in the early stages of a longitudinal study. Ascertaining the best methods to increase initial response rates and to reduce attrition rates is a matter of great urgency for project management. 4. Weakness of conceptual model The research plan does not define the basic concepts of health and development. While mentioning them, it frequently defaults to a deficit model that focuses on disease and im- pairment and the risk factors that contribute to them, rather than on the 

EXECUTIVE SUMMARY  factors that encourage healthy development. Late additions to the set of hypotheses that are addressed to healthy development are not well specified. Consequently, there is an imbalance of hypotheses, with specific hypotheses about disease conditions and vague hypotheses about social environments and children’s intellectual and social development. The same imbalance ap- pears in the measures selected. Little attention has been paid to outcomes in later childhood and adolescence that might have encouraged attention to additional or alternative exposures. 5. Weakness of certain data instruments The success of the study depends critically on the quality of the data instruments to be employed for assessment of environmental exposures and child outcomes. Our review revealed gaps, uncertainties, and insufficient rationales for a substantial number of instruments. Among the measures that appear most appropriate are those related to asthma and to biological exposures such as allergens and maternal physical activity. The list of proposed variables about which we have raised questions includes: birth defects, pregnancy outcomes (in- cluding fetal death), child mental health disorders, maternal depression, brain injuries, reproductive development outcomes, nonpersistent organic chemicals, pesticides, childhood infections, and the social environment in the home. The set of psychosocial measures selected appears particularly problematic. We hope that the issues we have raised about these measures will encourage a reconsideration and fresh articulation of the bases on which they were chosen, including the introduction of additional measures as substitutes or supplements when appropriate. We also suggest that ef- forts to assess the validity and reliability of the most problematic measures be made by the Vanguard Centers for their sites. Even when suitable measures have been selected, the timing of their deployment—as proposed in the research plan—will leave large gaps in the measured trajectories of child health and development. These gaps, particu- larly with respect to in-person home or clinic visits, will make it difficult to identify critical periods of exposure to various environmental agents. The period that will be most successfully studied is that of pregnancy, birth, and the first year of life because that is the period of most intense observation. At older ages, we urge more frequent measurement of key variables, at least for a subsample of the NCS participants. 6. Insufficient attention to racial, ethnic, and other disparities The Children’s Health Act asks the study to “consider health disparities among children,” a phrase that typically directs attention to racial and ethnic disparities and can also include language, socioeconomic, and geographic area disparities. While the study will gather a great deal of information that is relevant to understanding such disparities, the research design was not informed by a concern with understanding their basis. The absence of virtually any hypotheses about racial and ethnic disparities is striking. In 

 THE NATIONAL CHILDREN’S STUDY RESEARCH PLAN particular, there is no attention to generating data on how individuals from different groups may interact with health systems, a factor whose impor- tance has been suggested in many previous studies, nor on psychosocial experiences that differentiate among population groups. 7. Failure to adequately integrate data from medical records The use of health services is an important variable intervening between exposure and outcome. Given a particular exposure to environmental conditions, the quality of the medical services that are employed can make a large dif- ference in outcomes. High-quality information about the use of medical services would also help to address major questions in health policy. These would include, as just noted, the potential sources of racial and ethnic disparities in health outcomes. Data on the use of health services will be derived almost exclusively from retrospective reports of parents, which are often unreliable. While they may be expensive to collect and mobilize and are themselves imperfect, records from physicians and hospitals would provide an extremely valuable and sometimes indispensable supplement to parents’ reports. Medical records could also play an important and often central role in clarifying diagnoses and identifying patterns of child develop- ment. We urge that greater attention be paid to incorporating such data. 8. Failure to plan adequately for disclosure of risk to participants As soon as data collection begins, the NCS will face questions about the circumstances under which information about a child’s health and develop- ment, as well as his or her exposure to toxic agents, should be conveyed to participants. The study plans to provide information on conditions that are “clinically relevant and actionable,” but this is not a conventional concept and needs to be defined and made operational. There is insufficient detail in the research plan about how decisions will be made about what to disclose. Some of the decisions—for example, regarding transmitting information about fetal defects encountered through ultrasounds—are urgent to make. Clearer plans must also be developed regarding what parents and children need to be told about emerging research findings. 9. Failure to plan for rapid dissemination of data We think that the present plan is unwise in reserving a period of time for researchers associ- ated with the data collection phase of the study to have exclusive access to its data for analytic purposes. Such a practice slows the advance of science and violates increasingly widespread norms, including those prominent within the National Institutes of Health itself. We urge wide and rapid dissemination of the data produced by the study. The data will be used for many analytic purposes that cannot presently be anticipated. The NCS is a national study paid for by public dollars, and we think that the data and results should be made accessible to the entire research community (with appropriate protections to preserve confidentiality) as soon as practicable. 

EXECUTIVE SUMMARY  Achieving rapid data dissemination will require early and elaborate orga- nizational and budgetary planning. CONCLUDING OBSERVATION It is clear from our review that the NCS offers not only enormous potential, but also a large number of conceptual, methodological, and administrative challenges. In addition, funding uncertainties make it dif- ficult to plan beyond the relatively short period for which funds have been appropriated. Like the scientists associated with the study itself, we are eager for it to succeed. We present our recommendations in the hope that, as it goes forward, the NCS will achieve its intended objective to examine the effects of environmental influences on the health and development of American children. Recommendations The panel presents recommendations in 23 areas. The recommenda- tions are presented in the box grouped according to these areas, and keyed to the chapter in which they appear in the body of the report. 

 THE NATIONAL CHILDREN’S STUDY RESEARCH PLAN Goals Recommendation 2-1: The NCS should give priority attention to seeking ways to bolster the ability of the study to contribute to understanding of health disparities among children in different racial, ethnic, and other population groups, including the reestablishment of a working group to oversee this area and the encourage- ment of appropriate adjunct studies. Recommendation 2-2: The NCS should seek resources and develop methods to obtain more frequent in-person measures and medical and other administrative records data on study participants. Conceptual Framework Recommendation 2-3: The NCS should clearly define the key constructs of child health and development and more fully develop a conceptual framework for understanding child health and development over the course of infancy, childhood, and adolescence. Using the Vanguard Centers as Pilots Recommendation 2-4: We strongly urge the NCS to delay enrollment at new sites to make effective use of initial findings from participant enrollment and data col- lection in the Vanguard Center sites to improve study procedures, as appropriate, and to refine key concepts, hypotheses, and measures of outcomes and expo- sures. Throughout the life of the study, the NCS should use the Vanguard Centers to pilot test and experiment with data collection methods and instrumentation. Pregnancy Outcomes Recommendation 3-1: The NCS should consider replacing research on sub- clinical maternal hypothyroidism as a factor in adverse pregnancy outcomes with research on the effects of a broader set of maternal physical and mental health conditions, such as maternal depression, maternal perceived stress, and maternal periodontal disease. Recommendation 3-2: The NCS should develop refined, detailed protocols for investigating all pregnancy outcomes, specifically a detailed protocol for obtaining information on various types of pregnancy loss, before beginning data collection at the Vanguard Centers, given that pregnancy outcomes are among the first outcomes to be examined; many outcomes lack clarity in measurement; and there are important questions regarding the adequacy of statistical power and the planned data collection (for example, the need for prepregnancy measurements of some exposures). 

EXECUTIVE SUMMARY  Neurodevelopment and Behavior and Child Health and Development Recommendation 3-3: The NCS should develop a clearer rationale for the selec- tion of specific neurodevelopment and behavior disorders to be considered in the study and a clearer conceptual basis for the assessment of normal child health and development trajectories and outcomes. Clarity is needed to guide the choice of outcome measures and exposure measures and the frequency and types of contacts (at the home, in clinics) with study participants in order to obtain the best information possible within resource and burden constraints. Asthma Recommendation 3-4: The NCS should develop a clearer rationale for its hy- potheses about factors that may increase the incidence of asthma. These should focus on prenatal and early life risk factors. Obesity and Growth Recommendation 3-5: The NCS should reevaluate its main hypotheses to be addressed in the study of childhood obesity and consider adopting a broader approach that incorporates social and psychological factors as well as biogenetic ones. Such an approach would help the study identify the constellations of key factors and their interrelationships that are important to understand in order to develop the most effective public health measures to reduce childhood obesity. Injury Recommendation 3-6: The NCS should consider replacing research on repeated mild traumatic brain injury (rMTBI) with more nuanced research on other injury- related topics, such as environmental factors in childhood injuries and the effects of clinical response to injury (treatment or nontreatment). Hormonally Active Agents and Reproductive Development Recommendation 3-7: The NCS should develop refined and detailed protocols for studying reproductive development outcomes, which, as presented in the research plan, often lack clarity in measurement and research design. Outcomes that are measured at birth for which there is little time to refine research proto- cols require immediate attention. The NCS should use results from the Vanguard Centers, such as estimates of the prevalence of specific reproductive development outcomes, to assist in protocol development, and it should consider the usefulness of substudies of high-exposure population groups. 

10 THE NATIONAL CHILDREN’S STUDY RESEARCH PLAN Demographic and Socioeconomic Measures Recommendation 3-8: The NCS should add to its well-planned battery of de- mographic and socioeconomic measures questions on immigrant generation, languages spoken, and, if possible, the legal status of the parents and child. Chemical Exposure Measures Recommendation 3-9: The NCS should consider the use of personal air sam- pling methods for a subsample of participating women and their children for measuring exposure to air pollutants. Recommendation 3-10: The NCS should incorporate methodology to measure paternal exposure to environmental chemicals (both persistent and nonpersistent). More generally, the NCS should consider collecting for fathers, not only chemical exposures, but also biological samples and interview data on paternal character- istics that may affect children’s health and development to the same degree as it collects such information for mothers. Physical Exposure Measures Recommendation 3-11: The NCS should provide a clearer rationale for some of the housing and neighborhood conditions it proposes to measure and revisit its data collection plans to ensure that needed measures are obtained at devel- opmental stages when children may be more vulnerable to risk factors. The goal should be a set of measures and data collection plans that are optimal with regard to analytic utility and response burden. Psychosocial Exposure Measures Recommendation 3-12: The NCS should reconsider its psychosocial measures to ensure that they will provide high-quality data for outcomes of interest for child health and development. In the face of resource and respondent burden con- straints, the NCS should emphasize the quality and analytic utility of information, even if some measures must be dropped in order to substitute other assessments more desirable on various grounds. Recommendation 3-13: The NCS should dedicate a portion of funds to support research and development of reliable and valid instruments of key psychosocial measures that are practical and economical to administer. Biological Exposure Measures Recommendation 3-14: The NCS should review some of the proposed measures of biological exposures, such as maternal glucose metabolism and child cortisol levels, to ensure that the proposed times for data collection are appropriate for capturing the underlying exposure. 

EXECUTIVE SUMMARY 11 Genetic Measures Recommendation 3-15: The NCS should adopt a clear mechanism by which ge- netic association studies are internally and, optimally, externally validated before any results are published or released to the media. The NCS should also revise its proposed “established” candidate gene approach to take advantage of the new information emanating from the current wave of genome-wide association studies, with appropriate replication. Recommendation 3-16: The NCS should consider consolidating its genetics studies in order to reduce costs and to coordinate the best science at the least cost to the project. One approach would be to simply collect the biological samples and properly store them for later genetic analysis when a better selection of poly- morphisms and cost-effective genotyping across studies are possible. Missing Exposures Recommendation 3-17: The NCS should add measures of access to and qual- ity of services, including medical care, education, child care, and services, as potential mediators of health and development outcomes and to improve the as- sessment of information obtained through maternal reports. Data Linkage Recommendation 3-18: To facilitate linkages of NCS data with environmental exposures from other databases, such as measures of demographics, crime, government programs, and pollution, the NCS should develop a plan for geocod- ing the residential addresses from prebirth through adulthood of all participating children to standard census geographic units. In addition, the study should de- velop arrangements by which researchers, both inside and outside the NCS study centers, can access geocodes for respondent addresses and are encouraged to perform linkages and make their environmental information available to the NCS analysis community. Such arrangements must safeguard the confidentiality of NCS respondents. Sampling Design Recommendation 4-1: The NCS should consider modifying the sampling design to allow for flexibility in increasing the number of study participants in the event that the estimated number of screened households needed to reach 1,000 births per primary sampling unit (PSU) is incorrect. Recommendation 4-2: The NCS should consider the proposed household enu- meration approach to be experimental and should conduct carefully designed field studies to clearly establish the statistical and practical implications of the proposed adjudicated listing approach. 

12 THE NATIONAL CHILDREN’S STUDY RESEARCH PLAN Recommendation 4-3: To ensure a diverse exposure profile in the sample, the NCS should consider a careful assessment of variation in ambient exposure to chemical agents within each PSU. If the set of segments in a PSU can be classified by combined exposure to a group of important chemical agents, this information could then be used to form varying exposure-level strata for segment sampling in each PSU and thus ensure a range of ambient exposure to relevant environmental agents. Data Collection Recommendation 4-4: The NCS should consider ways in which the survey data collection could be consolidated into a smaller number of highly qualified survey organizations. Recommendation 4-5: Because of the complexity of the proposed organizational model for data collection and the difficulty of maintaining the quality and uniformity of data collection procedures across a large number of study sites, the NCS pro- gram office should establish and monitor strict standards for enrollment, retention, and data collection at each of the study sites and be prepared to take immediate corrective action if sites do not meet high-quality standards in data collection. Recommendation 4-6: The NCS should prepare a plan for monitoring progress of the study in reaching its sample size goals. As part of the plan, the NCS should take advantage of the experience of the Vanguard Centers to evaluate initial en- rollment rates, the effectiveness and potential respondent burden of the interview instrument, and the ability of the Vanguard Centers to obtain the required house- hold environmental measures reliably. Recommendation 4-7: To resolve issues that arise during data collection, the NCS should set aside sufficient resources to maintain an ongoing program of methods research and field experimentation. Among the issues that might be addressed in this research are the reliability and validity of previously untested survey questions and measurement strategies, the effectiveness of sample reten- tion procedures, predictors of response outcomes associated with sample initial recruitment and subsequent annual retention, error implications of unit nonre- sponse, adjustment strategies for unit nonresponse, and methods for dealing with item nonresponse. Data Analysis and Dissemination Recommendation 4-8: The NCS should begin planning for the rapid dissemi- nation of the core study data, subject to respondent protection, to the general research community and for supporting the use of the data after dissemination. The costs of implementing this plan should be estimated and set aside in future NCS budgets. Dissemination includes not only the publication of findings through reports and scientific papers and the production of documented data files for re- searchers, but also active support in the use of NCS data by the broadest possible range of qualified investigators. 

EXECUTIVE SUMMARY 13 Criteria for Giving Information to Participants Recommendation 5-1: The NCS should define the criteria and the process for deciding what individual clinical and research information, such as environmental assessments, test results, and survey scales, will be given to children and their families. Protection and Release of Information Recommendation 5-2: NCS and non-NCS investigators should be given equal access to the full NCS data as soon as they are cleaned and documented. To protect respondent confidentiality, all analyses should be performed with the kind of strict safeguards employed by the Census Bureau research data centers. Community Engagement Recommendation 5-3: The NCS should engage communities in selected study implementation, data analysis, and data interpretation activities that go beyond recruitment. The NCS should consider requiring every study center to formulate a more detailed plan to engage and collaborate with local communities.