Previous chapters, particularly Chapter 4, outline the need for an enhanced approach to science and technology in the US Environmental Protection Agency (EPA) that recognizes the challenge of characterizing and preventing effects on human health and ecosystems in the context of complex systems. With the development of new tools and approaches to collecting and processing large amounts of environmental and health data and for characterizing effects when knowledge is uncertain, it is imperative that a new way of thinking—embodied in the concepts of science that anticipates, innovates, takes the long view, and is collaborative—be integrated into scientific processes in EPA’s Office of Research and Development (ORD) and across its national research program areas.
In the United States, environmental management is conducted through a mosaic of federal, state, and local activities in multiple federal and state agencies, often through regionally distributed offices. Environmental decisions are made at multiple administrative levels in those agencies. Science questions arise throughout that environmental-management network and require access to the latest and best scientific information possible. In EPA’s program and regional offices, science is most often conducted in direct response to particular regulatory and programmatic needs and often operates on different timescales in contrast with longer-term discovery-oriented science in ORD. Efforts to enhance EPA science for the 21st century should not focus only on ORD but should incorporate efforts, resources, expertise, and scientific and non-scientific perspectives in program and field offices. Such efforts need to support the integration of both existing and new science throughout the agency; avoid duplication or, worse, contradictory actions; respect different sets of priorities and timeframes; and advance common goals. EPA also engages in activities to deliver science and provide decision support to nonfederal entities (for example, states and tribes), and decreasing budgets of tribal, state, and local environmental agencies
will make this function increasingly important. At the same time, EPA is itself increasingly resource constrained. As noted in Science Integration for Decision Making at the US Environmental Protection Agency, since 2004, the budget for ORD has declined 28.5% in real dollar terms (gross domestic product-indexed dollars) (EPA SAB 2012a).
To support enhanced leadership and to continually improve environmental science and engineering for the 21st century, the committee identified six key topics:
• Enhance agency-wide science leadership.
• Fully implement the recent restructuring of ORD.
• Coordinate and integrate science efforts within the agency more effectively.
• Strengthen scientific capacity inside and outside the agency.
• Deliver and support 21st century environmental science and engineering outside the agency.
• Support scientific integrity and quality.
Emerging challenges in ecosystem quality and human health necessitate the enhancement and broader use of science in the agency. The environmental challenges outlined in Chapter 2, such as climate change and degradation of surface waters from mixtures of contaminants, share many characteristics—they are transboundary, are multigenerational, and involve complex interactions of multiple stressors and feedback loops. They are affected by population growth, changes in land-use patterns, and technologic change. They constitute wicked problems—that is, problems that are difficult to characterize and to solve because of their complexity; lack of comprehensive understanding; controversy over causes, effects, and solutions; and interdependence. The rapidly emerging scientific techniques and approaches and their application described in Chapter 3 offer both opportunities and challenges for enhancing the science that EPA produces and applying it to the increasingly complex decisions that are necessitated by wicked problems.
The agency has shown an ability to evaluate new tools and integrate them into its activities in some instances, as described in Chapter 3 and Appendixes C and D, although the process has not been systematic or agency-wide. Also, the agency has made strides in recent years to reorganize and reorient its science activities in ORD with some success. The work of ORD scientists is often the most visible, and at times controversial, scientific interpretation and application in the agency. However, more than three-fourths of the scientific staff in EPA do not work within ORD (EPA SAB 2012b); these scientists are frequently placed in positions where they must apply and interpret science for equally controver-
sial decisions and must be able to access and understand the latest scientific techniques and approaches. There has been progress toward agency-wide science integration with the establishment of the Office of the Science Advisor, and further progress might be made with the shift of the science advisor position from within ORD to the Office of the Administrator in early 2012; however, the Office of the Science Advisor may need further authority from the administrator or additional staff resources to continue to improve the integration and coordination of science across the programs and regions throughout the agency.
As discussed in several places in this report, EPA has made important progress in human health and environmental science and engineering over the last few decades, and the environment is better today because of that progress. However, as the committee reviewed emerging challenges and scientific tools and evaluated the capacity of the agency to respond, the need for substantially enhanced science leadership throughout the agency became clear. When the committee speaks of enhanced science leadership, it is not just referring to the strengthened capacity of someone in a high-level position within EPA to whom the administrator has provided independence, authority, and resources, but also the internal support at all levels in the agency (including scientists, analysts, directors, and deputy and assistant administrators) to ensure that the highest-quality science is developed, evaluated, and applied systematically throughout the agency’s programs.
At least four independent reports in the last 20 years (EPA 1992, NRC 2000, GAO 2011, EPA SAB 2012b) have, on the basis of their own analyses recommended enhanced science leadership. Some of the specific recommendations included the need for the position of deputy administrator for science with sufficient resources and authority to coordinate scientific efforts in the agency (as noted above) and to build collaboration with external agencies and expertise; the establishment of an overarching issue-based planning process and a scientific agenda for major environmental issues that integrates and coordinates scientific efforts throughout the agency and that is regularly reviewed and updated; a coordinated approach to managing and strengthening EPA’s scientific workforce that will serve as a resource for the entire agency; and a strategy that promotes science integration by making it a more consistent priority, by strengthening management oversight, and by strengthening participation and support of EPA scientists. Most recently, the EPA Science Advisory Board (SAB) noted that
Narrow interpretations of legislative mandates and the organizational structure of the EPA’s regulatory programs have posed barriers, in many cases, to innovation and cross-program problem solving. EPA managers and staff in many interviews, especially in program offices, defined the success of their programs in terms of meeting statutory requirements and court-ordered deadlines. Although meeting legal mandates is essential, the EPA needs a broader perspective that extends beyond specific program objectives to achieve multiple environmental protection goals, including
sustainability. A narrow focus on “program silos” and defensibility can be a barrier to formulating and responding to problems as they occur in the real world. Such a limited approach can hinder integration of new scientific information into decisions and new applications of science to develop innovative, effective solutions to environmental problems (EPA SAB 2012b, p. 5).
In the committee’s analysis of the strengths and limitations of an enhanced agency-wide leadership position, it has concluded that successful implementation of the systems-based application of emerging tools and technologies to meet persistent and future challenges cannot be achieved under the current structure. Success will require leadership throughout the agency, in the programs and regions as well as in ORD. There will need to be clear lines of authority and responsibility, and regional administrators, program assistant administrators, and staff members at all levels will need to be held accountable for ensuring scientific quality and the integration of individual science activities into broader efforts across the agency.
Finding: The need for improvement in the oversight, coordination, and management of agency-wide science has been documented in studies by the National Research Council, The Government Accountability Office, and the agency’s own SAB as a serious shortcoming and it remains an obstacle at EPA. The committee’s own analysis of challenges and opportunities for the agency indicates that the need for integration of systems thinking and the need for enhanced leadership at all levels is even stronger than it has been in the past.
Recommendation: The committee recommends that the EPA administrator continue to identify ways to substantially enhance the responsibilities of a person in an agency-wide science leadership position. That person should hold a senior position, which could be that of a deputy administrator for science, a chief scientist, or possibly a substantially strengthened version of the current science advisor position. He or she should have sufficient authority and staff resources to improve the integration and coordination of science across the agency. If this enhanced leadership position is to be successful, strengthened leadership is needed throughout the agency and the improved use of science at EPA will need to be carried out by staff at all levels.
Whatever administrative arrangement is adopted, the following are suggestions of the types of responsibilities that the committee thinks should be associated with this position:
• Chairing and assuring that the work of the Science and Technology Policy Council is comprehensive and effective.
• Promotion of systems thinking and systems-oriented tools to address complex challenges ahead and the integration of this approach into every aspect of agency science and engineering (as described in Chapter 4).
• Ensuring that the scientific and technical staff throughout the agency (including program, regional, and research offices) have the expertise necessary to perform their duties whether in support of the agency’s research or in support of its role as a regulatory and policy decision-maker.
• Ensuring that the agency has in place a system for quality assurance and quality control of its scientific and technical work (including a system for consistent high-quality peer review).
• Ensuring that the best available scientific and technical information is being used to carry out the agency’s mission.
• Working to coordinate research and analytic efforts within and outside the agency to ensure that the best information is used in the most efficient manner.
• Encouraging and supporting interoffice and interagency science collaboration in order to solve problems and develop good solutions.
If the occupant of the position is to be successful, he or she will require sufficient staff and resources to act on behalf of the administrator to implement a coordinated budget and strategic planning process of the regional, program, and research offices to ensure that appropriate scientific and technical expertise and capabilities are available and used. The person in this position would also oversee the policies and procedures related to the operation of the agency’s federal advisory committees. The committee specifically recommends that the person in this position and his or her staff create, implement, and periodically update an integrated, agency-wide multiyear plan for science, its use, and associated research needs. Such a plan would bring together ORD, program, and regional science initiatives while being cognizant of the flexibility that is imparted through bottom-up initiatives undertaken in ORD, the program offices, and the regions.
The strengthening of science leadership is not without its challenges. For example, whether or not the position is held by a political appointee could affect the ability of the person in the position to be effective throughout the agency, especially with the other political appointees who head the programs that rely on science (and supervise many of the agency’s scientists). There is also the possibility that new procedures established from the central administration could serve to discourage innovation in science if not carefully applied. To a certain extent, the recent EPA decision to re-establish the position of science advisor as a non-political position distinct from ORD (as had been the case in earlier EPA administrations) will provide a test of how to overcome some of these challenges. However, the revised role of the current science advisor does not fully implement the committee’s recommendation unless that person is empowered
with the tools and support described above. Even with the full support of the administrator and senior staff, the effort will fail if the need to improve the use of science in EPA is not accepted by staff at all levels.
ORD often sets the stage for research and scientific assessment efforts throughout EPA. In 2011, the deputy administrator for ORD, Paul Anastas, announced a restructuring of the office in response to growing scientific challenges and recommendations from the agency’s scientific advisers. The SAB called for integrated transdisciplinary research at ORD, stating that “it will be essential for EPA as a whole, and not just ORD alone, to adopt a systems approach to research planning. It will also be essential to plan and conduct research in new, integrated, and cross-discipline ways to support this systems approach” (EPA SAB 2010). The ORD restructuring aims to
• Align ORD’s research with the agency’s strategic goals.
• Reorient ORD’s research to be guided by the concept of sustainability.
• Promote systems thinking and innovation.
• Couple excellence in problem assessment with excellence in solving problems.
• Encourage integrated, transdisciplinary research among ORD labs and through external funding.
The realignment consolidates 13 previous research sectors into four cross-cutting sectors of research and two overarching sectors, as shown in Table 5-1.
In October 2011, SAB and ORD’s Board of Scientific Counselors (BOSC) published a review of ORD’s structure (EPA SAB/BOSC 2011). SAB and BOSC noted the “impressive increase in transdisciplinary collaboration as well as coordination across ORD programs with the restructuring.” They also made note of ORD efforts to think about innovation operationally as a fundamental aspect of ORD research. SAB and BOSC gave ORD particular credit for having involved regional and program offices in designing the realignment and for giving serious consideration to ways of encouraging creativity among ORD scientists and engineers (EPA SAB/BOSC 2011).
Several key conclusions emerged from the SAB and BOSC review, including suggestions that
• EPA ensure that financial and staff resources are adequate to implement the restructuring and are secured to sustain the communication, stakeholder involvement, and integrated transdisciplinary collaboration that will be essential for its success.
|Former ORD Research Structure||Integrated ORD Research Structure|
|Global Change Research||Air, Climate & Energy|
Clean Air Research
|Human Health and Ecosystems Research||Safe and Sustainable Water Resources|
|Drinking Water Research
Water Quality Research
|Human Health and Ecosystems Research|
|Pesticides & Toxics Research
|Sustainable and Healthy Communities|
Land Research (Excluding Nanotechnology)
|Endocrine Disrupting Chemicals Research
Computational Toxicology Research
Human Health & Ecosystems Research
|Chemical Safety for Sustainability|
|Human Health Risk Assessment (NexGen)|
|Pesticides & Toxics Research
Land Research (Nanotechnology)
|Clean Air Research (Nanotechnology)
|Human Health Risk Assessment
|Human Health Risk Assessment
Source: Teichman and Anastas 2011.
• EPA continue to refine its implementation plans to ensure that the restructuring takes root. The agency needs to define clearly how ORD and program office research programs relate to one another and how they fit within the larger context of EPA and stakeholder science. A key aspect is ensuring that senior and junior scientists in ORD and the program offices are invested in the restructuring process.
• EPA develop clear metrics for the evaluation of progress of research divisions and their ability to respond to environmental challenges in a new and more solutions-oriented way. The long-term sustainability of the revised structure (in time and through administrations) will depend on the degree to which the agency can demonstrate that the reorganization leads to better science and better outcomes.
• ORD maintain close communication and working relationships with program offices to ensure that research in the agency continues to support programmatic needs. Regional and program offices should be engaged in evaluating ORD’s progress and performance.
The importance of delivering science to EPA decision-makers and supporting the scientific capacities and endeavors of program and regional offices is well-recognized in the agency. The agency should use scientific information in all its decisions. Science needs for decisions are identified within program and regional offices through various processes and can take two main forms-summaries and syntheses of existing science and the creation of new science to fill key gaps.
Existing science to inform and support decisions is usually acquired by EPA scientific staff (through a combination of professional networks and electronic tools). ORD’s Office of Science Policy (OSP) is charged with integrating and communicating scientific information that comes from or that supports ORD’s laboratories and centers (EPA 2012a). OSP’s Regional Science Program links ORD science to regional offices. The Regional Science Program’s Regional Science Liaison and Superfund and Technology Liaison locate scientists in regional offices to facilitate regional staff and management access to ORD science. The regional liaisons have regular communication with OSP to ensure communication between ORD and the regional offices (M. Dannel, EPA, personal communication, December 30, 2011). The EPA SAB Committee on Science Integration for Decision Making found that regional offices consider the liaisons to be important in science acquisition (EPA SAB 2012b).
OSP plays a key role in connecting program and regional offices to ORD research and in expanding the capacity of regional offices to conduct needed research. For a few programs, most notably several programs in the Office of Pesticide Programs, needed research can be required of regulated entities. However, that option is not available to most programs, and those programs and regions rely to various degrees on inhouse research. At the regional level, there are several mechanisms through which new science is supported. For example, the Regional Applied Research Effort Program, which allocates about $200,000 per year to each EPA region for collaborative research, funds near-term research (1-2 years) on high-priority, regional applied-science needs. It is also intended to foster collaboration between EPA regions and ORD laboratories and centers, to build a network between regions and ORD for future scientific interaction, and to provide opportunities for ORD scientists to apply their expertise to regional issues and explore new research challenges. The Regional Methods Program, for which about $600,000 per year is allocated, works to develop new monitoring and enforcement methods (EPA 2012b). It is analogous to the Regional Applied
Research Effort Program in that it provides the regions with near-term research support on high-priority, region-specific science needs and improves collaboration between regions and ORD laboratories and centers (EPA 2008). An example is EPA Region 8, where scientists used support from the Regional Methods Program to collaborate with EPA in developing a vitellogenin gene-induction method to produce a marker of exposure to endocrine disrupting chemicals (Keteles 2011). The Regional Research Partnership Program provides short-term training opportunities (up to 6 months) for regional technical staff to work directly with ORD scientists in ORD laboratories and centers. Regional Science Topic Workshops are held on high-priority topics, including green chemistry, water reuse, and children’s environmental exposures. The workshops are intended to identify research needs, initiate research partnerships, and improve information-sharing and coordination of existing research efforts. Through the Regional Research Partnership Program, OSP provides travel and relocation expenses for 10 regional scientists a year to be detailed to specific ORD laboratories for 4 to 12 weeks to work on high-priority research projects in direct collaboration with ORD scientists. The committee concludes that the Regional Science Program could improve the effectiveness of its delivery of ORD and program-office research to regional programs through additional liaisons with specific responsibility in this regard.
ORD is beginning to use social networking and information technology tools, as noted in Chapters 3 and 4 and Appendix D, to promote the development of science communities that cross internal organizational boundaries and extend outside the agency. For example, EPA SAB (2012b) found various electronic sources that are considered useful by the program and regional offices, including the Office of Solid Waste and Emergency Responses CLU-IN Web site (which provides a platform for training, seminars, and podcasts); a variety of forums sponsored by the Office of Solid Waste and Emergency Responses that support the Superfund and Resource Conservation and Recovery Act programs; the Economics Forum, hosted by EPA’s National Center for Environmental Economics (NCEE), to keep the agency and other interested parties informed about research; and the Environmental Science Connector, a Web-based tool designed for project management and information-sharing with EPA researchers and external collaborators. ORD is also experimenting with a Web-based collaborative platform called IdeaScale that allows its scientists and engineers to engage in an open, interactive conversation. Staff can share their ideas, then harness the input of their peers through online discussions and ranking tools to refine them. EPA is also developing IdeaScale sites for research programs, engaging both internal and external stakeholders to help in preparing new research frameworks. It is an interesting new approach, but there is little evidence that it has worked effectively to date, having had few users.
Despite the variety of efforts to support and coordinate science within the agency more effectively, the efforts focus on one-way interaction between ORD and program offices or regions and, as noted in several reviews, are not thoroughly coordinated. EPA SAB (2012b, p.7) noted, “ORD principally focuses on
ORD scientists, although it supports several small but important programs in the regions…. Program and regional offices manage their scientific workforces relatively independently, with some organizations providing stronger support than others.” Given the need for integrated, transdisciplinary, and solutions-oriented research to solve 21st century environmental problems, the existing structure focused on ORD as the “science center” that establishes the scientific agenda of EPA will not be sufficient; ORD only makes up a portion of EPA’s scientific efforts, and more than three-fourths of EPA’s scientific staff work outside ORD (EPA SAB 2012b). When science integration or collaboration occurs, it involves largely short-term needs and problems. Although ORD has surveyed regional and program offices for science and data needs and it will be necessary to continue to conduct regular and systematic assessments of regional and program offices to inform its planning, the focus on ORD planning alone will not be adequate to address science needs for 21st century challenges. As noted above, the development of strategic, coordinated multiyear agency-wide science integration plans, overseen by enhanced science leadership empowered by the administrator, are critical for the agency to coordinate and deliver science in and outside of the agency more effectively in the future. Such integrated plans would also assist the agency in determining where resources outside the agency may be used.
Science flourishes where scientists flourish, and scientists flourish where they have opportunities to work on interesting, challenging problems, interact synergistically with colleagues, have an impact, and earn recognition for their work. In seeking to strengthen its science capacity, EPA needs to attend to the structure of its research operations; to attract, retain, and develop scientific talent within the agency; to contribute to environmental-education efforts to build the talent pool for the future; to support science outside the agency; and to ensure that science is conducted with the utmost integrity. Those points are addressed below.
Enhancing Expertise in the US Environmental Protection Agency
As discussed in Chapters 2 and 3, EPA will need to continue to be prepared to address a wide array of environmental and health challenges and their complex interactions. In some cases, the agency will need to advance scientific understanding through inhouse research efforts; in others, it will need to assimilate and influence scientific efforts that are undertaken elsewhere. Strategic workforce planning when hiring new staff will help to ensure that EPA has expertise it needs in critical fields. Equally important, EPA should carefully attend to the challenge of continuing science education to ensure that scientists are productive throughout their careers even as the pace of change in scientific tools, techniques, and challenges increases.
Building and enhancing capacity of young scientists to be innovators, collaborators, and systems thinkers with a transdisciplinary perspective will require strong leadership, flexibility, and coordination. Given that a large percentage of EPA scientists in ORD and other program offices are near retirement, it is critical for the agency to recruit a new generation of scientists who are well versed in emerging tools (discussed in Chapter 3 and Appendixes C and D) and in cross-disciplinary collaboration and who have been mentored by current scientists. Mentoring will allow younger scientists to gain an understanding of years of research and regulatory science from older scientists. One specific example is in the field of statistics. Senior statisticians are important in EPA because they have the knowledge and experience to mentor inhouse junior statisticians and scientists, facilitate inhouse data analytic work, steer the agency to secure appropriate expert support from outside, and ensure the quality of agency’s statistical work. The best type of person to fill this senior position not only has advanced statistical expertise, but also has substantive knowledge in other fields and substantial teamwork experience.
To develop career paths and increase productivity of its newer scientists, EPA needs to be vigilant in engaging them and fostering their professional development. The committee supports ORD’s efforts to clarify requirements for promotion of scientists and engineers to senior levels (Anastas 2011). The promotion criteria require substantial achievement that displays high scientific quality, relevance to EPA’s mission, and impacts on decision-making. As is typical of expectations in most academic institutions, scientists and engineers seeking promotion to the GS-14 level are expected to be nationally recognized for their contributions and those seeking promotion to GS-15 to have international recognition. ORD’s promotion criteria now highlight expectations for transdisciplinary research, teamwork, and leadership (Anastas 2011).
EPA also needs larger and more senior cadres of scientists in fields in which it wants to play a strong leading role among federal agencies (NRC 2010a). In a recent example, EPA’s National Center for Computational Toxicology (NCCT) was established to address the lack of toxicity data on the many chemicals that are on the market and to do so in an efficient and cost-effective manner (see Chapter 3 and Appendix C for more information about EPA’s computational toxicology program). Buoyed by the guidance and affirmation it received from Toxicity Testing in the 21st Century (NRC 2007), ORD and NCCT leadership set an ambitious path to address their charge. In its first 5 years, the center has been able to break boundaries and build transdisciplinary collaborations with other federal partners and the private sector both in the United States and internationally. The science generated through the center’s collaborations has created momentum around computational toxicology research and influenced research investments by other agencies and organizations, including the chemical industry.
Optimizing resources, creating and benefiting from scientific exchange zones, and leading innovation through transdisciplinary collaborations to address the many challenges described in Chapter 2 will require forward-thinking
and resourceful scientific leadership at various levels in the agency. This includes EPA using all of its authority effectively, including pursuing permanent Title 42 authority, to recruit, hire, and retain the high-level science and engineering leaders that it needs to maintain a strong inhouse research program (NRC 2010a). It will also mean maintaining a critical mass of world-class experts who have the ability to identify and access the necessary science inside or outside EPA and to work collaboratively with researchers in other agencies.
Finding: Expertise in traditional scientific disciplines—including but not limited to statistics, chemistry, economics, environmental engineering, ecology, toxicology, epidemiology, exposure science, and risk assessment—are essential for addressing the challenges of today and the future. The case of statistics is one example where the agency is facing significant retirements and needs to have, if anything, enhanced expertise.1 EPA is currently attuned to these needs, but staffing high-quality scientists in these areas of expertise who can embrace problems by drawing from information across disciplines will require continued attention if EPA is to maintain its leadership role in environmental science and technology.
Recommendation: EPA should continue to cultivate a scientific workforce across the agency (including ORD, program offices, and regions) that can take on transdisciplinary challenges.
The committee recognizes that EPA already provides many unique opportunities to engage in high quality, collaborative, and interdisciplinary research. However, EPA can continue to build its capacity by cultivating a scientific workforce across the agency (including ORD, program offices, and regions) that can take on transdisciplinary challenges. Some options that EPA might explore to fulfill the recommendation above include:
• Build a stronger mentoring and leadership development program that supports young researchers and fosters the culture of systems-thinking research.
• Recruit young scientists who have expertise and interest in scientific concepts and tools relevant to systems thinking.
• Promote rotations through its laboratories and through the laboratories of other federal agencies and scientific organizations as valuable training experiences for new scientists in the areas of environmental health, science, and engineering.
1 ORD currently has 12 epidemiologists, 31 statisticians (mathematical or research), and 8 biologic and health statisticians (E. Struble, EPA, personal communication, July 13, 2012). These job titles typically require a certain amount of statistics course work and do not fully reflect statistical expertise across the entire agency. There are staff members with other job titles who also fulfill the data analysis role.
• Expand opportunities for internal networking, including opportunities for scientists and engineers to work between programs and offices.
• Encourage scientists and engineers to work in interdisciplinary teams and in new ways to provide information in a timely fashion.
• Implement programs to help scientists and engineers to acquire new skills and expertise throughout their careers, including educational opportunities, sabbaticals and other kinds of leave, and laboratory rotations.
• Provide opportunities for agency scientists to interact with colleagues in other agencies, in universities, in non-profit organizations, and in the private sector; such opportunities could include workshops, roundtables, participation in traditional research conferences, and longer-term exchanges with or as visiting scientists.
• Promote the visibility and recognition of scientific excellence across its divisions, programs, and locations by enhancing and highlighting its featured research and awards programs.
• Assess its current policies for retaining and hiring civil service employees. The agency should be nimble and should be able to easily hire or reassign employees to make sure it has specific expertise to understand emerging challenges and make use of new tools, technologies, and approaches in the appropriate offices, regions, and laboratories at the appropriate time.
There are several fields in which EPA lacks expertise and in which investments in additional expertise could provide substantial benefits to the agency and its mission. One key recognized need is for social, behavioral, and decision scientists. EPA’s economic, social, behavioral, and decision science staff consists almost entirely of economists. The agency is without strong expertise in social, behavioral, and decision sciences, though it does support some research in these areas through the Science to Achieve Results (STAR) program and procures economics research from contractors. Social research in EPA was historically funded by ORD (NRC 2000). In 2008, the economics and decision science extramural research program was transferred to NCEE (EPA SAB 2011). As part of the reorganization, decision sciences were eliminated altogether (EPA SAB 2011). Economics has remained a low priority (EPA SAB 2011) and the economics staff (about 100 economists) is a very small fraction of the agency’s professional staff (EPA, unpublished material, 20122 ).
The small representation of economics expertise and the virtual nonexistence of behavioral and decision scientists (nine social scientists, four psychologists, and one sociologist; 3 [P. Vaughn, EPA, personal communication, July 13,
2 The unpublished data was received from M. Bender, EPA, on July 13, 2012, as part of a data request made on behalf of the Committee on Science for EPA’s Future. This material is available by contacting the Public Access Records Office of the National Academies.
3 In addition to psychologists and sociologists, NCEE acknowledged that it was lacking expertise in behavioral economics and is pursuing the hiring of new staff in that field.
2012] in the professional staff are matters of concern, given EPA’s ever-present and increasing need to defend programs and initiatives on economic grounds, its concerns for environmental justice and community engagement in environmental decision-making, and its goal of becoming more transdisciplinary. In addition, and as noted above, economic, social, behavioral, and decision sciences can make important contributions to improving environmental policy decisions within the emerging integrated systems-based approach to environmental management and contributions to innovation in strategies for achieving environmental goals efficiently, equitably, and cooperatively. The importance of behavioral science to the conduct of environmental economics research, including environmental valuation, has been well established and has led to considerable research on the integration of behavioral sciences with environmental economics (Sent 2004; Shogren and Taylor 2008; Shogren et al. 2010).
In particular, behavioral and decision science deals with such issues as “framing effects” and the role of cognitive heuristics (see, for example, Kahneman et al. 1982). It provides the intellectual basis of modern methods of risk communication (see, for example, Morgan et al. 2002), and its insights and ideas should constitute a key complement of economics and decision analysis in such contexts as the design and assessment of the likely effectiveness of alternative regulatory strategies (Fischbeck and Farrow 2001). Those insights often arise from extensive experimental studies in both laboratory and field settings. Without staff in EPA who are experts in those subjects, not only can the agency not conduct such studies, but often it does not know how to ask the right questions or how to seek the right expertise and advice. That is unfortunate—social, behavioral, and decision sciences are essential in the design of survey research, the development of methods in expert elicitation to characterize the state of uncertain science (EPA 2009a), the development and evaluation of risk-communication methods, the development and use of mediation and other group-decision processes, the promotion of a variety of environmentally benign behaviors, and the anticipation of behavioral responses to alterative regulatory and other protective strategies. EPA needs to have staff with sufficient expertise or cross-disciplinary training to allow it to become an educated consumer of social, behavioral, and decisions sciences and to engage more effectively with external entities to conduct innovative science.
Finding: EPA’s economic, social, behavioral, and decision science staff consists almost entirely of economists. The agency is without strong expertise in social, behavioral, and decision sciences, though it does support some research in these areas through outside grants, collaborations, and procurement.
Recommendation: The committee recommends that EPA add staff who have training in behavioral and decision sciences and find ways to enhance the existing staff capabilities in these fields.
Options that EPA might explore to fulfill that recommendation include:
• Recruit several new staff who have earned advanced degrees in empirically-based behavioral and decision science. The new staff would need to have strong communication skills and would need to work with economists, natural scientists, and engineers in the agency to help to make regulatory and other agency policies that promote environmentally protective behaviors and that are more realistic. Their knowledge would assist the agency by helping it to make more informed choices in seeking outside contractors and advisers and to create stronger collaboration with academics in related fields. The committee suggests that the new staff be located within NCEE. The reason for that suggestion is that NCEE currently staffs the largest number of social scientists within the agency. The large interest in behavioral and decision sciences that exists now in economics broadly, as exhibited by the fields of behavioral and neuroeconomics, will contribute to making NCEE a productive location. More importantly, behavioral economics is an essential source of new insight in environmental economics research pertaining to the benefits of environmental protection and the design of incentives for environmental management. Co-locating behavioral scientists within NCEE will increase the capacity of economics staff to participate in the advances in environmental economics emerging from the integration of behavioral economics.
• Provide mechanisms for cross-disciplinary training of staff in core disciplines relevant to behavioral and decision science. The committee acknowledges that the number of staff in EPA who have advanced training in these fields is likely to remain modest even with a concerted recruitment effort, and it is important for staff scientists who work in adjacent disciplines to have enough familiarity to know what questions to ask (and whom to ask).
• Develop improved mechanisms for integrating economic, social, behavioral, and decisions sciences into the development of science to support environmental-management decisions.
Using Outside Expertise
EPA often needs to seek expertise and research from sources outside the agency when science needs cannot be met from within. Sources include other federal agencies (such as, the Department of Defense, the Department of Energy, the National Oceanic and Atmospheric Administration, and the US Department of Agriculture), government research organizations (such as the National Institute of Environmental Health Sciences), industrial research initiatives, universities, consultants, state and local governments, and nongovernment organizations (EPA SAB 2011). The international community is also a good resource for EPA, as there is a lot of high-quality environmental and human health research undertaken in Europe, Asia, and elsewhere. International collaboration is particularly important considering products and processes are becoming more global and considering many environmental problems, such as the transport of pollutants, are global in nature.
Strategic collaborations with other agencies and scientific institutions will be critical if EPA is to access the breadth of expertise necessary to address 21st century environmental challenges. For example, chemical and pesticide regulations are informed by hazard data derived from animal toxicologic studies. But as epidemiologic and biomonitoring studies generate more information that is relevant to risk assessment, a broader array of expertise will be required to interpret the new types of data and weigh their evidence relative to the more prevalent toxicologic data. EPA needs to have sufficient internal expertise and critical mass in epidemiology, biostatistics, and population-based research. However, rather than house large teams of epidemiologists and biostatisticians among its experts, EPA could build collaborative networks with the National Institute of Environmental Health Sciences and other agencies to undertake assessments. In fields in which it is unrealistic to have sufficient inhouse capacity, existing scientific staff at EPA will need to have adequate cross-disciplinary awareness to ask the right questions and identify appropriate collaborators. For example, if statistical expertise is needed from an outside source, the contractor or subcontractor that is hired should have adequate expertise in statistics (such as a PhD) to successfully meet EPA’s needs.
Building New Expertise Through Education
The future of EPA’s scientific enterprise depends on having a diverse body of capable and committed scientists and engineers to work in EPA and in research positions in other government agencies, academe, the nonprofit sector, and the private sector. Future scientists and engineers should understand the complex nature of environmental challenges and the transdisciplinary needs and opportunities for solutions. Furthermore, to achieve its mission of protecting human health and the environment, the agency will need to play a role in helping to educate and engage the public. Public understanding and engagement are especially critical in EPA’s efforts to achieve its aims by using nonregulatory approaches and in building ongoing support for the environmental science and engineering and protection efforts of the agency. Among other needs, the agency will need to educate stakeholders and the public about new scientific concepts and approaches that it develops or adopts, and to provide training for potential users of new tools and technologies. EPA has numerous valuable programs that are designed to increase the pipeline of future environmental engineers and scientists and to expand and improve environmental education more broadly. Early environmental education is important in creating champions for environmental protection and innovation in new science and technology who can work in the agency in the future.
The National Environmental Education Act of 1990 simultaneously established the Office of Environmental Education (OEE) in EPA and the National Environmental Education Foundation, a nonprofit corporation meant to leverage private support. The act authorized environmental-education grants, internship
and fellowship programs, and the Environmental Education and Training Partnership, which has worked to develop standards for environmental education. In 2009, OEE issued an Environmental Education Highlights report (EPA 2009b) that briefly describes some of the dozens of outreach and education programs that EPA leads. They include collaborations with schools, the Boys and Girls Clubs, the Girl Scouts, and the Parent—Teacher Organization to provide education and service opportunities focused on energy conservation, water conservation, recycling, and waste reduction. The Tools for Schools program in the Office of Air and Radiation has reached more than 60,000 schools with educational materials, training, and guidance on indoor air quality. The agency collaborates with the American Meteorological Society to provide training and outreach tools for broadcast meteorologists on air quality and watershed protection. Between scientific survey trips, EPA’s ocean survey vessel Bold hosts open houses at ports of call around the country.
OEE administers the National Network for Environmental Management Studies fellowships, which were established in 1986 and have supported more than 1,400 fellows. Network fellows receive support for undergraduate or graduate studies and work on EPA-supported and EPA-directed research projects. Students apply in response to requests for applications developed by EPA staff in Washington, DC, and in regional offices and laboratories around the country.
ORD also offers critical student support and encouragement through its People, Prosperity, and the Planet (P3) student grants and design competition, its Greater Research Opportunities undergraduate fellowship program, and the STAR graduate fellowship program (EPA 2012c). The Greater Research Opportunities program offers fellowships to juniors and seniors who are studying in environment-related fields in colleges and universities that do not receive large amounts of federal research funding. The fellowships provide academic support for up 2 years with a summer internship at EPA. The agency plans to award about $2 million worth of Greater Research Opportunities undergraduate fellowships in 2012 (EPA 2012d). The STAR fellowship program supports master’s and doctoral students who are working in environment-related fields. Students competing for STAR grants are required to submit original proposals on EPA-specified research topics that run the gamut from social sciences to engineering. More than 1,500 STAR fellowships have been awarded since the program began in 1995 (EPA 2012c). The P3 program offers grants to teams of college students who research and design innovative solutions to sustainability challenges (EPA 2012e). The teams can apply for $15,000 for Phase 1 development grants and up to $90,000 in Phase 2. Phase 2 grants are awarded at the National Sustainable Design Exposition in Washington, DC, each April (EPA 2011a).
Delivering Science Outside the Environmental Protection Agency
As state, local, and tribal environmental agency budgets decline, the agencies will rely increasingly on EPA for scientific support. EPA conducts pro-
grams that are intended to provide and communicate science and tools for decision-makers and practitioners outside EPA. Several of EPA’s large-scale regional research programs (for example, the Chesapeake Bay, Great Lakes, and Puget Sound programs) are designed specifically to develop and deliver science and decision support tools to help environmental authorities outside EPA. ORD conducts research programs to develop widely applicable decision-support tools. ORD’s Collaborative Science and Technology Network for Sustainability provides grants to explore “new approaches to environmental protection that are systems-oriented, forward-looking, preventive, and collaborative” (EPA 2011b). The Tribal Science Program supports community-based research in an effort to improve understanding of the relationship between tribal-specific factors and health risks posed by toxic substances in the environment (EPA 2011c). Web-based platforms are essential for delivering science and tools to state, local, tribal, and other non-EPA practitioners, and EPA has made an effort to take advantage of such platforms, such as the Health and Environmental Research Online (HERO) system (see below).
Since its founding, EPA has been challenged by the need to use the best available scientific information in developing policy and regulations. Critics of EPA’s regulations (as either too lax or too stringent) have sometimes charged that valid scientific information was ignored or suppressed, or that the scientific basis of a regulation was not adequate. EPA’s best defense against those criticisms is to ensure that it transparently distinguishes between questions of science and questions of policy in its regulatory decisions; to demand openness and access to the scientific data and information on which it is relying, whether generated in or outside of the agency; and to use competent, balanced, objective, and transparent procedures for selecting and weighing scientific studies, for ensuring study quality, and for peer review.
Distinguishing Science Questions from Policy Questions
In a memorandum on scientific integrity issued on March 9, 2009, President Obama declared that “political officials should not suppress or alter scientific or technologic findings and conclusions” (The White House 2009). After the president’s directive, EPA administrator Lisa Jackson stated in a memo issued on May 9, 2009, that
while the laws that EPA implements leave room for policy judgments, the scientific findings on which these judgments are based should be arrived at independently using well-established scientific methods, including peer
review, to ensure rigor, accuracy, and impartiality. This means that policymakers must respect the expertise and independence of the Agency’s career scientists and independent advisors while insisting that the Agency’s scientific processes meet the highest standards of rigor, quality, and integrity (Jackson 2009).
The Bipartisan Policy Center (2009) has recommended that the best means for regulators to reduce opportunities for inappropriate political intervention in scientific judgments and to avoid the perception that politicization of science had occurred is to distinguish clearly between science and policy questions in formal regulatory documents. EPA has done that well in recent reviews of the National Ambient Air Quality Standards, which separate the review of scientific information on health and welfare effects presented in its integrated science assessments from the policy-assessment documents that draw on the scientific information. Contining to promote that approach will support the distinction between science questions and policy questions and conducting periodic audits of rulemaking documents will help to ensure compliance with the distinctions. Given the uncertainties surrounding many complex environmental problems, it is important for the agency to be transparent about types of uncertainties involved in its assessments and to be clear about how both science and policy considerations inform ultimate decisions.
Increased Access to Scientific Information
One of the key elements of ensuring the credibility of science used in decision-making is maintaining the highest level of transparency, and making scientific information used in EPA decisions easily accessible, as much as possible, to all parties who are interested. That access includes
• Access to the full array of published scientific evidence. One example of important progress has been the recent development by EPA of a searchable electronic database, the HERO system, to give its own staff full access to the emerging scientific literature and give the public access to a searchable on-line database of citations of all studies reviewed in support of its regulations. This is a valuable tool that should continue to have support.
• Access to data. When regulatory stakeholders have legitimate interest in examining the data that underlie reported results, access to published articles or reports is not sufficient. Since the late 1990s, the Data Access Act (or Shelby Amendment) has required that recipients of federal research funding provide their research data to requesting parties if the federal government has used their research findings in developing regulations and the data are later requested under the Freedom of Information Act (OMB Circular A-110). That requirement allows requesting parties the opportunity to inspect and reanalyze data that were
used to support regulations. The Office of Management and Budget (OMB) circular that contains the requirement exempts preliminary analyses, drafts of papers, plans for future research, peer reviews, and communications with colleagues. It also exempts trade secrets, commercial information, and information that must be withheld to protect the privacy of research subjects. The Data Access Act is consistent with an interest in providing greater access to scientific information that underlies regulatory efforts but is limited in applying only to federally funded research (Wagner 2003; Wagner and Michaels 2004). It would be useful to extend requirements for data access to privately supported research that is submitted for regulatory purposes. As with publicly supported research, exemptions could be provided as necessary to protect the privacy of research subjects and legitimate proprietary interests.
• Access to EPA internal research. Concerns about access apply not only to externally sourced scientific information but to research data and findings that are developed through EPA’s internal research programs (Grifo 2009). Publication of EPA science not only helps to bolster the agency’s influence, it also provides legitimacy in the scientific community. EPA needs to encourage its own scientists to communicate and publish their results and to do so in a timely manner. Institutional barriers to the publication of results, particularly bureaucratic delays related to internal approvals and concerns about policy implications, should be addressed.
Ensuring the Quality of Scientific Information
In rule-making processes that rely on extensive reviews of scientific information, EPA generally imposes a strong preference for reliance on published, peer-reviewed studies. The agency’s peer review policy states that “peer review of all scientific and technical information that is intended to inform or support Agency decisions is encouraged and expected” (EPA 2006). The OMB Final Information Quality Bulletin for Peer Review (OMB 2004) and EPA’s internal Peer Review Handbook (EPA 2000) guide the peer-review process for internally generated scientific studies and tools. However, when EPA needs to go beyond peer-reviewed literature to fill information gaps, it may need to be more active in initiating external peer review to ensure that the identified externally generated information is reliable and to provide quality assurance for stakeholders.
EPA has used advisory groups both to review scientific research and to provide advice and expertise from outside the agency. For example, EPA’s National Advisory Council for Environmental Policy and Technology (NACEPT) was established in 1988 to use environmental-policy expertise outside the agency. The advisory council is an independent group of experts that has provided advice to EPA on a broad variety of topics, including workforce capacity, strategic planning, promotion of environmental stewardship, and strategies for improving access to environmental information (EPA 2012d). Various other
advisory committees, established under the Federal Advisory Committee Act, provide scientific advice on such issues as environmental justice and children’s environmental health. A 2009 review of EPA’s Office of Cooperative Environmental Management found that although committees like NACEPT were useful tools for the agency, there was a lack of coordination between other committees and agency advisory boards, such as SAB and BOSC (EPA 2009c). External advisory groups—including SAB, BOSC, and NACEPT—play an important role in helping EPA to ensure the credibility and quality of its scientific studies and science-based decisions. They will remain a valuable resource for the agency assuming the members of these bodies continue to be chosen based on the virtue of their expertise and experience and are appropriately tasked with providing advice that falls within the purview of scientific experts.
Even when the underlying science meets the highest standards of quality and integrity, judgment is used to select and weigh studies that will be used for decision-making. EPA has developed various guidelines to weigh studies and evaluate science, such as guidelines developed in response to sections 108 and 109 of the Clean Air Act. However, EPA has sometimes been criticized for its failure to describe clearly its criteria and methods to identify, evaluate, and weigh scientific studies. For example, National Research Council (NRC) reports over the last decade have evaluated health assessments developed for EPA’s Integrated Risk Information System (IRIS) and indicated a need to improve formal, evidence-based approaches to increase transparency and clarity for selecting datasets for analysis, and to focus more on uncertainty and variability (NRC 2005, 2006, 2010b).
Many of the above observations were reflected in the Review of the Environmental Protection Agency’s Draft IRIS Assessment of Formaldehyde (NRC 2011). In its review, the authoring committee of that report noted a lack of clarity and transparency in the methods used to assess the health effects of formaldehyde. Specifically, that committee found the assessment did not contain “sufficient documentation on methods and criteria for identifying evidence from epidemiologic and experimental studies, for critically evaluating individual studies, for assessing the weight of evidence, and for selecting studies for derivation of the [reference concentrations] and unit risk estimates” (NRC 2011). The report made several recommendations that were specific to improving the formaldehyde IRIS assessment, but also provided some suggestions for improving the IRIS process.
Deficiencies in EPA’s IRIS assessments have resulted in some critics casting doubt on the science used to support agency decisions. EPA is aware of those stakeholder criticisms and of the problems identified by the NRC (2005, 2006, 2010b, 2011), and it has announced improvements in the IRIS assessments that will be reviewed by the recently assembled NRC Committee to Review the IRIS Process. This example illustrates the need for formal evidence-based approaches that are clearly documented and well-reviewed; they can be protective of EPA’s science-informed policies.
This report has stressed the importance of sustaining and strengthening EPA’s present programs of scientific research, applications, and data collection while identifying and pursuing a wide array of new scientific opportunities and challenges. Both are needed to address the complexity of modern problems and both are essential to the agency if it is to continue to provide scientific leadership and high-quality science-based regulation in the years to come.
Specific recommendations related to agency budgets are outside the scope of this study, but the committee feels compelled to note, as did the report Science Advisory Board Comments on the President’s Requested FY2013 Research Budget (EPA SAB 2012a), that since 2004, the budget for ORD has declined 28.5% in real-dollar terms (gross domestic product—indexed dollars). The reductions have been even greater in a number of specific fields, such as ecosystem research and pollution prevention.
Finding: If EPA is to provide scientific leadership and high-quality science-based regulation in the coming decades, it will need adequate resources to do so. Some of the committee’s recommendations, if followed, will allow EPA to address its scientific needs with greater efficiency. But the agency cannot continue to provide leadership, pursue many new needs and opportunities, and lay the foundation for ensuring future health and environmental safety unless the long-term budgetary trend is reversed.
Recommendation: The committee recommends EPA create a process to set priorities for improving the quality of its scientific endeavors over the coming decades. This process should recognize the inevitably limited resources while clearly articulating the level of resources required for the agency to continue to ensure the future health and safety of humans and ecosystems.
It is clear that if EPA is to meet current, persistent, and future challenges and is to succeed in applying systems thinking throughout its scientific enterprise, it will have to continue to enhance its scientific capacity and improve coordination of science throughout the agency. In this chapter, the committee has described how EPA can enhance its agency-wide science leadership, take steps to continue the realignment of ORD to advance transdisciplinary research and support the agency’s strategic goals, strengthen internal scientific capacity and ties to the larger environmental science and engineering research community, and ensure the integrity of the scientific information the agency generates or uses.
Anastas, P. 2011. Technical Qualifications Board (TQB) Policy-Supplemental Guidance. Memorandum to Laboratory Directors and Center Directors, from Paul Anastas, Assistant Administrator for Research and Development, US Environmental Protection Agency, Washington, DC. June 9, 2011. Attachment: Additional Guidance for Promotion of ORD Scientists and Engineers in Research, Development, and Expert Positions, May 2010.
EPA (US Environmental Protection Agency). 1992. Safeguarding the Future: Credible Science, Credible Decisions: The Report of the Expert Panel of the Role of Science at EPA. EPA/600/9-91-050. US Environmental Protection Agency, Washington, DC. March 1992.
EPA (US Environmental Protection Agency). 2000. Peer Review Handbook. EPA 100-B-00-001. Office of Science Policy, Office of Research and Development, Washington, DC [online]. Available: http://www.epa.gov/peerreview/pdfs/prhandbk.pdf [accessed Apr. 18, 2012].
EPA (US Environmental Protection Agency). 2006. Peer Review and Peer Involvement in the US EPA [online]. Available: http://www.epa.gov/peerreview/pdfs/peer_review_policy_and_memo.pdf [accessed Apr. 12, 2012].
EPA (US Environmental Protection Agency). 2008. Smart Energy Resources Guide. EPA/600/R-08/049. US Environmental Protection Agency [online]. Available: http://www.arta1.com/cms/uploads/Smart%20Energy%20Resource%20Guide.pdf [accessed Apr. 19, 2012].
EPA (US Environmental Protection Agency). 2009a. Expert Elicitation Task Force White Paper, Draft. Science Policy Council, US Environmental Protection Agency, Washington, DC. January 6, 2009 [online]. Available: http://www.epa.gov/osa/pdfs/elicitation/Expert_Elicitation_White_Paper-January_06_2009.pdf [accessed Apr. 12, 2012].
EPA (US Environmental Protection Agency). 2009b. Environmental Education Highlights. US Environmental Protection Agency [online]. Available: http://www.epa.gov/enviroed/pdf/2009_EEHighlights.pdf [accessed Apr. 12, 2012].
EPA (US Environmental Protection Agency). 2009c. 20 Years of Shaping Environmental Policy at EPA. EPA 130-R-09- 003. Office of Cooperative Environment Management, US Environmental Protection Agency [online]. Available: http://nepis.epa.gov/Adobe/PDF/500025SF.PDF [accessed Apr. 12, 2012].
EPA (US Environmental Protection Agency). 2011a. Funding Opportunities. Extramural Research, US Environmental Protection Agency [online]. Available: http://www.epa.gov/ncer/rfa/2012/2012_p3.html [accessed Apr. 19, 2012].
EPA (US Environmental Protection Agency). 2011b. Collaborative Science and Technology Network for Sustainability (CNS), Extramural Research, US Environmental Protection Agency [online]. Available: http://epa.gov/ncer/cns/ [accessed Apr. 19, 2012].
EPA (US Environmental protection Agency). 2011c. Collaborative Community and Regional Programs. US Environmental Protection Agency [online]. Available: http://epa.gov/ncer/cns/programs.html [accessed Apr. 19, 2012].
EPA (US Environmental Protection Agency). 2012a. Science Policy. US Environmental Protection Agency [online]. Available: http://www.epa.gov/osp/ [accessed Apr. 19, 2012].
EPA (US Environmental Protection Agency). 2012b. Regional Methods (RM) Program. US Environmental Protection Agency [online]. Available: http://www.epa.gov/osp/regions/rm.htm [accessed Apr. 18, 2012].
EPA (US Environmental Protection Agency). 2012c. Fellowships. Extramural Research, US Environmental Protection Agency [online]. Available: http://www.epa.gov/ncer/fellow/ [accessed Apr. 18, 2012].
EPA (US Environmental Protection Agency). 2012d. Fall 2012 EPA Greater Research Opportunities (GRO) Fellowships for Undergraduate Environmental Study. Greater Research Opportunities (GRO) Program, Office of Research and Development, US Environmental Protection Agency[online]. Available: http://www.epa.gov/ncer/rfa/2012/2012_gro_undergrad.html [accessed May 17, 2012].
EPA (US Environmental Protection Agency). 2012e. P3: People, Prosperity and the Planet Student Design Competition for Sustainability. US Environmental Protection Agency [online]. Available: http://www.epa.gov/ncer/p3/ [accessed Apr. 18, 2012].
EPA SAB (US Environmental Protection Agency Science Advisory Board). 2010. Office of Research and Development Strategic Research Directions and Integrated Transdiciplinary Research. EPA-SAB-10-010. Memo to Lisa P. Jackson, Administrator, US Environmental Protection Agency, from Deborah L. Swackhamer, Chair, Science Advisory Board, US Environmental Protection Agency, Washington, DC. July 8, 2010 [online]. Available: http://yosemite.epa.gov/sab/sabproduct.nsf/E989ECFC125966428525775B0047BE1A/$File/EPA-SAB-10-010-unsigned.pdf [accessed Apr. 13, 2012].
EPA SAB (US Environmental Protection Agency Science Advisory Board). 2011. Science Advisory Board Comments on The President’s Requested FY 2012 Research Budget. EPA-SAB-11-007. Memo to Lisa P. Jackson, Administrator, from Deborah L. Swackhamer, Chair, Science Advisory Board, and Jerold Schnoor, Chair, SAB Research Budget Work Group, US Environmental protection Agency, Washington, DC. June 2, 2011 [online]. Available: http://yosemite.epa.gov/sab/sabproduct.nsf/c91996cd39a82f648525742400690127/9BE9A90F43A8DD1D852578A30069D7E5/$File/EPA-SAB-11-007-unsigned.pdf [accessed Apr. 18, 2012].
EPA SAB (US Environmental Protection Agency Science Advisory Board). 2012a. Science Advisory Board Comments on the President’s Requested FY 2013 Research Budget, May 3, 2012. Science Advisory Board, US Environmental Protection Agency, Washington, DC [online]. Available: http://yosemite.epa.gov/sab/sabproduct.nsf/0/1190D2161DBCAD3B852579F3005FC0CF/$File/EPA-SAB-12-006-unsigned-SS.pdf [accessed Aug. 16, 2012].
EPA SAB (US Environmental Protection Agency Science Advisory Board). 2012b. Science Integration for Decision Making at the US Environmental Protection Agency(EPA). Final Report, July 6, 2012. Science Advisory Board, US Environmental Protection Agency, Washington, DC [online]. Available: http://yosemite.epa.gov/sab/sabproduct.nsf/fedrgstr_activites/8AA27AA419B1D41385257A330064A479/$File/EPA-SAB-12-008-unsigned.pdf [accessed July 22, 2012].
EPA SAB/BOSC (US Environmental Protection Agency Science Advisory, and Board and Board of Scientific Counselors). 2011. Office of Research and Development (ORD) New Strategic Research Directions: A Joint Report of the Science Advisory Board (SAB) and ORD Board of Scientific Councilors (BOSC). EPA-SAB-12-001. Science Advisory Board, and Board of Scientific Councilors US Environmental Protection Agency, Washington, DC. October 21, 2011[online]. Available: http://www.epa.gov/osp/bosc/pdf/StratResDir111021rpt.pdf [accessed Apr. 13, 2012].
Fischbeck, P.S., and R.S. Farrow, eds. 2001. Improving Regulation: Cases in Environment, Health, and Safety. Washington, DC: Resources for the Future.
GAO (US Government Accountability Office). 2011. Environmental Protection Agency: To Better Fulfil It’s Mission, EPA Needs a More Coordinated Approach to Managing its Laboratories. July 2011. GAO-11-347. Washington, DC: US Government Accountability Office [online]. Available: http://www.gao.gov/assets/330/321850.pdf [accessed April 13, 2012].
Grifo, F. 2009. Testimony of Francesca T. Grifo, Senior Scientist with the Union of Concerned Scientists, Director of the Scientific Integrity Program Before the US Senate Committee on Environment and Public Works “Scientific Integrity and Transparency Reforms at the EPA”, June 9, 2009 [online]. Available: http://www.ucsusa.org/scientific_integrity/solutions/agency-specific_solutions/ucs-testimony-to-senate-June-2009.html [accessed Apr. 18, 2012].
Jackson, L.P. 2009. Scientific Integrity. Memo to all EPA Employees, from Lisa P. Jackson, Administrator, US Environmental Protection Agency, Washington, DC. May 9, 2009 [online]. Available: http://blog.epa.gov/administrator/2009/05/12/memo-to-epa-employees-scientific-integrity/ [accessed Apr. 18, 2012].
Kahneman, D., P. Slovic, and A. Tversky. 1982. Judgment under Uncertainty: Heuristics and Biases. Cambridge: Cambridge University Press.
Keteles, K. 2011. Science at EPA: A Regional Office Perspective. Presentation to the Third Meeting on Science for EPA’s Future, August 8, 2011, Washington, DC.
Morgan, M.G., B. Fischhoff, A. Bostrom, and C.J. Atman. 2002. Risk Communication: A Mental Models Approach. Cambridge: Cambridge University Press.
NRC (National Research Council). 2000. Strengthening Science at the US Environmental Protection Agency: Research-Management and Peer-Review Practices. Washington, DC: National Academy Press.
NRC (National Research Council). 2005. Health Implications of Perchlorate Ingestion. Washington, DC: National Academies Press.
NRC (National Research Council). 2006. Health Risks from Dioxin and Related Compounds: Evaluation of the EPA Reassessment. Washington, DC: National Academies Press.
NRC (National Research Council). 2007. Toxicity Testing in the 21st Century: A Vision and a Strategy. Washington, DC: National Academies Press.
NRC (National Research Council). 2010a. The Use of Title 42 Authority at the US Environmental Protection Agency: A Letter Report. Washington, DC: National Academies Press.
NRC (National Research Council). 2010b. Review of the Environmental Protection Agency’s Draft IRIS Assessment of Tetrachloroethylene. Washington, DC: National Academies Press
NRC (National Research Council). 2011. Review of the Environmental Protection Agency’s Draft IRIS Assessment of Formaldehyde. Washington, DC: National Academies Press.
OMB (Office of Management and Budget). 2004. Final Information Quality Bulletin for Peer Review [online]. Available: http://www.whitehouse.gov/sites/default/files/omb/ memoranda/fy2005/m05-03.pdf [accessed Apr. 19, 2012].
Sent, E.M. 2004. Behavioral economics: How psychology made its (limited) way back into economics. Hist. Polit. Econ. 36(4):735-760.
Shogren, J.F., and L.O. Taylor. 2008. On behavioral-environmental economics. Rev, Environ. Econ. Policy 2(1):26-44.
Shogren, J.F., G.M. Parkhurst, and P. Banerjee. 2010. Two cheers and a qualm for behavioral environmental economics. Environ. Resour. Econ. 46(2):235-247.
Teichman, K., and P. Anastas. 2011. Science for EPA’s Future: Innovative Thinking, Creative Solutions. Presentation at the Second Meeting on Science for EPA’s Future, June 17, 2011, Washington, DC.
The Bipartisan Policy Center. 2009. Improving the Use of Science in Regulatory Policy. Science for Policy Project Final report. August 5, 2009 [online]. Available: http://bipartisanpolicy.org/sites/default/files/BPC%20Science%20Report%20fnl.pdf [accessed Apr. 18, 2012].
The White House. 2009. Scientific Integrity. Memorandum for the Heads of Executive Departments and Agencies, from President Obama, May 9, 2009 [online]. Available: http://www.ucsusa.org/assets/documents/scientific_integrity/President-Obama-Scientific-Integrity-Memo.pdf [accessed Apr. 19, 2012].
Wagner, W.E 2003. The “bad science” fiction: Reclaiming the debate over the role of science in public health and environmental regulation. Law Contemp. Probl. 66(Fall): 63-124.
Wagner, W., and D. Michaels. 2004. Equal treatment for regulatory science: Extending the controls governing the quality of public research to private research. Am. J. Law Med. 30(2-4):119-154.