Suggested Citation:"Appendix A: Workshop Agenda." National Academies of Sciences, Engineering, and Medicine. 2017. Principles and Practices for Federal Program Evaluation: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/24831.
×
Appendix A
Workshop Agenda
Principles and Practices for Federal Program Evaluation
October 27, 2016
Keck Center of the National Academies of Sciences, Engineering, and Medicine
500 Fifth Street, NW, Washington, DC
Room 100
Thursday, October 27, 2016
Open Session, 9:00am–4:30pm
| 9:00am | Call to Order |
| Breakfast available outside the meeting room | |
| Welcome and Introduction | |
| Connie Citro, Director, Committee on National Statistics | |
| Christine Fortunato, Administration for Children and Families | |
| Purpose of the Workshop | |
| Russ Whitehurst, Brookings Institution, Committee Chair | |
| The steering committee for the workshop will facilitate discussion on principles and practices for federal program evaluation, to include reviews of extant policies issued by the Administration for Children and Families, the Institute for Education Sciences, the Chief Evaluation Office in the Department of Labor, and other federal agencies. Throughout the workshop we will consider ways to build upon these documents, including ways to institutionalize the principles, with the goal of bolstering the integrity and protecting the objectivity of the evaluation function in federal agencies, which is essential for evidence-based policy making. | |
Suggested Citation:"Appendix A: Workshop Agenda." National Academies of Sciences, Engineering, and Medicine. 2017. Principles and Practices for Federal Program Evaluation: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/24831.
×
| Scope of the Workshop:Evaluations of interventions, programs, and practices intended to affect human behavior, carried out by the federal government or its contractual agents, domestic and abroad, and leading to public reports sponsored by the federal government that are intended to provide information on their impacts, cost, and implementation. | |
| 9:20am | History of Federal Program Evaluation |
| Moderator: Howard Rolston, Abt Associates (Committee Member) | |
| Discussants: Jean Grossman, Princeton University; Ron Haskins, Brookings Institution; Larry Orr, Johns Hopkins University | |
| History of federal program evaluation and its successes, challenges, and vicissitudes from a variety of perspectives, including formal federal government evaluation leaders, evidence-based policy advocates, and social policy researchers and producers. | |
| 10:20am | Break |
| 10:35am | Review of Present Principles by Topic and Agency Coverage |
| Moderator: Brian Harris-Kojetin, CNSTAT Deputy Director | |
| Discussants: Naomi Goldstein Administration for Children and Families (ACF); Jack Molyneaux, Millenium Challenge Corporation (MCC); Ruth Neild, Institute of Education Sciences (IES); Demetra Nightingale, Department of Labor (DOL) | |
| Exploration of several prominent evaluation agencies and their approaches to protecting the integrity and objectivity of evaluation work through formal guidance and other means. | |
| 11:30am | What More Do We Need to Understand/What’s Missing from Current Evaluation Principles & Practices: A Discussion |
| Moderator: Russ Whitehurst, Brookings Institution (Committee Chair) | |
Suggested Citation:"Appendix A: Workshop Agenda." National Academies of Sciences, Engineering, and Medicine. 2017. Principles and Practices for Federal Program Evaluation: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/24831.
×
| (For discussion panels, each moderator will open the session with initial remarks that will lay out a coherent framework for the task and issues at hand. The steering committee members will offer their opinions, and then attendees will have an opportunity to share their thoughts and insight. We encourage active audience participation during these sections.) | |
| 12:00pm | Lunch |
| 1:00pm | Issues and Challenges for Implementing Principles & Practices: A Discussion |
| Moderator: Judy Gueron, MDRC President Emerita (Committee Member) | |
| Although the five principles in current agency guidance—rigor, relevance, transparency, independence, and ethics—seem uncontestable, challenges arise in balancing them, especially since advancing high-quality evaluations requires both obtaining sustained funding and engaging the best talent. This discussion will focus on the components necessary to advance high-quality evaluations, protect the infrastructure that supports them, and ensure that the evaluations produce results that a broad community of politicians, practitioners, and funders consider objective and useful. | |
| 2:00pm | How Do We Institutionalize the Major Principles? (Discussion) |
| Moderator: Bill Sabol, Westat (Committee Member) | |
| Discussant: Bethanne Barnes (OMB) | |
| Even with broad support for enhancing the principles outlined in the previous discussion—quality, utility, transparency, independence, and high ethical standards—these goals cannot be achieved by the efforts of evaluation staff alone. Rather, these principles need to be institutionalized into agency practices so that evaluators are protected against efforts to quash them. Similarly, ensuring an adequate funding base for evaluations requires agency-level commitments to developing knowledge both about what works and the circumstances under which something works or does not work. Institutionalizing these principles may require legislative changes to statutory authorities, new organizational entities, or new |
Suggested Citation:"Appendix A: Workshop Agenda." National Academies of Sciences, Engineering, and Medicine. 2017. Principles and Practices for Federal Program Evaluation: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/24831.
×
| relationships between organizations. This discussion will focus on approaches that can be taken to institutionalize quality, relevance, and independence, the opportunities and challenges associated with various approaches, and pathways and priorities to implement the changes. | |
| 2:45pm | Break |
| 3:00pm | Garnering Support and Maintaining Focus: A Discussion |
| Moderator: Rebecca Maynard, University of Pennsylvania (Committee Member) | |
| Discussant: Jon Baron, Arnold Foundation | |
| Discussion will center on how to develop guidance that will directly serve the interests of evaluation offices within federal agencies while simultaneously mitigating the potential resistance from offices and organizations whose interests may be threatened by the formation of such a document (e.g., advocacy organizations and special interest groups). The discussion will consider vehicles and resources that can maximize support for objective evaluation across federal, state, and local levels of government, including executive and legislative branches. | |
| 4:00pm | Future of Principles & Practices for Federal Program Evaluation: A Discussion about Next Steps |
| Moderator: Russ Whitehurst, Brookings Institution (Committee Chair) | |
| This concluding session will focus on themes from the preceding sessions and consider potential future steps for articulating and strengthening principles and practices for federal program evaluation and the evaluation function itself. | |
| 4:30pm | Adjourn |
