Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Prepub blication C Copy Uncorrrected Pro oofs Indic cators fo Monit or toring Undergr U raduate STEM Educati ion Committe on Develo ee oping Indica ators for Und dergraduate STEM Educ cation Mar B. Rosenb rk berg, Margar L. Hilton and Kenne A. Dibner, Editors ret n, e Board on Science Edu S ucation Division of Behavioral an Social Sc D B nd ciences and E Education A Consensus Study R Report of ADVANCE CO OPY NOT FOR PUBLIC RELEAS BEFORE T SE E Wedn nesday, Decembe 13, 20 17 D er 11:00 am. E 1 EST
PREPUBLICATION COPY, UNCORRECTED PROOFS THE NATIONAL ACADEMIES PRESS 500 Fifth Street, NW Washington, DC 20001 This study was supported by Contract/Grant No. 1533989 between the National Academy of Sciences and the National Science Foundation. Any opinions, findings, conclusions, or recommendations expressed in this publication do not necessarily reflect the views of any organization or agency that provided support for the project. International Standard Book Number-13: xxxx International Standard Book Number-10: xxxx Digital Object Identifier: https://doi.org/10.17226/24943 Additional copies of this publication are available for sale from the National Academies Press, 500 Fifth Street, NW, Keck 360, Washington, DC 20001; (800) 624-6242 or (202) 334-3313; http://www.nap.edu. Copyright 2017 by the National Academy of Sciences. All rights reserved. Printed in the United States of America Suggested citation: National Academies of Sciences, Engineering, and Medicine. (2017). Indicators for Monitoring Undergraduate STEM Education. Washington, DC: The National Academies Press. doi: https://doi.org/10.17226/24943
PREPUB BLICATION COPY, UN N NCORRECT TED PROOF FS The Nattional Acaddemy of Sci iences was establishe in 1863 b an Act o Congress, s ed by of , signed by President Lincoln, as a private nongover b a e, rnmental in nstitution t advise th to he nation on issues re o elated to science and technology Members are electe by their t y. ed peers fo outstandi contributions to re or ing esearch. Dr Marcia M r. McNutt is prresident. The Nattional Acad demy of Engineering was establi w ished in 196 under th charter of 64 he the Nati ional Acadeemy of Scie ences to bri the prac ing ctices of enngineering to advising the g nation. Members are elected by their pe M eers for ext traordinary contributi y ions to engineering. Dr. C. D. Mote, Jr., is presi . J ident. The Nattional Acad demy of Me edicine (forrmerly the Institute of Medicine) was ) establish in 1970 under the charter of the Nation Academ of Sciences to advi hed 0 e f nal my ise the nation on mediical and heealth issues. Members are elected by their peers for distinguished contr ributions to medicine and health Dr. Victo J. Dzau is president o h. or s t. The thre Academi work to ee ies ogether as the Nationa Academi of Sciences, t al ies Engineeering, and Medicine to provide independen objectiv analysis and advise to M nt, ve e the nation and connduct other activities to solve co r omplex probblems and inform pub blic policy decisions. The Nationa Academie also enco al es ourage edu ucation and research, recogniz outstand ze ding contrib butions to knowledge, and increa public understand k , ase ding in matte of scien ers nce, engineering, and medicine. Learn more about the Nationa Academi of Scien m al ies eering, and Medicine at nces, Engine d www.na ationalacad demies.orgg.
PREPUBLICATION COPY, UNCORRECTED PROOFS Consensus Study Re eports published by the National Ac e cademies of Sciences, Engineering, and f Medicine document the evidence-based co e onsensus on the studyâ statemen of task b an âs nt by authoring committe of experts. Report typically include findings, c g ee ts y conclusions, and recomme endations based on inf formation gathered by the comm g y mittee and t the commit tteeâs deliberat tions. Each report has been subje ected to a rigorous an independ nd dent peer-reeview process and it repres a sents the po osition of the National A e Academies o the statem on ment of task k. Proceedings publish by the National Academie hed e es of Sci iences, Enngineering, and Medicine chronicle the presen e ntations an discussio nd ons at a workshop, symposium orm, other eve convene by the Na ent ed ational Acad demies. The statement and opinions contained in e ts proceedings are tho of the participants and are no endorsed by other p ose p ot d participants, the planning committee, or the Nati ional Academies. For information about other products an activitie of the N nd es ademies, please National Aca visit www w.nationalac cademies.or rg/about/wh hatwedo.
PREPUBLICATION COPY, UNCORRECTED PROOFS COMMITTEE ON DEVELOPING INDICATORS FOR UNDERGRADUATE STEM EDUCATION Mark Rosenberg (Chair), President, Florida International University Heather Belmont, School of Science, Miami Dade College Charles Blaich, Center of Inquiry and the Higher Education Data Sharing Consortium, Wabash College Mark Connolly, Wisconsin Center for Education Research, University of Wisconsin, Madison Stephen Director, Provost Emeritus, Northeastern University Kevin Eagan, Higher Education Research Institute, University of California, Los Angeles Susan Elrod, Academic Affairs, University of Wisconsin, Whitewater Kaye Husbands Fealing, School of Public Policy, Georgia Institute of Technology Stuart Feldman, Schmidt Sciences, Schmidt Philanthropies, Palo Alto, California Charles Henderson, Department of Physics and Mallinson Institute for Science Education, Western Michigan University Lindsey Malcom-Piqueux, Center for Urban Education, University of Southern California Marco Molinaro, Center for Educational Effectiveness University of California, Davis Rosa Rivera-Hainaj, Academic Affairs, Our Lady of the Lake University Gabriela Weaver, Faculty Development, University of Massachusetts, Amherst Yu Xie, Princeton Institute for International and Regional Studies Princeton University Margaret Hilton, Study Director Kenne Dibner, Deputy Study Director Brenezza Garcia, Consultant Leticia Garcilazo Green, Senior Program Assistant Heidi Schweingruber, Director, Board on Science Education FM-v
PREPUBLICATION COPY, UNCORRECTED PROOFS BOARD ON SCIENCE EDUCATION Adam Gamoran (Chair), William T. Grant Foundation, New York, New York Sunita V. Cooke, MiraCosta College Melanie Cooper, Department of Chemistry, Michigan State University Rodolfo Dirzo, Department of Biology, Stanford University Rush Holt, American Association for the Advancement of Science, Washington, DC Matthew Krehbiel, Achieve, Inc., Washington, DC Michael Lach, Urban Education Institute, University of Chicago Lynn S. Liben, Department of Psychology, Pennsylvania State University Cathy Manduca, Science Education Resource Center, Carleton College John Mather, Goddard Space Flight Center, National Aeronautics and Space Administration Tonya Matthews, Michigan Science Center, Detroit Brian Reiser, School of Education and Social Policy, Northwestern University Marshall âMikeâ Smith, Carnegie Foundation for the Advancement of Teaching, Stanford, CA Roberta Tanner, Thompson School District (retired), Loveland, Colorado Suzanne Wilson, Neag School of Education, University of Connecticut Heidi Schweingruber, Director FM-vi
PREPUBLICATION COPY, UNCORRECTED PROOFS Acknowledgments This Consensus Study Report represents the work of many individuals, especially those who served on the committee, wrote papers for it, and participated in the committeeâs open sessions. The first thanks are to the committee members for their deep knowledge and contributions to the study. This report was made possible by the important contributions of the National Science Foundation (NSF). We particularly thank Susan Singer, the former division director of NSFâs Division of Undergraduate Education. The committee benefited from presentations by, and discussions with, the many individuals who participated in our three fact-finding meetings, in January, February, and April 2016. We thank Jordan Matsudaira, Cornell University; Alexei Matveev, Southern Association of Colleges and Schools; Alicia Dowd, Pennsylvania State University; Beethika Khan, National Center for Science and Engineering Statistics; Matthew Wilson, National Science Board; Jeff Gold, California State University Office of the Chancellor; Chris Rasmussen, San Diego State University; Emily Miller and Josh Trapani, Association of American Universities; and Shirley Malcom, American Association for the Advancement of Science. The committee also thanks the experts who discussed the public comment draft during the committeeâs October 2016 public meeting: Adam Gamoran, William T. Grant Foundation; Susan Singer, Rollins College; Annette Parker, South Central College, Minnesota; Linda Slakey, University of Massachusetts, Amherst. and convener, Coalition for Reform of Undergraduate STEM Education; Kacy Redd, Association of Public and Land-Grant Universities; Susan Ambrose, Northeastern University, Boston, Massachusetts; Jillian Kinzie, Indiana University; Mica Estrada, University of California, San Francisco; Deborah Santiago, Excelencia in Education; and Lee Zia, NSF Division of Undergraduate Education. In addition, the committee benefited from the many individuals and organizations that provided written comments on the public comment draft. This Consensus Study Report has been reviewed in draft form by individuals chosen for their diverse perspectives and technical expertise. The purpose of this independent review is to provide candid and critical comments that will assist the National Academies of Sciences, Engineering, and Medicine in making its published report as sound as possible and to ensure that it meets institutional standards for objectivity, evidence, and responsiveness to the study charge. The review comments and draft manuscript remain confidential to protect the integrity of the deliberative process. We thank the following individuals for their review of this report: Ann Austin, Department of Educational Administration, Michigan State University George R. Boggs, President emeritus, Palomar College, San Marcos, CA Linnea Fletcher, Department of Biotechnology, Austin Community College, Austin, TX Adam Gamoran, President, W.T. Grant Foundation, New York, NY Judith Harackiewicz, Department of Psychology, University of Wisconsin-Madison Joan Herman, Graduate School of Education and Information Studies, University of California, Los Angeles Paul R. Hernandez, Department of Learnings Science and Human Development, West Virginia University Monika E. Kress, Department of Physics and Astronomy, San Jose State University FM-vii
PREPUBLICATION COPY, UNCORRECTED PROOFS Sally F. Mason, President Emerita, University of Iowa Andrew M. Penner, School of Social Sciences, University of California, Irvine Carl E. Wieman, Department of Physics, Stanford University Although the reviewers listed above provided many constructive comments and suggestions, they were not asked to endorse the content of the report nor did they see the final draft of the report before its release. The review of this report was overseen by Greg J. Duncan, School of Education, University of California, Irvine, and Paul R. Gray, Department of Electrical Engineering and Computer Sciences, University of California, Berkeley (emeritus). They were responsible for making certain that an independent examination of this report was carried out in accordance with the standards of the National Academies and that all review comments were carefully considered. Responsibility for the final content of this report rests entirely with the authoring committee and the National Academies. Thanks are also due to the project staff: Margaret Hilton, Kenne Dibner, Heidi Schweingruber, and Leticia Garcilazo Green, and to our consultant, Brenezza Garcia. Staff of the Division of Behavioral and Social Sciences and Education also provided help: Eugenia Grohman substantially improved the readability of the report, Kirsten Sampson Snyder expertly guided the report through the report review process, and Yvonne Wise masterfully managed the production of the report. Mark B. Rosenberg, Chair Committee on Developing Indicators for Undergraduate STEM Education FM-viii
PREPUBLICATION COPY, UNCORRECTED PROOFS Contents Summary 1 Introduction Interpreting the Study Charge Vision A Focus on the National Level Equity, Diversity, and Inclusion Goals and Objectives Measures and Indicators Undergraduate STEM Education Evidence-Based STEM Educational Practices and Programs Measuring College Quality in an Era of Accountability Employment Outcomes The STEM Workforce Learning Outcomes Goals of the Indicator System Study Approach and Organization of the Report References 2 Conceptual Framework for the Indicator System A Systems View of Higher Education Goals for Undergraduate STEM Education Goal 1: Increase Studentsâ Mastery of STEM Concepts and Skills Goal 2: Strive for Equity, Diversity, and Inclusion Goal 3: Ensure Adequate Numbers of STEM Professionals Articulating Goals as Objectives The Federal STEM Education Strategic Plan Criteria for Identifying Objectives The Objectives Proposed Indicators Conclusion References 3 Goal 1: Increase Studentsâ Mastery of STEM Concepts and Skills Objective 1.1: Use of Evidence-Based Educational Practices Both in and outside of Classrooms Importance of the Objective Proposed Indicators Objective 1.2: Existence and Use of Supports that Help STEM Instructors Use Evidence-Based Educational Practices Importance of the Objective Proposed Indicators Objective 1.3: An Institutional Culture that Values Undergraduate STEM Instruction FM-ix
PREPUBLICATION COPY, UNCORRECTED PROOFS Importance of the Objective Proposed Indicators Objective 1.4: Continuous Improvement in STEM Teaching and Learning Importance of the Objective Challenges of Measuring Continuous Improvement References 4 Goal 2: Strive for Equity, Diversity, and Inclusion Objective 2.1: Equity of Access to High-Quality Undergraduate STEM Educational Programs and Experiences Importance of the Objective Proposed Indicators Objective 2.2: Representational Diversity among STEM Credential Earners Importance of the Objective Proposed Indicators Objective 2.3: Representational Diversity among STEM Instructors Importance of the Objective Proposed Indicators Objective 2.4: Inclusive Environments in Institutions and STEM Departments Importance of the Objective Proposed Indicators References 5 Goal 3: Ensure Adequate Numbers of STEM Professionals Objective 3.1: Adequate Foundational Preparation for STEM for all Students Importance of the Objective Proposed Indicator Objective 3.2: Successful Navigation into and through STEM Programs of Study Importance of the Objective Proposed Indicators Objective 3.3: STEM Credential Attainment Importance of the Objective Proposed Indicator References 6 Existing Data Sources and Monitoring Systems Overview Public Data Sources The Integrated Postsecondary Education Data System The Beginning Postsecondary Students Longitudinal Study The National Survey of Postsecondary Faculty National Student Loan Data System State Unit-Record Data Systems Proprietary Data Sources National Student Clearinghouse Higher Education Research Institute Surveys FM-x
PREPUBLICATION COPY, UNCORRECTED PROOFS National Survey of Student Engagement Community College Survey of Student Engagement Faculty Survey of Student Engagement Monitoring Systems Science and Engineering Indicators Proprietary Monitoring Systems Data for Each Indicator Indicator 1.1.1: Use of Evidence-Based STEM Educational Practices in Course Development and Delivery Indicator 1.1.2: Use of Evidence-Based STEM Educational Practices outside the Classroom Indicator 1.2.1: Extent of Instructorâs Involvement in Professional Development Indicator 1.2.2: Availability of Support or Incentives for Evidence-Based Course Development or Course Redesign Indicator 1.3.1: Use of Valid Measures of Teaching Effectiveness Indicator 1.3.2: Consideration of Evidence-Based Teaching in Personnel Decisions by Departments and Institutions Indicator 2.1.1: Institutional Structures, Policies, and Practices that Strengthen STEM Readiness for Entering and Enrolled College Students Indicator 2.1.2: Entrance to and Persistence in STEM Academic Programs Indicator 2.1.3: Equitable Student Participation in Evidence-Based STEM Educational Programs and Experiences Indicator 2.2.1: Diversity of STEM Degree and Certificate Earners in Comparison with Diversity of Degree and Certificate Earners in All Fields Indicator 2.2.2: Diversity of Students Transferring from 2- to 4-Year STEM Programs in Comparison with Diversity of Students in 2- Year STEM Programs Indicator 2.2.3: Time-to-Degree for Students in STEM Academic Programs Indicator 2.3.1: Diversity of STEM Instructors in Comparison with the Diversity of STEM Graduate Degree Holders Indicator 2.3.2: Diversity of STEM Graduate Student Instructors in Comparison with the Diversity of STEM Graduate Students Indicator 2.4.1: Students Pursuing STEM Credentials Feel Included and Supported in Their Academic Programs and Departments Indicator 2.4.2: Instructors Teaching Courses in STEM Disciplines Feel Included and Supported in Their Departments Indicator 2.4.3 Institutional Practices Are Culturally Responsive, Inclusive, and Consistent across the Institution Indicator 3.1.1: Completion of Foundational Courses, Including Developmental Education Courses, to Ensure STEM Program Readiness FM-xi
PREPUBLICATION COPY, UNCORRECTED PROOFS Indicator 3.2.1: Retention in STEM Degree or Certificate Programs, Course to Course and Year to Year Indicator 3.2.2: Transfers from 2- to 4-Year STEM Programs in Comparison with Transfers to All 4-Year Programs Indicator 3.3.1 Percentage of Students Who Attain STEM Credentials over Time, Disaggregated by Institution Type, Transfer Status, and Demographic Characteristics Summary and Conclusions References 7 Implementing the Indicator System Option 1: Create a National Student Unit Record Data System Option 2: Expand NCES Data Collections Option 3: Combine Existing Data from Nonfederal Sources Conclusions Research, Evaluation, and Updating of the Proposed Indicator System A Note of Caution References Appendixes A Public Comments on Draft Report and Committee Response B Possible Formulas for Calculating Selected Indicators C Agendas: Workshop and Public Comment Meeting D Biographical Sketches of Committee Members and Staff FM-xii