Chapter 1 introduced the study charge and briefly described trends in the larger social, economic, educational, and scientific and technological context that may influence the quality of undergraduate STEM education. In this chapter the committee focuses more narrowly on dimensions of undergraduate STEM education that are closely related to student learning and success, presenting a simplified conceptual framework to guide its development of indicators.
As background for discussing the framework, the committee notes its conceptual process of arriving at the indicators proposed in this report. First, the committee adopted a systems perspective on higher education. Then, it identified three overarching goals for improving undergraduate STEM education, asking: What are the key targets that represent the best leverage toward the committee’s vision for undergraduate STEM education? After identifying these goals, the committee then operationalized each one by identifying specific objectives, or elements of the goal, that need to be addressed in order to meet the goal in its entirety. Identifying these discrete objectives, described below, allowed the committee to move forward with developing specific indicators, designed to measure progress toward meeting the specified objectives, and ultimately, to monitor the status and quality of undergraduate STEM education. The committee’s conceptual framework represents the process of students moving through higher education as institutions seek to produce graduates capable of meeting the grand challenges of society (as mentioned in Chapter 1): see Figure 2-1. This overall framework will enable readers to envision the conceptual basis for the proposed indicator system, indicating how each
goal (and supporting objectives) maps onto the higher education system in all its complexity: see Figure 2-2.
As a first step in developing the conceptual framework, the committee adopted an organizational systems perspective (Katz and Kahn, 1966, 1978). Viewing U.S. undergraduate education as a complex, open system facilitates understanding of how to improve it. For example, Austin (2011) used a systems approach to identify and understand factors influencing individual STEM faculty members’ decisions about adopting evidence-based teaching strategies. Adopting a similar systems perspective, the committee’s conceptual framework begins with a generic process model of the higher education system: see Figure 2-1. The model has four components: inputs (students entering higher education); processes (educational experiences of the students); environment (which shapes the process); and outcomes (students leaving higher education with skills and knowledge).
The committee recognizes that the process of undergraduate STEM education is not always linear, as depicted in Figure 2-1. Students follow increasingly complex trajectories through undergraduate education, transferring across institutions, dropping out for periods, and switching into and out of STEM majors at different times (National Academies of Sciences, Engineering, and Medicine, 2016a). STEM instructors—including faculty members, adjunct faculty, graduate teaching assistants, and all others who teach undergraduates—work within multiple, interacting contexts and feedback loops that influence decisions about teaching. Austin (2011) viewed
these contexts—including departments, colleges, institutions, and such external groups as accrediting associations, parents and employers, and state and federal governments—as “levels” of the system that influence instructors’ work. She also observed that various elements in organizations can serve either as useful levers for or as barriers to change. Key levers in colleges and universities that may encourage or discourage adoption of evidence-based teaching strategies include evaluation and reward systems, workload allocation, professional development opportunities, and the strategic use of leadership practices. Thus, linear approaches to change that address only one factor or intervention are unlikely to lead to sustained adoption of evidence-based teaching practices. Austin (2011) concluded that, in such complex organizations, change efforts are most likely to be effective when they use both a “top-down” and a “bottom-up” approach, consider the multiple factors and contexts that influence instructors’ work, and strategically use multiple levers of change.
The committee recognizes that the environment surrounding undergraduate STEM includes not only the external groups and individuals mentioned above (accreditors, parents, employers), but also others, such as disciplinary bodies and K-12 educators. However, the committee’s generic process model focuses on the most immediate components of the higher education environment—departments, colleges, and institutions—reflecting its charge to develop indicators for undergraduate STEM education.
Following from this model of higher education as a complex system, the committee addressed its charge to “identify objectives for improving undergraduate STEM education and a set of indicators to document the status and quality of undergraduate STEM education at the national level over multiple years.” The committee identified three overarching goals for improving the quality of undergraduate STEM education. It then drew on relevant literature to identify objectives related each of the three goals. The specific targets for improvement reflected in these goals and objectives provide a focus for monitoring the status of undergraduate STEM education over time.
GOAL 1: Increase Students’ Mastery of STEM Concepts and Skills by Engaging Them in Evidence-Based STEM Educational Practices and Programs. Engage undergraduate students in STEM learning experiences and programs backed by research and supported by evidence.
GOAL 2: Strive for Equity, Diversity, and Inclusion of STEM Students and Instructors by Providing Equitable Opportunities for Access and Success. Broaden participation such that the students participating in undergraduate STEM programs are representative of the demographics of the national population of undergraduate students. Ensure that STEM learning environments are inclusive and effectively engage and educate diverse learners.
GOAL 3: Ensure Adequate Numbers of STEM Professionals by Increasing Completion of STEM Credentials as Needed in the Different STEM Disciplines. Increase the number of graduates and certificate holders to meet the grand challenges of society.
In developing goals and the objectives that follow from them, the committee considered not only its basic framework (refer to Figure 2-1) but also other models of change in higher education (e.g., Elrod and Kezar, 2015, 2016; Henderson, Beach, and Finkelstein, 2011). These various models of undergraduate education as a complex, interacting system were helpful as the committee considered the most important levers for improvement and identified objectives to be monitored through an indicator system.
In addition, the committee derived its goals in part from a similar set of statements in the recent report, Monitoring Progress Toward Successful K–12 STEM Education (National Research Council, 2013) which in turn followed a related report on K–12 STEM education (National Research
Council, 2011). Although there are clear parallels between the goals discussed in that pair of reports and the committee’s three goals, the committee’s goals reflect the different challenges and contexts of the K–12 and the higher education sectors. In response to policy makers’ questions and increasing accountability pressures, the higher education sector is particularly concerned about students’ outcomes, especially the employment outcomes that are reflected in Goal 3. However, ensuring adequate numbers of STEM professionals (Goal 3) will not be possible without first attending to the STEM educational processes and environment reflected in Goals 1 and 2.
These three goals are interconnected and mutually supportive, targeting improvement in various elements of the undergraduate education system and the interactions of these elements that together will enhance students’ success in STEM education. Advancing the goals will require strategic use of multiple change levers within and across the multiple levels of the higher education system, using both top-down and bottom-up approaches (Austin, 2011). The goals are applicable to all varieties of undergraduate STEM educational experiences and are designed to enhance those experiences to the greatest extent possible. The systems perspective reflected in these goals is also essential in developing indicators to monitor progress, because an educational indicator system not only measures an educational system’s inputs, processes, and outputs, but also suggests how they work together to produce an overall effect on students (Odden, 1990, pp. 24-25; Shavelson, McDonnell, and Oakes, 1991).
A growing body of research has identified the STEM teaching and learning experiences and equity and inclusion strategies that support all students’ mastery of STEM concepts and skills and persistence to graduation. Widely deploying these evidence-based processes is essential to ensure adequate numbers of STEM professionals. As noted in Chapter 1, the most rapidly growing groups within the general population are often underrepresented in STEM education and employment fields. These groups provide an untapped resource of talent, and Goal 2 focuses on changing the educational processes and environment to increase their engagement and success in undergraduate STEM education (Summers and Hrabowski, 2006; National Academy of Sciences, National Academy of Engineering, and Institute of Medicine 2011; National Academies of Sciences, Engineering, and Medicine, 2016a). Thus, advancing the three complementary goals will sustain a robust STEM workforce that contributes to national economic growth and international competitiveness (President’s Council of Advisors on Science and Technology, 2012; Xie and Killewald, 2012). The rest of this section discusses the committee’s three goals in more detail.
As noted in Chapter 1, parents, employers, policy makers, and society at large often ask whether, and to what extent, students are learning the content, skills, and abilities that will serve them for their lives and careers after graduation. And as discussed in Chapter 1, there is no simple way to answer this question.
Although there are no agreed-upon measures of students’ STEM learning, an abundance of research has demonstrated that certain common approaches to teaching, learning, and co-curricular programs can improve student learning and degree completion in STEM disciplines (Fairweather, 2012; Kober, 2015; National Research Council, 2012). At the same time, research has shown that poor instructional strategies often discourage persistence in STEM programs of study even among students who are academically capable to engage in them and were originally interested in STEM fields (Correll, Seymour, and Hewitt, 1997). However, the strategies that have evidence of effectiveness have not yet been widely implemented (National Research Council, 2012). Goal 1 addresses this problem, calling for broad implementation of teaching approaches and programs that researchers have identified as most effective for helping students master core STEM concepts and skills. Doing so will require that institutions and their academic units examine their underlying approaches to teaching and curricular design (Elrod and Kezar, 2015, 2016; Henderson, Beach, and Finkelstein, 2011; Weaver et al., 2015). Institutions will also need to consider the educational experiences they offer outside the classroom, such as internships, mentoring, and advising.
The work of engaging students in evidence-based teaching and learning experiences and co-curricular programs rests partly on the shoulders of instructors and staff—those who are on the front lines in education. However, as noted above, the work of these individuals is embedded in a complex system that involves norms, resources, evaluation systems, and reward and recognition practices in departmental, institutional, and disciplinary cultures—the educational environment depicted in Figure 2-1 (Austin, 2011; Weaver et al., 2015). Thus, increasing the use of evidence-based STEM educational practices can only happen with support from departmental and institutional cultures. Such support includes real alignment between institutions’ statements about the value of undergraduate teaching and learning and the explicit valuing of teaching by those institutions. Reward and recognition structures will need to be part of that explicit valuing and robust, reliable forms of evaluating instruction will have to be put in place to make that possible. These approaches will also allow the improvement of students’ educational experiences to be implemented in a scholarly way, based on existing literature and depending on evidence for continuous improvement.
Goal 2 involves broadening participation so that the students who participate in postsecondary STEM programs are representative of the national population who could participate in those programs and ensuring that STEM learning environments are inclusive and effectively engage and educate diverse learners. Equity, diversity, and inclusion are distinct concepts; yet, all three are critically important to ensuring that the STEM educational system meets the nation’s needs and serves all people (Association of American Colleges & Universities, 2015; Withem et al., 2015).
The goal of striving for equity, diversity, and inclusion so that STEM learners are as diverse as the country’s national talent pool and that STEM workforce opportunities are equally available to all is both ethical and critical to continuing national innovation and global competitiveness. In comparison with previous generations of undergraduates, today’s undergraduate students are more likely to be female, Black, Hispanic, from low-income families, and single parents (National Academies of Sciences, Engineering, and Medicine, 2016a). Although recent data show that these populations are as interested in STEM fields as their white peers, they are far less likely to complete STEM degrees. Retaining diverse students in STEM that reflect the national population is essential to achieving the increased numbers of STEM students called for in Goal 3.
Many of today’s most challenging scientific and technical issues are global in nature and can best be addressed by combining diverse expertise across disciplinary boundaries, along with community perspectives (National Research Council, 2015). Recent research suggests that science teams comprised of ethnically and geographically diverse members may be more effective than those that are more homogeneous (Freeman and Huang, 2014a,b). More broadly, as the national economy continues to recover from recession, providing equitable employment opportunities for women, minorities, and people with disabilities would facilitate economic growth and reduce income inequality, according to OECD (2016).
Goal 3 seeks to increase the numbers of students who complete STEM credential and degree programs, both to meet demand for STEM professionals in some fields of STEM and to prepare these graduates to participate fully in society. The committee assumes that engaging students in evidence-based STEM educational practices (Goal 1) and ensuring their full inclusion and equity (Goal 2) will increase the number of students who receive STEM credentials (Goal 3). Progress toward Goal 3 will be influenced by many factors, such as admission review processes, summer “bridge” programs,
and recruitment practices. In proposing Goal 3, the committee assumes that a more technologically oriented society requires more people with expertise in science and engineering for economic success and to meet global competition (President’s Council of Advisors on Science and Technology, 2012; Xie and Killewald, 2012). But it does not assume that all STEM graduates will be part of the STEM workforce to reap these economic benefits. Rather, the committee thinks that increasing the number of people with an understanding of STEM ideas and ways of thinking will benefit all segments of society. As discussed in Chapter 1, STEM knowledge and skills are valuable in a broad range of occupations, beyond those formally classified as STEM occupations (Carnevale, Smith, and Melton, 2011; Rothwell, 2013).
The conceptual framework represented by the committee’s generic process model and its three overarching goals could be articulated as many different potential objectives for improving undergraduate STEM education. To identify what it considers to be the most important objectives for improving the quality of undergraduate STEM, the committee took several steps, which are discussed below.
Following the committee charge to consider the federal STEM education strategic plan as a starting point, we reviewed the plan of the National Science and Technology Council (NSTC) (2013), which included the goal of enhancing undergraduates’ STEM experiences as a way to reduce student attrition from STEM majors and thus help achieve the prior federal goal of graduating 1 million additional STEM majors over the next decade (President’s Council of Advisors on Science and Technology, 2012). The committee’s framework follows a similar approach.
As noted in Chapter 1, NSTC identified four strategic objectives for improving students’ undergraduate experiences and reducing attrition: (1) promoting evidence-based instructional practices; (2) improving STEM experiences in community colleges; (3) expanding undergraduate research experiences; and (4) advancing success in the key gateway of introductory mathematics. The committee adopted NSTC objective (1), modifying it to incorporate aspects of NSTC objective (3) (see below). Because its charge encompasses STEM education at both 2-year and 4-year institutions, the committee’s proposed objectives aim for improvement at both types of institutions and we did not adopt a specific objective similar to NSTC objective (2). The committee adopted NSTC objective (4) but broadened it to address
retention of students in key gateway courses in all STEM fields. The committee notes that elements of NSTC objectives (2) and (3) were specific to the federal government’s role, calling for increased federal support of certain aspects of undergraduate STEM and do not represent broad national objectives for the U.S. higher education system as a whole.
Students’ attainment of STEM credentials (e.g., certificates, degrees) and development of STEM knowledge and skills are complex processes, influenced by a variety of factors that operate within and across multiple layers of the educational system. Although each student (i.e., background, cognitive and social-psychological characteristics, and level of preparation) is a central actor in these processes, it is now widely understood that a student’s experiences, the larger college environment, and the instructors and staff all play a critical role in a student’s progress (e.g., Astin, 1993; Braxton, 2000; Kuh et al., 2007; Tinto, 1993). This is true for students in all fields, and it has been specifically demonstrated for students in STEM fields (Xie, Fang and Shauman, 2015).
To identify the most important objectives for improving undergraduate STEM within this complex system, the committee reviewed research related to its three goals, focusing on the factors identified in the research as most critical for advancing these goals. Drawing on the literature review in a related National Academies study (National Academies of Sciences, Engineering, and Medicine, 2016a), the committee considered factors at multiple levels of the higher education system. To weigh the importance of various factors emerging from the literature related to each goal, the committee adopted the following criteria for identifying objectives:
- Evidence of importance or efficacy to STEM educational outcomes: To what extent is there evidence to link the objective to the desired outcomes? The committee sought to identify the most important, high leverage points within the higher education system depicted in Figure 2-1.
- Applicability across multiple institution types. To what extent is the objective relevant to all of the diverse types of 2-year and 4-year, public, and private higher education institutions in the United States? This criterion reflects the committee’s charge to develop objectives for improving undergraduate STEM at both 2-year and 4-year institutions and to develop a national indicator system relevant across all types of institutions.
- Emphasis on first 2 years. To what extent is the objective relevant to the first 2 years of undergraduate STEM? This criterion reflects the committee’s charge to focus on the first 2 years of undergraduate STEM. Given that STEM course-taking and performance during the first 2 years of college are key determinants of persistence in STEM (Bettinger, 2010; Chen and Soldner, 2013) and that much attrition from STEM programs occurs within the first 2 years (Chang et al., 2008; Seymour and Hewitt, 1997), these years are critical for improving student success in STEM. Thus, the framework emphasizes objectives relevant to the first 2 years, while still leaving room to include highly important processes or characteristics relevant beyond the first 2 years.
The committee also considered several cross-cutting issues. As mentioned in Chapter 1, institutions of higher education across the country have enormously different aims and missions related to STEM education and, relatedly, serve hugely diverse student populations. As a result, STEM coursework and curricular standards vary from institution to institution, and students who may be prepared to do the work in one institution may be ill-equipped to meet the standards of another. Across the objectives listed below it is critical to consider the necessary differences in how institutions meet their own stated goals in light of the preparation and expectations of their student populations.
The committee selected 11 objectives for improving undergraduate STEM, grouped under the committee’s three overarching goals.
GOAL 1: Increase Students’ Mastery of STEM Concepts and Skills by Engaging Them in Evidence-Based STEM Educational Practices and Programs.
1.1 Use of evidence-based STEM educational practices both in and outside of classrooms
1.2 Existence and use of supports that help instructors use evidence-based STEM educational practices
1.3 An institutional culture that values undergraduate STEM education
1.4 Continuous improvement in STEM teaching and learning
GOAL 2: Strive for Equity, Diversity, and Inclusion of STEM Students and Instructors by Providing Equitable Opportunities for Access and Success.
2.1 Equity of access to high-quality undergraduate STEM educational programs and experiences
2.2 Representational diversity among STEM credential earners
2.3 Representational diversity among STEM instructors
2.4 Inclusive environments in institutions and STEM departments
GOAL 3: Ensure Adequate Numbers of STEM Professionals.
3.1 Foundational preparation for STEM for all students
3.2 Successful navigation into and through STEM programs of study
3.3 STEM credential attainment
These objectives and their relationship to the three goals are shown in Figure 2-2. The objectives are designed to improve the quality in each component of the basic conceptual framework: inputs, processes, environment, and outcomes. However, the objectives primarily target improvement of the educational processes, environments, and outcomes. Although the inputs, the incoming students, influence the quality of undergraduate STEM education, some of the characteristics of the students reflect K–12 preparation, which lies outside the scope of the study charge.
The detailed framework shown in Figure 2-2 illustrates students’ entrance to 2-year or 4-year colleges, their STEM-related learning experiences inside and outside the classroom, the environments that surround students and instructors, and student outcomes, including credentials and knowledge of STEM concepts and skills.
The objectives identified in the detailed framework drove the committee’s development of indicators: Table 2-1 presents the committee’s proposed indicators in concert with the committee’s framework and objectives. The next three chapters of the report describe those objectives and indicators.
In this chapter, the committee has proposed a conceptual framework for the indicator system. Beginning with a model of higher education as
TABLE 2-1 Framework, Objectives, and Indicators
|GOAL 1: Increase Students’ Mastery of STEM Concepts and Skills by Engaging Them in Evidence-Based STEM Educational Practices and Programs|
|Process||1.1 Use of evidence-based STEM educational practices both in and outside of classrooms||1.1.1 Use of evidence-based STEM educational practices in course development and delivery|
|1.1.2 Use of evidence-based STEM educational practices outside the classroom|
|Environment||1.2 Existence and use of supports that help STEM instructors use evidence-based learning experiences||1.2.1 Extent of instructors’ involvement in professional development|
|1.2.2 Availability of support or incentives for evidence-based course development or course redesign|
|Environment||1.3 An institutional culture that values undergraduate STEM instruction||1.3.1 Use of valid measures of teaching effectiveness|
|1.3.2 Consideration of evidence-based teaching in personnel decisions by departments and institutions|
|Process||1.4 Continuous improvement in STEM teaching and learning||No indicators: see “Challenges of Measuring Continuous Improvement” in Chapter 3|
|GOAL 2: Strive for Equity, Diversity, and Inclusion of STEM Students and Instructors by Providing Equitable Opportunities for Access and Success|
|Input||2.1 Equity of access to high-quality undergraduate STEM educational programs and experiences||2.1.1 Institutional structures, policies, and practices that strengthen STEM readiness for entering and enrolled college students|
|2.1.2 Entrance to and persistence in STEM academic programs|
|2.1.3 Equitable student participation in evidence-based STEM educational practices|
|Outcome||2.2 Representational diversity among STEM credential earners||2.2.1 Diversity of STEM degree and certificate earners in comparison with diversity of degree and certificate earners in all fields|
|2.2.2 Diversity of students who transfer from 2-year to 4-year STEM programs in comparison with diversity of students in 2-year STEM programs|
|2.2.3 Time to degree for students in STEM academic programs|
|Environment||2.3 Representational diversity among STEM instructors||2.3.1 Diversity of STEM instructors in comparison with diversity of STEM graduate degree holders|
|2.3.2 Diversity of STEM graduate student instructors in comparison with diversity of STEM graduate students|
|Environment||2.4 Inclusive environments in institutions and STEM departments||2.4.1 Students pursuing STEM credentials feel included and supported in their academic programs and departments|
|2.4.2 Instructors teaching courses in STEM disciplines feel supported and included in their departments|
|2.4.3 Institutional practices are culturally responsive, inclusive, and consistent across the institution|
|GOAL 3: Ensure Adequate Numbers of STEM Professionals by Increasing Completion of STEM Credentials as Needed in the Different STEM Disciplines|
|Process||3.1 Adequate foundational preparation for STEM for all students||3.1.1 Completion of foundational courses, including developmental education courses, to ensure STEM program readiness|
|Process||3.2 Successful navigation into and through STEM programs of study||3.2.1 Retention in STEM programs, course to course and year to year|
|3.2.2 Transfers from 2-year to 4-year STEM programs in comparison with transfers to all 4-year programs|
|Outcome||3.3 STEM credential attainment||3.3.1 Number of students who attain STEM credentials over time, disaggregated by institution type, transfer status, and demographic characteristics|
a complex system, the committee identified three overarching goals for improving the quality of undergraduate STEM education. It then built on the federal STEM education strategic plan and drew on relevant literature to articulate each goal into more specific objectives. The specific targets for improvement reflected in these goals and objectives provide a focus for monitoring the status of undergraduate STEM education over time.
CONCLUSION 1 Improving the quality and impact of undergraduate STEM education will require progress toward three overarching goals:
Goal 1: Increase students’ mastery of STEM concepts and skills by engaging them in evidence-based STEM educational practices and programs.
Goal 2: Strive for equity, diversity, and inclusion of STEM students and instructors by providing equitable opportunities for access and success.
Goal 3: Ensure adequate numbers of STEM professionals by increasing completion of STEM credentials as needed in the different STEM disciplines.
Association of American Colleges & Universities. (2015). Committing to Equity and Inclusive Excellence: A Campus Guide for Self-Study and Planning. Washington, DC: Author.
Astin, A.W. (1993). What Matters in College? Four Critical Years Revisited. San Francisco, CA: Jossey-Bass.
Austin, A. (2011). Promoting Evidence-Based Change in Undergraduate Science Education. Paper commissioned by the Board on Science Education. Available: http://sites.nationalacademies.org/cs/groups/dbassesite/documents/webpage/dbasse_072578.pdf [June 2016].
Bettinger, E. (2010). To be or not to be: Major choices in budding scientists. In C.T. Clotfelter (Ed.), American Universities in a Global Market (pp. 69–98). Chicago, IL: University of Chicago Press.
Braxton, J.M. (Ed.). (2000). Reworking the Student Departure Puzzle. Nashville, TN: Vanderbilt University Press.
Carnevale, A.P., Smith, N., and Melton, M. (2011). STEM: Science, Technology, Engineering, and Mathematics. Washington, DC: Georgetown University Center on Education and the Workforce. Available: https://cew.georgetown.edu/wp-content/uploads/2014/11/stem-complete.pdf [July 2017].
Chang, M.J., Cerna, O., Han, J., and Sáenz, V. (2008). The contradictory roles of institutional status in retaining underrepresented minorities in biomedical and behavioral science majors. The Review of Higher Education, 31(4), 433–464.
Chen, X., and Soldner, M. (2013). STEM Attrition: College Students’ Paths into and out of STEM Fields. Washington, DC: U.S. Department of Education.
Correll, S.J., Seymour, E., and Hewitt, N.M. (1997). Talking about leaving: Why undergraduates leave the sciences. Contemporary Sociology, 26, 644. doi10.2307/2655673.
Elrod, S., and Kezar, A. (2015). Increasing student success in STEM. Peer Review, 17(2). Available: https://www.aacu.org/peerreview/2015/spring/elrod-kezar [June 2016].
Elrod, S., and Kezar, A. (2016). Increasing Student Success in STEM: A Guide to Systemic Institutional Change. Washington, DC: Association of American Colleges & Universities.
Fairweather, J. (2012). Linking Evidence and Promising Practices in Science, Technology, Engineering, and Mathematics (STEM) Undergraduate Education: A Status Report for The National Academies National Research Council Board on Science Education. Available: http://sites.nationalacademies.org/cs/groups/dbassesite/documents/webpage/dbasse_072637.pdf [June 2016].
Freeman, R.B., and Huang, W. (2014a). Collaborating with People Like Me: Ethnic Co-Authorship within the U.S. NBER Working Paper No. 19905. Cambridge, MA: National Bureau of Economic Research. Available: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.702.2502&rep=rep1&type=pdf [February 2018].
Freeman, R.B., and Huang, W. (2014b). Strength in diversity. Nature, 513, 305.
Henderson, C., Beach, A., and Finkelstein, N. (2011). Facilitating change in undergraduate STEM instructional practices: An analytic review of the literature. Journal of Research in Science Teaching, 48(8), 952–984. doi10.1002/tea.20439.
Katz, D., and Kahn, R.L. (1966). The Social Psychology of Organizations. New York: John Wiley & Sons.
Katz, D., and Kahn, R.L. (1978). The Social Psychology of Organizations. 2nd ed. New York: John Wiley & Sons.
Kober, N. (2015). Reaching Students: What Research Says About Effective Instruction in Undergraduate Science and Engineering. Washington, DC: The National Academies Press. Available: http://www.nap.edu/catalog/18687/reaching-students-what-research-says-about-effective-instruction-in-undergraduate [June 2016].
Kuh, G.D., Kinzie, J., Buckley, J., Bridges, B., and Hayek, J.C. (2007). Piecing Together the Student Success Puzzle: Research, Propositions, and Recommendations. (ASHE Higher Education Report, vol. 32, issue 5). San Francisco, CA: Jossey-Bass.
National Academies of Sciences, Engineering, and Medicine. (2016a). Barriers and Opportunities for 2-Year and 4-Year STEM Degrees: Systemic Change to Support Students’ Diverse Pathways. Washington, DC: The National Academies Press. Available: http://www.nap.edu/catalog/21739/barriers-and-opportunities-for-2-year-and-4-year-stem-degrees [June 2016].
National Academies of Sciences, Engineering, and Medicine. (2016b). Science Literacy: Concepts, Contexts, and Consequences. Washington, DC: The National Academies Press.
National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. (2011). Expanding Underrepresented Minority Participation: America’s Science and Technology Talent at the Crossroads. Washington, DC: The National Academies Press. Available: http://www.nap.edu/catalog/12984/expanding-underrepresented-minority-participation-americas-science-and-technology-talent-at [June 2016].
National Research Council. (2011). Successful K-12 STEM Education: Identifying Effective Approaches in Science, Technology, Engineering, and Mathematics (STEM). Washington, DC: The National Academies Press. Available: http://www.nap.edu/catalog/13158/successful-k-12-stem-education-identifying-effective-approaches-in-science [June 2016].
National Research Council. (2012). Discipline-Based Education Research: Understanding and Improving Learning in Undergraduate Science and Engineering. Washington, DC: The National Academies Press. Available: https://www.nap.edu/catalog/13362/discipline-based-education-research-understanding-and-improving-learning-in-undergraduate [July 2017].
National Research Council. (2013). Monitoring Progress Toward Successful K-12 STEM Education: A Nation Advancing? Washington, DC: The National Academies Press. Available: https://www.nap.edu/search/?term=Monitoring+Progress+Toward+Successful+K-12+STEM+Education%3A+A+Nation+Advancing%3F.+&x=16&y=6 [July 2017].
National Research Council. (2015). Enhancing the Effectiveness of Team Science. Washington, DC: The National Academies Press. Available: http://www.nap.edu/catalog/19007/enhancing-the-effectiveness-of-team-science [June 2016].
National Science and Technology Council. (2013). Federal Science, Technology, Engineering, and Mathematics (STEM) 5-Year Strategic Plan. Washington, DC: Author. Available: https://www.whitehouse.gov/sites/default/files/microsites/ostp/stem_stratplan_2013.pdf [June 2016].
Odden, A. (1990). Educational indicators in the United States: The need for analysis. Educational Researcher, 19(5), 24–29.
OECD. (2016). OECD Economic Surveys: United States. Available: http://www.oecd.org/eco/surveys/United-States-2016-overview.pdf [June 2016].
President’s Council of Advisors on Science and Technology. (2012). Engage to Excel: Producing One Million Additional College Graduates with Degrees in Science, Technology, Engineering and Mathematics. Washington, DC: Author. Available: https://www.whitehouse.gov/sites/default/files/microsites/ostp/pcast-engage-to-excel-final_feb.pdf [March 2016].
Rothwell, J. (2013). The Hidden STEM Economy. Metropolitan Policy Program at Brookings Institution. Available: http://www.brookings.edu/~/media/research/files/reports/2013/06/10-stem-economy-rothwell/thehiddenstemeconomy610.pdf [April 2015].
Seymour, E., and Hewitt, N. (1997). Talking about Leaving: Why Undergraduates Leave the Sciences. Boulder, CO: Westview Press.
Shavelson, R.J., McDonnell, L., and Oakes, J. (1991). What are educational indicators and indicator systems? Practical Assessment, Research and Evaluation, 2(11). Available: http://pareonline.net/getvn.asp?v=2andn=11 [July 2017].
Summers, M.F., and Hrabowski III, F.A. (2006). Preparing minority scientists and engineers. Science, 311(5769), 1870–1871.
Tinto, V. (1993). Leaving College: Rethinking the Causes and Cures of Student Attrition. (second ed.). Chicago, IL: University of Chicago Press.
Weaver, G.C., Burgess, W.D., Childress, A.L., and Slakey, L. (2015). Transforming Institutions: Undergraduate STEM Education for the 21st Century. West Lafayette, IN: Purdue University Press.
Withem, K., Malcom-Piqueux, L., Dowd, A.C., and Bensimon, E.M. (2015). America’s Unmet Promise: The Imperative for Equity in Higher Education. Washington, DC: American Association of Colleges & Universities.
Xie, Y., Fang, M., and Shauman, K. (2015). STEM education. Annual Review of Sociology, 41, 331–357.
Xie, Y., and Killewald, A.A. (2012). Is American Science in Decline? Cambridge, MA: Harvard University Press.
This page intentionally left blank.