2

Conceptual Framework for the Indicator System

Chapter 1 introduced the study charge and briefly described trends in the larger social, economic, educational, and scientific and technological context that may influence the quality of undergraduate STEM education. In this chapter the committee focuses more narrowly on dimensions of undergraduate STEM education that are closely related to student learning and success, presenting a simplified conceptual framework to guide its development of indicators.

As background for discussing the framework, the committee notes its conceptual process of arriving at the indicators proposed in this report. First, the committee adopted a systems perspective on higher education. Then, it identified three overarching goals for improving undergraduate STEM education, asking: What are the key targets that represent the best leverage toward the committee’s vision for undergraduate STEM education? After identifying these goals, the committee then operationalized each one by identifying specific objectives, or elements of the goal, that need to be addressed in order to meet the goal in its entirety. Identifying these discrete objectives, described below, allowed the committee to move forward with developing specific indicators, designed to measure progress toward meeting the specified objectives, and ultimately, to monitor the status and quality of undergraduate STEM education. The committee’s conceptual framework represents the process of students moving through higher education as institutions seek to produce graduates capable of meeting the grand challenges of society (as mentioned in Chapter 1): see Figure 2-1. This overall framework will enable readers to envision the conceptual basis for the proposed indicator system, indicating how each

goal (and supporting objectives) maps onto the higher education system in all its complexity: see Figure 2-2.

A SYSTEMS VIEW OF HIGHER EDUCATION

As a first step in developing the conceptual framework, the committee adopted an organizational systems perspective (Katz and Kahn, 1966, 1978). Viewing U.S. undergraduate education as a complex, open system facilitates understanding of how to improve it. For example, Austin (2011) used a systems approach to identify and understand factors influencing individual STEM faculty members’ decisions about adopting evidence-based teaching strategies. Adopting a similar systems perspective, the committee’s conceptual framework begins with a generic process model of the higher education system: see Figure 2-1. The model has four components: inputs (students entering higher education); processes (educational experiences of the students); environment (which shapes the process); and outcomes (students leaving higher education with skills and knowledge).

The committee recognizes that the process of undergraduate STEM education is not always linear, as depicted in Figure 2-1. Students follow increasingly complex trajectories through undergraduate education, transferring across institutions, dropping out for periods, and switching into and out of STEM majors at different times (National Academies of Sciences, Engineering, and Medicine, 2016a). STEM instructors—including faculty members, adjunct faculty, graduate teaching assistants, and all others who teach undergraduates—work within multiple, interacting contexts and feedback loops that influence decisions about teaching. Austin (2011) viewed

these contexts—including departments, colleges, institutions, and such external groups as accrediting associations, parents and employers, and state and federal governments—as “levels” of the system that influence instructors’ work. She also observed that various elements in organizations can serve either as useful levers for or as barriers to change. Key levers in colleges and universities that may encourage or discourage adoption of evidence-based teaching strategies include evaluation and reward systems, workload allocation, professional development opportunities, and the strategic use of leadership practices. Thus, linear approaches to change that address only one factor or intervention are unlikely to lead to sustained adoption of evidence-based teaching practices. Austin (2011) concluded that, in such complex organizations, change efforts are most likely to be effective when they use both a “top-down” and a “bottom-up” approach, consider the multiple factors and contexts that influence instructors’ work, and strategically use multiple levers of change.

The committee recognizes that the environment surrounding undergraduate STEM includes not only the external groups and individuals mentioned above (accreditors, parents, employers), but also others, such as disciplinary bodies and K-12 educators. However, the committee’s generic process model focuses on the most immediate components of the higher education environment—departments, colleges, and institutions—reflecting its charge to develop indicators for undergraduate STEM education.

GOALS FOR UNDERGRADUATE STEM EDUCATION

Following from this model of higher education as a complex system, the committee addressed its charge to “identify objectives for improving undergraduate STEM education and a set of indicators to document the status and quality of undergraduate STEM education at the national level over multiple years.” The committee identified three overarching goals for improving the quality of undergraduate STEM education. It then drew on relevant literature to identify objectives related each of the three goals. The specific targets for improvement reflected in these goals and objectives provide a focus for monitoring the status of undergraduate STEM education over time.

In developing goals and the objectives that follow from them, the committee considered not only its basic framework (refer to Figure 2-1) but also other models of change in higher education (e.g., Elrod and Kezar, 2015, 2016; Henderson, Beach, and Finkelstein, 2011). These various models of undergraduate education as a complex, interacting system were helpful as the committee considered the most important levers for improvement and identified objectives to be monitored through an indicator system.

In addition, the committee derived its goals in part from a similar set of statements in the recent report, Monitoring Progress Toward Successful K–12 STEM Education (National Research Council, 2013) which in turn followed a related report on K–12 STEM education (National Research

Council, 2011). Although there are clear parallels between the goals discussed in that pair of reports and the committee’s three goals, the committee’s goals reflect the different challenges and contexts of the K–12 and the higher education sectors. In response to policy makers’ questions and increasing accountability pressures, the higher education sector is particularly concerned about students’ outcomes, especially the employment outcomes that are reflected in Goal 3. However, ensuring adequate numbers of STEM professionals (Goal 3) will not be possible without first attending to the STEM educational processes and environment reflected in Goals 1 and 2.

These three goals are interconnected and mutually supportive, targeting improvement in various elements of the undergraduate education system and the interactions of these elements that together will enhance students’ success in STEM education. Advancing the goals will require strategic use of multiple change levers within and across the multiple levels of the higher education system, using both top-down and bottom-up approaches (Austin, 2011). The goals are applicable to all varieties of undergraduate STEM educational experiences and are designed to enhance those experiences to the greatest extent possible. The systems perspective reflected in these goals is also essential in developing indicators to monitor progress, because an educational indicator system not only measures an educational system’s inputs, processes, and outputs, but also suggests how they work together to produce an overall effect on students (Odden, 1990, pp. 24-25; Shavelson, McDonnell, and Oakes, 1991).

A growing body of research has identified the STEM teaching and learning experiences and equity and inclusion strategies that support all students’ mastery of STEM concepts and skills and persistence to graduation. Widely deploying these evidence-based processes is essential to ensure adequate numbers of STEM professionals. As noted in Chapter 1, the most rapidly growing groups within the general population are often underrepresented in STEM education and employment fields. These groups provide an untapped resource of talent, and Goal 2 focuses on changing the educational processes and environment to increase their engagement and success in undergraduate STEM education (Summers and Hrabowski, 2006; National Academy of Sciences, National Academy of Engineering, and Institute of Medicine 2011; National Academies of Sciences, Engineering, and Medicine, 2016a). Thus, advancing the three complementary goals will sustain a robust STEM workforce that contributes to national economic growth and international competitiveness (President’s Council of Advisors on Science and Technology, 2012; Xie and Killewald, 2012). The rest of this section discusses the committee’s three goals in more detail.

Goal 1: Increase Students’ Mastery of STEM Concepts and Skills

As noted in Chapter 1, parents, employers, policy makers, and society at large often ask whether, and to what extent, students are learning the content, skills, and abilities that will serve them for their lives and careers after graduation. And as discussed in Chapter 1, there is no simple way to answer this question.

Although there are no agreed-upon measures of students’ STEM learning, an abundance of research has demonstrated that certain common approaches to teaching, learning, and co-curricular programs can improve student learning and degree completion in STEM disciplines (Fairweather, 2012; Kober, 2015; National Research Council, 2012). At the same time, research has shown that poor instructional strategies often discourage persistence in STEM programs of study even among students who are academically capable to engage in them and were originally interested in STEM fields (Correll, Seymour, and Hewitt, 1997). However, the strategies that have evidence of effectiveness have not yet been widely implemented (National Research Council, 2012). Goal 1 addresses this problem, calling for broad implementation of teaching approaches and programs that researchers have identified as most effective for helping students master core STEM concepts and skills. Doing so will require that institutions and their academic units examine their underlying approaches to teaching and curricular design (Elrod and Kezar, 2015, 2016; Henderson, Beach, and Finkelstein, 2011; Weaver et al., 2015). Institutions will also need to consider the educational experiences they offer outside the classroom, such as internships, mentoring, and advising.

The work of engaging students in evidence-based teaching and learning experiences and co-curricular programs rests partly on the shoulders of instructors and staff—those who are on the front lines in education. However, as noted above, the work of these individuals is embedded in a complex system that involves norms, resources, evaluation systems, and reward and recognition practices in departmental, institutional, and disciplinary cultures—the educational environment depicted in Figure 2-1 (Austin, 2011; Weaver et al., 2015). Thus, increasing the use of evidence-based STEM educational practices can only happen with support from departmental and institutional cultures. Such support includes real alignment between institutions’ statements about the value of undergraduate teaching and learning and the explicit valuing of teaching by those institutions. Reward and recognition structures will need to be part of that explicit valuing and robust, reliable forms of evaluating instruction will have to be put in place to make that possible. These approaches will also allow the improvement of students’ educational experiences to be implemented in a scholarly way, based on existing literature and depending on evidence for continuous improvement.

Goal 2: Strive for Equity, Diversity, and Inclusion

Goal 2 involves broadening participation so that the students who participate in postsecondary STEM programs are representative of the national population who could participate in those programs and ensuring that STEM learning environments are inclusive and effectively engage and educate diverse learners. Equity, diversity, and inclusion are distinct concepts; yet, all three are critically important to ensuring that the STEM educational system meets the nation’s needs and serves all people (Association of American Colleges & Universities, 2015; Withem et al., 2015).

The goal of striving for equity, diversity, and inclusion so that STEM learners are as diverse as the country’s national talent pool and that STEM workforce opportunities are equally available to all is both ethical and critical to continuing national innovation and global competitiveness. In comparison with previous generations of undergraduates, today’s undergraduate students are more likely to be female, Black, Hispanic, from low-income families, and single parents (National Academies of Sciences, Engineering, and Medicine, 2016a). Although recent data show that these populations are as interested in STEM fields as their white peers, they are far less likely to complete STEM degrees. Retaining diverse students in STEM that reflect the national population is essential to achieving the increased numbers of STEM students called for in Goal 3.

Many of today’s most challenging scientific and technical issues are global in nature and can best be addressed by combining diverse expertise across disciplinary boundaries, along with community perspectives (National Research Council, 2015). Recent research suggests that science teams comprised of ethnically and geographically diverse members may be more effective than those that are more homogeneous (Freeman and Huang, 2014a,b). More broadly, as the national economy continues to recover from recession, providing equitable employment opportunities for women, minorities, and people with disabilities would facilitate economic growth and reduce income inequality, according to OECD (2016).

Goal 3: Ensure Adequate Numbers of STEM Professionals

Goal 3 seeks to increase the numbers of students who complete STEM credential and degree programs, both to meet demand for STEM professionals in some fields of STEM and to prepare these graduates to participate fully in society. The committee assumes that engaging students in evidence-based STEM educational practices (Goal 1) and ensuring their full inclusion and equity (Goal 2) will increase the number of students who receive STEM credentials (Goal 3). Progress toward Goal 3 will be influenced by many factors, such as admission review processes, summer “bridge” programs,

and recruitment practices. In proposing Goal 3, the committee assumes that a more technologically oriented society requires more people with expertise in science and engineering for economic success and to meet global competition (President’s Council of Advisors on Science and Technology, 2012; Xie and Killewald, 2012). But it does not assume that all STEM graduates will be part of the STEM workforce to reap these economic benefits. Rather, the committee thinks that increasing the number of people with an understanding of STEM ideas and ways of thinking will benefit all segments of society. As discussed in Chapter 1, STEM knowledge and skills are valuable in a broad range of occupations, beyond those formally classified as STEM occupations (Carnevale, Smith, and Melton, 2011; Rothwell, 2013).

ARTICULATING GOALS AS OBJECTIVES

The conceptual framework represented by the committee’s generic process model and its three overarching goals could be articulated as many different potential objectives for improving undergraduate STEM education. To identify what it considers to be the most important objectives for improving the quality of undergraduate STEM, the committee took several steps, which are discussed below.

The Federal STEM Education Strategic Plan

Following the committee charge to consider the federal STEM education strategic plan as a starting point, we reviewed the plan of the National Science and Technology Council (NSTC) (2013), which included the goal of enhancing undergraduates’ STEM experiences as a way to reduce student attrition from STEM majors and thus help achieve the prior federal goal of graduating 1 million additional STEM majors over the next decade (President’s Council of Advisors on Science and Technology, 2012). The committee’s framework follows a similar approach.

As noted in Chapter 1, NSTC identified four strategic objectives for improving students’ undergraduate experiences and reducing attrition: (1) promoting evidence-based instructional practices; (2) improving STEM experiences in community colleges; (3) expanding undergraduate research experiences; and (4) advancing success in the key gateway of introductory mathematics. The committee adopted NSTC objective (1), modifying it to incorporate aspects of NSTC objective (3) (see below). Because its charge encompasses STEM education at both 2-year and 4-year institutions, the committee’s proposed objectives aim for improvement at both types of institutions and we did not adopt a specific objective similar to NSTC objective (2). The committee adopted NSTC objective (4) but broadened it to address

retention of students in key gateway courses in all STEM fields. The committee notes that elements of NSTC objectives (2) and (3) were specific to the federal government’s role, calling for increased federal support of certain aspects of undergraduate STEM and do not represent broad national objectives for the U.S. higher education system as a whole.

Criteria for Identifying Objectives

Students’ attainment of STEM credentials (e.g., certificates, degrees) and development of STEM knowledge and skills are complex processes, influenced by a variety of factors that operate within and across multiple layers of the educational system. Although each student (i.e., background, cognitive and social-psychological characteristics, and level of preparation) is a central actor in these processes, it is now widely understood that a student’s experiences, the larger college environment, and the instructors and staff all play a critical role in a student’s progress (e.g., Astin, 1993; Braxton, 2000; Kuh et al., 2007; Tinto, 1993). This is true for students in all fields, and it has been specifically demonstrated for students in STEM fields (Xie, Fang and Shauman, 2015).

To identify the most important objectives for improving undergraduate STEM within this complex system, the committee reviewed research related to its three goals, focusing on the factors identified in the research as most critical for advancing these goals. Drawing on the literature review in a related National Academies study (National Academies of Sciences, Engineering, and Medicine, 2016a), the committee considered factors at multiple levels of the higher education system. To weigh the importance of various factors emerging from the literature related to each goal, the committee adopted the following criteria for identifying objectives:

1. Evidence of importance or efficacy to STEM educational outcomes: To what extent is there evidence to link the objective to the desired outcomes? The committee sought to identify the most important, high leverage points within the higher education system depicted in Figure 2-1.
2. Applicability across multiple institution types. To what extent is the objective relevant to all of the diverse types of 2-year and 4-year, public, and private higher education institutions in the United States? This criterion reflects the committee’s charge to develop objectives for improving undergraduate STEM at both 2-year and 4-year institutions and to develop a national indicator system relevant across all types of institutions.

3. Emphasis on first 2 years. To what extent is the objective relevant to the first 2 years of undergraduate STEM? This criterion reflects the committee’s charge to focus on the first 2 years of undergraduate STEM. Given that STEM course-taking and performance during the first 2 years of college are key determinants of persistence in STEM (Bettinger, 2010; Chen and Soldner, 2013) and that much attrition from STEM programs occurs within the first 2 years (Chang et al., 2008; Seymour and Hewitt, 1997), these years are critical for improving student success in STEM. Thus, the framework emphasizes objectives relevant to the first 2 years, while still leaving room to include highly important processes or characteristics relevant beyond the first 2 years.

The committee also considered several cross-cutting issues. As mentioned in Chapter 1, institutions of higher education across the country have enormously different aims and missions related to STEM education and, relatedly, serve hugely diverse student populations. As a result, STEM coursework and curricular standards vary from institution to institution, and students who may be prepared to do the work in one institution may be ill-equipped to meet the standards of another. Across the objectives listed below it is critical to consider the necessary differences in how institutions meet their own stated goals in light of the preparation and expectations of their student populations.

The Objectives

The committee selected 11 objectives for improving undergraduate STEM, grouped under the committee’s three overarching goals.

GOAL 1: Increase Students’ Mastery of STEM Concepts and Skills by Engaging Them in Evidence-Based STEM Educational Practices and Programs.

GOAL 2: Strive for Equity, Diversity, and Inclusion of STEM Students and Instructors by Providing Equitable Opportunities for Access and Success.

GOAL 3: Ensure Adequate Numbers of STEM Professionals.

These objectives and their relationship to the three goals are shown in Figure 2-2. The objectives are designed to improve the quality in each component of the basic conceptual framework: inputs, processes, environment, and outcomes. However, the objectives primarily target improvement of the educational processes, environments, and outcomes. Although the inputs, the incoming students, influence the quality of undergraduate STEM education, some of the characteristics of the students reflect K–12 preparation, which lies outside the scope of the study charge.

The detailed framework shown in Figure 2-2 illustrates students’ entrance to 2-year or 4-year colleges, their STEM-related learning experiences inside and outside the classroom, the environments that surround students and instructors, and student outcomes, including credentials and knowledge of STEM concepts and skills.

PROPOSED INDICATORS

The objectives identified in the detailed framework drove the committee’s development of indicators: Table 2-1 presents the committee’s proposed indicators in concert with the committee’s framework and objectives. The next three chapters of the report describe those objectives and indicators.

CONCLUSION

In this chapter, the committee has proposed a conceptual framework for the indicator system. Beginning with a model of higher education as

TABLE 2-1 Framework, Objectives, and Indicators

a complex system, the committee identified three overarching goals for improving the quality of undergraduate STEM education. It then built on the federal STEM education strategic plan and drew on relevant literature to articulate each goal into more specific objectives. The specific targets for improvement reflected in these goals and objectives provide a focus for monitoring the status of undergraduate STEM education over time.

REFERENCES

Austin, A. (2011). Promoting Evidence-Based Change in Undergraduate Science Education. Paper commissioned by the Board on Science Education. Available: http://sites.nationalacademies.org/cs/groups/dbassesite/documents/webpage/dbasse_072578.pdf [June 2016].

Bettinger, E. (2010). To be or not to be: Major choices in budding scientists. In C.T. Clotfelter (Ed.), American Universities in a Global Market (pp. 69–98). Chicago, IL: University of Chicago Press.

Braxton, J.M. (Ed.). (2000). Reworking the Student Departure Puzzle. Nashville, TN: Vanderbilt University Press.

Carnevale, A.P., Smith, N., and Melton, M. (2011). STEM: Science, Technology, Engineering, and Mathematics. Washington, DC: Georgetown University Center on Education and the Workforce. Available: https://cew.georgetown.edu/wp-content/uploads/2014/11/stem-complete.pdf [July 2017].

Chang, M.J., Cerna, O., Han, J., and Sáenz, V. (2008). The contradictory roles of institutional status in retaining underrepresented minorities in biomedical and behavioral science majors. The Review of Higher Education, 31(4), 433–464.

Chen, X., and Soldner, M. (2013). STEM Attrition: College Students’ Paths into and out of STEM Fields. Washington, DC: U.S. Department of Education.

Correll, S.J., Seymour, E., and Hewitt, N.M. (1997). Talking about leaving: Why undergraduates leave the sciences. Contemporary Sociology, 26, 644. doi10.2307/2655673.

Elrod, S., and Kezar, A. (2015). Increasing student success in STEM. Peer Review, 17(2). Available: https://www.aacu.org/peerreview/2015/spring/elrod-kezar [June 2016].

Elrod, S., and Kezar, A. (2016). Increasing Student Success in STEM: A Guide to Systemic Institutional Change. Washington, DC: Association of American Colleges & Universities.

Fairweather, J. (2012). Linking Evidence and Promising Practices in Science, Technology, Engineering, and Mathematics (STEM) Undergraduate Education: A Status Report for The National Academies National Research Council Board on Science Education. Available: http://sites.nationalacademies.org/cs/groups/dbassesite/documents/webpage/dbasse_072637.pdf [June 2016].

Freeman, R.B., and Huang, W. (2014a). Collaborating with People Like Me: Ethnic Co-Authorship within the U.S. NBER Working Paper No. 19905. Cambridge, MA: National Bureau of Economic Research. Available: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.702.2502&rep=rep1&type=pdf [February 2018].

Freeman, R.B., and Huang, W. (2014b). Strength in diversity. Nature, 513, 305.

Henderson, C., Beach, A., and Finkelstein, N. (2011). Facilitating change in undergraduate STEM instructional practices: An analytic review of the literature. Journal of Research in Science Teaching, 48(8), 952–984. doi10.1002/tea.20439.

Katz, D., and Kahn, R.L. (1966). The Social Psychology of Organizations. New York: John Wiley & Sons.

Katz, D., and Kahn, R.L. (1978). The Social Psychology of Organizations. 2nd ed. New York: John Wiley & Sons.

Kober, N. (2015). Reaching Students: What Research Says About Effective Instruction in Undergraduate Science and Engineering. Washington, DC: The National Academies Press. Available: http://www.nap.edu/catalog/18687/reaching-students-what-research-says-about-effective-instruction-in-undergraduate [June 2016].

Kuh, G.D., Kinzie, J., Buckley, J., Bridges, B., and Hayek, J.C. (2007). Piecing Together the Student Success Puzzle: Research, Propositions, and Recommendations. (ASHE Higher Education Report, vol. 32, issue 5). San Francisco, CA: Jossey-Bass.

National Academies of Sciences, Engineering, and Medicine. (2016a). Barriers and Opportunities for 2-Year and 4-Year STEM Degrees: Systemic Change to Support Students’ Diverse Pathways. Washington, DC: The National Academies Press. Available: http://www.nap.edu/catalog/21739/barriers-and-opportunities-for-2-year-and-4-year-stem-degrees [June 2016].

National Academies of Sciences, Engineering, and Medicine. (2016b). Science Literacy: Concepts, Contexts, and Consequences. Washington, DC: The National Academies Press.

National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. (2011). Expanding Underrepresented Minority Participation: America’s Science and Technology Talent at the Crossroads. Washington, DC: The National Academies Press. Available: http://www.nap.edu/catalog/12984/expanding-underrepresented-minority-participation-americas-science-and-technology-talent-at [June 2016].

National Research Council. (2011). Successful K-12 STEM Education: Identifying Effective Approaches in Science, Technology, Engineering, and Mathematics (STEM). Washington, DC: The National Academies Press. Available: http://www.nap.edu/catalog/13158/successful-k-12-stem-education-identifying-effective-approaches-in-science [June 2016].

National Research Council. (2012). Discipline-Based Education Research: Understanding and Improving Learning in Undergraduate Science and Engineering. Washington, DC: The National Academies Press. Available: https://www.nap.edu/catalog/13362/discipline-based-education-research-understanding-and-improving-learning-in-undergraduate [July 2017].

National Research Council. (2013). Monitoring Progress Toward Successful K-12 STEM Education: A Nation Advancing? Washington, DC: The National Academies Press. Available: https://www.nap.edu/search/?term=Monitoring+Progress+Toward+Successful+K-12+STEM+Education%3A+A+Nation+Advancing%3F.+&x=16&y=6 [July 2017].

National Research Council. (2015). Enhancing the Effectiveness of Team Science. Washington, DC: The National Academies Press. Available: http://www.nap.edu/catalog/19007/enhancing-the-effectiveness-of-team-science [June 2016].

National Science and Technology Council. (2013). Federal Science, Technology, Engineering, and Mathematics (STEM) 5-Year Strategic Plan. Washington, DC: Author. Available: https://www.whitehouse.gov/sites/default/files/microsites/ostp/stem_stratplan_2013.pdf [June 2016].

Odden, A. (1990). Educational indicators in the United States: The need for analysis. Educational Researcher, 19(5), 24–29.

OECD. (2016). OECD Economic Surveys: United States. Available: http://www.oecd.org/eco/surveys/United-States-2016-overview.pdf [June 2016].

President’s Council of Advisors on Science and Technology. (2012). Engage to Excel: Producing One Million Additional College Graduates with Degrees in Science, Technology, Engineering and Mathematics. Washington, DC: Author. Available: https://www.whitehouse.gov/sites/default/files/microsites/ostp/pcast-engage-to-excel-final_feb.pdf [March 2016].

Rothwell, J. (2013). The Hidden STEM Economy. Metropolitan Policy Program at Brookings Institution. Available: http://www.brookings.edu/~/media/research/files/reports/2013/06/10-stem-economy-rothwell/thehiddenstemeconomy610.pdf [April 2015].

Seymour, E., and Hewitt, N. (1997). Talking about Leaving: Why Undergraduates Leave the Sciences. Boulder, CO: Westview Press.

Shavelson, R.J., McDonnell, L., and Oakes, J. (1991). What are educational indicators and indicator systems? Practical Assessment, Research and Evaluation, 2(11). Available: http://pareonline.net/getvn.asp?v=2andn=11 [July 2017].

Summers, M.F., and Hrabowski III, F.A. (2006). Preparing minority scientists and engineers. Science, 311(5769), 1870–1871.

Tinto, V. (1993). Leaving College: Rethinking the Causes and Cures of Student Attrition. (second ed.). Chicago, IL: University of Chicago Press.

Weaver, G.C., Burgess, W.D., Childress, A.L., and Slakey, L. (2015). Transforming Institutions: Undergraduate STEM Education for the 21st Century. West Lafayette, IN: Purdue University Press.

Withem, K., Malcom-Piqueux, L., Dowd, A.C., and Bensimon, E.M. (2015). America’s Unmet Promise: The Imperative for Equity in Higher Education. Washington, DC: American Association of Colleges & Universities.

Xie, Y., Fang, M., and Shauman, K. (2015). STEM education. Annual Review of Sociology, 41, 331–357.

Xie, Y., and Killewald, A.A. (2012). Is American Science in Decline? Cambridge, MA: Harvard University Press.

This page intentionally left blank.