into a framework of criteria and benchmarks to assess instruction; and (3) how departments and institutions of higher learning might use such a framework to assess their STEM programs and to promote ongoing improvements.

Workshop participants would focus on four questions regarding undergraduate STEM education at the classroom, departmental, and institutional level: (a) what characteristics and indicators should be included in a comprehensive evaluation instrument that could serve as the basis for recognizing exemplary STEM courses and academic programs; (b) what are the desired student outcomes of such STEM courses that can indicate course effectiveness; (c) what qualities of organization, governance, and incentive structures can be identified at the departmental and institutional levels that promote quality STEM education; and (d) how can such qualities be used as the basis for creating indicators and benchmarks for the evaluation of institutions and departments?

To sharpen and focus these questions, breakout groups at the workshop were asked to define “appropriate measures of undergraduate learning” by developing a list of desired student learning outcomes for each science discipline. The logic here was that student success in achieving defined learning outcomes could serve as an indicator of the effectiveness of a particular course or an instructional approach. Further, still using student learning outcomes as a criterion of success, workshop participants were challenged to identify characteristics and indicators that should be included in a comprehensive evaluation instrument or framework for recognizing a hypothetical “exemplary” STEM course. To investigate how departments and institutions of higher learning might use such a framework to assess their STEM programs, workshop participants were instructed to identify qualities of organization, governance, and incentive structures at the departmental and institutional levels that promote quality STEM education, and to consider how such qualities could be used to create a set of indicators and benchmarks for the evaluation of institutions and departments.

As an initial step in thinking about appropriate measures of undergraduate learning in STEM disciplines, workshop participants were asked first to identify a few “exemplary” programs that were known by reputation to be effective in achieving desired learning outcomes. Participants then outlined characteristics that would enable an observer to classify these programs as effective. These characteristics, which are summarized in Chapter 3, could be included in a comprehensive evaluation instru-