the inventory developers draw on existing research about misconceptions, whereas in engineering, the concept inventories drive the definitions of the misconceptions (Reed-Rhoads and Imbrie, 2008).
Reed-Rhoads identified gaps in the research related to engineering concept inventories. First, she explained that concept inventories have been used only in the basic engineering courses so far, which means that upper division courses and subject areas are sparsely represented. In addition, although some research indicates that examinees’ attitudes and beliefs about a field of study might influence assessment results in that field (Gal and Ginsburg, 1994), few of the engineering concept inventories have related instruments that measure the affective and cognitive domains.
Another gap in the research is that engineering concept inventories have not been extensively studied for the various types of bias that might be included in the questions (Reed-Rhoads and Imbrie, 2008). These biases include how gender, race/ethnicity, native language, and culture might affect student scores on the inventories. The understanding of bias in engineering concept inventories is limited because not enough students from different subpopulations have used the instrument; with such low sample numbers, the statistics for each subgroup are not reliable. However, Reed-Rhoads noted that although women are the most underrepresented population in engineering, enough women have used the concept inventories to allow for some statistical testing related to gender bias.
Reed-Rhoads also observed that the relationships among concept inventories is important but not well understood. She emphasized the need to track students’ conceptual development, which requires greater knowledge of how the concept inventories fit together. She argued that this need is becoming increasingly important as concept inventories proliferate.
The final gap relates to helping faculty members use concept inventories to change their practices. To this end, Reed-Rhoads and her colleagues created a community of inventory developers, faculty members, and students called ciHUB (short for concept inventory hub) to provide access to resources that can facilitate collaboration and the use of research-based tools to improve instruction.
Karen Cummings delivered a presentation by Paula Heron (University of Washington) on work by Heron and her colleagues in the University of Washington’s Physics Education Group.2 This group conducts a coordi-
See the workshop paper by Heron (see http://www.nationalacademies.org/bose/Heron_CommissionedPaper.pdf).