format matters more if the purpose is to evaluate learning for instruction, and it is very important if the purpose of the concept inventory is to assess learning for research.
Teri Reed-Rhoads (Purdue University) observed that although engineering lags behind science in terms of developing concept inventories, the few engineering concept inventories available are increasingly being used for such purposes as accreditation, grant proposals, and grant project accountability. In addition, she explained that engineering faculty members are beginning to use concept inventories to facilitate changes in pedagogy aimed at increasing student learning.
Reed-Rhoads defined engineering concept inventories as those that are developed by engineers, either on their own or in collaboration with others. Using this definition, Reed-Rhoads identified 21 engineering concept inventories, 6 of which she labeled as science, technology, engineering, and mathematics (STEM) concept inventories, which were developed by or in conjunction with engineers and focused on nonengineering-related subjects.1
Discussing the relative maturity of engineering concept inventories, Reed-Rhoads pointed out that many more examinees have taken the statics concept inventory than the other engineering-related concept inventories, and that its growth has been exponential. For example, between year 2 and year 3 of its existence, the cumulative number of examinees for the statics inventory jumped from about 300 to about 1,700, further increasing to 2,700 in year 4 (Reed-Rhoads and Imbrie, 2008). In contrast, the cumulative number of examinees for the systems and signals inventory steadily grew from about 300 in year 1 to about 500 in year 2 to slightly more than 800 in year 3. She also explained that, because concept inventories take years to develop (as noted by Libarkin), there is often a significant lag time between their development and a discernible effect on instructional practices.
In engineering, concept inventory developers initially were slow to analyze the psychometric properties of engineering concept inventories, said Reed-Rhoads. She observed, however, that developers are increasingly collaborating with psychometricians to analyze and validate their instruments. She also noted that the research base on students’ engineering misconceptions is lagging behind those in some of the other sciences. This lag complicates the development of the concept inventories; in other disciplines
The specific concept inventories are listed in the workshop paper by Reed-Rhoads (see http://www.nationalacademies.org/bose/Reed_Rhoads_CommissionedPaper.pdf).