NCAT requires each participating institution to conduct a rigorous evaluation of the impact of the redesign on learning outcomes as measured by student performance and achievement. National experts have provided consultation and oversight regarding the assessment of learning outcomes to ensure that the results are as reliable and valid as possible. To date, results show improved student learning in 72 percent of the redesigns, with the remaining 28 percent showing learning outcomes equivalent to traditional formats. Other qualitative outcomes achieved in the course redesigns include increased course-completion rates, improved retention, better student attitudes toward the subject matter, increased student performance in downstream courses and increased student satisfaction with the mode of instruction.
All NCAT redesign projects compare student learning outcomes in the traditional format with those achieved in the redesigned format. This is done by either running parallel sections of the course in the two formats or comparing baseline data from an offering of the traditional course to a later offering of the redesigned course, looking at differences in outcomes from before and after. The four measurement methods used to assess student learning are Comparisons of Common Final Exams, Comparisons of Common Content Items Selected from Exams, Comparisons of Pre- and Post-tests, and Comparisons of Student Work Using Common Rubrics.
NCAT requires each participating institution to establish a team of faculty and staff to conduct the redesign. Each team develops a detailed cost analysis of both the traditional and the redesigned course formats. All 120 course redesigns have reduced costs—by 37 percent, on average, with a range of 9 percent to 77 percent. The 120 redesigned courses have affected more than 160,000 students nationwide and produced a savings of about $9.5 million each year.
Each team analyzes and documents “before and after” course costs using activity-based costing. NCAT developed a spreadsheet-based course planning tool (CPT) that allows institutions to (1) determine all personnel costs; (2) identify the tasks associated with preparing and offering the course in the traditional format and determine how much time each type of personnel spends on each of the tasks; and (3) identify the tasks associated with preparing and offering the course in the redesigned format, and determine how much time each type of personnel spends on each of the tasks. The CPT then automatically converts the data to a comparable cost-per-student measure. At the beginning of each project, baseline cost data for the traditional course and projected redesigned course costs are collected; actual redesigned course costs are collected at the end.
Completing the CPT allows faculty members to consider changes in specific