The National Center for Academic Transformation (NCAT) is an independent nonprofit organization that describes itself as “dedicated to the effective use of information technology to improve student learning outcomes and reduce the cost of higher education.”1 NCAT works with institutions to analyze their instructional models and identify ways to efficiently leverage technology, faculty, graduate students, peer interactions, and other learning resources to improve quality and efficiency. In the NCAT model, students are transformed from “passive note takers” in a standard lecture format to active participants in their own learning, leveraging a set of tools and resources. Based on a review of the participating institutions, NCAT has identified six redesign models that vary in format, student experience, and use of technology.
Course redesigns focus primarily, but not exclusively, on large-enrollment, introductory courses in multiple disciplines, including 16 in the humanities (developmental reading and writing, English composition, fine arts, history, music, Spanish, literature, and women’s studies), 60 in quantitative subjects (developmental and college-level mathematics, statistics, and computing), 23 in the social sciences (political science, economics, psychology, and sociology), 15 in the natural sciences (anatomy and physiology, astronomy, biology, chemistry, and geology) and 6 in professional studies (accounting, business, education, engineering, and nursing).
________________
1This appendix draws heavily from the NCAT Web site: http://www.thencat.org/ [November 2011].
QUALITY
NCAT requires each participating institution to conduct a rigorous evaluation of the impact of the redesign on learning outcomes as measured by student performance and achievement. National experts have provided consultation and oversight regarding the assessment of learning outcomes to ensure that the results are as reliable and valid as possible. To date, results show improved student learning in 72 percent of the redesigns, with the remaining 28 percent showing learning outcomes equivalent to traditional formats. Other qualitative outcomes achieved in the course redesigns include increased course-completion rates, improved retention, better student attitudes toward the subject matter, increased student performance in downstream courses and increased student satisfaction with the mode of instruction.
All NCAT redesign projects compare student learning outcomes in the traditional format with those achieved in the redesigned format. This is done by either running parallel sections of the course in the two formats or comparing baseline data from an offering of the traditional course to a later offering of the redesigned course, looking at differences in outcomes from before and after. The four measurement methods used to assess student learning are Comparisons of Common Final Exams, Comparisons of Common Content Items Selected from Exams, Comparisons of Pre- and Post-tests, and Comparisons of Student Work Using Common Rubrics.
COST
NCAT requires each participating institution to establish a team of faculty and staff to conduct the redesign. Each team develops a detailed cost analysis of both the traditional and the redesigned course formats. All 120 course redesigns have reduced costs—by 37 percent, on average, with a range of 9 percent to 77 percent. The 120 redesigned courses have affected more than 160,000 students nationwide and produced a savings of about $9.5 million each year.
Each team analyzes and documents “before and after” course costs using activity-based costing. NCAT developed a spreadsheet-based course planning tool (CPT) that allows institutions to (1) determine all personnel costs; (2) identify the tasks associated with preparing and offering the course in the traditional format and determine how much time each type of personnel spends on each of the tasks; and (3) identify the tasks associated with preparing and offering the course in the redesigned format, and determine how much time each type of personnel spends on each of the tasks. The CPT then automatically converts the data to a comparable cost-per-student measure. At the beginning of each project, baseline cost data for the traditional course and projected redesigned course costs are collected; actual redesigned course costs are collected at the end.
Completing the CPT allows faculty members to consider changes in specific
instructional tasks, make decisions about how to use technology (or not) for specific tasks, visualize duplicative or unnecessary effort and complete a cost/benefit analysis regarding the right type of personnel for each instructional task. The CPT consists of four worksheets: Instructional Costs per Hour, used to determine all personnel costs involved in the course and to express each kind of cost as an hourly rate; Traditional Course Activities and Cost, used to determine the activities involved in and the costs of preparing and delivering the course in its traditional format; Redesigned Course Activities and Cost, used to determine the activities involved in preparing and delivering the course in its redesigned format when it is fully operational; and Annual Cost Comparison, used to compare the annual costs of the traditional course and the redesigned course.
Among the assumptions of the NCAT Model are the following:
Developmental costs are not included. NCAT’s model compares the before costs (current/historical/traditional) and the after costs (what the course will cost when it is fully operational)—i.e., it asks the team to plan what the redesigned course will look like at the end of the developmental process. It does not include the up-front developmental costs of either the traditional or the redesigned course.
Institution-wide support services, administrative overhead, infrastructure and equipment costs are not included. The assumption is that these costs are constant—are part of the campus environment—for both the traditional and redesigned courses. Campus networking, site licenses for course management systems and desktop PCs for faculty, for example, are part of the campus-wide IT environment. Costs that are particular to the course are included in “other costs.”
EXAMPLES (CASE STUDIES)
Research University: Louisiana State University
Louisiana State University (LSU) redesigned College Algebra, a three-credit course enrolling 4,900 students annually. The university moved all structured learning activity to a computer lab where students work with an instructional software package—which includes interactive tutorials, computational exercises, videos, practice exercises, and online quizzes—and receive just-in-time assistance from instructors and undergraduate assistants. Instructors also meet students in a once-a-week focus group which builds community among students and instructors.
The redesign reduced costs by serving the same number of students with half of the personnel used in the traditional model. Section size stayed at 40-44 students, but the number of class meetings each week was reduced from three to one. The redesigned format allows one instructor to teach twice as many students as in the traditional format without increasing class size and without increasing workload. In the traditional format, each instructor taught one three-day-a-week
section with 44 students. In the redesigned format, that same instructor teaches two sections of 44 students and spends four hours tutoring in the lab. This can be accomplished because the class only meets once a week and because no hand-grading is required. While the cost of adding tutors in the computer lab as well as increased time for coordination and systems administration reduced the net savings, the redesign reduced the cost-per-student from $121 to $78.
Learning outcomes were measured by comparing medians on a common final exam. Final exam medians for traditional fall sections ranged from 70 percent to 76 percent. After redesign, the final exam median in fall 2006 was 78 percent, the highest ever achieved. In the traditional format, final exams were graded by individual instructors and partial credit was allowed. In the redesigned format, final exams were group-graded, which means that grading was more consistent from one section to the next, and partial credit was not allowed, yet exam medians were higher. The success rate (grades of C or better) for College Algebra in the five years prior to the redesign averaged 64 percent. In fall 2006, after full implementation of the redesign, students had a success rate of 75 percent.
Comprehensive Four-Year Institution: The University of Southern Mississippi
The University of Southern Mississippi (USM) redesigned World Literature, a course enrolling more than 1,000 students each term, in order to eliminate course drift and inconsistent student learning experiences. The traditional course was offered in 16 sections of about 65 students each: 8 sections taught by full-time faculty and 8 by adjuncts. The redesign placed all students in a coherent single online section and replaced the passive lecture environment with media-enriched presentations that required active student engagement. A course coordinator directed the team teaching of four faculty members, each of whom taught his or her area of expertise of four weeks, and four graduate teaching assistant (GTA) graders. The faculty team offered course content through a combination of live lectures with optional attendance and required Web-delivered, media- and resource-enhanced presentations.
USM reduced the number of sections from 30 to 2 and increased the number of students in each section from 65 to 1,000. These changes enabled the university to reduce the number of faculty teaching the course from 16 to the equivalent of 2 full-time faculty and 4 GTAs. USM eliminated adjuncts completely. The course is now taught 100 percent by full-time faculty supported by GTAs for writing assignment grading. By making these changes, six full-time faculty were freed to teach other courses, and the funds previously used to hire adjuncts were made available for a variety of academic enhancements in the department. The cost-per-student was reduced from $70 to $31.
Student performance improved on multiple dimensions in the redesign. Grades on weekly quizzes with common content showed an increase in grades
of C or better from 68 percent in the traditional course to 88 percent in the redesigned course. Writing scores of C or better increased from 61 percent to 77 percent. Essay scores increased in from 7.11 in the traditional mode to 8.10 in the redesigned format. In the traditional version of the course, faculty-taught sections typically retained about 75 percent of their students, while adjunct- and GTA-taught sections retained 85 percent. In the fall 2003 semester of full implementation of the redesign, retention was 87 percent, with all students being taught solely by full-time faculty. At the same time, the rate of D and F grades dropped from 37 percent to 27 percent in the redesigned course.
Private, Liberal Arts Institution: Fairfield University
Fairfield University, a comprehensive Jesuit university with 3,100 full-time undergraduates, redesigned its general biology course to improve the quality of student learning and to reduce its cost. The traditional biology course was taught in a multiple-section model, with 35-40 students per section, and met three times per week with a three-hour lab. Four faculty members provided lectures; additional faculty and professionals were needed to staff the labs. The cost of offering the traditional course to 260 students annually was $131,560, which translates to a per-student cost of $506.
Significant cost savings were realized from reducing faculty time in three major areas: (1) materials development for lectures; (2) out-of-class course meetings; and (3) in-class lectures and labs. The number of faculty needed to teach the course declined from seven to four. Faculty time was reconfigured to support a division of teaching responsibilities so that the four faculty members now teach from their areas of expertise. Faculty time devoted to this course decreased from 1,550 hours to 1,063 hours. Consolidation of the seven lecture sections into two in the redesigned course and the introduction of computer-based modules in the lecture and laboratory resulted in a cost-per-student reduction from $506 to $350.
Students in the redesigned course performed significantly better on benchmark exam questions. The questions on the new exams were designed to test higher order thinking and allow students to synthesize material from the basic concepts.
In addition, specific exam questions incorporated in the second year genetics course (required of all biology majors) were used to measure the retention of key concepts and compare the performance of traditional students and redesign students. Students from the redesigned course performed significantly better on this set of questions than did students from the traditional course (88 percent correct vs. 79 percent correct, respectively). The DFW rate dropped: only 3 percent of the students in the redesign dropped the course compared to an average of 8 percent in previous years. The number of students who decided to enroll in the second semester of the course approached 85 percent compared to less than 75 percent in previous years.
Community College: Tallahassee Community College
Tallahassee Community College (TCC) redesigned College Composition, a required course serving approximately 3,000 students annually. The traditional format, which combined lecture and writing activities in sections of 30 students each, made it difficult to address individual needs. The redesign used technology to provide diagnostic assessments resulting in individualized learning plans; interactive tutorials in grammar, mechanics, reading comprehension, and basic research skills; online tutorials for feedback on written assignments; follow-up assessments; and discussion boards to facilitate the development of learning communities. The classroom was restructured to include a wide range of learner-centered writing activities that fostered collaboration, proficiency, and higher levels of thinking. By shifting many of the basic instructional activities to technology, faculty could focus the classroom portion of the course on the writing process.
TCC reduced the number of full-time faculty involved in teaching the course from 32 to 8 and substituted less expensive adjunct faculty without sacrificing quality and consistency. In the traditional course, full-time faculty taught 70 percent of the course, and adjuncts taught 30 percent. In the redesigned course, full-time faculty teach 33 percent of the course, and adjuncts teach 67 percent. Overall, the cost-per-student was reduced from $252 to $145, a savings of 43 percent. Full-time faculty were freed to teach second-level courses where finding adjuncts was much more difficult.
Five final out-of-class essays were selected randomly from the traditional and redesigned sections and were graded by an independent group of faculty using an established holistic scoring rubric. Students in the redesigned composition course scored significantly higher (p = 0.03) and had an average score of 8.35 compared to 7.32 for the students in the traditional course. Students in redesigned sections had a 62 percent success rate (grades of C or better) compared with 56 percent in traditional sections, representing a 13.6 percent decrease in the DFW rate. Success rates of students in the second-level English course increased (79.3 percent success for redesigned compared to 76.1 percent for traditional).
