Final Report(as of 9/19/13))
Impact on Students
In the redesign, did students learn more, less or the same compared to the traditional format?
No significant difference
The Towson University team compared the final exam grades earned by students in the traditional format and in the redesigned format. The two formats were taught using comparable homework assignments and the final exam was prepared and distributed by the course coordinator. An independent external evaluator interviewed students in both the traditional and redesign groups. Although students complained about the redesign model and the effort it involved, we did not see a significant difference in their final exam scores. The common final exam was designed by the course coordinator. An analysis of the scores from the pilot sections shows no difference between student scores on the traditional and redesign sections on their common final exam. Both class averages were 64%. Full implementation of the redesign over the spring semester showed final exam grade averages similar to the traditional semesters.
Improved Retention
The final grades for both formats are reported above.
During the pilot semester, the redesign section had a DFW rate of 23% and the traditional sections had a DFW rate of 36%. The redesign section experienced difficulties with the online learning homework system and many students had to be awarded extra credit to compensate for this loss. The apparent gain in the redesign scores could be attributed to this inflation. During the full implementation, our DFW rates were 22% which is less than our historical (measured over three years before the redesign implementation) DFW rate of 35% for this course. For the lab course, our DFW was 14%. We have no comparable control to measure changes because we split the lab and lecture during the redesign phase.
Other Impacts on Students
Faculty observed that: 1) students were more likely to seek assistance from the learning assistant than the professor 2) they struggled with the pilot semester redesign implementation because of several issues with the software and 3) changing the software during the full implementation reduced complaints drastically and students were more willing to devote time to learning
One major impact of the redesign on the students was the mandatory standardization of the entire course. We have reduced course drift and the grading variation in the department significantly since the implementation of course redesign. This was a major step in ensuring fairness in a “gatekeeper” course for the Allied Health majors.
Impact on Cost Savings
Were costs reduced as planned?
The actual cost savings to the university was higher than the team anticipated. The average cost-per-student in the traditionally offered course was $236. The projected cost-per-student for the redesign was $117, a 50% reduction. The full implementation actually decreased the cost-per-student to $88, a 63% reduction. These savings were offset somewhat by a slight increase in the direct cost of instruction from $11,728 in Spring 2012 (pre-redesign) to $12,269 in Spring 2013 (full implementation of redesign), an increase of 5%. The increase was due to the decrease in adjunct faculty teaching the course (in effect, one adjunct faculty was replaced by the Department Chair, whose hourly salary is much higher) and the employment of Undergraduate Learning Assistants in the computer laboratory.
The total number of sections of the course remained constant, but the average section size increased from 24 to 27, resulting in an overall enrollment increase of 12%. Thus there was a modest increase in the efficiency of instruction.
Lessons Learned
Pedagogical Improvement Techniques
What techniques contributed most to improving the quality of student learning?
- Establishing a chemistry computer laboratory. The availability of a dedicated computer lab which replaced three hours of the wet lab every alternate week helped students with practicing the lecture material. Since lab and lecture topics were coordinated better to accommodate the online learning, students were more prepared for their exams.
- Incorporating web-based exercises. The Towson University team used Cengage’s OWL online homework in the pilot semester during the pilot semester. This was quickly replaced with Connect by McGraw Hill, which has been a more user friendly program.
- Classroom management technology. Blackboard was used to provide students with instant access to the course syllabus, announcements, important dates, lecture notes and review exercises. Students reported that the Blackboard tools were helpful throughout the course and in preparation for exams.
Cost Reduction Techniques
What techniques contributed most to reducing costs?
- Restructuring lecture and lab meeting times: In the pilot semester, two 50-minute lecture sessions were offered weekly compared to three 50-minute lecture sessions in the traditional format. The third lecture session was scheduled in a computer lab to enable students to do practice problems and ask clarifying questions. This structure caused serious problems with scheduling more than 60 students in a computer lab space which could accommodate no more than 30 students. During full implementation, we changed this structure to replace weekly lab meetings with the computer lab. This eliminated the problem of finding alternate times for students, as the lab periods were part of their predetermined schedules and lab classes were designed to hold no more than 28 students for safety reasons.
- Staffing changes: Adjunct instructors and a few tenure-track faculty members have traditionally been involved in this chemistry course. In the redesign, the TU team employed undergraduate learning assistants (ULAs), who provided on-demand assistance to students in the computer laboratory. Although the tasks accomplished by the ULAs could have freed the professor from course related contact duties in the pilot phase, the technology concerns were major challenges which required more time to be spent on assigning partial credit and providing opportunities for extra credit.
- Course coordination: The redesign team rotated the task of generating course assignments. The coordinator was responsible for posting practice problems, notes, calendar dates, and announcements on Blackboard in addition to making a common lab syllabus. One of the PI’s was charged with making a common lecture schedule and to determine the order of topics to be covered.This eliminated duplication of effort and ensured that teaching toward a common set of learning objectives occurred.
Implementation Issues
What implementation issues were most important?
- Technology concerns: In the pilot phase of the redesign, we used an online learning homework system which appeared to have too many problems associated with it- including wrong information being disseminated. Additionally, the online homework systems required students to enter information using specific formatting, often penalizing them harshly for trivial errors not having the right amount of spaces or the exact syntax required by the software. This caused major problems with student learning and grading.
- Course drift:Reduction of course drift was one of the most successful aspects of our redesign effort. We were able to use common lab quizzes for all sections of the cohort and also develop a common final exam. Since the computerized lab quizzes were available on the common lab website, it was easier to enforce the grading rubrics and level of difficulty across all sections. The redesign forced us to look at our variable grading metrics and ensured that all faculty came to a consensus on the order of topics, level of difficulty of concepts to be introduced and assignment of grade boosting extra credit on exams and homework assignments.
Sustainability
Will the redesign be sustained now that the course redesign project is over ?
While we reduced course drift significantly, there was no evidence to show that we reduced costs for the institution or had an impact on student learning. We learned several important lessons as part of the redesign- specifically lessons on what we could not do. Given that we now have a split lab and lecture course with separate grades for both components and the assignment of lab hours to computer practice exercises might help students who are struggling with the course, this may be a good chance to reinvent course redesign and try an alternative system of combining online case studies and wet labs for the Allied Health majors.
The Carnegie course redesign grant provided funds to furnish a new student computer lab and to hire Undergraduate Learning Assistants to provide help in the lab. The student computer lab will still be available for CHEM 121 classes. An alternative funding source within Towson University will need to be provided to continue the use of Learning Assistants.