The Clickers are Coming
A proposal to assess the use of Personal Response Systemsin large classes at UVic
Nov 19, 2005
Principal Investigator: Mary Sanseverino, Computer Science and Learning and Teaching Centre
Co-investigators: Dr. Penelope Codding, Chemistry, Dr. Sara Ellison, Astronomy and Physics, Dr. Tim Pelton, Education, Dr. Leslee Francis Pelton, Education, Dr. Rosaline Canessa, Geography
Index
Executive Summary
1.0 Introduction
1.1Response System Specifics
1.1.1 Business Model Differences
1.2Why use a "Clicker" system?
2.0 The Pilot Proposal
2.1 Research Questions
2.2 Methodology
3.0 Project Costs
4.0 Conclusion
Bibliography
Appendix
Executive Summary
The clickers are coming – is UVic ready? Personal Response devices, euphemistically known as "clickers," are starting to make themselves known on campus. These handheld wireless devices allow students to respond to multiple choice questions posed by the instructor. The responses are gathered by one or more wireless receivers attached to the instructor's computer and are then aggregated and shown to the students. Simple as it sounds, this can make a dramatic change in the way students engage with course material.
The worth of these systems appears to be in benefits to learning by increasing classroom interaction and enhancing student engagement. This seems to be especially true in large classes. Since the technology is easy to incorporate into lecture formats, and since some publishers (notably McGraw-Hill Ryerson) are packaging them with textbooks, we can expect to see them proliferating on campus in the next few years. Indeed, in the current term (Sept 2005), two instructors of large, first year classes will be trying out two different systems. Already there are questions about how to install some of the technology required, and, of course, who is going to pay for it.
Although this technology can be relatively inexpensive for students (under $50.00 / year), it is still going to require correct management on the part of the institution. If we do not step in early, perform tests and studies, and anticipate the distribution and use problems, we could see a plethora of different clicker systems arriving on campus. Certainly a wide variety of systems will lead to support issues – to say nothing of relying on students to remember which clicker to bring to what class!
This project is designed to look at use (student and instructor perspectives), support (distribution, hardware, software, training), and cost (student and institution). Our team of investigators has deep experience in all aspects of this study. In keeping with the focus of improving the student experience at UVic (especially the first year experience), the technology will be used to stimulate student engagement within large, first year classes. These classes are all taught by senior, experienced faculty. And all classes are in the large, first year cohort – an important group to actively engage. As well, Leslee Francis-Pelton and Tim Pelton bring a wealth of educational assessment experience to the team, especially in the area of personal response systems in education.
The timing is right for UVic to take a proactive stand and get both the infrastructure and campus-wide organization for this technology in place, as well as making suggestions for best practices in our large, first year classes. Properly done, this will afford UVic students and faculty easy and inexpensive integration of this technology into teaching and learning. It will also demonstrate to our first year cohort that we care deeply about what we teach at UVic. Indeed, that we care enough to thoroughly test and evaluate new teaching ideas.
Project Timeline
In January 2006, four instructors will use an RF clicker system in at least one of their large first year classes. The classes are as follows:
Dr. Penelope Codding, Chemistry 102 S01, 215 students
Dr. Sara Ellison, Astronomy 120 Y03, approx 84 students
Dr. Rosaline Canessa, Geography 101A, approx 135 students
Mary Sanseverino, Computer Science 100 S01, approx 125 students
Total of approx 550 students.
PROJECT TIMELINE
TIME FRAME / ACTION(S) / RESULT(S)Sept 2005 /
Pilot project proposal reviewed by VPAC office, CASS, Learning and Teaching Centre, Dean of Science, Dean of Engineering /
Money to run the pilot project agreed upon.
Oct 2005 /
Hardware and software acquired and tested /
Hardware and software are distributed to instructors.
Systems tested in teaching classrooms. CASS involved.
Oct 2005 / Prepare Ethics request for study / Ethics request
Nov 2005 /
Faculty workshops /
Instructors meet to explore / train / discuss strategy on use of system. LTC involved.
Nov 2005 / Design Questionnaire for students / Questionnaire
Dec 2005 / Questionnaire given to Anthropology students and Economics students using IR systems. / Data to be analyzed.
Jan 2006 – Mar 2006 / Systems deployed in the three classes
Invitation to other educators to sit in on selected classes and see the clicker systems at work. / Instructors keep problem log
Instructors note pros/cons/ideas
Provide working examples for other educators.
Late Mar 2006 – April 2006 / Conduct questionnaires and focus groups.
Aggregate grade comparisons from courses.
Confer with Bookstore
Confer with CASS
Confer with LTC / Data to be analyzed.
April 2006 – June 2006 / Data analysis and recommendations / Report to VPAC
Scholarly papers in progress.
Presentation at several conferences and workshops.
Project Costs:
550 RF clickers and registration @ 25.00 US$13,750.00US
Shipping $700.00 US
Total US $14,450.00 US
Sub Total CAD$17,100.00 CAD (1 USD = 1.17825 CAD)
Resale 500 (some will be lost) @ 12.00 CAD($6600.00 CAD)
Total CAD $10,500.00 CAD
A cost of approximately 19.00 CAD / student (excluding taxes).
It will be possible to recoup some money on the clickers by reselling them for the Sept term – either to students at UVic, or to other institutions. Reselling at 12.00 CAD / clicker certainly seems possible as the price new is 15.00 US.
As we are working with students from various faculties, and are looking at a number of institutional questions, there are many groups that could be involved. We have approached and received monetary support from the following groups:
A). The Deans of the Faculties of Science, Engineering, and Social Sciences have committed a total of $3150.00 to this study
B). The Learning and Teaching Centre contributes $1500.00
C). CASS contributes $2500.00
D). VPAC's office contributes $5000.00
Total from all sources – (Additional is to cover taxes) $12,150.00
1.0 Introduction
The clickers are coming – is UVic ready? Personal Response devices, euphemistically known as "clickers," are starting to make themselves known on campus. These handheld wireless devices allow students to respond to multiple choice questions posed by their professor. The responses are gathered by one or more wireless receivers attached to the instructor's computer and are then aggregated and shown to the students. The questions can be provided in a number of ways: paper, tied into presentation software (i.e. PowerPoint), spoken, chalkboard, overhead transparency, etc.
The worth of these systems appears to be in benefits to learning by increasing classroom interaction (Horowitz, 1988), (Burnstein, R; Lederman, 2001), (Guthrie, R; Carlin, 2004). Certainly, these systems are not the "silver bullet" that will solve all the problems surrounding student engagement, but they do appear to be a useful instructional technology application. Since the technology is easy to incorporate into lecture formats, and since some publishers (notably McGraw-Hill Ryerson) are packaging them with textbooks, we can expect to see them proliferating on campus in the next few years. Indeed, in the upcoming term (Sept 2005), two instructors of large, first year classes will be trying out two different systems. Already there are questions about how to install some of the technology required, and, of course, who is going to pay for it.
Although this technology can be relatively inexpensive for students (under $50.00 / year), it is still going to require correct management on the part of the institution. If we do not step in early, perform tests and studies, and anticipate the distribution and use problems, we could see a plethora of different clicker systems arriving on campus. Certainly a wide variety of systems will lead to support issues – to say nothing of relying on students to remember which clicker to bring to what class!
This project is designed to look at use (student and instructor perspectives), support (distribution, hardware, software, training), and cost (student and institution).
1.1Response System Specifics
Personal Response Systems (clickers) have been used in education since the early 1980's. Working at IBM , Horowitz (1988) showed how useful they were in learning situations. However, those systems were cumbersome, hard-wired together, and expensive. Today the wireless revolution has changed all of these downsides, and we now have a tool that is light and portable, durable, and relatively inexpensive.
There are two types of systems currently in use – those that rely on infrared (IF) technology to transmit responses, and those that rely on radio frequency (RF). While somewhat less expensive (e.g. one system has a clicker cost of 5.00 US for IR, versus 12.00 for RF), the difference in usability, especially for large classes, is great. In order to work with numbers above 40 to 60 students, or in rooms that are wide rather than long, the IR systems need multiple receivers. In some cases, the system must be networked together in a given room, and then connected to the instructor's computer. In other cases it is possible to simply daisy-chain multiple receivers together, and then spread them out in the classroom at the start of every class. Moreover, as IR works on line-of-site, the students must point their clickers directly at a receiver. Also, florescent light can interfere with the IR signal. In contrast, the RF system uses only one small receiver that plugs into a USB port on the instructor's computer, and can handle responses from up to 1000 clickers.
Another concern with IR vs. RF is the ability of the clicker to let the student know when an answer has been received by the receiver. In the case of most IR systems, some sort of visual cue on the projected computer screen at the front of the room is used. This is fine for smaller classes, but for classes over 50, there is only so much space that can be given over to providing visual cues. This means that the cues either have to cycle around, or the entire screen has to be given over to providing these cues.
With the RF system, the cues that the answer has been received are right on the clicker unit itself. Moreover, access time (clicker to receiver) is very fast – hundreds of answers received, registered, and processed in a few seconds.
For the reasons noted above, we propose using only an RF system for this project. Although IR is cheaper "on-paper," the support issues surrounding testing, placing, and servicing additional receivers would outweigh the initial savings. This is not to say that IR systems will not work for larger classes. They certainly do, however, for this project, we want the system setup to be as easy as possible, and require the least amount of assistance from IT support agencies on campus as possible.
From the instructor's perspective, the system must work equally well with a Macintosh operating system as with a Windows system. Running a Windows simulator on a Mackintosh system will not do. A typical system would integrate with PowerPoint for both question and response display. The system should also integrate with Excel for grading purposes.
The systems to choose from include the following:
1). HITT Classroom Response System:
2). Quizdom:
3). eInstruction:
4). TurningPoint:
5). Interwrite:
Of these, only eInstruction offer RF systems that work in both the Macintosh and Windows worlds, and that integrate with PowerPoint and Excel. Quizdom is beta testing a Macintosh system and my well have one available for use sometime in 2006.
1.1.1 Business Model Differences
The business model eInstruction uses is quite different from most of the other vendors. The other vendors sell the clickers and receivers at a one-time cost, and the instructor is responsible for entering class rosters. that match clickers to students. eInstruction sells the clickers at a one-time cost, gives the receivers to instructors for free, and charges a term by term registration fee to students. This fee activates the clicker, creates the class roster of clickers / names, and provides a conduit for instructors to communicate with students via a web interface. It does, however, look like there is some ability to have a site-license – PurdueUniversity has negotiated one. In any event, there appear to be pros and cons involved in these models. This project will look at and report on all of the models.
1.2Why use a "Clicker" system?
Good teachers care about whether their students learn. Good teachers frequently ask themselves "do they get it?" Yet when an instructor asks if "anyone has any questions" the response is often a resounding silence. This does not necessarily mean the students are "getting it," often the exact opposite! Clicker systems allow instructors to poll the thinking of all students in the class, at any time in the class, without students having to risk the embarrassment of a wrong answer, or in speaking out in front of classmates.
If used correctly, clicker systems can promote more interactivity in the classroom. Students become more actively engaged in their learning. When compared with the traditional lecture format, interactive classes provide more learning gains for students (Hake, 1998), (Duncan, 2005).
Douglas Duncan (2005) reports that the correct use of clickers in the classroom can have the following benefits:
- Measure what students know before instruction starts (pre-assessment)
- Measure student attitudes
- Find out if students have done required readings
- Get students to confront common misconceptions
- Transform the way instructors do demonstrations
- Increase students' retention of what is being taught
- Test students' understanding
- Make some kinds of grading and assessment easier
- Facilitate testing of conceptual material
- Facilitate discussion and peer instruction
- Increase class attendance
As with any instructional technology, clickers can be misused. Moreover, they are not the only way to increase student interaction in a classroom. However, when used correctly, they do look like an effective instructional tool. Given the potential benefits, it is not surprising to see so many faculty keen on trying them out. Support around introducing, training, and assisting faculty with this type of instructional technology will certainly be required.
2.0 The Pilot Proposal
During the January to April 2006 term three instructors will use an RF clicker system in at least one of their large first year classes. The classes are as follows:
Dr. Penelope Codding, Chemistry 102 S01, 215 students
Dr. Sara Ellison, Astronomy 120, approx 84 students
Dr. Rosaline Canessa, Geography 101A, approx 135 students
Mary Sanseverino, Computer Science 100 S01, approx 125 students
Total of approx. 550 students.
All classes are part of multi-sectioned courses. Astronomy 120 and Chemistry 102 have other professors teaching other sections. Computer Science 100 has Mary Sanseverino teaching both sections, although only one section will have the clickers.
All of the participating instructors have agreed to use the clickers at least once a week in class, if not more. All instructors will record pros and cons, and log any problems encountered with the system.
At the student end, every student will sign out a clicker for the duration of the course. As this is a pilot, there will be no cost to the students. However, if the student looses his or her clicker, they will have to pay for a replacement.
Not only will the pilot look at support, deployment, and faculty issues, it will also attempt to answer a number of student-based research questions around improved learning. The timeline for this proposal is six months – January to April 2006 for the in-class use portion, then May to June 2006 for analysis, write-up, and dissemination of information.
This pilot will deliver a comprehensive report to the Vice President Academic evaluating what we have learned and what the most cost-effective technology and options might be (assuming it works well enough) The report will look not only at impact, but cost feasibility (eg: will students pay, as part of course materials costs?), preferred technology, technical support needs, instructor learning and training needs, etc.
2.1 Research Questions
As mentioned, this proposal has a research agenda. Not only will it address support and logistics questions, but it will look at faculty training, student acceptance, and student performance issues.
TABLE 1: Research Questions
STUDENT ISSUES1). Any evidence for improved student performance -- indicators
include:
- Students attending more classes
- Improved performance on exams
- Better understanding of material
- Students feel they "get more out of" classes
- Loss / forget to bring
- Registration – ease of registering, problems and resolution
- Overall usability
- Would students pay for clickers
- If yes, how much
- What would change opinions
FACULTY ISSUES
1). Use issues:
- Ease of setting up receiver software
- Ease of use with hardware
- Ease of use in the class
- Ease of problem solving
-- in class
-- for self - Ease of use / accuracy of class roster preparation
- Problem log
- Development and training recommendations
- Changes to teaching
- Overall pros / cons
1). Hardware / Software support requirements across campus
2). Distribution possibilities
3). Vendor Business Models that would work within UVic environment.
4). Possibilities to build our own systems.
5). Development and training issues
2.2 Methodology
The pilot project would see UVic purchase approximately 550 eInstruction RF clickers and registration. Each instructor would get a receiver and the eInstruction Classroom Response System software at no charge. Each instructor would be responsible for handing out the clickers to the students. Students would then be responsible for registering their clickers into the correct classes via a website.