S207 AL Tablet PC Trial

The use of Tablet PCsfor marking physics assignments

Jonathan Underwood* and Stuart Freake,

Department of Physics and Astronomy, The Open University

(* now at Department of Physics and Astronomy, UniversityCollegeLondon)

May 2007

Abstract

Marking and commenting on electronic tutor marked assignments (eTMAs) requires a simple and efficient method for tutors to annotate students’ scripts. We report here on a pilot project that provided associate lecturers (ALs) for a Level 2 physics course with Tablet PCs and appropriate software that enabled them to input handwritten annotation, including equations and diagrams, to pdf files containing students' assignment solutions. A survey of ALs views indicated that this method of marking eTMAs is effective and acceptable to both ALs and students. Some ALs expressed concern about additional time required for marking; this may not be an issue when ALs become more familiar with this system, but needs to be kept under review.

1Introduction

The majority of Open University courses are assessed by a combination of tutor marked assignments (TMAs), which are submitted by students at intervals during the presentation of a course, and an end-of course examination. The TMAs are marked by associate lecturers (ALs, also known as tutors), who are based at different location around the country and are responsible for a group of about 25 students. The traditional method of producing and marking TMAs is paper based: a student writes the answer to their assignment on paper (or word-processes it and prints it out), they use the postal system to mail it to their AL, who marks and comments on the assignment by annotating the student’s script and completing a cover form (the PT3 form). The AL sends marked assignments and PT3 forms to the Open University in Milton Keynes, where the marks are recorded in a database, a sample of scripts are copied so that each AL’s marking can be monitored, and the scripts are then posted back to students.This system involves shuffling large quantities of paper around the country and there are delays and expenses associated with the postal system and with the transcription of marks and the copying of assignments for monitoring. Now that OU students and ALs are expected to have access to a computer and the internet, some of the delays and costs can be reduced by handling TMAs electronically.

The OU has developed a system for handling electronic tutor marked assignments (eTMAs), and is encouraging course teams and students to use this system wherever possible. The eTMA system allows students to submit electronic files containing their assignments over the internet to a file handling system on a server in Milton Keynes. ALs download the eTMA files for their students from the server, mark and comment on the eTMAs offline, and then return the marked scripts over the internet to the server in Milton Keynes, where marks are automatically recorded in the appropriate database and a sample of scripts is made available for monitors to download for monitoring. Students can then download their marked assignment from the central server. This system has many advantages for students, ALs and the OU over the paper-based assignment system that relies on the post office for moving the assignments around the country, but it also has some disadvantages.

Gibbs and Simpson (Learning and Teaching in Higher Education, Issue 1, 3 (2004-2005) established a list of conditions under which assessment effectively supports learning. A key finding of this work was that it is essential that “feedback is timely in that it is received by students while it still matters to them and in time for them to pay attention to further learning or receive further assistance”. This issue was also highlighted in the ‘seven principles of good practice in undergraduate education’ (Chickering, A.W. & Gamson, Z.F. (1987) Seven principles to good practice in undergraduate education, Racine, W.: The Johnson Foundation Inc. and Chickering, A.W. & Gamson, Z.F. (1991) Applying the seven principles to good practice in undergraduate education, San Francisco: Jossey-Bass). The eTMA system in principle allows ALs to give more rapid feedback to students following the cut-off date for the assignment than does the paper-based system.

However, the benefits of rapid feedback could be more than offset if the time required for students to produce an eTMA were significantly longer than to produce a paper TMA, so that their time is diverted from learning physics, and if a longer time were required for ALs to mark eTMAs to the same standard as paper TMAs, so that the quality and quantity of feedback provided to students is reduced. Assignments for physics and astronomy courses (and courses in some other disciplines too) generally involve large amounts of mathematical notation, equations and diagrams. If these are to be produced and submitted by students electronically, and marked by ALs using the OU's eTMA system, then students and ALs need to be able to incorporate equations and diagrams simply and efficiently. However, many OU students do not have the skills required to incorporate equations and diagrams into a word-processed file, and for some of those who do have these skills the additional time required compared with handwriting an assignment is prohibitive. The same factors apply to the marking of eTMAs by ALs, though in this case the time factor is generally the major issue. Keyboard and mouse based tools, such as equation editors and line drawing programs, are much more time consuming to use compared to annotating a paper script with a pen or pencil.

The pilot project described in this report was set up to investigate one possible way in which ALs could mark physics eTMAs easily, and could incorporate equations and diagrams just as they would if they were marking a paper assignment.A key requirementwas that ALs should be able to handle eTMAs submitted by students in a variety of file formats,since it was regarded as important that students should be allowed to use whichever software packages they were most comfortable with when preparing their eTMA.

2The project

In March 2006 the Physics Innovations Centre for Excellence in Teaching and Learning (πCETL) at the Open University agreed to fund a pilot project to investigate the use of Tablet PCs by ALs for marking assignments. All of the ALs who were tutoring a Level 2 physics course, S207 The physical world, were invited to participate in the project. Since the project was set up after the start of the course, participation by both ALs and students had to be optional, and 19 of the 24 ALs agreed to take part. The participating ALs were provided with a Tablet PC (Toshiba Tecra M4), with appropriate software installed, to enable them to add handwritten annotation to a student’s eTMA file.

Students were informed about the pilot project at the end of March, and told about some of thebenefits of using the eTMA system. They were told that not all ALs were participating, that student participation was optional, and that they should discuss with their AL whether eTMA submission was possible and what formats were acceptable for their eTMA files. It was left up to individual ALs to decide how much encouragement they gave to students to submit their assignments electronically.

The timing of the approval of the project and of despatch of the Tablet PCs meant that electronic submission was not possible for the first assignment (TMA01), but most of the ALs were prepared to accept eTMAs for the second and subsequent assignments (TMA02 – TMA07).

2.1Marking procedures with the Tablet PC

ALs had a number of options for marking and commenting on eTMAs using the Tablet PC.

2.1.1Ink annotation of a pdf file using PDF Annotator

This is the option that we anticipated that most of the ALs would use. The Tablet PCs were sent out with the PDF Creator application installed (software authored by Philip Chinery and Frank Heindörfer, free download from This application creates a “virtual printer” that can be used to convert into pdf formatany file that be printed. The AL simply opens a student's eTMA file – which requires that an appropriate application is installed on the Tablet PC – selects Print and chooses PDF Creator as the printing device. 'Printing' the file then produces a pdf version of the file, which is stored in a location specified by the AL, rather than printing out on paper. It is also possible to combine a number of different files into a single pdf file.

The AL then opens the pdf file from the PDF Annotator application (Grahl Software Design, which was also preinstalled on the Tablet PC. With the computer in the tablet configuration, the pdf file can be annotated by writing on the screen with the tablet pen in exactly the same way that a paper assignment would be annotated with a conventional pen, but with various advantages:

- it is easy to change pen colour and line width – so it's possible to colour code different types of comments;

- it is easy to erase or correct comments;

- it is easy to move or resize comments;

- it is possible to save comments as macros that can then be inserted easily elsewhere in the same document or in other documents;

- it is easy to insert blank pages for additional comments, or to append files (such as specimen answers).

When the AL has finished marking, they save the file in pdf format and follow the standard procedures for returning the eTMA to the OU server. The student downloads the marked assignment from the server and opens the file using Acrobat Reader.

One facility that was not available in PDF Annotator in 2006was the input of text using the keyboard. When this application was running, the keyboard was disabled, so even if the computer were in the laptop configuration rather than the tablet configuration, it was only possible to annotate a document using the tablet pen.However, an upgrade to PDF Annotator in May 2007 (version 1.5.0.108) now allows simple text to be keyed into a text box, and ALs will make use of this facility when marking the later TMAs for the 2007 presentation of the course.

2.1.2Ink annotation using Microsoft Word

A second option for eTMA marking is to use the Ink tools available when Word 2003 (which was installed on the computers that were provided) and subsequent releases of Word are installed on a Tablet PC. With the computer in the tablet configuration, it is possible to use the Ink Annotation tool to write with the tablet pen anywhere in an open Word document, and, as with PDF Annotator, it is possible to change pen colour and line width, to erase ink marks, and to reposition and resize comments. In contrast to the PDFAnnotator approach described above, employing Microsoft Word for marking requires that the student submits the TMA in a file format compatible with the version of Word used by the AL.

Alternatively,the Ink Comment tool can be used to insert comments in the margin. This is similar to using the normal Insert Comment feature in Word, except that the comments are handwritten in a balloon in the right marginusing the tablet pen. These comments can include equations and diagrams.

When the AL has finished marking the script, the file is saved in .doc format for return to the student via the eTMA system.

An important advantage of marking and commenting using Word is that the AL is not restricted to using ink since, unlike PDF Annotator, keyboard entry is always available while the programme is running. It is possible to switch back and forth between using the keyboard to enter for simple text in the laptop configuration and using the tablet pen to enter equations and figures in the tablet configuration. Switching between the two configurations of the screen takes about 15 seconds, which is not an unreasonable overhead, but the switchover is somewhat clumsy and it is not clear whether the hinge mechanism for the screen is designed for such frequent use. An alternative procedure tried by some ALs is to connect a USB keyboard to the Tablet PC, so that both keyboard entry and tablet pen entry are possible in the tablet configuration.

3Evaluation

After completion of the 2006 presentation of S207, a brief questionnaire was emailed to the ALs who had participated in the pilot project, and 15 of the 19 participants responded.

Number of eTMAs submitted The extent to which the ALs had engaged with eTMA marking during the year was equal to the number of TMAs that their students submitted electronically, since the participating ALs were required to mark electronically all of the eTMAs that they received. There was a large variation between the numbers of eTMAs marked, as shown in Figure 1. Eight ALs received four or fewer eTMAs during the year, but twoALs received over 30, and one of them received 44. The total number of eTMAs submitted for each assignment increased for the first few assignments, as shown in Figure 2, then remained fairly steady for TMAs 04 – 06, and dropped slightly for the final TMA. However, the percentage of TMAs that were submitted electronically increased throughout the year, reaching 33% for the final TMA.

Figure 1 Number of eTMAs marked by each of the 19 participating associate lecturers.

Figure 2 Total number of eTMAs received for TMAs 02-07, and eTMAs as a percentage of the total number of TMAs submitted to the 19 participating associate lecturers.

Format of the eTMA files submitted ALs reported that most of the eTMAs were submitted as Word doc files, but some students submitted pdf files and occasionally there were filesproduced with Star Office or MS Works. Some students inserted scanned diagrams into Word files, and a few students scanned a complete handwritten TMA and submitted it either as a set of jpg files or combined into a single pdf file.

Marking procedures used by ALs Most ALs converted students' files to pdf (if they were not already in pdf format) and used the PDF Annotator application to annotate them. Several commented that they liked this way of working because it mirrored their usual method of marking paper TMAs. Four ALs used PDF Annotator for files that were submitted in pdf format, but when a student submitted a Word readable file they opened the file in Word 2003 and used the standardWord facilities to insert comments,track changes and insert text boxes, plus the facility provided with the Tablet PC software to use ink annotations and ink comments for equations and diagrams. One AL indicated he preferred this method to using PDF Annotator as it enabled him to use the keyboard for text entry and he could type text much faster than he could write.

Difficulties in learning how to use the marking system ALs were asked whether they had problems learning how to use the Tablet PC or the associated software. Even though the only information that they had received was the standard user manuals for the computer and the software, there were few reported problems. One person was averse to the use of an additional computer and did not complete the setting up of the machine because they were put off by 'problems' discussed by colleagues on the S207 AL FirstClass forum; however, they received no eTMAs so had no impetus to engage with the system. A few tutors said they would have appreciated more guidance on the conversion of files to pdf format and on the use of PDF Annotator, and one commented that the use of Print File in PDF Creator to produce a pdf file was counterintuitive. A couple of ALsthought there should be explicit guidance about how to insert pages into a script for additional comments. The other comments made by ALs were related to difficulties experienced in learning how to use the standard eTMA file handling system. One AL handed the training materials to his (presumably IT-literate) son, who then trained him (and received the training fee).

Ergonomic considerations Six tutors said that it took them some time to find a comfortable way to mark using the Tablet PC. Three of them found that it was difficult to use the Tablet PC while wearing varifocal glasses, and two of them overcame this problem by using single prescription lenses while marking. Two ALsworked with the tablet on their laps and another found that they had to use a lower working surface. In future it would be appropriate to provide some 'ergonomic' advice to ALs as part of a brief guide to the use of the Tablet PC for eTMA marking.

Ongoing problems and disadvantages of using the Tablet PC for marking The main ongoing problems and disadvantages that ALs mentioned were:

• some ALs found that it took longer to mark eTMAs, particularly those who only marked a small number of eTMAs and had to relearn how to use the system each time;
• it was difficult to trawl through pages to get an overview to ensure that the comments were appropriate, or difficult to lay out pages and compare several scripts to ensure consistency;
• it was not as convenient as paper for marking while waiting in the car;
• it was not possible to use the keyboard while running PDF Annotator;ALs wanted the ability to insert both typed comments and handwritten comments on the students’ eTMA script; (this is something that is now possible with the May 2007 release of version 1.5.0.108 of the application);
• it is not possible to cut and paste in PDF Annotator;
• students who previously submitted early, perhaps because of uncertainties about the postal service, now tended to delay submission until just before the deadline.

Advantages of using the Tablet PC The ALs were asked about the advantages of using the Tablet PC compared with paper marking, and the points they mentioned were: