DRAFT10/07/181

______

Integrating Cognitive Tools for Peer Help:

the Intelligent IntraNet Peer Help-Desk Project

Jim Greer, Gordon McCalla, John Cooke, Jason Collins, Vive Kumar, Andrew Bishop, Julita Vassileva

Department of Computer Science, University of Saskatchewan

Abstract

Recent research activities at the ARIES Laboratory have been focused on developing cognitive tools to encourage and promote peer help in workplace training and post-secondary education. This work has been motivated by the desire to provide cost effective advice, help, coaching, and mentoring for learners in a variety of contexts. Peer help can be very cost effective, is deeply situated in a shared context, and can provide positive and authentic learning experiences for both helper and helpee.

One initiative, called the Cooperative Peer Response project, better known as "CPR for Students", provides a suite of WWW-based tools to facilitate cooperative learning, peer help, and expert help within a University course. The CPR discussion forum encourages peer help, while other more knowledge-based components focus on system-generated help. CPR has been deployed in a number of University courses and has proven to be an effective learning support tool.

Another initiative, called the Peer Help System, better known as PHelpS, provides a facility for locating, from somewhere in the organization, a peer helper who is ready, willing, and able to provide help. A capable helper is one who is likely to know the answer (or willing to help find the answer) to a particular help request. A prototype of PHelpS has been deployed in a distributed workplace environment and preliminary experiments suggest that it is an effective training and performance support tool.

These two initiatives, together with various related spin-off projects, have led to an attempt to integrate several cognitive tools into a new style of Intelligent IntraNet Help-Desk facility. Such a Help-Desk draws together a variety of cognitive tools, particularly tools for peer help, into a comprehensive environment to support many styles of learning. This chapter outlines the design of the Intelligent IntraNet Help-Desk focusing on the proven peer-help tools that act as its structural skeleton.

1Introduction

This paper presents a somewhat different example of computers as cognitive tools than those presented in the other chapters of this book. Instead of designing an artificial environment (based on a certain cognitive theory) supporting human learning, we decided to create a tool which supports a phenomenon which happens naturally in any learning context: peer help.

Many authors have pointed out the advantages of peer help as a learning method (Presley et al., 1992). First, it is deeply situated in a shared context and can therefore provide a stronger learning experience for the person asking for help. Second, it is a way to make learning happen Òjust in timeÓ, i.e. when the problem arises. Third, it promotes processes of self-explanation (Chi et al. 1994) and reflection in the helper, and in this way ÒreciprocalÓ learning (Nichols, 1993) takes place. Fourth, it is cost effective, since it uses the learners themselves as a teaching resource. And finally, it facilitates social interaction in a group of learners and helps to create knowledge-anchored personal relationships among them.

Peer-help happens naturally within every group of learners. When the group of learners is distributed, however, obstacles arise for this process to happen. Learners who need help may not know whom to ask for help, since they are not able to identify which member of the group is knowledgeable; in fact they may not even know the other learners. If they did know a potential helper, they wouldnÕt know whether or not the helper was currently available (without asking), which means a loss of time and a loss of the immediate context in which the problem has arisen.

Exactly this phenomenon occurs in distributed learning environments. Such environments are becoming increasingly popular, especially for organizations providing training courses for their employees on IntraNets. In distributed IntraNet learning environments employees can take a course at their place of work, during their preferred time. In this way companies save a lot of money by not taking people away from their job, by not needing to find replacement workers, and by not hiring instructors or paying travel expenses. Slowly, but steadily, Universities are moving towards this model of education too, by providing access to web-based course materials.

The natural process of peer help in distributed learning environments can be supported in several ways:

by providing an appropriate forum where people can ask subject-related questions and give answers, discuss problems of common interest either synchronously (like a chat-facility), or asynchronously (like a newsgroup-facility);

by finding appropriate peer helpers who are knowledgeable and likely to be able to answer a learnerÕs question (like an electronic dating system!);

by providing a shared workspace for helper and helpee where they can share the context of the problem;

by helping the helper better understand the problem of the person asking for help and suggesting a pedagogically appropriate way of helping. For example, sometimes it is pedagogically better not to give any help, but to encourage the learner to try harder to solve the problem alone.

We have addressed the first two ways of supporting peer help by developing and integrating a set of cognitive tools. The first one, called the Cooperative Peer Response (CPR) system, provides a suite of WWW-based tools to facilitate cooperative learning, peer help, and expert help within a University course. The CPR discussion forum (the newsgroup-facility) encourages peer help, while other knowledge-based components focus on system-generated help. CPR has been deployed in a number of University courses and has proven to be an effective learning support tool.

Another tool, called the Peer Help System (PHelpS), contributes to the second way of supporting peer help. PHelpS provides a facility for locating somewhere on the network a peer helper who is ready, willing, and able to provide help, i.e. one who is likely to know the answer (or willing to help find the answer) to a particular help request. A PHelpS prototype has been deployed in a distributed workplace environment and preliminary experiments suggest that it is an effective training and performance support tool.

The advantage of using an IntraNet instead of the Internet is that in this way access is restricted to students attending the course only, so that they can communicate with their peers and teachers only. In this way students are protected from possibly disturbing comments made by occasional ÒvisitorsÓ, who are not involved in the course. This makes it easier to track down what students are doing and to collect information about the topics that were discussed. This information is used both for improving the peer-help facility and as a feed back to teachers to adapt the course accordingly. It also eliminates some concerns about the privacy of user information gathered during studentsÕ work with the system.

CPR and PHelpS, together with various related spin-off projects, have led to an attempt to integrate several cognitive tools into a new style of Intelligent IntraNet Peer Help-Desk facility. Such a Help-Desk draws together a variety of cognitive tools, particularly tools for peer help, into a comprehensive environment to support many styles of learning. This chapter outlines the design of the Intelligent IntraNet Help-Desk focusing on the proven peer-help tools that act as its structural skeleton.

The Chapter is organized as follows: section 2 motivates the need for and benefits of peer help in university teaching and learning. Section 3 presents the two main resources of the integrated help-desk: the Cooperative Peer Response system (CPR) and the Peer Help System (PHelpS). Section 4 presents the architecture and functioning of the integrated help desk. Section 5 provides a discussion and comparison with related work. Finally, section 6 outlines future research directions.

2Why Peer Help in University Teaching?

Universities, experiencing large growths in student enrollment, are faced with the difficult problem of providing adequate help resources for their members i.e. the staff, faculty and students. Help resources are needed at an institution-wide and also at a course-specific level, due to the limited time of instructors to provide help and answer questions. Computer technology offers several approaches to facilitating and providing the necessary personalized help resources that can be made available to a mass audience. By deploying IntraNets in Universities different kinds of resources (lecture notes, exercises, quizzes, syllabii, etc.) can be made available on request to any student. There are numerous positive examples of implementing on-line course materials and discussion groups at other universities, for example, the Virtual-U Project (Virtual-U, 1997), WebCT (Goldberg, 1997), Quorum (Canas et al., 1995).

However, merely providing access to appropriate material via a network doesnÕt solve the problem of providing help. One way to decrease the load of teachers is to provide conditions for students to help each other. Peer help has many pedagogical advantages. For example, it promotes the socializing of students in the context of work and increases their motivation by giving social recognition for their knowledge and helpfulness. Computer technology can be applied to support peer help. There are many Computer Supported Collaborative Work (CSCW) tools that facilitate communication among peers. However, they rarely ever provide personalized help on demand. We propose to develop cognitive tools that support peer help in an individualized way and just in time. Two such tools are described in the next section.

3Cognitive Tools for Peer Help

3.1The Cooperative Peer Response Project (CPR)

Created in the Computer Science department at the University of Saskatchewan in 1996, the Cooperative Peer Response system, also referred to as CPR, was an attempt to meet some of the urgent help needs of students in the department. The Cooperative Peer Response system provides the following: a discussion forum (a newsgroup-like facility) where students can post questions or comments; an interface to support rapid responses from peers, tutors, or instructors; an accounting system to recognize students who provide and use peer help; an FAQ facility; tools to aid the moderator in constructing and organizing the FAQ; and a database of potential peer helpers. The system is customizable to a particular course, providing peer help and FAQ facilities via the Web. Figure 1 presents the complete CPR architecture (the dimmed components are currently under development).

At the core of the CPR system is a database, that tracks interactions between the user and the system, that is used to create and maintain web pages, and that keeps a profile of each user. To keep the posted material relevant to the particular class, access to the system is restricted to registered users. CPR uses only standard HTML tags on all of the pages, and is therefore accessible to both textual and graphical browsers. Three sources of help information (modules) are available in CPR: a rapid response area, called Discussion Forum, an archived section called the Moderated FAQ, and a User Adapted help resource.

Figure 1: The CPR system architecture

Discussion Forum

The presentation and function of the Discussion Forum is similar to a newsgroup, with the content provided by the users themselves. Links to the latest questions and their associated replies are stored on this one HTML page, which is used to organize and index the information found in each separate question file. The most recent questions are placed at the top of the page, and the most recent replies to a particular question are placed at the top of that particular list of replies. This threaded scheme is used in both the Discussion Forum index page (Figure 2) and the individual HTML page for each particular question and its replies (Figure 3).

Figure 2: Discussion Forum Web page

Each entry in the Discussion Forum consists of a class-specific category or topic; an author (who may be anonymous); subject and keywords (both optional); the posting time; and the text of the question, comment or reply. If a student posts a question, a peer can post a reply immediately. Thus, help can be available to the user on demand. The class experts can be extremely useful in this medium, increasing the possibility of many good solutions being proposed for the same problem. The moderator is also equipped with tools to quickly remove/add entries to the Discussion Forum. This makes it easier to correct erroneous postings. The role of the moderator is to install and maintain the CPR site (including handling the users and maintaining the web pages), to handle any unanswered postings, and to correct any erroneous items. In particular, he or she has to enter the categories that will be used to organize the postings.

Figure 3: A Question and its Corresponding Answers Web page

Moderated FAQ

The Discussion Forum periodically needs to be pruned to remove outdated entries. Some of these entries might be discarded, but others should be entered into an archive Ð the Moderated FAQ Ð for future use. The design of the Moderated FAQ pages is similar to a traditional FAQ, with a list of class topic categories, cross postings of messages in relevant categories, and a table of contents to organize the postings.

A problem with traditional FAQs is maintaining the consistency of messages that have been posted under several topics, or easily adding to or removing entries from one or more FAQ subtopics. CPR handles this by manipulating the structure of the FAQ pages through modification of the underlying database. This method also makes it easier to maintain consistency when the actual text of the FAQ entry has been modified. Only the moderated FAQ entry itself needs to be updated as the changes are reflected in all of the relevant class category pages.

The job of the moderator is made easier because the main task is to organize and maintain the archive by selecting the best peer questions and answers, rather than to dream up questions and provide answers to all of the questions. Peers will answer the majority of the questions and only the really difficult ones need the moderator. The role of the moderated FAQ is to serve as a repository of reference material available to the class. The information can be expected to be accurate since it was edited by a knowledgeable individual before inclusion, usually after being proposed by classmates.

Evaluation

The first version of CPR containing these two modules (i.e. without the adaptive help facility) has been in use in several computer science courses during the following semesters: fall 1996, spring 1997 and summer and intersession 1997. Throughout each of these courses, different approaches to evaluating the use and user satisfaction with the system were taken. For each of the three semesters data were collected from actual observation of the system in use, from questionnaires completed by the users, and from Web server statistics. The students who used the system had high praise for it, but complained about the lack of participation of their peers in the discussion forum. The majority of the students thought that the system should be available in all of their computer science courses.

Adaptive Help Facility

One of the problems with a CPR-type system that students complained about was that their questions were sometimes not answered. The time delay in obtaining a useful response was also a complaint. Response time was heavily dependent on who else was on-line at that time as well as whether or not these other users were able and if so, willing to respond. Sometimes when questions were answered, the responses given were not what were required. Subsequent on-line communication was needed to refine these responses until a useful answer was obtained. These problems highlight the need for automatic help response and for integrating the system with a peer-help system like PHelpS, which would find a peer-helper, with whom the student could communicate on-line, and receive a just-in-time help. For this reason CPR is currently being upgraded with an adaptive help facility (see Figure 1).

In order to provide an automatic help facility, it is necessary for CPR Òto understandÓ the studentÕs question. Two of the things that are necessary are a concept map of the topics covered in the course, and a keyword-synonym dictionary for finding matches to user-entered keywords. A concept map underlying the course was included in CPR and the threads and FAQs were indexed with respect to the concepts. Using the concept map as a guide, terms that can be used as keywords have to be identified. These terms are sought by the system in the question posted by the student.