Addressing the Challenges of Teaching Research Skills and Argumentation

Addressing the Challenges of Teaching Research Skills and Argumentation:

A Blueprint for Collaboration

Abstract

In this project, a composition instructor and a librarian collaborated to teach information retrieval and evaluation skills in support of rhetorical argumentation. Their success and lessons learned offer a blueprint for effective faculty/librarian collaboration and for the integration ofinformation literacy and traditional composition instruction.

Introduction

At PennState, English 15 – Rhetoric and Composition is a general education course that must be completed by all undergraduates. The battery of essay formats taught in this course includes a rhetorical argumentation essay that must be supported by reliable sources. At the Delaware County Campus, librarians regard this assignmentas the perfect opportunity to introduce research skills that involve not only database search strategy, but also critical evaluation of the relevance, authority, timeliness, and bias of sources. In most cases, English 15 instructors schedule one 50-minute class period for the librarian tolecture and demonstrate database search strategy and basic criteria forevaluating the quality of their sources. Guiding students in evaluating sources as they develop their arguments is left to instructors during subsequent class sessions and individual consultations.Both librarians and English 15 instructors have found this strategy ineffective. Despite our best efforts, most students do not transfer the skills covered in library workshops to their final project database searching and source selection.

Both student and institutional challenges presented obstacles to our teaching effectiveness. Most studentslacked motivation, research experience, and the ability to transfer skills taught inlecture/practice exercises to their actual assignments. We noted that undergraduate motivation was stronglygrade-driven, while attendance and participation in library workshops offered no grade points or penalties. As novice researchers, theytypically used very broad search strategies and their choice of sources was more often based on convenience than quality or relevance. When they failed to apply the library skills from workshop exercises to subsequent research assignments, most evident was their inability to evaluate sources based on authority and bias.

Institutional challenges included:

1) Lack of immediate context for the library sessions (assignments often did not coincide with the library sessions)

2) No Standard Practice: Generally, librarians work more closely with students in evaluating sources when they come to the reference desk for help. No standard practice had been established to ensure that all students get this additional attention. Lack of consistency between the library sessions and individual English 15 instructors followup instruction. Lack of consistency between English 15 research instruction and students’ research efforts in other courses.

3) Limited Time: Very rarely are additional class periods available for librarians and instructors to work with composition students in library labs as they search for and evaluate sources.lack of library session time to adequately cover the databases and their individual retrieval techniques (Boolean commands, etc.)

4) The Growing Diversity and Complexity of Electronic Databases makes this approach even more ineffective and increased the need to build a stronger source evaluation component in basic library skills training for English composition students.

The major challenges that led to our decision to create the library tutorials included:

A New Strategy: Web-based Tutorials

Web-based Tutorials

We sought the answer to our problem in Web-based Tutorials.

• Students can complete them Anytime and Anywhere

… and being self-paced, students can spend as much or as little time as needed on each skill.

• Tutorials are perfect for presenting basic concepts, and with basic HTML, you can design immediate Feedback and Assessment components.

• If students could develop Minimum Competencies through the use of Web-Tutorials prior to our one precious class meeting, the librarian could spend class time on more complex database search strategies and careful evaluation of search results.

• Fortified with a basic knowledge of HTML, a great freeware quiz that graded and reported student performance, and an Instructional Development Specialist who is a wizard, we thought, ‘This is the answer… let’s do it!”

Despite the recognition of these problems, no University or campus-wide consensus suggested alternatives to the existing approaches. After a couple of semesters using this approach and several discussions, we decided to attempt a different approach.

Since research today is primarily conducted “on-line” and since PennState advocates including technology in our classes and educating students in the latest technology available, naturally, our discussions included the possibility of on-line, web-based tutorials. During the summer of 2001, together with Matt Bodek, DelawareCounty’s Coordinator of Instructional Services, who would be responsible for programming the tutorials, we discussed how to design a better approach to teaching information retrieval and evaluation skills utilizing some web-based, interactive modules to supplement classroom pedagogy.

In this project, a librarian and a freshman composition instructor collaborated to teach information retrieval and evaluation skills more effectively with the assistance of Web-based tutorials. By assigning the Library Tutorials for Argument Essay Research prior to the library class meeting, we expected students to arrive with minimum competencies in database searching and information evaluation. With this achieved, the librarian could use the class meeting to move beyond the basics and focus on the complexities and subtleties of searching and evaluating sources and develop student skills that would be more lasting.

Specifically, our goals were to:

Since English 15 emphasizes the rhetorical pattern of argumentation, our composition instructor, Moyer, wanted a tutorial designed around and argumentative research paper which would:

1) introduce the mechanics of information retrieval

2) explain the value of sources and how to evaluate them

3) explain the role evidence plays in argumentation

4) provide some means of testing students’ information retrieval and evaluation skills

5) encourage students to use scholarly books and journals along with reputable newspapers

6) provide an opportunity for students to somehow use the tutorial to begin an actual search for sources for an individual research project and evaluate the quality of those sources

Of course, he wanted to do all this without taking precious class time from other components of the course. A web-based tutorial seemed ideal because students could work on it following their own schedules ( a big plus since many students held part-time jobs). He also hoped that other instructors would embrace the idea of a common tutorial and even contribute ideas for creating it.

Our librarian, Ware, envisioned a tutorial that would satisfy Moyer’s particular wants, but one that would also be used by other instructors, as well. Given her experience in assisting other instructors in designing their research projects and helping scores of students retrieve information for various courses, Ware wanted a tutorial or tutorials that would serve more than one professor’s class – she wanted tutorials that would serve different courses for different assignments. Her driving objectives were:

1) to teach basic database content, information retrieval, and evaluation skills using a web-based tutorial that students could access at their convenience and at their own pace;

2) to ensure instructors and librarians spoke with one voice when teaching information retrieval, evaluation skills, and the language of argument – terms like authority, relevance, objectivity, and timeliness:

3) to provide real-life topics and “authentic” tasks for grade credit to hold students’ interest;

4) to provide analysis of students’ work with immediate feedback.

The English Composition Objectives and the Library Objectives for the tutorial matched almost perfectly.

Objectives included in both categories were:

• Cover Basic Databases

• Teach Source Evaluation

• Use Consistent Terms (Tutorial should include the same terms used in the course textbook, class lectures, and library lectures)

• Use Course-Related Tasks (to be highly relevant to student’s immediate needs)

• Develop Transferable Skills (transferable to other courses and other assignments)

• Be Interactive (Provide immediate feedback to students)

Objectives specific to English Composition were:

• Save Class Time (Teach research skills in more depth without using additional class time)

• Earn Course Credit (Stress the value of learning research skills by awarding grade points for participation)

Objective specific to the Library were:

• Cover Periodical Types (Focus on the differences in content and style of popular magazines, news magazines, review and opinion magazines, and scholarly journals)

• Teach Search Strategy (Teach students how to revise search strategies as they review search results)

Our Tutorials

With Matt Bodek’s help, we created a web-based class assignment consisting of three database tutorials and quizzes, three evaluation tutorials and quizzes, and worksheets for students to document their search strategies.

(Originally??) The first database tutorial and quiz covered the online catalog (CAT) with a focus on finding books. The second database tutorial and quiz covered the popular and multidisciplinary database, ProQuest Direct, as a source of journal and magazine articles. The third tutorial and quiz covered Academic Universe (Lexis nexis) as a source of newspaper articles.

The evaluation tutorials and quizzes likewise covered books, journals and magazines, and newspapers and taught students how to evaluate their database search results for relevance, authority, bias, and timeliness. The journal and magazine module described the differences between scholarly and professional journals, news and opinion magazines, and popular magazines. It also offered recommendation for using them and explained the differences between primary and secondary sources.

Worksheets were provided for each tutorial

In collaboration with the Instructional Design Specialist we agreed on the:

• Tutorial content and design

• 3 parts: Database Search Strategy, Article Evaluation, Worksheets

• include 3 databases: books, journals, & newspapers

• cover periodical types/primary secondary sources

• include online evaluation of full text articles (ProQuest durable links)

• exclude web searching

• Use research and evaluation terminology consistently (One Voice)

• in tutorial, in library workshop, in class, in text

• terms: credible evidence, relevance, authority, bias, timeliness

• Utility for Assessment and Feedback

• freeware program, ExamMail, an automatic exam-marking program by John Oysten and Carsten Schafer, © 1996

• perl script installation by Matt Bodek and Jeff Stine

• Guided Worksheets (students’ independent searches after database tutorial)

• analysis of quiz results

• Incentives

• class credit, tutorial sample sources accepted in final paper

•Scheduling

• coordinate tutorial deadlines with research assignment deadlines

Successes

Those students who completed the tutorials conscientiously and in a timely fashion clearly benefited from the exercise. They knew the vocabulary of argumentation, they understood the need for relevant, authoritative, objective, and timely sources, and they were familiar with the basic databases and retrieval commands. This translated into faster and more productive workshops conducted by librarians.

Our efforts were rewarded with a number of Successes:

Most student demonstrated Minimum Competence

• during the library lecture:

•Students were able to recall basic content and search commands of the databases covered in the tutorial.

• in the classroom:

•Student questions reflected an understanding of the databases.

•Students used the terminology of research and evaluation.

•Students expressed a desire to find the best sources.

Upon reports of success, more faculty assigned the Tutorial

• additional English composition sections used the tutorials

• Freshman Seminar courses used the tutorials

Problems

We found that we overestimated some students’ computer literacy, some students’ willingness to read the material, some students’ ability to follow directions, and some students’ honesty. Some of our students come from schools where computer literacy is not stressed and some do not have access to computers at home, so we overestimated their computer skills. Some students were not motivated nor mature enough to complete the tutoials on their own. A few never complete any of the tutorials, quizzes or worksheets. Several students failed to read the directions for doing the tutorials or had problems understanding the directions. Several students attempted to get around the assignments by circumventing the system’s authentication requirements. Rather than read the material and answer the questions, they would fake ids to enter the program, take the quizzes, and learn the right answers, and then they would redo the quizzes using their proper ids.

For the first year using the tutorials, we allowed too much time for the students to complete the tutorials, and lost the value of them; particularly in terms of their connection to what was going on in class lectures and discussions. Also, initially, a couple of students using older computer hardware and software from home had trouble accessing the tutorials.

Despite our best efforts, we made some Mistakes:

We Overestimated:

• student computer literacy

• freshman student motivation and maturity

• freshman student willingness to read

• the completion timeline (too much time was allotted for completion)

We also Underestimated:

• authentication requirements for remote use of Web-Tutorials containing documents from licensed databases

• the corruption of quiz data due to student attempts to “beat the test.”

Lessons Learned

Over the course of semesters, we concluded:

1) students had to receive some grade credit for tutorials to prompt them to take the tutorials seriously

2) the “general” tutorial approach needed to be replaced with one that included a specific and immediate “hands-on” assignment to provide context to reinforce the material;

3) the number of databases student should be introduced to needed to be limited (going beyond the basic ones simply overloaded students)

4) the tutorials had to include an evaluation component because, despite instructor’s efforts, students often did not evaluate sources successfully

5) the tutorials had to be more consistent with the individual instructor’s specific teaching of research and with their assignments so students would see connections between the tutorials and their instructor’s teaching and subsequent assignments

6) the library tutorials and English 15 research instruction had to lay the groundwork for and be consistent with research instruction and students’ research efforts in other courses.

The most important Lessons We Learned were:

•High Tech/Personal Touch

•as you develop high tech learning tools, also incorporate a personal touch

• particularly with freshman students, include tutorial tasks that require human contact through individual or group consultations.

• Prime your Audience

for more meaningful participation:

• sell the importance of the tutorial in advance

• anticipate attempts to subvert, trick the system; remind students that there are people behind the computer programming to discourage the urge to outwit the machine.

• Keep It Simple

• limit successive screens of text (3-4)

• keep online instructions brief and direct

• limit choices of practice exercises; too many choices lead to confusion.

Over the course of a couple of semesters, we concluded:

1) Students had to receive some grade credit for the tutorials to prompt

Our future plans include:

• Revise and Update the Tutorial

• update the content; make the text shorter and more concise

• improve navigational prompts

• revise quiz format to deter misuse

• better coordinate tutorials with sequence of writing assignments

• make evaluation section a separate tutorial; enhance it with an interactive section on rules of argumentation and logical fallacies

• Conduct Formal Program Assessment

• student surveys

• instructor surveys

• quiz data analysis

• longitudinal study

Goals of Freshman Composition at PennState (English 15)

PennState requires all freshman students to complete a one-semester writing course, Rhetoric and Composition (English15). The goals of English 15 include teaching students library research skills and argumentation. Library research skills include information retrieval and evaluation of sources; argumentation skills include students understanding and evaluating arguments found in sources and using them to help develop written arguments of their own. Composition instructors may accomplish these goals as they see fit. At the Penn State, Delaware County Campus, the librarians conduct individual library tutorial sessions on information retrieval for the classes of instructors interested in this service. These in-library tutorials consist of an overview of the library resources including location of materials, instruction to electronic databases, and information retrieval. Depending on individual instructors, these tutorials might or might not include basic research techniques for different databases. For the most part, these “introductory” sessions consist of basic retrieval techniques to help students find sources for any assignment given by any English 15 instructor. Discussion of students evaluating sources and developing their own arguments is left to individual instructors to address in regular class sessions. Both librarians and English 15 instructors have found this strategy ineffective.