Measuring the effectiveness of library information literacy instruction
Elise Y. Wong, Sharon Radcliff, Gina Kessler Lee, and Suellen Cox
Librarians at Saint Mary’s College of California
September 2014
Abstract:
Our project aimed to strengthen the connection between English Composition courses and library instruction sessions to achieve information literacy learning goals. To this end, SMC librarians compared two versions of library instruction in ENGL5 “Argument & Research” sections to measure the effectiveness of embeddingthe “information evaluation and research practices” and “critical thinking” learning outcomes into our library instruction.
Executive summary:
In April 2013, Saint Mary's College of California (SMC) was one of the 75 libraries accepted to be part of a nationwide assessment project called "Assessment in Action" sponsored by Association of College and Research Libraries.Our project targeted the restructured ENG 5 and the research was also used as evidence to inform WASC (Accrediting Commission for SchoolsWestern Association of Schools and Colleges) of the College's initiative of assessing student proficiency in Information Evaluation and Research Practices (IERP).
The SMC Campus team consisted of nine members: four librarians, four Composition faculty members, and the Director of Educational Effectiveness. The team composition was chosen due to the subject of our research. The Director of Educational Effectiveness reviewed our rubrics and recommended best practices for designing assessment methodologies for this project.
Our primary inquiry question was:
How effective are our methods of embeddingthe “information evaluation and research practices” and “critical thinking” learning outcomes into our library instruction?
SMC librarians compared two versions of library instruction in ENGL5 “Argument & Research” sections.The traditional (TRAD) library session focused on IERP while the experimental (EXP) focused on both IERP and Critical Thinking.The 5 learning outcomesevaluated were:
IERP1: Develop search strategies and use library catalogs and databases to find relevant
materials for research
IERP2: Practice evaluating sources critically
CT2a: Seek and identify confirming and opposing evidence relevant to own hypothesis
IERP3: Evaluate and synthesize evidence for the purpose of drawing valid conclusions
IERP4: Demonstrate academic honesty and safeguard the intellectual property of others
by properly integrating and citing sources
The project was in full swing in spring 2014. We had 3 faculty volunteers, each with 2 sections of ENG5. Each faculty was assigned to one traditional and one experimental library session.
Project SAILS is a standardized assessment test for information literacy skills. Among the 8 skill sets evaluated in the test, SMC students performed better than institution-type benchmark on "Developing a research strategy" and "Using finding tool features." The students performed about the same as the institution-type benchmark on "Searching," "Retrieving sources," "Evaluating sources, and "Documenting sources." Students performed worsethan the institution-type benchmark on "Selecting finding tools" and "Understanding economic, legal, and social issues." The SAILS report also ranked SMC student performance in the order from best to worst. Within SMC,Students performed best in "Documenting sources" and worst in "Developing a research strategy."
We collected 75 papers from students who attended the library session. Based on our analysis, the most frequently used sources by students were: popular articles from the Web (38.71% EXP over 34.22% TRAD),scholarly articles from the library (20.42% TRAD over 17.3% EXP), books (10.56% EXP over 9.28% TRAD), popular articles and from library (13.53% TRAD over 10.26% EXP), and general & authoritative Websites (15.24% EXP over 12.73% TRAD).
Students from both experimentaland traditional sessions preferred quoting to paraphrasing. However, the traditional session students had a higher percentage of paraphrasing (43.43%) over quoting (56.57%) over experimental session students (35.64% paraphrasing, 56.57% quoting).
The extended research papers were analyzed based on the quality of the work cited page, citations, inclusion of alternative viewpoints, and how well students integrate evidence to support their claims.The papers were analyzed thru a blind review and norming process by the librarians.We rated them from 1 to 4. (1 beginning, 2 developing, 3 accomplished, 4 exemplary)
Students from traditional sessions did better in all learning outcomes. However, none has achieved the “accomplished” level.We examined the results further to see why they were contrary to our hypothesis. By comparing the learning outcomes among the 3 faculty groups, we found that ST, being the traditional session, was an honors class, which skewed our results.If we only compared the F & G groups, we would find that students from experimental sessions perform better than the traditional sessions.
At the completion of our project, it was clear to us that our library sessions should be restructured to align more specifically with the teaching of the new ENG5.
Contribution:
Our project builds on the collaboration between the library and the English Composition program. The project contributes to the campus-wide assessment initiatives to measure the institutional learning outcomes for the undergraduate programs. The assessment projects conducted by librarians over the years haveset an example for the rest of the campus to followthe culture of assessment in higher education. Information literacy is a key contributing factor to student success. Library information literacy instruction is the most logical channel to educate students to strengthen their skills in achieving “Critical Thinking” and “Information Evaluation and Research Practices” learning goals at SMC. The project explores new ways in library instruction to bridge the gap between knowledge and practice, and measures how well students transfer what they learned from course to course.
Conclusion, implications, and recommendations:
Based on the qualitative and quantitative assessments from the data collected, librarians will use the results to guide changes to library classroom instruction that will improve student achievement of the learning outcome. The project hopes to demonstrate that the two learning goals “Critical Thinking” and “Information Evaluation and Research Practices” needs to go hand in hand in contributing to student proficiency in information literacy. The library hopes that the successful collaboration with the Composition program will serve as incentive to promote future assessment initiatives to administrators and faculty of the schools, departments, programs, and courses on campus.