This is a draft version for “Chu, S. & Law, N. (2008). The Development of Information Search Expertise of Research Students. Journal of Librarianship and Information Science, 40(3): 165-177.”

Authors:

Samuel Kai-Wah Chu

Faculty of Education, the University of Hong Kong, Pokfulam, Hong Kong

Nancy Law

Faculty of Education, the University of Hong Kong, Pokfulam, Hong Kong

Title: The Development of Information Search Expertise of Research Students

Abstract:

This study identifies the development of information search expertise of twelve beginning research students (six in education and six in engineering) who were provided with a set of systematic search training sessions over a period of one year. The study adopts a longitudinal approach in investigating whether there were different stages in the development of search expertise for the research students, andexamines the contributing factors in their progress from one stage of expertise to the next. The inclusion of both education and engineeringstudents allows the researcher to examine whether search expertise development is different for students in different domains. This study provides a model, which relates students’ growth and development in research expertise to information search expertise. The resultsshow that every student went through three stages of information needs during graduate studies, and the changes in information needs reflects the students’ growth in knowledge of their subject. It was also found that the research students had problems in finding relevant information sources and that they needed to at least achieve a competent level of expertise in order to effectively locate information.

Introduction

Few researchers have taken the longitudinal approach in studying search behavior over time. Yuan (1997) wrote, “Systematic examinations of end-user searching behavior over time are noticeably absent from the literature” (p.219), and six years later, Vakkari, Pennanen, and Serola (2003) still found this to be true. This study adopts a developmental, longitudinal approach to investigate whether there were different stages in the development of search expertise for the research students, and to examine the factors which contributed to the new student’s progress from one stage of expertise to the next. One year was spent on noting the correlation between changes in students’ search behavior and the development of their information search expertise.The research also looks for possible connections between the students’ progress through different stages of their studies and their corresponding stages of searching skills development.

Research students, especially Doctor of Philosophy (PhD)students, are the focus of this study. Such students must select an original research topic and be comprehensive and current in their literature reviews on areas of knowledge related to their project, andsince they need to conduct rigorous database searches to support their studies, they are ideal subjects for information search studies such as the present one. However, relatively few studies have examined the information search behavior of research students (Barry, 1997; Libutti & Kopala, 1995; Morner, 1993; Zaporozhetz, 1987).

This study thus focuses on how students’ develop in (1) their knowledge of databases/sources, and (2) their knowledge of information search skills. Its objective is to identify the critical changes indicating qualitative progress in the students’ development of information search expertise as reflected through a diverse set of collected data.

Literature review

Research indicates that students’ information search skills are initially inadequate (Bates, 1977; Chang & Perng, 2001; Drabenstott & Weller, 1996; Hildreth, 1997; Tillotson, 1995), even at the PhD level (Barry, 1997; Hess, 1999; Morner, 1993; Zaporozhetz, 1987). How, then, can students improve their skills so that they can find the necessary information?

To help students advance from being novices to becoming experts in information searching, it is first necessary to identify the differences between the two. Since a good understanding of how people become experts may help novices shorten their learning curve (Larkin, McDermott, Simon, & Simon, 1980), many researchers during the past four decades have carried out studies on expertise, following two major approaches: (1) novice-expert comparison, and (2) developmental studies. Most of the research has taken the novice-expert approach (Ahmed, McKnight, & Oppenheim, 2004; Chase & Simon, 1973; Chi, Glaser, & Farr, 1988; De Groot, 1965, 1966; Holscher & Strube, 2000; Hsieh-Yee, 1993; Kuhlthau, 1999; Marchionini, Dwiggins, Katz, & Lin, 1993); although there has been some consideration for the developmental approach (Campbell, Brown, & DiBello, 1992; Campbell & Dibello, 1996; Dreyfus & Dreyfus, 1980, 1986, 2005; King & Clark, 2002; Walton, Adams, Goldsmith, & Feldman, 1988; Wiedenbeck & Scholtz, 1996).

Novice-expert research has contributed to componential accounts of expertise which have helped to identify the general characteristics of experts, and it has been noted that experts have a substantial body of organized knowledge in their own domain of expertise (Chase & Simon, 1973; Marchionini, et al., 1993). This knowledge alone, however, is not sufficient in helping students to advance from being novice to becoming expert, and developmental research methods, or more specifically, longitudinal methods tracing learning over time are also necessary (Campbell, Brown, & DiBello, 1992).

Campbell et al. (1992) used structured interviews to elicit information about the learning histories of expert programmers and their evaluations of other programmers’expertise, and used tape diaries to follow the course of learning on the Smalltalk language of individual programmers for up to two months. Based on these data, the three researchers proposed that Smalltalk learning involves students’ going through a sequence of seven stages:

Interacting with the visual interface

-Syntax rules and order of precedence

-Locating classes and methods

-Class versus instance distinction

-Model-Pane-Dispatcher

-Object-oriented design

-Grandmaster level (p. 286)

-

Campbell et al. (1992) concluded that a longitudinal, developmental study, such as the present one, has practical applications for skill development. Their issue-based levels, developmental model offers clear targets for what people should be learning when acquiring knowledge of Smalltalk.

Some studies on information search have also applied the developmental approach (Halttunen, 2003; Halttunen & Jarvelin, 2005; Vakkari, Pennanen, & Serola, 2003; Yuan, 1997). Yuan (1997) examined the search behavior of 25 law studentsfor over one year and found that they improved their search speeds and made use of more search commands and features during this period. Vakkari et al. (2003) investigated how 22 undergraduate students in psychology changed in their use of search terms and tactics while writing a research proposal over a period of three months. The students used more search terms, but remained constant in their use of search tactics and operators. Halttunen (2003) and Halttunen and Jarvelin (2005) studied the development of information searching skills of 57 first year undergraduate students’ over a period of three months, and found that the students made steady progress in their use of operators and in the construction of facets.

Problem Statement

None of the developmental studies reviewed above focused on how search expertise can be developed over time, nor did they involve students at the PhD level. The present study, therefore, adopts a developmental, longitudinal approach to investigate how students (mainly PhD students) develop search expertise as they progress through the stages of their studies. It focuses on how students develop (1) their knowledge of databases/sources, and (2) their information search skills.

The objective of this research was to identify the critical changes which indicated qualitative progress in the students’ development of information search expertise as reflected through a diverse set of collected data. More specifically, the research analyzed the students’ growth and development in research and information search expertise and investigated the relationship between the two. It looked at the correlation between students’ progress in subject knowledge and changes in their information needs, and at the same time, it tried to delineate different levels of information search expertise. Similar to Campbell et al.’s (1992) proposition, the ability to delineate different levels of information search expertise will offer clear targets for what people should be learning when acquiring information search skills.

Procedures

Existing literature indicates that the general approach for understanding expertise in a certain field is to use task analysis to analyze the various elements and levels of a subject domain, usually through the help of an expert in the field (Campbell et al., 1992; Feldman, 1994; Wiedenbeck & Scholtz, 1996). For example, Walton et al. (1988) in their research on juggling expertise consulted an expert in the domain, a professional street juggler, to help separate different stages of juggling into eight levels: from raw beginner to expert. The researchers also produced a set of twelve elements that students in their research used to self-rate their juggling skills – timing, accuracy, height, and others. Ratings were done on a 9-point Likert scale with five labels at alternating points: from very poorly (0) to very well (8). The expert’s understanding of the knowledge structure of information search may or may not match the developmental changes as experienced by the students. Therefore, instead of using a pre-determined number of elements and levels of information search expertise for graduate research, this study took a naturalistic approach in examining the students’ development in search expertise. It posited the major elements of expertise and constructed the different levels of searching skill from the data collected.

In measuring students’ knowledge of sources/databases, researchers have mainly used surveys (Cole, 1992; Cool & Xie, 2000; Majid & Tan, 2002),although search logs (Cool & Xie, 2000) and interviews (Vakkari, 2000) have also been adopted. In this study, surveys were conducted to gather standardized data elements from students regarding their level of familiarity with (and the importance they placed on) various sources/databases. This information formed a general yet useful base for analyzing students’ knowledge of sources/databases over time. Direct observation and the “think aloud” protocol were also used to capture students’ procedural knowledge at different points in time to discover the progress in their ability to use correct sources and search commands. These methods were supplemented by interviews with students in order to understand the reasons behind their actions, and to clarify the choices they made.

Researchers measuring students’ knowledge of information searching have relied mainly on search logs (Bates, Wilde, & Siegfried, 1993; Marchionini et al., 1993; Vakkari, Pennanen, & Serola, 2003), but they have also used surveys (Yuan, 1997), think aloud protocols (Marchionini et al., 1993), and interviews (Vakkari, Pennanen, & Serola, 2003). This study used surveys, direct observation, think aloud protocols, and interviews to measure the students’ search abilities over time.In order to investigate the process whereby the research students moved from being novice to becoming expert in information search, the researcher observed the growth and development of search capabilities of the students in five different research meetings over a period of one year. The participants were surveyed three times during the period, and additional data was gathered through taping theparticipants’ verbalization of their thoughts, through direct observations, and through interviews. This approach can be regarded as a “basic interpretive qualitative study” (Merriam, 1998, 2002), which seeks to:

discover and understand a phenomenon, a process, or the perspectives and world views of the people involved… The analysis usually results in the identification of recurring patterns (in the form of categories, factors, variables, themes) that cut through the data or in the delineation of a process. (Merriam, 1998, p.11)

Sample selection

The study adopted the purposeful sampling method, “based on the assumption that the investigator wants to discover, understand, and gain insight and therefore must select a sample from which the most can be learned” (Merriam, 1998, p.61). Twelve research students from The University of Hong Kong (HKU) participated in this study. Ten of whom were PhD students (five each from the disciplines of education and engineering) and two were Master of Philosophy (MPhil) students (one from education, one from engineering). Examining students in two disparate subject areas enabledthe researcher to investigate the effects of disciplines on the development of searching expertise. Eleven of the twelve students were in the first year of their graduate program when they first joined the study, and one was in her third year. Beginning research students were preferred because these students at HKU would typically need to search many information sources for their research. The third year student within the sample was also considered a suitable research subject since she indicated that she still needed to find more information for her thesis. To minimize the gender effect, there were three male and three female students from each of the two disciplines.

Duration of study and arrangement of the research meetings

One year was spent on noting the correlation between changes in students’ search behavior and the development of their information search expertise. In order to obtain data in settings familiar to the students, meetings took place where students normally performed their information searching, either in their offices or in a computer lab where the HKU library’s databases could be accessed.

The researcher met with the students six times (approximately once every two months) individually during the year. The first five one-and-a-half hour meetings were similarly structured for all the students and included some brief training on information search. Each of the first three meetings focused on the kinds of materials to be searched, and students selected one of the following source types depending on their research needs and interests at the time: theses, books, journals and conference proceedings, orfreely available Web resources. For the fourth and fifth meetings, students used whatever sources they liked. The sixth meeting was a brief follow-up interview, used for clarifying various aspects of the collected data.

Training session

Tailor-made and systematic training was provided five times to the students during five meetings over a period of about one year. Other less formal support was given through emailor telephone conversations with the students between the meetings. This arrangement was quite different from other studies in which students received one or more training sessions within a single day (Yuan, 1997; Bates, Wilde & Siegfried, 1993).

In each of these meetings,the student would search databases according to their own information needs for fifteen minutes twice on their own. Each of these search sessions would be followed by a fifteen- to twenty-minutetraining session by the researcher. This model of training is closely modeled on Vygotsky’s (1978) theory of scaffolding in the “zone of proximal development”.

The “zone of proximal development” is the difference between what a student can do with assistance (from the teacher or an expert) and what the student can accomplish alone without help. Scaffolding refers to the assistance offered to the student that has enabled him/her to successfully complete a task that he/she could not have accomplished otherwise (James, 2003). In this present study, the expert (the researcher who is a reference librarian and proficient at information search) provided scaffolding support to the students (novice searchers) throughout the research period. The interaction between the researcher and the student during each meeting should help the student to expand his or her current zone of proximal information search expertise.

Feedback from students indicated that this type of training was preferred as it facilitated their gradual improvement in search skills, as education student CD commented:

I have made much improvement in my information search skills…I didn’t learn it all in any single meeting. After meeting with you five times, my search skills built up gradually. If it was only once or twice, I think it would not have been too useful. Now after meeting with you so many times, though I am still not very familiar with some of the search skills…my level of awareness in information search is higher now.

Data collection and analysis

The growth and development of each of the twelve students was traced by gathering data from them through surveys, direct observations, verbalizations of their own thinking processes, and interviews. To collect information about the students’ perceptions of the importance of, and their familiarity with, various sources/databases and to identify their search skills, they were asked to fill out a survey at the beginning of the first meeting and at the end of the third and fifth meetings[1]. Conducting the survey three times over the one year period enabled us to see the changes in the students’ perceptionsof various sources/databases and search skills. At the same time, in each of the first five meetings, data about how the students conducted their searches was collected through direct observations, wherebythe researcher and an assistant observed the interaction of the students with the databases. The students were also asked to think aloud and data was collected via tape recording.