Kknwown Item author Searchers

Information Services
Observational survey of Voyager OPAC users
A study into searching behaviour at Cardiff University

1.Introduction

1.Aims

The overall aim of this pilot study has been to devise a methodology for gaining information about customer success when using the Voyager online catalogue. The project aimed to identify the main ways in which customers use the OPAC, and then to assess the effectiveness of the techniques they employed. The analysis will inform the approach that Information Services take to facilitating customer use of Voyager. It is also hoped that the method devised can be applied to similar studies of other types of electronic resource or used as a method for a campus wide assessment of OPAC use.

1.Contents

1 Introduction1

2 Method2

3 Results4

4 General comments about Voyager12

5. Conclusion13

Acknowledgements14

Bibliography15

Appendix 116

Appendix 217

Appendix 319

Appendix 420

Appendix 534

Appendix 635

2Method

2.1The observational survey

Customers were observed while they used the OPAC for their own purposes. Each potential participant was asked if they would mind being observed whilst using the OPAC then answering some questions after they had finished. Participants were then asked for what purpose they intended to use the OPAC. This was asked to ensure that the intention of the potential participant was to browse or search and not to check their accounts. It also set a context against which to observe the searches and therefore aided the process of assessing their effectiveness. During their search the participants were asked to behave as though the observer was not present but were not discouraged from making comments.

This survey initially involved observing users of the OPAC without any preconceived structure to the survey format. Field notes were taken documenting the exact searching or browsing process, search terms used and number of hits. Any problems experienced by the participant or any comments made were also noted down at the time. After 15 participant observations had been conducted a structured format was given to the observation sheet. This was partly for ease of processing the data as the criteria could be used as a means of collating the results and partly so that the structure could be easily adapted by other library staff when surveying in the future.

The new, structured, survey retained the question from the original survey – What are you looking for? – which would still be asked of the participant before the searching began but added a Time Began and Time Ended prompt as it was felt that the time taken to search was an important factor in the effective use of the OPAC. See appendix 1 for a copy.

2.2Questionnaire

The questionnaire was designed both to complement the observational survey and to provide information about the respondent which would lead to a more rounded set of results. The questionnaire was developed and tested alongside the development of the observational survey structure as questions were asked of the first 15 respondents after they had finished their searches. For a copy of the finalised questionnaire see Appendix 2. Some of the data which has been collected but not included in this report can be found in Appendices 5 and 6.

2.3Making sense of the data

After conducting the survey the results of the observation were measured against a set of criteria in order to assess the customers' use of the Voyager OPAC. The criteria were established after the first 15 surveys had been completed and were therefore based on events that had occurred during these surveys. For a list of the criteria used see Appendix 3.

Deciding on the effectiveness of a search also involved asking other library staff members the situations in which they would choose a particular type of search. This was to ensure that the opinion of the observer of effective searching matched that of other library staff. It was the general consensus that:

•library staff tended to choose keyword for most searches. This is selected when all the details or only sketchy details about an item are known. This search type was also used for browsing a subject area.

•Out of the index searches the journal title search was used the most.

•Author and title searches were sometimes used for more specific searching.

•The advanced search was less used. It was felt that in most cases a search could be satisfied without the need to use the advanced search option.

•The classmark search was rarely used and was only chosen if that was the only information known about an item.

Once this had been established it was possible to make a brief ‘on the spot’ assessment of the survey data in readiness for understanding the results. The criteria were used to examine the observational data. In some cases this meant performing the searches again using the same keywords used by the customer for purposes of assessment in order to see whether another search might have been more effective or successful.

To ease the process of analysing the observational data a grid was drawn up for each of the groups of searchers and browsers that had been identified from the observations. The data was then input in the grids under headings. This system helped to identify patterns in the data and simplified the analysis process. See Appendix 4.

The quantifiable data from the questionnaire were coded and analysed using SPSS. Frequency tables and cross tabulations were used to make sense of the questionnaire data.

3Results

This section presents the main results of the observational surveys and questionnaires. It begins with a demographic overview of the survey sample. This is followed by more general observations concerning the respondents' experience and confidence with using Voyager. A more detailed overview of the participants in the sample and their use of Voyager is then presented. This includes identifying categories and groups of searchers within the sample, a discussion of the trends within each group and an assessment of the efficiency of their searches.

3.1The sample

Using the final version of the observation sheet and the questionnaire, twenty-eight respondents were observed searching the OPAC and then interviewed. As demonstrated in tables 1 and 2, the survey sample consisted of a wide-ranging set of users. Respondents included members of the University of varying status from a range of ten departments and a variety of courses. Eight visitors were also included in the sample, six of whom described themselves as students from other universities. Amongst members of the university the sample consisted mainly of staff, research and taught postgraduate students. Three undergraduates attending the university were interviewed therefore all members of the university apart from non-academic staff were represented. The small number of undergraduates in the sample can be attributed to the time of year in which the study was performed. The sample consisted mainly of members of the SOCSI, HISAR, RELIG and ENCAP departments which was expected because the study was conducted in the Arts and Social Studies Resource Centre which holds items for these departments as well as, PSYCH, EUROS and WELSH.

Table 1 – University department and position in university

University Dept. / Academic Staff / Postgraduate Research / Postgraduate
Taught / Under graduate / Visitor (student) / Visitor (other) / Total
HISAR / 1 / 3 / 4
ENCAP / 2 / 1 / 1 / 1 / 5
SOCSI / 2 / 1 / 3
JOMEC / 1 / 1
ENGIN / 1 / 1
CPLAN / 1 / 1
WELSH / 1 / 1 / 2
RELIG / 1 / 1
CLAWS / 1 / 1
BIOSI / 1 / 1
Visitor / 6 / 2 / 8
Total / 5 / 6 / 6 / 3 / 6 / 2 / 28

Table 2 – Courses pursued by sample members

Course / Numbers
BA English / 1
BA Mechanical Engineering / 1
BE Religious Studies / 1
MA History / 1
MA Medieval British Studies / 1
MA Medieval History / 1
MA Philosophy / 1
MSc Criminology / 1
MSc Population and development / 1
MSc Social Work / 1
PhD / 5
Not applicable – staff / 5
Not applicable - visitor / 8

3.2General observations

The questionnaire asked two questions of the sample concerning the frequency of their use of Voyager. Respondents were asked a closed question regarding how long they had been using Voyager and an open question, which was later coded for analysis, which asked the respondents to approximate how often they used Voyager. 22 of the respondents replied that they had been using Voyager for one to two years. Two said they had been using it for six months to a year. Three said they had been using Voyager for less than six months and finally one replied that it was the first time they had used Voyager. Of the four respondents who claimed they had been using Voyager for less than six months or it was their first time three were visitors to the university and one was a postgraduate research student. The majority of the respondents can be said to have been regular users of the OPAC. Six of the respondents stated that they used the OPAC daily, ten claimed they used it at least two or three times a week. Eight used it weekly, two used it once a fortnight and one used it less than once a fortnight. The final respondent was not applicable to this question because they were using Voyager for the first time. All of those using Voyager on less than a weekly basis were visitors to the library. Every member of the university in the sample said they used Voyager at least once a week.

As shown in table 3, the study revealed that 23 out of the 28 respondents claimed that they were self taught in using Voyager or had learned to use it through ”trial and error”. Only four respondents answered that they had received some kind of formal training through a library induction or a leaflet about Voyager. The final respondent claimed to have been taught at another university where Voyager is also used. Seven of those who claimed themselves to be self-taught were visitors but this question has shown the limited degree to which Voyager training has disseminated throughout the university population. Amongst post-graduates and staff in the sample the levels of those who received some sort of training in Voyager are very low. Out of the three undergraduates, two did claim to have received al library induction or a leaflet but one also claimed to be self-taught.

Table 3 – how respondents learne to use Voyager and their position in the university

How learned to use Voyager / Academic Staff / Postgraduate research / Postgraduate taught / Under graduate / Visitor (student) / Visitor (other) / Total
Self-taught / 5 / 5 / 5 / 1 / 5 / 2 / 23
Library induction / 1 / 1 / 1 / 3
Learnt at another university / 1 / 1
Leaflet / 1 / 1
Total / 5 / 6 / 6 / 3 / 6 / 2 / 28

As shown in Table 4, all of those who had received formal training in Voyager claimed to be confident that they could find information about an item on Voyager if the library had it in stock. Only 16 out of the 23 who were self-taught claimed to be confident in answer to this question. Four out of twenty three felt they could probably find information about an item on Voyager and three did not feel confident at all. This indicates that lack of training, within this sample, affected user confidence in finding information on Voyager.

Table 4 – How respondents learned to use Voyager / confidence in ability to find information

How learned to use Voyager / Confident using Voyager to find information? / Total
Yes / Probably / No
Self-taught / 16 / 4 / 3 / 23
Library induction / 3 / 3
Learnt at another university / 1 / 1
Leaflet / 1 / 1
Total / 21 / 4 / 3 / 28

High frequency of use of the OPAC, in the sample, did not necessarily relate to user confidence in finding information using Voyager. As table 5 shows, three of those users who claimed to use the OPAC daily were confident they could find information about an item if the library had it stock. Two felt only that they probably could find information and one said they were not confident at all. Eight out of ten of those respondents that used the OPAC two to three times a week were confident. All those who used it once a fortnight or less than once a fortnight were also confident. The respondent using Voyager for the first time felt that they could probably find information about an item if it was in stock.

Levels of confidence and levels of search success are not always equal. For example, respondent number eight performed an author search and went straight to the record they wanted without the need to browse. They used the search type correctly and with relevant search terms. When asked to assess their confidence in finding information on Voyager respondents expressed a lack of confidence in using it to find items in stock. It may have been that the respondent was lucky during the search that was observed but they appeared sure of what they were doing and therefore it seems more likely that they had underestimated their capabilities.

Table 5 – How often respondents use Voyager and their confidence

How often do you use Voyager / Confident using Voyager / Total
Yes / Probably / No
Daily / 3 / 2 / 1 / 6
2 or 3 times a week / 8 / 1 / 1 / 10
Weekly / 7 / 1 / 8
Once a fortnight / 2 / 2
Less than once a fortnight / 1 / 1
Not applicable / 1 / 1
Total / 21 / 4 / 3 / 28

3.3Within sample groups and their characteristics

The searches performed by participants in the sample were divided into two categories depending upon their goals. The two categories were browse searches and known item searchers. Known item searchers were classified as those intended to find a specific item. Details about an item such as the author, title or journal title were known to the respondent. This category included participants who were searching from memory or for items on a list of references. Browse searchers were categorised as those searching for items on particular topic or by a certain author. No specific details, such as title, were known. Where the respondent knew the author the aim was to find additional items by or about that particular author.

-17 respondents performed know item searches

-7 can be classed as browsers

-4 used the OPAC for both known item and searches and for browsing. These were categorised as combination serchers.

An example of a combination searcher was one participant who performed author searches for a know list of items and then proceeded to browse for other items by that particular author. This searching and then browsing behaviour meant the respondent could not be classified as simplay a browser ro a searcher. Another ran an author search for a specific title by Virgnina Woolf and then proceeded to browse for books on translation suing the keyword option. Where respondents had performed a combination of browse searches and known item searches during the same session their individual searches were categorised separately into the two groups.

Once the sample had been placed into the two categories their individual searches were grouped according to the search type they had chosen. Table 5 shows the groups into which the respondents have been placed and the number who chose to use a particular type of search. The most popular type of seach chosen by known item searchers was the author search. The second most popular was the title search. Keyword was third most popular, followed by journal title. It had been expected that Keyword would be the most popular option.

All of those respondents who performed purely browse searches chose to use keyword along with three of the combination searchers who were using this search type to browse. The combination searcher who used the author search type to search for known items and then to browse has been included in the author group fo both categories. Journal title and title were not chosen by the browsers as a search type.

None of the respondents in either of the categories ever used a classmark or advanced search. Two of the respondents searching for known items chose the advanced search but abandonded the search before running it. A reason why the advanced search was not chosen by any participants in the sample might have been offered by one of the users who abandoned the search type. They commented that they did not understand the screen. The other respondent who attempted an Advanced search said the screen gave too many options. Thus, difficulty with using the advanced search might be a reason why it is not chosen. Another might be that the users feel that they are successful enough with the search types they are already using.

Table 6 – Type of search against aim of respondents

Type of search / Aim of respondent
Known item search / Browse
Author / 12 / 1
Title / 8 / 0
Keyword / 5 / 10
Journal Title / 3 / 0
Classmark / 0 / 0
Advanced / 0 / 0

In the next section the characteristics of each group are discussed separately. Further details about each of the groups can be found in Appendix 4.

3.4Known item searchers

Four search options were used by known item searchers. These were (in decreasing order of popularity):