5
Collection Evaluation
Collection evaluation is a continuing, formal process for systematically analyzing and describing the condition of a library’s collection and to indicate areas needing improvement. Evaluations are conducted to provide several kinds of important information to libraries. They help clarify the library’s goals in the context of its mission and budget, supply data used to set funding priorities, and build a base for long-range planning and administration. But why is collection assessment conducted? (Collection evaluation and collection assessment will be used interchangeably in this paper.) This paper will answer this question by presenting the benefits of evaluation, the various evaluation techniques with their strengths and shortcomings.
Benefits of Conducting Collection Evaluation
Collection evaluation provides library administrators with a management tool for adapting the collection, an internal analysis tool for planning, a tool to respond systematically to budget changes, and a communication tool and data for resource sharing with other libraries. Library staff can also benefit by having a better understanding of the collection, a basis for more selection collection development, improved communication with similar libraries, and enhanced professional skills in collection development. For libraries involved in resource sharing, collection evaluation is essential in determining how each library fits into the system and what should be expected for each library’s further growth in the context of the cooperative relationship.
It goes without saying that before any meaningful evaluation of the library’s collection can take place, the goals and purposes of the collection should be stated in a collection development policy. Since the development of a collection can be very subjective, a written policy about what is to be selected or rejected becomes essential to evaluation.
The Importance of Planning
When planning a collection evaluation, it is important to carefully define the goals for the program, choose the most appropriate method(s) to be used, and establish what information is needed. An evaluation can be fully comprehensive or it can focus on specific areas, depending on the library’s needs (and the resources available to carry it out – evaluations can be expensive!). It can be very tempting to gather all sorts of information because it seems interesting, but it may be useless if it does not fit into the parameters of the evaluation. Be sure that everyone participating in the evaluation understands what is expected and when tasks should be completed.
Evaluation Techniques
There are a number of standard techniques for obtaining evaluation, but they can all be considered as either collection-centered or client-centered. Collection-centered techniques examine the content and characteristics of the collection to determine the size, scope, and/or depth of a collection, often in comparison to an external standard. Client-centered techniques measure how the collection is used by library users.
A. Collection-centered Techniques
Collection-centered techniques are employed to examine an existing collection and to compare its size, scope, depth, and significance with external criteria. The methods include:
1. list-checking
2. shelf scanning
3. compiling statistics
4. application of standards
5. formula method
6. conspectus evaluation
List-Checking
This method compares the collection to authoritative lists of what is available and appropriate for a particular type of collection. There are hundreds of possible lists to use and It is important to carefully interpret Presumably, a high percentage of titles found indicates an excellent collection, although at this time there is no agreed upon interpretations or weightings of percentages held. The type of list selected depends on the type of collection being evaluated and the purposes of the evaluation. For example, a “basic” collection serving an undergraduate program may be checked against “standard” lists developed for this type of collection. It is crucial that the list match as closely as possible the libraries’ objectives. Types of lists include:
F standard catalogs and basic general lists such as Books for College Libraries and Choice’s Opening Day Collection
F printed catalogs of the holdings of important and specialized libraries
F specialized bibliographies and core lists
F current lists such as acquisition lists of major libraries, books of selected publishers, or annual subject compilation
F lists of types of materials, such as reference works, periodicals, etc
F authorized lists prepared by governmental authorities or professional associations
F lists, usually in current journals or review sources of evaluated publications at the forefront of current research
Procedures
F Decide on lists appropriate to the subject and goals of the library;
F Decide whether to check the lists completely or by sampling;
F Assign staff responsibility and check lists against card catalog or online public access catalog;
F Record the number of titles held that are listed in the bibliography being checked;
F Determine percentage of library holdings in relation to number of titles on each list;
F Analyze results and integrate findings with results of other techniques to determine the collection level.
The advantages of list-checking include:
? a wide variety of published lists is available;
? many lists are backed by the authority and competence of expert librarians or subject specialists;
? provides concrete, objective picture of holdings;
? may be carried out by support staff/procedure of searching lists is easy;
? provides specific items that may be purchased to strengthen collection;
? quantitative results useful for budget justification, accreditation studies, etc.
The disadvantages of this technique include:
q many lists are not revised and become out-of-date;
q lists representing the viewpoint of one individual or group may not represent the subject well;
q lists may not be as representative of the library’s subjects or purposes or the interest of its users;
q may be difficult to evaluate their validity, usefulness, relevance, etc.
q in some subject areas, lists may be hard to locate or compile.
It is important to carefully interpret the qualitative data that result from checking lists, considering who assembled the list and for what purpose. List checking can help the library staff understand the size and scope of possible materials, and it can be helpful in assessing what should be added to the collection. Because there are many possible lists to check and they are quickly outdated, this can a time- and labor-intensive method, especially if you do not have an automated system. Misleading results can occur when the same “best book” lists are used for selection as well as evaluation, when published lists are written for a very different audience than the library’s community, and when lists don’t include works owned by the library that are as good or better for the local community than the materials on the list.
Shelf Scanning
This technique involves examining the collection directly. With this procedure, a person physically examines materials on the shelf; the person then draws conclusions about the collection’s size, scope, depth, and significance; its recency; and its condition. Preservation, conservation, restoration, or replacement of materials may be taken into consideration in this process. Examination of date due slips provides samples of actual use.
Procedures
1. Select classification area. Work with someone if possible.
2. Gather necessary tools.
3. Determine scope: examine every item or a sample.
4. Determine location of other materials in the collection that need be considered: electronic documents, periodicals, vertical files, commercial databases, reference tools, etc.
5. Look for such things as:
F physical condition of the materials (weeding needed?)
F types of materials
F language
F serial runs: complete? Broken? Bound? With our without deficiencies?
F scope, extent of the collection
F special problems
F multiple copies
6. Record findings.
The advantages of using this technique include:
? providing immediate, relevant results
? can be done quickly
? provides overall view of the size, scope and quality of the collection
? builds on knowledge and expertise of the evaluator
? evaluator sees what the users see
? can be used to accomplish goals other than evaluation
The disadvantages of this technique are:
q subjectiveness and impressionistic, does not produce quantitative or comparable results
q persons knowledgeable in a subject area and its literature are required and such individuals may be difficult to locate, unable to invest time, or charge fees greater than the library can afford
q results may be biased if conducted by the librarian who developed the collection or currently selects for it
q some materials may not be on the shelf
This technique is well-suited to smaller libraries and areas of a collection that don’t fit into the classification scheme. It has the advantage of providing relevant information quickly, but it can be highly subjective, especially if the person doing the evaluation also does the selection. Working in a team should be encouraged. Direct examination should not be used as the sole evaluation technique. Shelf-scanning should be conducted after the shelflist data have been collected; the two techniques complement each other to provide a reliable characterization of the collection. Be sure to examine the entire collection, including periodicals, audiovisuals, and reference works. You should make notes on items that should be weeded, but do not weed as you assess. Depending on the size and type of you r collection, you may not need to examine every item; sometimes a sample works just as well.
Compiling Statistics
Traditionally, the main comparative methods for evaluation of collection strengths were aggregate figures on collection size and material expenditures. This information is typically reported in annual reports. The following statistics are typically collected and reported by libraries:
F Size: this may be the measure of titles, volumes, or portions of the shelflist in specific call number ranges
F Net volumes, titles, or units added: this measures growth rates in total collection size, specific portions of the collection, or in various formats, for example, books, microforms, or periodicals
F Expenditures for library material: this may include money spent for all material or for specific formats or portions of the collection. Figures may be annual and may be expressed in pesos or a proportion of the total or institution budget
The advantages of collecting statistics include:
? some statistics may be easily maintained, for example, shelflist counts performed every 4-5 years
? if proper records have been kept, they are easily available
? if clearly defined, they may be widely understood and comparable
? automated library systems should be able to extract this easily and perform some preliminary analysis
The disadvantages of this technique include:
q statistics may be recorded improperly
q clear definitions of units may be lacking
q statistical records may not be comparable
q significance of statistics may be difficult to interpret
Application of Standards
This technique analyzes government standards or an accrediting group’s standards. Such standards vary a great deal in format and specificity. The advantages of using this technique include:
? for the appropriate type of library, the standard will generally relate closely to the library goals
? standards are generally widely accepted and authoritative
? they may be promulgated and used for evaluation by accrediting agencies, funding agencies, and others;
? they may be very persuasive in generating support for the library.
The disadvantages of applying standards include:
q some are stated generally and are difficult to apply
q they may require a high degree of professional knowledge and judgment
q knowledgeable people may disagree in application or results
q minimum standards may be regarded as maximum standards
Formula Method
Various formulas have been conducted to assess the quality of library collections. Among the formulas that have received wide attention include: Clapp-Jordan, the ACRL formula for college libraries, the ACRL formula for college libraries. Mention can also be made of the formula presented during PAARL’s seminar Procedures depending upon the formula used. Advantages associated with formulas include:
? greater potential for in-depth comparison between libraries
? greater ease in preparation and interpretation
The disadvantages of this technique are:
q in ability to assess qualitative factors that are important in the relationship between the library collection and patron needs
q lack of standard definitions of what to measure, e.g., there is no uniformity in use of the terms titles and volumes.
Evaluation by Outside Expert
A knowledgeable person from outside the library staff can be enlisted to survey a portion of the collection and provide qualitative data. Outside experts include consultants, other librarians, a faculty member, or a library user with specialized knowledge, surveys a portion of the collection. If you choose to use an outside expert, be sure that the expert understands your assessment goals and that you are asking for advice which may or may not be implemented. That person’s contributions should always be recognized. The advantages of this method include:
? brings a fresh perspective to the collection
? facilitates mutually advantageous communication between librarians and administrators, staff and users
The disadvantages of this technique include:
q evaluation may be impaired by bias, narrow or specialized view, or lack of understanding of the library’s collection policy
q subjectivity of the evaluator
q may be difficult to find an outside expert, or expert may not be available when needed
Conspectus Evaluation
The term “conspectus” was developed by the Research Libraries Group in the United States to refer to a standardized means of evaluating library collections in each of the subjects, categories, and divisions. Developed in the 1970’s and 1980’s, it was originally intended for large academic libraries, but it has been adopted and used by other types and sizes of libraries, as well. The conspectus method provides a framework to describe their collection strengths and current collection intensities. The evaluation levels indicate the appropriateness of the collection to support certain subjects at certain levels, so they have immediate relevance for collection development, for example, if you have a level 3 collection but the academics are running strong research programs, you’d better improve that area of the collection. Conspectus uses a Collecting Level Indicator (0 to 5), or summary, of a library’s Current collecting Level (CL); Acquisition Commitment (AC) and Collecting Goal (GL). Libraries can also supply additional information about strong collections, notable items in the collection, number and median age of items held, and the selection policy for a particular part of the collection.
N.B. Refer to previous notes on the collection code levels
The advantages of conspectus evaluation include:
? the detailed subject breakdown of the conspectus allows for more finely delineated collection descriptions
? collections and collecting patterns are described in comparable terms
? conspectus values are easily accessible online or in paper format
? cooperative collecting or preservation policies can be developed using the conspectus as an instrument to map collection strengths