EVALUATING DIGITAL LIBRARIES1
An Evaluation of the Digital Public Library of America and the Smithsonian’s History Explorer as Effective Digital Libraries
Christine Skirka
Rutgers University
Abstract
The concept of a digital library is defined and reasons for evaluation are explored. Several criteria are set forth as a framework for evaluation of two distinct digital libraries: the Digital Public Library of America and the Smithsonian’s History Explorer. The six different criteria are defined and the individual libraries are outlined and evaluated using the criteria. The analysis of the digital libraries is very cursory and there is overall commentary about the two libraries and their respective goals. Improvements are suggested and remarks are made for future research.
Introduction
Digital libraries are relatively new compared to traditional libraries, and they continue to grow and develop in ways that are not easy to predict. The basis of these predictions is hard to describe because digital libraries are severely lacking in a major way: evaluation. Evaluation of digital libraries is something that can be time consuming and expensive, so it often is not a priority for the institution. I can see the reasons behind this, but that does not mean efforts should not be made to do an analysis on a somewhat regular basis, even if that means every five years.
An initial discussion will attempt to define what exactly a digital library is and what its main attributes are. I also plan to evaluate two different digital libraries as an overall resource to see if they are effective. I will do this using several different criteria which look at the overall website, content, access, user needs, and organization, among other things. Ultimately, after a review of the libraries, commentary will be made about if the digital libraries are effective in terms of the goals they have described on their website.
The digital libraries that I have chosen are exemplary examples of resources that can be used by many people for various purposes. I think they are a good representation of possible digital libraries because they represent a public library and one which evolved from a museum. I wanted to look at libraries from different institutional types to see if their goals might be the same. Each digital library has a different goal, but all libraries share the ambition of making information available to the users.
Evaluation of digital libraries is important because it gives the institution the ability to monitor if the library is being used to the best of its ability. Digital libraries can be created and not evaluated and function just fine. Evaluation allows for a more in-depth view of the overall resources and their usage. Looking at the different facets of any digital library and the roles that the various user types play permits the digital library to appeal to the governing body for increased funds to further develop collections, add staff, or upgrade materials.
What is a digital library?
A clear-cut definition for “digital library” is something that is very hard to come by, let alone define based on my own personal observations. I think that a major difficulty in writing a definition comes from the fact that a digital library cannot be fit into one perfect little box. Digital libraries are often extensions of physical libraries or collections. They have developed in the digital age to allow access to people who would otherwise never have access to materials that have long been kept on shelves or in drawers in institutional repositories. Each institution which has created a digital library has made an attempt to showcase its most important and valued materials so that people might find interest in them and share them with others. Because of this, it is difficult to determine where a physical library’s purview ends and the digital library begins.
Several people have made great effort to attempt to define a digital library. In a report on a workshop which investigated the social aspects of digital libraries conducted by Christine Borgman et al. (1995), a rather lengthy, but clear definition is presented. This report defines a digital library using two complementary concepts:
1. Digital libraries are a set of electronic resources and associated technical capabilities for creating, searching and using information. In this sense they are an extension and enhancement of information storage and retrieval systems that manipulate digital data in any medium (text, images, sounds; static or dynamic images) and exist in distributed networks. The content of digital libraries includes data, metadata that describe various aspects of the data (e.g. representation, creator, owner, reproduction rights) and metadata that consist of links or relationships to other data or metadata, whether internal or external to the digital library.
2. Digital libraries are constructed – collected and organized – by a community of users, and their functional capabilities support the information needs and uses of that community. They are a component of communities in which individuals and groups interact with each other, using data, information and knowledge resources and systems. In this sense they are an extension, enhancement and integration of a variety of information institutions as physical places where resources are selected, collected, organized, preserved and accessed in support of a user community. These information institutions include, among others, libraries, museums, archives and schools, but digital libraries also extend and serve other community settings, including classrooms, offices, laboratories, homes and public spaces. (p. 4)
While this definition is too lengthy to be included in most materials that might explore digital libraries, it does highlight many important aspects of digital libraries that should be included in any definition.
Borgman’s definition starts by pointing out the technical capabilities. Most people, myself included, might not think of the technical aspect of a digital library initially though it is important to note that a digital library cannot exist without technology. The definition then links the technology to “digital data of any medium,” (Borgman, 1995). While the definition does go on to identify specific document types, it also is ambiguous enough to allow for media type extensions in the future. Allowing for unlimited extensibility has led to this definition still being relevant 20 years after it was written. Borgman et. al. then go on to identify the importance of data and metadata in a digital library. Discussing the content in these terms is something that I would have considered earlier since that information makes up the entire library. The goal of the entire first point seems to be that of the actual library. They discuss the technological components that make up a digital library and what their needs and extensibility might be.
The second point goes on to look at the needs of the user and the surrounding community. It is important to note that these two components are separate because they do, in fact, complement one another. The library cannot exist without the technology, and will be useless without a community of users. The definition states that digital library collections “support the information needs and uses of that community,” (Borgman, 2995). This is interesting to note because it implies that no two digital libraries should be the same. They should be adapted and modified to meet the needs of the community which it is aimed to serve. A digital public library in California has a completely different user community than that of a small digital library from a historic commission in Ohio. The same user group might under some circumstance use both libraries, but their research goals would likely vary for each library.
This definition also goes on to discuss the division between the digital and physical library. “In this sense they are an extension, enhancement and integration of a variety of information institutions as physical places where resources are selected, collected, organized, preserved and accessed in support of a user community,” (Borgman, 1995). The use of the word “enhancement” shows that digital libraries are important additions to the library as we have known it for centuries. The library has always been a physical place, but a digital library breaks out of the traditional confines to be something complementary. This also lends validity to an institution which some might be skeptical of. Being that digital libraries are a rather new concept, people might not trust that they are reliable or go-to sources of information. Many of us have been brought up to go to the physical library for any host of reasons, but now we are able to conduct much of the same research without ever leaving the living room.
Another shorted definition of digital libraries has been set forth by Donald J. Waters. In his article “What are digital libraries?” (1998), he explores a working definition which was created by the Digital Library Federation (DLF). The definition states that:
Digital libraries are organizations that provide the resources, including the specialized staff, to select, structure, offer intellectual access to, interpret, distribute, preserve the integrity of, and ensure the persistence over time of collections of digital works so that they are readily and economically available for use by a defined community or set of communities. (Waters, 1998)
Unlike Borgman’s definition, this one brings up the topic of the specialized staff that is necessary to maintain and develop the collection, as well as provide assistance and guidance to those who want to use it. This is an interesting, yet very important difference in the two definitions. Borgman’s definition does address important issues, but fails in that it lacks any acknowledgement that specialized staff is necessary for digital library creation, maintenance, and customer support.
Another interesting difference between Borgman’s original definition and the one from the DLF is that they analyze the need for digital libraries to maintain “persistence over time…[and] …that they are readily and economically available,” (Waters, 1998). This notion that digital libraries be maintained and updated over time is an important feature of this definition. A digital library can be created today, but if efforts are not made to sustain it with most recent technological advances, then it is useless. The issue of economics also lends some accountability to those creating the digital library to produce something within certain fiscal constraints. Access to information is always a good thing, but the requests for new materials versus the cost need to be assessed and monitored. If the digital library is not being used to its utmost potential, then there needs to be discussion about if developing a new collection would be helpful or harmful overall. Ideally, an added collection would increase usage, but if usage is down, then it might not be economically sound to use money or other resources in this way. The funds might be better allotted to increased marketing or user outreach, for instance.
In an article which explores what digital libraries are, Christine Borgman (1999) states, “The focus of the DLF definition is on the digital library as an organization who services include the provision of information resources in digital forms,” (p. 236). While somewhat implied, this statement reflects that the responsibility of the digital library lies in maintaining the digital files in the collections. This does not necessarily include the maintenance of the original physical items. Based on this, I would assume that the responsibility of maintaining the original items lies with the owning institution, whether it be the physical library, or the digital library for born digital resources.
Using a combination of the two suggested definitions of “digital library”, I think a happy medium can be found. I think any digital library definition needs to discuss staff, content, technology, users, and future growth. A better definition might be something like, “a digital library is a set of digital resources which is intended to appeal to the designated community through a variety of materials. This library is an enhancement of a physical library or may be a standalone institution with plans to expand to further support the user community if necessary. The technology that supports the library primarily supports information storage and retrieval systems. Technologies should sustain any medium and contain metadata which creates links to other data. Specialized staff is needed to control the digital library and support future expansion and technological upgrades.” While not perfect, this definition covers all of the important ideals that a digital library should strive to achieve.
Scope of evaluation
An evaluation of any digital library cannot be done without first identifying several different criteria for analysis. This gives the researcher a definitive method by which to look at digital libraries so that there is no variation in the assessment of the library being evaluated. I believe that if enough digital libraries start using similar methods of evaluation criteria that a standard could eventually be developed. In doing this, there is potential for more digital libraries to allocate the resources needed to complete this process. I have found with various library projects, the worst part is getting started, especially if there are no previously established methods to guide my work. Implementation of a standard would reduce the overall difficulty of completing the evaluation because the guesswork would be removed. Once one evaluation is complete, the subsequent ones become easier because they can ideally be streamlined. Once staff gets used to the process, evaluation can become second nature in some ways, and can possibly be done as the library is being added to and maintained. The staff will know how to recognize collections that are not being used or that are not fully developed. They will also be able to identify and highlight the strengths of the digital library thus encouraging more people to look at them thus increasing usage.