Usability Issues in Metasearch Interface Design

Usability Issues in Metasearch Interface Design

Presented by the LITA Human-Computer Interface Interest Group

Renaissance Orlando Resort

Friday, June 25, 2004 2 pm to 5 pm

The program began with an introduction by Lisa Martincik (University of Iowa Libraries), Co-Chair of the Interest Group with Paul Soderdaul. Martincik suggested we follow the interface usability rule of computer game developers: is the interface intuitive and can it be learned quickly?

The preconference was organized around four expert panelists, each speaking for 20 minutes. There were opportunities at the end of each presentation and at the end of the session for questions and answers. The participating audience was made up of over 40 attendees from academic and public libraries as well as content-providing vendors. The format and diversity of the speakers, as well as the audience, made for a lively and provocative afternoon.

“Metasearch and the Public Portal”

Brenda Bailey-Hainer, Colorado State Library

Interface as Portal. The Colorado Virtual Library provides an interface for library information and catalogs, public records, geospatial data, historic documents and web sites. Its complexity points to the importance of the portal as a tool for libraries in helping the public make sense of information. The functionality of metasearch is critical to this interface/portal. The public portal can provide access to resources of two types: those that are relatively static – OPAC, websites, GIS maps, digitized materials from local collections, and knowledge databases from reference transactions. These are the resources we most often think about when developing metasearch interfaces. But we might also incorporate interactive services into our portals such as online classes or online homework help. Public libraries, in particular, have challenges not only in presenting a broad range of collections and services but also in serving a community diverse in age and language.

Usability has two faces. We tend to think most about the public usability face of a portal, but the back end component is an important aspect of usability for librarians, content managers and web designers. There may be a different measure of success for different users. The public just wants it fast. They don’t care about the technical challenges of metasearch, they just expect the behavior and speed of Google. The back end workers need flexibility and easy maintenance. Increasingly, the capacity for statistical output (numbers) as well as facility for conducting outcome-based evaluations (a survey instrument) may be required.

Future development. What we expect of metasearch has grown and evolved. Challenges for metasearch engines and interfaces are speed, consistent application of standards, aggregating and presenting results, differing record structures, and interoperability. We need to crosswalk between different levels of specificity. And how do we integrate searching with services? How do we include new viewpoints for interfaces – GIS, temporal, special access? For example, the teen view varies quite a bit from the genealogists’ view and that is different from the K-12 educators’ view. In addition to the standard information gathering for assessment (comments submitted via web site, focus groups and usability testing) Colorado has developed a pool of “cultivated reviewers” who regularly provide feedback on the portal as it develops.

“Usability and Metasearch at Rochester”

Stanley Wilder, University of Rochester

Serial Failure.The University of Rochester is driven by its awareness of the phenomena they call serial failure. This is the single most important access issue today. Metasearch is the most obvious tool to reduce serial failure. Examples fromRochester demonstrate how usability testing influenced the metasearch interface design. Other insights from usability testing were that the library interface was too complex with two many choices to be made, including confusion over when to use the catalog and when to use databases—these became assumptions for further development.

Developing the Out-of-the-Box Interface. There were several problems with the out-of-the box metasearch interface provided by Endeavor. The initial changes made by Rochester were to reduce branding and to reduce the steps in the selection process for databases. The process for choosing a database from a subject-oriented list is a task that does not correspond to any other experience that students have on the web. Students need to be reassured that they are on the right track. To respond to that need, Rochester named screens for associated tasks.

Current work.Rochester is currently working ondistributing the metasearch box throughout the web site. In course pages, the course name is important since students don’t necessarily associate their class work with a course number. Current tab marked Articles is still problematic, since students expect that the articles will be displayed at this point.The MyLibrary type of functionality is a distraction that doesn’t fit into the needs of the student, requiring them to reserve a corner of their brain for this specialized function.

Partnerships. Rochester and Endeavor have a partnership; currently they are working together to develop a different “out of the box” product.An IT divide is developing. That divide is characterized by the libraries who use search tool as it and the modifying library. It is imperative that coalitions be built between libraries together and with vendors.

Questions

Q. How do you choose databases for the metasearch subject areas?

A. This is determined to some extent by what you CAN you link to and what databases can provide you with results quickly. Broad subject coverage is good, as is provision of full text quickly.

Q. Were reference librarians resistant to the metasearch interface?

A. Rochester went from strong opposition to support from public services librarians. Wilder described how they show it off now. Their involvement in the design was critical to this buy in. Rochester sees it’s top audience as undergraduates and their top task, to find articles.

“Usability Issues in Metasearch Interface Design: perspective of an information provider”

Oliver Pesche, Chief Strategist, EBSCO

Current challenges for vendors. What does metasearching mean for vendors? Practically, there is an increase on system load. Although serendipitous content may be discovered for end users, vendors have gone from supporting one user on one database at a time to supporting 10 users on 20 databases all at the same time. This may be possible for the larger information provider but not sustainable for the small database provider. The library gets to decide what databases get searched and what is displayed from those databases. Faster searches get displayed first, but is their ranking consistent?

Relevancy is ranked on a number of elements, including, frequency of words within a single document, frequency of words relative to the document and to the database as a whole. Other databases may take into account the fields keywords appear in and factors such as proximity. But relevancy between databases is not necessarily comparable, creating problems when results of multiple databases are presented together.

How standards can help. Standards are important in addressing problems of presentation, system load and usage statistics. How can vendors maintain the integrity of presentation in order to comply with copyright and licensing agreements? The vendor wants and needs to retain some control over the user experience and presentation to users. Usage statistics present another challenge. Can you isolate metasearch activity so that it doesn’t inflate your usage statistics? They need to be able to isolate metasearch traffic. If vendor knows metasearch is coming in, they can get rid of certain overhead of computing resources, like authentication or setting up of user profile. They can tailor responses to the specific needs of the metasearch users. Finally, what direction should investments take to enhance content for users?

Questions

Q. Where will work on classification schemes go if the market is not there for specialized subject schemes and functions?

A. Librarians need to speak out for that kind of valued-added data classification.

“Getting to Know U”

Erza Schwartz, President, Art & Tech Inc. – The User Interface Practice

Balancing differing needs. The challenge for vendors is in designing an interface to satisfy the needs of many clients: management, marketing, finance and developers. Librarians are not the end user.

User needs are evolving to include new requirements for functionality: portability, direct manipulation of results, sharing capability, abilities to organize and present information in different ways. These requirements present a conundrum for vendors that need to balance the provision of “just in case” functionality but avoid an interface that becomes too complicated for end users with too many buttons. In designing the user interface, we need to think broadly and balance these concepts.

User mental models. What is the mental model for searching? There are new search engines all the time. Do students know what metasearch means or do they care how it works? They just want to move on. They don’t want to understand the search; they just want the results. We as librarians want to understand the process. An agricultural metaphor helps us to understanding the complexity of the mental model for searching: If you are a magnet, finding the needle in the haystack is not hard; what is hard is finding the particular piece of hay in the stack that you want. If it is there at all how do you distinguish it from the others?

Barriers to searching. Barriers make it hard to get the user from here to there. Permission for access is denied, or the incorrect keyword is used. It’s hard to describe what you want. Dealing with search results presents other barriers. Results may look promising, but the student wonders about the authenticity and value of the results. Is this the real thing? Perhaps it is valuable but needs massaging or cleanup. Perhaps it needs to be shared. Vendors are trying to build to interact that satisfies the needs of everyone. To solve that is challenge, they need to have real people use the system.

Final question and answer session

Q. How much of the burden of designing the user interface should be taken on by libraries and how much from the vendor?

A. Librarians need to be involved because they are the intermediaries; vendors are trying to anticipate the needs of librarians and end-users.

We are in a transitional time. Eventually, we as libraries will be able to give clear signals to vendors as to what we need. Standards are the tool for overcoming challenges.

Q. If metasearch engine skims results, you run the risk of losing those 51-100 records that where the best one might have been found?

Q. Can relevancy ranking be standardized?

A. That information may be proprietary. How to normalize “bests” against the others? How to compare such various databases like WorldCat, Google, and EBSCO? Index the database as a single thing; return results that are a amalgam that analyzes the results in a value-added way. Need to meet the student at where they are and contextualize the search for students.

Q. Where does the OPAC fit into that? Will metasearching make cataloging go away?

A. The role of the library in standardizing metadata, particularly for resources like institutional repositories, is critical. Provide better access through mapping of subject heading and semantic indexing. How can we get our print collections back in front of students?

Q. What are some ideas for getting public service librarians on board with metasearch?

A. Get reference librarians to watch usability testing; they need to see this in a context where they are not in a position to “help”. Have public services staff find attempt to find articles on their web site in fields with which they are unfamiliar in order to experience what students may experience.

Notes provided by:

Nancy Turner

Electronic Resources Librarian

Syracuse University Library