A Qualitative Study of the Influence of Visual Aids on Search-Result Document Preferences Regarding Online Purchasing Decisions
Matthew E. Carroll
INLS 172: Losee
Spring 2002 Term Paper
Introduction
The type of Information Retrieval this study concentrates on concerns goods for sale or auction and how visual aids help in the selection process from among the search results. It does not necessarily look at how the information is retrieved, but how that information is assimilated after it has been retrieved and how important visual aids are in that assimilation. The reasons for looking at this include a number of factors, foremost among them my desire to study how individuals use visual aids and how important such aids are for conveying information when an individual may not know much about the item in question.
It has been said that interacting with information on the Web starts with browsing and searching, continues with selecting, digesting, and assimilating information, terminates with generating new information, and begins anew.1 Here, I take part of that circle, and look at how a user selects, digests, and generates new information from set search queries with the possible aid of visual artifacts. To accomplish this task, this study looked at two Websites to see how visual aids helped or otherwise influenced users in deciding which items from the search-result lists were most desirable. To study the influence of visual aids, I performed contextual interviews with various users who interacted with the different Websites while they were supervised.
Overview of the Test
To gather data for this test, I decided to conduct the test as a contextual inquiry and to gather qualitative data from various users who participated in the study. A series of questions were designed, and additional questions to the users were permitted if deemed necessary, based off of feedback gotten throughout the process. Non-solicited observations of the users were also noted. There was no time constraint on the test; users could take as long as they wanted to browse and the interview process continued until all relevant questions were asked.
For the purposes of this test two websites were used: EBay.com and Autotrader.com. I chose these two websites because searches through their engines provide a wide variety of result documentation regarding a similar item, and visual aids are prevalent but not always present nor similar in the results.
About the Websites
EBay is “the leading online marketplace for the sale of goods and services by a diverse community of individuals and businesses.”2 It was founded in 1995 for the purpose of allowing anybody to sell just about anything, and today boasts over 42 million registered users. When items are posted for sale, the seller has the option to include visual aids, such as pictures, with the item descriptions. EBay recommends that all sellers include visual aids in their descriptions, since “in general, items listed with photos get more bids and sell for higher prices than those that do not.”4
AutoTrader allows people and organizations to list new and used cars for sale, and is “the only used car website with over 1.5 million used vehicles listed for sale by private owners, dealers, and manufacturers.”5 Although it is not specified on the website why people or organizations should list their cars for sale with photos included, AutoTrader does charge more for listings that include a photo and has three tiers of listing options that range from no visual aid, to one photo, to up to nine photos, with prices for the three options ranging from 25 to 70 dollars for the initial listings.6 This implies thatvisual aids are important and help in the selling process by intriguing browsers.
About the Search Terms
The two items I used for my searches were the term “hk695” for EBay, and a 1997 Toyota Tercel within 50 miles of the ZIP code 27514 for AutoTrader. I chose these two terms for several reasons.
The term “hk695” is actually a model name for a computer-speaker set produced by harman/kardon, and is otherwise known as the Champagne speaker set, due to the flute shape of the speakers, which resemble tall champagne glasses. The term hk695 is not widely known and the text descriptions from the search results include a good amount of technical speaker jargon that the average consumer does not necessarily understand. The inability of the user to make sense of the description lends importance to the visual aids for the particular item.
The Toyota, being an automobile, is a more common item with jargon more familiar to the general public. Specifically, it is a Japanese subcompact automobile. The documents retrieved in a search follow a more rigid layout than the EBay documents, as AutoTrader has set categories that organize the information about the vehicle presented (thus the detailed search term listed above). Visual aids such as photos are optional and seemingly less important as the user is generally more familiar with the jargon associated with automobiles (like mileage; and such short-hand terms as PW, AT, A/C, for power windows, automatic transmission, and air conditioning.)
Besides attempting to pick search items that would vary in the general knowledge among the users, I also looked at the difference in price between the two search items. There is a significant monetary difference between these two items: the Toyota costing well over $6000 and the computer speakers averaging about $60. Although the users were instructed to disregard the price when choosing the most desirable items, I wanted to establish a wide discrepancy between the two terms users would deal with to possibly get different reactions to the term descriptions and visual aids.
Methodology
To test how visual aids help in the selection process from among the search results, I first found two searches that would produce a moderate number of results, ranging from 10 – 20 returned documents for each query. This was done by searching a series of terms before the survey was administered to find search results that would not be too long or produce too few results. The above stated terms were chosen because they met this criteria and ensured that while there would be a large enough data set for the user to browse through, that data set would not be too large and thus possibly overwhelming. At the beginning of the test, before the user started, I educated each user about the two websites, how they work, and showed them examples of the user interfaces (UI) found on each. This included describing how the results are listed and what certain icons in the UI, like the camera icon, which represents a visual aid in the description and is found on both websites, mean.
After ensuring that the user was familiar with the layout and purpose of a particular site, the user was instructed to start the search. The test was administered on a computer with Internet access, and the search terms were set up beforehand on two separate browser windows on the same computer. Users were instructed to browse the search results for each term and to make a decision as to which of the items they most preferred. They were instructed to disregard price as a concern in influencing their final decisions and only concentrate on the item descriptions and visual aids.
As the users browsed the search results, I noted how they navigated the search results and how their eyes looked over the site, and recorded any verbal comments they made. They were also asked if anything in particular stood out to them as they looked over the results and recorded their answers. After the user had made their choice for which items they liked best, they were asked how they came to their conclusions. I also asked how much the visual aids influenced their decisions, and if they offered any additional information not provided by the text descriptions. Finally, they were asked if the visual aids made the items they looked at any more or less attractive to the user as a potential final choice.
Limitations and Bias
Some limitations faced in this study include the small size of the data set looked at and the small size of users tested (four users and two search result sets). Within the sites looked at, EBay offered an additional limitation as documents would appear and disappear as time lapsed during the study. AutoTrader was more stable in this regard as its listings have longer listing dates and the prices on the items in the search results do not change as they do with EBay.
Another limitation faced in this study is the differing levels of experience among the users regarding the Internet, World Wide Web, and the two websites used for this study. Although an effort was made to recruit users who had a basic understanding of the Internet, some users were wholly unaware of how the sites used for this study worked before being oriented for the purposes of this study, while others were intimately aware of how they worked.
Users also had personal opinions about the items in the search results that ranged from general like or dislike of the item in question to personal preferences regarding such things as color and style. To test how intrusive this was to the test, users were also asked to search for an item that interested them on each site, and their reactions were also noted according to the same criteria as the control items.
Results
Users involved in the test had wide ranging opinions about the items based on different factors, most notably page design, description layout, and visual aid presentation.
Users and hk695
With EBay and the hk695 search results, users generally stopped reading the text descriptions after they looked at the first one or two items in the search results. When asked about this, they stated that the descriptions all offered the same information, and that they were unfamiliar with most of the technical terms in the descriptions. Some users began to associate similar term descriptions from the initial search result list as similar items and skipped those documents altogether, assuming that they were the same item listed multiple times.
It was observed that the more they looked, the more users began to fixate on visual aids and page layout, and grew frustrated at the lack of variety among the search results. Indeed, many of the items in the search results were posted by the same original sellers and had a recognizable common layout. Thus, search result items that offered divergent layouts immediately drew more attention and the users spent more time looking over the selected document. Among the final items selected as the most preferable from the search results, the items with unique page layouts generally were chosen.
As for the actual images used, within the search results there were two basic images to be found on every search result (Appendix B). Users split evenly in their preference for “stock” images, which one might find in a catalogue, and “real” images, like those that do not appear as professional or sterile. In reality, both images seemed to be stock images and were found throughout the search results, and their content differed only in that in one image the speakers appeared isolated on a white background, while in the second common image, the speakers appeared in their original plastic wrapping with a room in the background. Some users preferred the reality the second image offered, while others preferred the cleanliness that the first showed. Regardless of their preference, all users ultimately chose search result selections that had an accompanying visual aid for the EBay search.
Users and the Toyota Tercel
The AutoTrader portion of the survey occurred after the EBay search and users had by then grown acquainted to the expectations of the test. Like the EBay portion of the survey, the search criteria were set before the user arrived and the user was instructed to peruse the results and choose a favorite item from among the search results. The big difference between the AutoTrader site and the EBay site was the diversity of the search result descriptions and visual aids. Site layout was more conformed, but the general descriptions varied to the point of complete uniqueness, and most items listed had a unique visual representation, although some had a stock photo for their visual aid.
With the second set of search results, users unanimously clicked first on a search result with an accompanying visual aid, either by chance or design. For most of their perusal of the documents provided, users seemed to wholly ignore listings without related visual aids. When asked about their choices in initially choosing documents to look at, most users reasoned that they wanted to see what they were looking at, especially if they were going to purchase the item in question.
Users did pay more attention to the text descriptions of the items and read them more than with the EBay search results, mainly because there was more variation among the descriptions of the listed items. Some users stated that they based their final decisions on such things as the mileage of the vehicle, but most users commented on things inferred from visual aids. Some users looked exclusively at the visual aids and ignored altogether the text descriptions unless they liked how the car looked, thus giving clear precedence to the visual aspects of the search results. Some users even became hostile towards documents that did not use visual aids or that used stock images to represent the car being presented in the text description, saying things like “that’s rude!”
One user noted that the visual aid was especially useful in providing information about the items in the search result list because of ambiguities within the text descriptions. One user noted that while two documents listed their cars as green, one car was “an ugly green, not a nice green.”
Overall, visual aids seemed much more important to the users with the automotive search than they were with the speaker search, and users were not as content with the stock photos, or even “bad” photos that did not appeal to the users, as visual aids for the AutoTrader search. These “bad” photos were generally fuzzy, poorly lit, or from a corner-on perspective and did not adequately show the car in the users’ opinions. Stock images were not seen as being as favorable when it came to searching cars as they were when searching for speakers.
When the users were asked about the importance of visual aids between the speaker and the auto searches, some users stated that they cared more about what a car looked like and that they got more attached to a car than to a set of speakers. Although users were asked to discount price, most users stated that for such an expensive item visual aids were more important in helping them reach a decision as to which items were most appealing to them.
Conclusion
Having had users search through the search results of two diverse items that sometimes involved visual aids and sometimes did not, and which had a range of qualities, it seems apparent that visual aids are always important, but in varying degrees depending on the item at hand. For the speakers on the EBay search, after users saw one visual representation of the item, additional pictures did not seem to matter as much, although the context of the photo and the users’ personal opinions became important factors, especially when text descriptions proved vague or unintelligible. The same holds true for the Toyota search on AutoTrader, where some users would not select an item because of personal preferences about color or the context that the auto was in. Regardless of the users’ preferences concerning the visual aids, they all used them to further their understanding of the particular item at hand and to help them form opinions of that item.
Although I attempted to limit price’s impact on the survey, visual aids became more important as it increased, indicating that when users are involved in more expensive decisions, they desire more to see what they are actually purchasing. This indication is further supported by the price increases associated with the number of visual aids sellers can use when listing their autos on the AutoTrader website. Users did not get very emotional over the various images associated with the hk695 listings, but the same users became irritated at poor visual aids when looking at the Toyota Tercel, even when they expressed no fondness for the car itself.