Cited As: Du, H., Chu, S.K.W., Gorman, G.E., & Siu, F.L.C. (in press). Academic social bookmarking: An empirical analysis of the Connoteausers. Library & Information Science Research.
Academic social bookmarking: An empirical analysis of Connotea Users
- Introduction
When people explore the Internet, one of the challenges they face is remembering and retrieving items that they have previously found to be useful. A common approach to managing information found on the Web is the use of personal bookmarks stored on a user’s own computer (Lund, Hammond, Flack & Hannay, 2005; Millen, Feinberg & Kerr, 2005]. Recent research suggests that social bookmarking (SB) offers a more effective means of resolving the difficulty associated with web information discovery and management (Hotho, Jaschke, Schmitz & Stumme, 2006). SB refers to the common indexing of objects and resources, which enables online users to classify useful websites andcreate shared bookmarks of online resources (Storey, Cheng, Bull & Rigby, 2006),as well asdiscover resourcesfrom other people’scollections (Yanbe, Jatowt, Nakamura & Tanaka, 2007). By allowing users to categorize websites with the use of freely assigned (or controlled) keywords (“tags”), users are able to find and browse their own bookmarks easily (Godwin-Jones, 2006). User-generated tags may further be shared among all users in anSB website, thereby increasing socialization within the community (Goh, Chua, Lee & Razikin, 2009). These tags are also known as “social tags”or “folksonomies”.The term folksonomycomesfrom the words folk and taxonomy, andis used to describe the vocabularyused for creating and managing social tags(Dye, 2006).Social bookmarks may serve as individual reminders, shared lists, or collective resource libraries (PankeGaiser, 2009).With the increasing adoption of SB tools and services, it becomes important to understand users’ opinions and their perspectives regarding the use of SB.
- Problem statement
Some SB sites, such asConnotea ( and CiteULike ( have been targeted specifically to academic audiences(Gordon-Murnane, 2006). Different user groups may have different bookmarking preferences and interaction patterns (Lin & Tsai, 2011; NiuHemminger, 2011), and so it is possible that academic SB users may have some bookmarking behaviors which distinguish them from general users. For instance, exploratory searches,characterized by less well defined inquiries that emphasize analytical strategies, are often employed byacademic researchers(Marchionini, 2006).The popularity of SB services is associated with their potential for improving online information search, exploratory as well as fact-finding (Yanbe et al., 2007), and so one would expect academic users to be attracted by the exploratory search capabilities of SB systems. However, a recent study indicates that the numbers of visitors to connotea.org shows a decreasing trend. (Compete 2010). Such a declineinpopularity highlights the urgency inunderstanding academic users’ bookmarking behaviors and their perceptions of bookmarking, so that SB toolscan be designed to be more useful for academicaudiences.
Prior studies of SBhave mainly looked attag growth and entropy (GolderHuberman, 2006; Kittur et al., 2007), tag choice (Sen et al., 2006), tag classification (Al-Khalifa & Davis, 2007), and discussions of the potential academic use of SB tools (Gordon-Murnane, 2006). While substantial research has explored the patterns and organization of tagging and bookmarking behavior of SB users, empirical evidence about how SB usage is affected by different types of users is yet to be explored. Few studies have tried to understand the motivations or incentives behind the behavior of user groups (e.g., Arakji, Benbunan-FichKoufaris, 2009), and even lessis known about those whose purposes are of an academic or scientific nature. To address this apparent gap in knowledge, the current study collected academic users’ opinions and perspectives on the use of Connotea. The findings are expected to contribute to the understanding and further development of SB as a tool for information management in academic communities.
- Literature review
SB has gained increasing popularity possibly due to two distinguishing characteristics which enhance its social utility. The first characteristic is the use of tags created by users to organize bookmarks using labels that are meaningful to them (Millen, Yang, Whittaker & Feinberg, 2007). In contrast with the hierarchical and static nature of traditional taxonomies and classification systems, the organizational structure of the tagging used in SB servicesis emergent and dynamic. The other important characteristic is that bookmark collections typically become public information visible to other people, who may use them for their ownpurposes (Rader & Wash, 2008; Riddle, 2005). People can share their discoveries and retrieve related information from others who have similar interests (Mejias, 2005). It has been suggested that SB may be a potentially low-cost application for group information management (Grudin, 2006). SB systems provide support for search activities that range from simple “fact-finding” to more exploratory forms of search (Millen et al., 2007).One of the attractions of social tagging systems is that users require no specific skills to obtain the benefits of information organization (Jaschke, Hothos, Schmitz, Canter & Stumme, 2008).
Research issues relating to SB can be categorized into five main topics as shown in Table 1. As SB systems allow users to use their own keywords for bookmark tagging, many researchers have been interested in examining tagging patterns. Tag growth and dynamics, tag choice and tag semantics are frequent foci for SB research. Golder and Huberman (2006) identified regularities in users’ frequency of tagging, their selection of tags and also their activities in SB websites. Factors affecting users’ choice of particular words for tagging were also explored. Similarly, Sen et al. (2006) observed that pre-existing tags affected the kinds of tags that were subsequently created by users. However, this observation was countered by Kipp(2006), who observed that tag selection was not a direct imitation of pre-existing tags, but rather an individual idiosyncratic process.
[Insert Table 1 about here]
Scholars such as Al-Khalifa and Davis (2007) have attempted to classify tags into meaningful semantics. Cattuto, Benz, Hotho and Stumme (2008) suggested relationships between user-created tags and introduced a method to measure such relationships. Sen et al., (2006)investigated the classification of tags, providing insights on tag choices. Redden (2010), Sun, Suryantoand Liu (2007) and Seki, Qin and Uehara (2010) focused on potential applications and future developments of SB tools, with the aim of improving current SB systems and developing more advanced ones with new features and broader applications. Based on participants’ bookmarking and peer interaction patterns in an SB system called “WeShare”, Lin and Tsai (2011) defined four types of user groups:lurkers (seldom performed collective information searching), actives (made more effort to managetheir own bookmarks than to review the bookmarks created by others), quoters(made more effort toreview others’ bookmarks than to createtheir own ones), and critics (focused on commenting the bookmarks shared from others).
Research which looks at user perspectives and opinions directlyis distinctly sparse when compared with indirect studies of user behavior as reflected in SB activity. This study is a contribution to that underexplored arena.
- Research methods
- Connotea
SB sites that have gained significant popularity include StumbleUpon, Delicious (formerly del.icio.us), Blinklist, CiteULike, and Connotea. A comparison of the number of visits among five popular academic SB systems (Compete, 2011) indicated that Connotea was the most widely-used in the year 2010 (see Figure 1). Connoteais designed to enablescholars to manage citations and scholarly articles (Rethlefsen, 2008). As with most SB systems, Connotea allows users to tag items, and a user's collection can be made public and shared among colleagues and workgroups across the world (Hammond et al., 2005; Rethlefsen, 2008). Connotea incorporates special functionalities targeted tomanaging references, such as the automatic collection of metadata.
[Insert Figure 1 about here]
4.2.Participants
To examine perceptions of academic users with a range of familiarity with SB systems, three groups of participants were recruited: experienced users, users with a moderate level of experience, and novice users. Experienced users (Group I) were those who: (a) hadcreated at least 100 bookmarks on the Connotea website, and (b) hadmade their emails public (e.g., through their bookmarks); thus allowing a means of contact for recruitment. Group I users were recruited from 101 potential participants that were identified on the Connotea website. Thirty-nine responses were collected, yielding a response rate of 38.6%. Moderately experienced users (Group II) included information management students in the University ofHong Kong (HKU) who had experiences using a SB system for group projects prior to their participation in this study. Out of 74 potential Group II participants, 48 responses were collected (65% response rate). Novice users (Group III) included part-time or full-time research assistants in HKU who had no previous experience of using any SB applications prior to participating in the research. Forty potential participants belonged to the last group were invited to register for an account on the Connotea website and use the SB tool for two months. After this period, participants were invited to respond to the online survey, and 32 responses were collected (80% response rate). All potential participants were invited by email to respond to an online survey, and informed consent was obtained from each who agreed to join the study. The research procedures were reviewed and approved by the Ethics Review Committee of the university. A total of 119 users, representing the three user groups, participated. Table 2 illustrates the academic background of all the participants in the study.
[Insert Table 2 about here]
4.3.Online survey
The online survey consisted of both close-ended and open-ended questions that investigated users’ bookmarking and tagging behaviors(Appendix A). These questions focused on (1) background of the participants and their habits inusing SB systems, particularly Connotea; (2) their ratings ofthe utilities and features of Connotea; and (3) their opinions on the effectiveness of Connotea. Bookmarking frequency was measured using a 6-point ordinal rating scale with the following values: 1 (never), 2 (once a month or less), 3 (once every two weeks), 4 (once to twice a week), 5(three to six times a week), and 6 (once daily or more). Participants’ perceptions of the utilities and policies of the SB system were examined using a 4-point Likert-type scale, where 1 referred to “strongly disagree” and 4 denoted “strongly agree”. A balanced 4-point Likert scale which eliminates a neutral mid-point has been shown to minimize social desirability bias and uncertainty in participants’ responses (Garland, 1991). Ratings of 1-2 were interpreted as negative responses and ratings of 3-4 as positive responses. Particular attention was directed towards the users’ perceptions of using the SB website for information discovery and management.
Open-ended questions were also provided so that participants could further elaborate on their scale-type answers when needed. As a result, participants #25 (in Group I), #41 (in Group II), and #28 (in Group III), also provided further written comments in the survey. For those participants who had indicated their willingness to be interviewed, follow-up interviews were arranged to collect in-depth opinions concerning their experiences with using the SB services. Two participants from Group I, one from Group II, and three from Group III were interviewed in the follow-up investigation.Appendix B lists a summary of the open-ended answers and follow-up interview responses by academic participants in the three different groups.
4.4.Handling of “don’t’ know”option
The survey also provided a don’t know option for each question. A number of participants answered with this value for some questions, largely due to their unfamiliarity with the SB features (Figure 2).
[Insert Figure 2 about here]
These data points were excluded in the analysis. This treatment les to differences in sample size, and may havevarying effects onthe results. The percentage of don’t know answers in each question was calculated, and has beenmarked in Tables 4-10 in four different shades of grayfor ease of reading. The darker the color in the results tables the higher the percentage of data points being removed from analysis because of the don’t knowresponse and the greater the possible impact on the predictability of the results. Fortunately, the majority of the results in Tables 8, 9, and 10 have no shading. The greatest impacts appear in Table 4 (results for group I), Table 5 (results for the question “It is easy to use” for Group II and III), Table 6 (the result forthe question “Using the search box with the choice Find Exact URL” for Group I), andTable 7(the result of the question “It is useful for managing information in a group” for Group I).
4.5.Analysis
Responses to the online survey were collected and analyzed using non-parametric tests. Descriptive statistics were used to measure central tendency and inferential statistics used the Kruskal-Wallis test to examine the differences in the responses of the three participant groups. Statistical significance was set at p< .05. Statistical analyses were done using SPSS 18.0. Responses to open-ended questions were summarized qualitatively.
- Results
- Users’ perceptions ofbookmarking and tagging behaviors
Table 3 shows five kinds of bookmarking behaviors of the three groups of academic users. Creating bookmarks received the highest rating on average by all groups (Group I = 3.26, Group II = 2.63, and Group III = 2.53), while “copying bookmarks from other users”got the lowest mean rating (Group I = 1.68, Group II = 1.96, and Group III = 1.63). This result suggests that academic users in generalare more likely tocreate their own bookmarks but less likely to use bookmarks created by others.In addition, experienced users (Group I) were found to have significantly higher frequency (once every two weeks) of creating bookmarks than the other two groups (once a month or less).
[Insert Table 3 about here]
Experienced users (Group I) created more tags per bookmarkcomparing to the other two groups (Figure 3). Over 20%of group I users created 4 or more tags per bookmark, but only a small portion (2.1%)of moderately experienced users (Group II) and no novice users (Group III) did so.A majority of Group I(65.5%) and Group II (59.6%) users created 2-3 tags per bookmark.However, Group III users tended to create much fewer tags: 63.3% of them created just 1 tag per bookmark and 36.7%of them created 2-3 tags per bookmark. Connotea policies require users to create at least one tag per bookmark, and it appears that most novice users comply with this minimum requirement without going further.
[Insert Figure 3 about here]
Academic users’ ratings (from 1 = strongly disagree to 4 = strongly agree') on the usefulness of tags and bookmarks are summarized in Table 4. Participants had generally positive perception about the seven utilities of tags/bookmarks, in which “use tags to find your own bookmarks”received the highest ratings of usefulness by all groups: Group I (M = 3.78), Group II (M = 3.17) and Group III (M = 3.07). In addition, experienced users (Group I) were found to give a significantly higher rating on this utility than the other two groups. “Form a group with friends or colleagues for sharing bookmarks” was rated the least useful by both Group I (M = 2.93) and Group II (M = 2.98) users. Novice users (Group III) rated this utility the second least useful (M = 2.80), andrated “using tags in sharing bookmarks with friends or colleagues”the least useful utility (M = 2.73). This finding suggests that academic users commonly do not share bookmarks with their friends or colleagues.
[Insert Table 4 about here]
5.2.Information discovery
All groups of users perceived the automatic collection of bibliographic information to be useful and easy to use. However, some users commented on a limitation of the automatic collection of bibliographic information, notingthat the SB website was sometimes inconsistent with the corresponding reference of an article, thus requiringadditional time to correct the information manually(Tables 5 and 6).
[Insert Tables 5 and 6 about here]
Generally speaking, positive ratings of usefulness were given by the three groups of userson alleight types of the search functions. Thisindicates that participants perceived the various search functions useful insupporting information discovery.Significant difference among the three user groupswas found only in their views regarding the usefulness of the “related users”function.While moderately experienced users regarded this function the least useful and noviceusers thought of it the second least useful,experiencedusers considered it the third most useful of the eight search functions. All three groups rated the function “using the search box with the choice My Library” the most useful search functions.This was consistent with the earlier finding that participants preferred using self-created bookmarks over copying bookmarks created by other users. Compared to the other types of search functions, finding information “through other users who have similar interests” also received relatively high ratings by both Group IIand Group IIIusers.Some users pointed out that this function could save time screening relevant information and bookmarks. Both Group Iand Group IIIusers rated the function “using the search box with the choice Find exact URL”, the least useful among the eight search functions, while Group II users rated “using Related users” the least useful.