Does Web 2.0 Matter? Investigating How Learning Environment Design Affects User Community Engagement
Anthony Cocciolo, Hui Soo Chae, Gary Natriello, Teachers College, ColumbiaUniversity
525 West 120th Street, New York, NY10027USA
, ,
ABSTRACT
In this session, Cocciolo, Chae, and Natriello discuss the impact of Web 2.0 design patterns on user community engagement in online learning environments. Specifically, this study compares two similar online resource sharing systems for the same community across two continuous periods of time, with one system explicitly designed with Web 2.0 design patterns and the other not. Results of this analysis reveal that relative knowledge experts participated at a rate twice that of knowledge novices in the non-Web 2.0 environment. However, the level of engagement shifted significantly with the introduction of the Web 2.0 environment, where knowledge novices not only participated proportionately to their total numbers, but also came to perform the role of key knowledge facilitators. The emergence of the novice as key knowledge facilitator was underscored through a Social Network Analysis.
Overview
The purpose of this study is to investigate how learning environment design decisions affect user community involvement. Specifically, this study will compare two similar online resource sharing systems for the same community across two continuous periods of time, with one system explicitly designed with Web 2.0 design patterns and the other not. Both systems provide the basic functionality of sharing intellectually authored materials, such as publications, working papers, research data, and audio/video content. The primary question was, how do learning environments designed with Web 2.0 design patterns influence community participation compared to non-Web 2.0 environments?
The context for this investigation is the community of 5,500 students, faculty and staff at Teachers College, Columbia University, a large graduate and professional school of education located in New York City. Within this study, the two systems that were compared are a) PocketKnowledge (PK) ( and b) Community Program Collections (CPC). PocketKnowledge was designed explicitly with Web 2.0 design patterns, most notably the patterns that a) users control their own data, b) users should be trusted, c) flexible tags are preferable to hierarchical taxonomies, d) the attitude should be playful, and e) the expectation that the software gets better when more people use it (O’Reilly, 2005). Figure 1 is a screen capture from PocketKnowledge whichillustrates some of these concepts, such as the primacy of the individual user in the context of the system as a whole. The other system, Community Program Collections, did not specifically employ Web 2.0 design patterns. Instead, it used more traditional hierarchical models, such as a) organizing information based on a taxonomy derived from institutional structures (e.g., programs and departments), b) lack of user control over their own content (e.g., a user cannot remove their content from the site), and c) centrality of authority (i.e., a user can only suggest content to be added to the collection; however, ultimate authority resides with an institutional librarian). Figure 2 is a screen capture from Community Program Collection which illustrates how information is organized according to institutional structures.
Figure 1: A user’s view of his collection of materials in PocketKnowledge
Figure 2: Community Program Collections is organized by institutional structures
This paper will first discuss the design differences inherentin each system, followed by a usage analysis. Several findings will be presented, leading to the conclusion that design dramatically impacts community usage patterns.
Systems Design
The design differences exhibited in PocketKnowledge and CPC are most evident when viewed in terms of affordances and constraints (Norman, 1988). With a web-based system, this refers specifically to those functions and features that allow a user to accomplish some action, as well as the barriers (intentional and unintentional) that the system enforces. One particularly salient constraint that CPC enforces is the inability for users to directly post materials to the system, but rather to make “suggestions” for addition. Before a user is allowed to make a suggestion, a warning message is displayed in caps and bold that reads “IMPORTANT – PLEASE READ CAREFULLY” as well as a three paragraph statement on copyright (see Figure 3).
Figure 3: CPC shows a warning message and information on copyright before a suggestion can be made
After a user makes a suggestion, the system displays the following message:
Thank you for your suggestion. We will review the item, and if possible, make it a part of the Arts & Humanities Program Collection. You will receive an e-mail either way.
This particular set of constraints highlights certain attitudes towards the end-user. First, the warning message in bold and caps indicates that the system distrusts that the user will read the copyright statement. Secondly, the system reinforces the knowledge authority relationship between library or university and the individual by allowing users only to make “suggestions” and if a suggestion if made, it must be “reviewed”. Given these set of constraints, it is plausible to believe that many users, especially those who are less confident in their knowledge expertise (e.g., students), would hesitate to make a suggestion out of fear of being rejected by the knowledge authority.
This design approach is in sharp contrast to PocketKnowledge, which allows any user to post any files instantly and trusts that an addition does not violate copyright laws. Figure 4 shows the “add an item” window, which asks the user if the file violates copyright laws, giving the option for cases where the user “doesn’t know” if copyright would be violated. This type of design choice illustrates that the system trusts the user to a high-degree, with the realization that true violations of copyright are relatively rare and can be handled on a case-by-case basis. In sum, this singular example illustrates a broad distinction in design approach between PocketKnowledge and CPC system.
Figure 4: PocketKnowledge includes a simplified copyright compliance policy
Methods
Data was collected from Community Program Collections from November 2004 to August 2006. Data for PocketKnowledge was gathered from September 2006 to January 2007. We used a Social Network Analysis (SNA) to shed light on the sociotechnical interaction network formed by PocketKnowledge. Following the work of Barab, MaKinster, Moore, and Cunningham (2001), the term sociotechnical interaction network looks “to capture the complex sociotechnical arrangement involved in a technology-intensive project, emphasizing the reciprocal character of the interaction among people, among people and equipment, and even among sets of technical structures and political climates” (p. 73).
During the five months data collection period, N=497 distinct users shared files on PocketKnowledge by either uploading a file or by downloading a file. The analysis does not include downloads from users who were not logged in. The usage data was converted to a matrix and visualized as a network using NetDraw (Boragatti, 2002). Network attribute data, which indicated institutional role (e.g., masters-level student) and color, was added to indicate user/node role. Additional analyses were conducted with NetDraw, including segmenting the network into components and illuminating key actors using cutpoints analysis. Table 1 presents the distribution of individuals over the set of institutional roles.
Table 1: Role distribution of user population
Library / Doctoral Student / MA Student / Faculty/instructor / Staff / Other2 / 151 / 216 / 35 / 17 / 76
In addition to the SNA of PocketKnowledge, an analysisof “Suggested Additions” to the CPC separated by users’ institutional role was completed. Optimally, a SNA of CPC would have also been completed. However, the system did not log detailed enough user data to perform the SNA. Despite this limitation, “Suggested Additions” data was sufficient for the purposes of this study.
Results
The PocketKnowledge interaction network can be decomposed into several components as shown in Figure 6. These include: 1) isolated actors (users who only use the system to store their own work and choose not to share with others), 2) a large and varied community of actors and interactions, and 3) close-knit communicators who are isolated. Visual inspection of Figure 6 reveals that many of the actors are connected via the central library node. This phenomenon illustrates the importance of community members and institutional groups who are specifically responsible for communicating knowledge or content (in this case, the academic library).
Figure 6. Components of PocketKnowledge interaction network
Upon close examination, there are many more key members of the network than Figure 6 might suggest. A cutpoints analysis was conducted to reveal those key actors/nodes whose removal would leave the network divided into unconnected systems. As described by Hanneman (1997), “cutpoints may be particularly important actors -- who may act as brokers among otherwise disconnected groups.” Figure 7, which shows the cutpoints or key facilitators, reveals that there are other actors—in addition to those highlighted in Figure 6—who play a significant role in knowledge sharing.
Figure 7. PocketKnowledge cutpoints, or key facilitators of sharing
In the context of this study, cutpoints/key facilitators are actors who enable knowledge sharing through their existence on the network. This analysis reveals that knowledge facilitators occur in approximate proportion to their total numbers within the system. For example, ~21.6% of all cutpoints are faculty, and ~7% of all actors are faculty. Similarly, ~62.1% of all cutpoints are students, where ~73.8% of all actors are students (see Table 2). Our analysis also reveals that students (i.e., novice) play an equally important role in facilitating knowledge sharing as do faculty (i.e., experts). Moreover, the analysis indicates that novice learners (i.e., students) are able to come to occupy the role of the expert facilitator, gradually “fashioning relations of identity as a full practitioner” (Lave & Wegner, 1991, p. 121).
Table 2: Cutpoints (or key actors) by role, % of all cutpoints, and % of all users
Library / Doctoral Student / MA Student / Faculty/instructor / Staff / OtherCutpoints / 2 / 12 / 11 / 8 / 2 / 2
% of all cutpoints / 5.4% / 32.4% / 29.7% / 21.6% / 5.4% / 5.4%
% of all users / .4% / 30.3% / 43.5% / 7.0% / 3.4% / 15.3 %
Whereas PocketKnowledge allowed users from different levels of expertise to occupy key roles, CPC (the non-Web 2.0 environment) prompted involvement predominantly from knowledge experts (faculty) at a rate twice that of knowledge novice (student). This is illustrated in Figure 8, which shows the higher proportion of faculty who choose to make content suggestions over students, even though students far outnumber faculty in population size.
Figure 8: Number of “Suggested Additions” to CPC by role
Findings and Conclusion
The findings from our research indicate that Web 2.0 design patterns such as increased user control and freedom over the learning environment, combined with a decrease in authority structures, led to conditions that prompted engagement from both knowledge expert and novice. Additionally, the sociotechnical interaction network prompted by the Web 2.0 environment better mirrors that of a Community of Practice (CoP) in the sense that novice learners (in a relative sense) are able to come to occupy the role of the expert performer (Lave & Wegner, 1991, p. 121). Much like a CoP, experts are not dispensed with, but rather novice learners are provided “with opportunities to make the culture of practice theirs” (p. 95).
In conclusion, this study reveals that Web 2.0 environments can positively impact community engagement and involvement as compared to non-Web 2.0 environments. In particular, Web 2.0 environments prompt use from members of varying levels of experience and expertise, most markedly by providing added opportunities for knowledge novices to play key roles.
References
Barab, S. A., MaKinster, J. G., Moore, J. A., & Cunningham, D. J. (2001). Designing and Building an On-line Community: The Struggle to Support Sociability in the Inquiry Learning Forum. Educational Technology Research and Development,49(4), 71-96.
Borgatti, S.P. (2002). NetDraw: Graph visualization software. Harvard: Analytic Technologies.
Buckley, W. (1967). Sociology and modern system theory. Englewood Cliffs, NJ: Prentice-Hall.
Hanneman, R. A., & Riddle, M. (2005). Introduction to social network methods. Riverside, CA: University of California, Riverside. Retrieved November 30, 2006 from
Lave, J. & Wenger, E. (1991). Situated learning: Legitimate peripheral participation. Cambridge: Cambridge UP.
Norman, D. (1988). The Psychology of Everyday Things. New York: Basic Books.
O’Reilly, T. (2005). What Is Web 2.0: Design Patterns and Business Models for the Next Generation of Software. O’Reilly Network. Retrieved January 30, 2007, from
Wasserman, S., & Faust, K. (1994). Social network analysis: Methods and applications. Cambridge: Cambridge University Press.
Wenger, E. (1998). Communities of practice: Learning, meaning, and identity. New York: Cambridge University.