NUMBERS TELL LIES; THREADS DON’T?

Ng Wai Kong, Marnisya Rahim, Marsyitah Ismail

Wawasan Open University, Penang, Malaysia

Abstract

It is always taken for granted that with LMS functioning, at least as a complement to existing ODL courses, the number of viewership is a good indicator of the ‘quality’, ‘acceptability’ of the on-line materials to the students. With more sophisticated statistical information tools available, course managers have migrated to the more informative threads as a measure of course participation. It is also postulated that more threading will occur with so-called more ‘talkative courses’ than the ‘straight-faced’ technology courses. Then it was also postulated that students would participate more if they were given better guidance by the tutor. This paper attempts to ‘debunk’ many current notions about LMS participation, and that only with a fully e-mode of learning will such postulates be fulfilled.

Introduction

A Learning Management System (LMS) is a software package that enables the management and presentation of online content to learners. Most LMSs are web-based to facilitate “anytime, any place” access to learning contents and interactions with the content provider. LMSs are thus useful and essential delivery systems for most Open and Distance Learning (ODL) institutions. In fact, as the LMS technology improves and become easier to deploy and use, this mode of delivery will become ubiquitous and almost dominant soon. While many ODL practitioners may still harbour fears of discrimination and access to technology, somehow we are getting the message that ODL students are up to par in accessing our LMS sites. Equity as an issue can now be placed in the backwaters. More importantly, ODL practitioners and course coordinators specifically should pay more attention to the use of this LMS technology.

WawasanLearn – the LMS of Wawasan Open University

Wawasan Open University (WOU) is the latest entrant in the competitive world of ODL. The first semester only began in January 2007, and courses were prepared only a year ahead of its launch. Right at the start, WOU’s course materials are predominantly print-based and are also considered to be the most comprehensive in terms of design and house style!The course materials normally consist of five units of printed self-instructional materials (SIM) that are written around a template guide with activities and interactions mainly driven through the use of icons. Evaluation strategies are built into the materials through many ‘self-tests’. Based on experience with the WOU template, the senior author also contributed to the development of ODL template distributed by COL (Roffey, 2006). In addition, there are also Student Guides, Tutor Guides with elaborations on the five tutorials convened for each course, the Tutor Marked Assignments (TMAs), and specimen examination questions. As a supplementary course support, WOU also developed a learning portal for the courses through WawasanLearn that is specifically developed for registered students. WawasanLearn is developed using the free open source web-based software platform named Moodle. In fact, a majority of ODLs have started migrating from proprietary packages and this includes the Open University of the United Kingdom serving 200,000 users. (

For us,WawasanLearn will initially focus on learning content management through the involvement of course coordinators (CC) as the subject matter experts and instructional designers, to test out e-delivery platforms more practically. It is thus a lean mechanism to create enough content just-in-time to meet the needs of individual learners or groups of learners (as in a tutorial group).

Figure 1: An example of WawasanLearnfront page for Course TCC 101 Computing 1

Moodle affords the institution a platform for e-learning. Of course being in the education field, there are always people using different names for the almost similar purposes! Imagine terms, apart from Learning Management System (LMS), like Course Management System (CMS), Learning Content Management System (LCMS), Virtual Learning System (VLS), Learning Support System (LSS) or Learning Platform (LP).

What does WawasanLearn provide?

Being a first attempt at e-learning, WOU is not too ambitious in exploiting the potential of Moodle, and thus for the first semester the site supports tutor-student forums, student chats and enquiries, and also carries additional resources like PowerPoint slides, Word and Adobe files, as well as up-to-date announcements, events, and links to course related off-site locations.

What is in there for the student?

1. Course Syllabus where an overview for all theimportant components given in the course guide will be available online. This will enable the students to review directly the syllabus whenever they are online.

2. Latest newsupdate provided by the various departments from the registry’s examination time table to the help guide on how to install the e-brary readerresourcesfrom the library services.

3. Discussion forumsfrom various topics like forming study groups among the students to the useful tips and review onthe final examination by the course coordinator

4. Web references tosupport the units discussed in the printed materials.

5. Downloadable video and audio files for topics that require observations and reflection highlighted from the units.

Most of these are catering at the front end of the platform. The main criteria for evidence of utility and feedbacks will be the frequency of visits and interactions. Moodle has its ‘back end’ procedures for the administrators to monitor and inform the stakeholders on the use of WawasanLearn. Since the arrival of the statistics function on Moodle 1.6.3, stakeholders as well as administrators would have the chance to collect and analyse important data about the number of hits and activities that goes on the WawasanLearn. For example, the increasing amount of students and tutors logging in a few days before and after every tutorial could be a sign that both groups are anticipating on the preparation and reviewing the tutorial session. This could also indicate that most students in certain courses or classes would rather view then participate in forums. These statistical features are vital in the long term to evaluate the effectiveness of the tutor, course coordinatorin terms of supporting the students on the course delivery.

Courses Supported by WawasanLearn

School of Business and Administration
  1. Microeconomics
  2. Principle and Practices of Management
School of Science and Technology
  1. Computing I
/ School of Foundation Studies
  1. Learning Skills and University Studies
  2. Foundation English
  3. Foundation Mathematics
  4. University Mathematics
  5. Bahasa Kebangsaan A

Some characteristics of the courses presentation on WawasanLearn:

1.Mathematics – very often providing the ‘solve a problem’ type of contents; queries posted and answered. Generally this and other subjects tend to be brief in interactions.

2.Business – widely based on organized themes and CC-initiated forums. Tutor monitoring (like generating feedbacks to the overall opinions on topics discussed) the types of activities and postings. For most cases, the students are attracted to the on-line queries because of the need to confirm their interpretation on the conceptsapplied on the case studies given. Although this could have generated major discussions and multiple long threads, it needs to be carefully monitored to justify the quality of the discussion. Why it attracts on-line queries? Is the course really web-enabled compared to others? Does it enjoin more inputs, chats, etc? Is Business and similar courses likely to be more participatory or ‘talkative’?

3.Others – mostly foundation or language courses like English, Bahasa Kebangsaan A where by multiple choices are most commonly used to test the application of grammar and sentence structure that are considered important in the language context.

Postulates about the use of LMS

We are of the opinion that not all courses (subjects) are created equal when placed in a web-environment and its ancillary facilities. Thus it is possible that:

  1. Courses that are of a practical and/or utilitarian nature tend to engender more ‘talks’ than courses that are considered plain and difficult (to discuss).
  2. Course tutors who actively participate in monitoring and feedback on the LMS would encourage bigger traffic flow and also improve the quality of the forum discussion.

These postulates highlight the authors’ disenchantment with the LMS and how these virtual platforms had addressed such issues like social learning. Notice that all LMSs have almost similar interfaces and facilities. These being:

  • Forums
  • Content managing (resources)
  • Quizzes with different kinds of questions
  • Database activities
/
  • Surveys
  • Chat
  • Glossaries
  • Peer assessment
  • Multi-language support

What all of these implies is that somewhere in the minds of the platform creators, all learners learning on-line are intuitively following the procedures encapsulated in the interface design. By default, this approach is seemingly ‘linear’, and that users more or less are trapped within the virtual environment as represented by the facilities made available on the website. Siemens (2004) suggests that this mode of interface presentation is “valid from a management and control standpoint, but antithetical to the way in which most people learn today.”He suggests that learning is “not a process to be managed” but it is “by nature multi-faceted and chaotic.” This brings us to another issue; even with formalized inflexible LMS interfaces, accidental learning does occur and it flies in the face of what Siemens claims as otherwise. Now what is accidental learning or accidental pedagogy? Accidental pedagogy refers to learning within a Learning Management System that occurs by pure accident. It promotes the idea that the concept of an LMS that is flawed and is in fact a content management system!

Siemens lists a number of effective alternatives to LMS including:

  • modular tools so that functionality can be expanded;
  • social tools to encourage individual expression and connect learners and content;
  • integration of synchronous tools; and
  • learner focused tools that encourages learning ecology.

The types of tools suggested include blogs, wikis, CMS, social tools, networking tools, collaborative spaces, and connection making protocols

Frequencies of use of LMS by Course

Moodle’s statistical tools allow the administrator to retrieve and interpret a number of frequency tabulations, and these are:

1. Most active courses – based on the overall total number of activities involved based on a period of time

Figure 2 illustrates that the number of activities performed by students and tutors (be it just viewing forums or downloading resources) will be a function of the number of students in the course.

Figure 2: Frequency of activities for courses

2.Most participatory (enrolments) – calculated using the ratio from active users compared to the total user from the course

Figure 3: Participation ratio of users in the courses

The graph in figure 3 is a good indicator in identifying the participation of the overall users in a course. For example, the number of students actively logging in and performing activities are relatively small compared to the total student enrolled for WUC131. Hence, this serves as a good way to see the number of students that has logged in and is actively participating for each course.

3. Most participatory (views/posts) – calculated using the ratio from the total posts made to the course and total view on the course

Figure 4: Participation ratio by course

The graph in figure 4serves as a useful indicator for viewing the participation of the course users at period of time. The participation ratio is taken based on the number of user views and posts for all of the forums in a course. Here, the stakeholders and administrators would be able to see that WUC111 has the most active forum participation posts compared to the other courses.

4. The students’ and tutors’activities and posts

The graph in Figure 5 shows the overall activities for both teachers (tutors) and students over a period of time. The overall activities range from viewing materials to uploading resources. Here, we can conclude that the students’ activities are obviously more than tutors’ and that student and tutor activities are slightly parallel with each other. So, during the indicated time, there are active interactions between the two groups.

Figure 5: Frequency of All Activities for a specific time frame

The graph in Figure 6 shows the two specific activities for students over a period of time. The activities consist of viewing and posting to the forums. Here, we can conclude that the students’ views are obviously higher during the weekend of their tutorials. This could probably be because the tutors post on some updates or materials related to the tutorials on the forums. So, students would rather view than posts questions during that period.

Figure 6: Student Views and Posts for a specific course over time

5. Statistical interpretations

The statistical use of the LMS is to assist administrators on monitoring students and teachers (course coordinators and tutors) activities based on a weekly and monthly basis. By obtaining this important information, we could evaluate if the tutors are responding to the students’ forums or if students prefer viewing than participating in forums. The number of active students compared to the overall number of students logging in and participating in the activities is also one of the advantages on using the statistics for the LMS. For example, if the graphs between the students and teachers have nearly the same pattern, then we could conclude that the two groups are probably responding or interacting with one another.

The graph in Figure 7 shows the overall course activities range from viewing materials to uploading resources for both teachers (tutors) and students over a period of time. In the example, we are looking at the course BBM101. A simple conclusion can be made here. The parallel lines indicate there is aparallel in intensity of activity between students and tutors for the consecutive three months. This is to say that as the number of views increase for students, so does the tutors. However, as stated in the title of the paper, numbers or specifically frequencies are not a reliable indicator of use or acceptability of the LMS.

The graph in Figure 8 shows a more detail perspective of the students’ activities in terms of views and postings to forums over a period of time. It is obvious that students in this case from BBM101, prefer to view than participating in discussions especially during the weekend of their tutorials.

Thus, absolute numbers do not tell the story on LMS utility. We are more concerned with the quality of use with the LMS.

Figure 7: Frequency of Student and Teacher Activity over time

Figure 8: Frequency of Student Viewing and Posting over time

The graph in Figure 9 shows a more detailed perspective of the tutors’ activities in terms of views and postings to forums over a period of time. It is obvious that tutors, in this case for BBM121, showed responsibility in monitoring the forums in their classes and participating whenever it matters in the discussions.

Figure 9: An example of active participation and posting for a course over time

Figure 10: A non-parallel frequency of posting by students and tutors

A more interesting graphical display is when the student and teacher posts are not parallel over time! Figure 10 illustrates the different frequencies of postings by students and tutors as seen in the course BBM 101. This could come about from delayed turnaround time taken by the tutors to respond all of their forum postings for the students. But, there are a few times where the line intercepts which can be interpreted as both tutors and students having the same number of posts during those specific times.

It must be noted that on a per view basis and subsequent participation through postings, the number of participation from students always fall to single or double digits revealing that while we may have a large number of students for a course and potentially big numbers of viewers, the probability of converting viewers to participants is usually around 20-25%. Also as surmised, viewer-participant will only act if the threads are interesting and rewarding in terms of enriching their experience. Then we also note that a forum chat tends to be rather ‘lean’ in that when once a query or remark is posted and if this is similar to what a viewer-participant intends to post, there is no possibility that that student will ‘re-post’ the query or remark.

6. Threads

The above discussions are based on the frequencies of activities or posting or viewings. These are only a set of numbers that are generated over a specific time frame, but they do not tell us the quality of the activities. To do this we need to look at the threads which are related to the contents of the postings in the forums of the course, or in the chat facilities in the sidebars of the courses.

Threads or commonly known as discussion threads are seen as the ‘life’ supporting interactions amongst users in a web-environment as these threads are more public and almost instantaneous (if the student is actively responding) compared to a time-delayed email that is mostly private in nature. For us at WawasanLearn, we need to distinguish threads that are in the open public chat rooms and those that are initiated by the course coordinator or tutor in the forum discussion platforms.