Usability Issues in ICT Assisted Learning

Usability Issues in ICT Assisted Learning

Mikovec,Z., Cmolik, L., Slavik,P.

Czech Technical University in Prague,

Department of Computer Graphics and Interaction

Karlovo nam.13,

121 35 Praha 2

Czech Republic

E-mail:xmikovec|cmolikl|

Abstract

One of key problems that users meet when interacting with any software product is easiness of communication with such a product – we speak about usability issues. This problem has not been addressed properly in case of software products used in ICT assisted learning. In this paper we will deal with usability issues in this field. There are various systems that support ICT supported learning like e-learning, m-learning, t-learning etc. We will present experience with usability testing in t-learning (iDTV – Interactive Digital Television) environment. The topic that was taught in the iDTV learning module was cultural heritage. This topic offers a possibility to use various media (text, pictures, videos etc.) together with different ways of interaction. In such a way the requirements for usability in this environment are rather complex. The paper describes the usability issues handled in individual stages of the learning module development. Description of experiments and the results obtained will be presented in the paper. Last but not least also set of recommendations (derived from findings we discovered during usability testing) that led to the improvements of this particular user interface will be presented.

Introduction

Various forms of learning supported by information technologies of all kinds exist for decades. Many approaches have been developed that define methodologies how to create content of courses, how to evaluate learner’s performance and how to obtain feedbacks that could be used for course improvement. These approaches are heavily influenced by appearance of new technologies that allow to develop new methods of the course delivery to the learners.

The individual technologies offer large variety of ways by means of which the information are delivered to the learners in more or less attractive ways (video, 3D animation, virtual reality etc.). The common denominator of these ways (through which the information is delivered) is interaction between the (computer) system and the learner. Up to now only little attention has been paid to usability issues concerning educational programs. In this paper we will deal with usability in standard way – roughly speaking the usability can be measured as the easiness the user can communicate with program [2].

Usability in the learning environment has several specific features like specific characteristics of individual groups of learners (kids, teenagers, seniors etc.), specific characteristics of devices that are used for education (notebook, PDA, TV set etc.) and also environment where the educational process takes place (home, public transport, school etc.). All these features (and some others too) influence the comfort of the user during communication with the system. In case when this comfort is low the user must concentrate himself/herself on communication with the system and not on the learning stuff that is delivered (what can substantially decrease the efficiency of the learning process). In other words the main goal of usability testing is evaluation of user interface by means of which the user communicates with the program.

Figure 1 – TV remote control.

Our research was done in the field of iDTV (Interactive Digital TV) [1]. Here we had to develop special user interface where the user communicates with the course by means of remote control – see Figure 1. It is obvious that the possibility for interaction is much more limited than in the case of traditional interaction devices like keyboard, etc. [3]. There are several ways of interaction like cursor movement, direct interaction, etc. In such a way we can interact with objects on the screen (e.g. menus) – see Figure 2. This limited way of interaction puts high requirements on the quality of the design of the user interface. This means that usability testing should be involved in all phases of the user interface design and implementation.

Figure 2 – The user interface of iDTV course

Usability testing

There exist standardized methods for usability testing that are used for testing general software products [5]. In this paper we will describe how these general methods were used for usability testing of educational programs. The guidelines developed for this type of programs were applied at CTU Prague for usability testing of educational programs in iDTV (Interactive Digital TV) environment. Special attention was dedicated to the testing of specially developed user interface for iDTV environment. Such an interface was based on standard remote control for TV (the users were familiar with such a device) where some specific interaction features were added [4].

The testing consisted of two phases. In the first phase the qualitative test on a group of 7 participants was performed. Based on the test results of this first phase the user interface was improved with the intention to minimize influence of the usability issues of user interface on the qualitative test (second phase).

In the second phase the quantitative test on 37 participants was performed with the aim to perform the statistical analysis of the results obtained(see Table 1 and Table 2). In this case we had to concentrate ourselves to two main aspects: low familiarity with modern technologies in general, and health problems related to their age (visual and hearing impairments, etc.). Participants of the testing were divided into two groups. Participants in both groups filled two identical questionnaires; the first was filled before the participant saw the course and the second after the course. Then the difference in number of right answers can be used as a measure of gained knowledge. The experiment was organized in such a way that the short term memory of the participants was used.

The participants in the first group (see Table 1) were not allowed to interact with the course (e.g. answer the questions – the traditional passive TV watching was used in this case) in contrast to the participants in the second group (see Table 2) who were allowed to interact with the course. Thus the difference, if any, of gained knowledge between the groups may indicate the influence of interactivity on the learning process.

participant # / 1 / 2 / 3 / 4 / 5 / 6 / 7 / 8 / 9 / 10 / 11 / 12
correct answers before / 5 / 4 / 2 / 4 / 2 / 4 / 5 / 3 / 4 / 6 / 3 / 3
correct answers after / 7 / 9 / 9 / 7 / 8 / 8 / 9 / 8 / 9 / 8 / 6 / 6
improvement / 2 / 5 / 7 / 3 / 6 / 4 / 4 / 5 / 5 / 2 / 3 / 3

Table 1. Results of participants who did not interact with the educational course

participant # / 1 / 2 / 3 / 4 / 5 / 6 / 7 / 8 / 9 / 10 / 11 / 12 / 13 / 14 / 15 / 16 / 17 / 18 / 19
correct answers before / 2 / 2 / 4 / 2 / 5 / 3 / 3 / 1 / 4 / 2 / 4 / 4 / 1 / 2 / 5 / 3 / 7 / 6 / 5
correct answers after / 7 / 7 / 4 / 5 / 7 / 5 / 9 / 2 / 4 / 7 / 5 / 9 / 9 / 9 / 9 / 6 / 9 / 8 / 9
improvement / 5 / 5 / 0 / 3 / 2 / 2 / 6 / 1 / 0 / 5 / 1 / 5 / 8 / 7 / 4 / 3 / 2 / 2 / 4

Table 2. Results of participants who did interact with the educational course

Results

The collected data were statistically processed with the following results:

The number of right answers after the course was significantly bigger than the number of right answers before the course in both groups. That is that participants in both groups have gained knowledge during the course. The difference of gained knowledge in the groups is not significant. Therefore we cannot say what influence has the interactivity on the learning process. Only an assumption can be made.

The mean value of samples in the first group (4.083) is bigger than mean value of samples in the second group (3.421). The variance of the samples in the first group (2.447) is lesser that variance of samples in the second group (5.257). From these facts we can make assumption that the interactivity was disturbing for the participants and they did not pay enough attention to the content, because they had to focus also on the interaction itself.

The main finding from the experiment is that the quality of user interface plays a significant role in the quality of the learning process. This means that the poor user interface occupies user’s mind and the remaining mental capacity is not in general sufficientfor problem solving. This is especially obvious in situations when the problem (e.g. answering question) should be solved within limited amount of time.

In another set of experiments the users worked with two kinds of user interfaces. The first user interface was designed in an intuitive way. The users performed tests with this user interface and several flaws were detected in the user interface design. In the design of the second user interface these findings were taken into account.

The improved user interface was much easier to use for the users than the first one (both from the point of easiness of use and from the point of time necessary to perform the task). Our hypothesis (derived from these experiments) is that the quality of the user interface in t-learning courses can influence the quality of the learning process. To prove the correctness of this hypothesis will require extensive experiments. These experiments should include various learning topics (history, physics, etc.) with conceptually different user interfaces.

Conclusion

In this paper the set of experiments was described. These experiments have shown that the quality of user interface influence the efficiency of the learning process. The user attention is divided between the learning process and the interaction with the user interface. The poor quality of the user interface leads in poor performance of the user in the learning process. The role of the user interface in the learning process was shown in the first experiment. The role of the quality of the user interface in the user mental load was shown in the second experiment. The future work should be concentrated on experiments that interconnect findings from both types of mentioned experiments (influence of user interface on the learning process and mental load when using various user interfaces).

Acknowledgements

The research is conducted within the framework of the ELU project (Enhanced Learning Unlimited) and is funded by European Commission within FP6 (IST-4-027866). See

This research has been partially supported by MSMT under research program MSM 6840770014.

This work has been partly supported by the MSMT under the research program LC-06008 (Center for Computer Graphics).

References

1. Bates, P. J., 2003. A Study into TV-based Interactive Learning to the Home. Available at:

2. Nielsen, J. 1993. Usability Engineering. Academic Press, London.

3. ELU project. Enhanced Learning Unlimited. Project funded by European Commission within FP6 (IST-4-027866). See

4. Cmolik, L., Mikovec, Z., Slavik, P. 2008: Prototyping Tool for T-learning Courses. To appear in proceedings of the Didmattech 2008 conference.

5. Kuniavsky, M. Observing the User Experience 2003. Morgan Kaufmann Publishers, San Francisco.