The effects of individual differences and visual instructional aids on disorientation, learning performance and attitudes in a Hypermedia Learning System
Abstract
Research suggests that certain visual instructional aids can reduce levels of disorientation andincrease learning performance in, and positive attitudes towards, HLS for learners with specific individual differences. However, existing studies have looked at only one or two individual differences at a time, and/or considered only a small number of visual instructional aids. No study has considered the impact of the three most commonly studied individual differences – cognitive style, domain knowledge and computer experience – on learning performance, disorientation and attitudes in a HLS incorporating a full range of visual instructional aids. The study reported here addresses this shortcoming, examining the effects of, and between, these three individual differences in relation to learning performance, disorientation and attitudes in two HLS versions: one that incorporated a full set of visual instructional aids and one that did not. Significant effects were found between the three individual differences with respect to disorientation, learning performance and attitudes in the HLS that provided no instructional aids, whereas no such effects were found for the other HLS version. Analysis of the results led to a set of HLS design guidelines, presented in the paper, andthe development of an agenda for future research. Limitations of the study and their implications for the generalizability of the findings are also presented.
Keywords: Hypermedia Learning; Individual Differences; Visual Support; Disorientation, Learning Performance; Attitudes
- Introduction
Hypermedia Learning Systems (HLS) are being increasingly used in Higher Education [55] to support students’ access to learning material in a flexible way. A defining feature of HLS is that the learning material is presented using a non-linear structure [29], allowing students to determine their own path through the material [4]. Allowing learners to decide the sequence in which they encounter the learning content has been suggested to offer improvements in learning and cognitive flexibility [53]. However, some users have difficulty in navigating through HLS to find the information that they need and, as a result, experience disorientation[6]. A consequence of disorientation is that learners can miss at least some of the relevant content in the system, which may hinder their learning performance[36]. Performing less well may lead to these learners having a negative attitude towards HLS and also to have less interest in learning using these types of learning system [22].
Research findings suggest that not all learners are comfortable using, satisfied with or learn effectively from HLS, implying that the value of HLS varies depending on the individual and may be influenced by various characteristics of the learner [26]. This means that the individual differences that these characteristics represent become important when designing and developing HLS. A range of studies have looked at the relationship between individual differences and student learning in HLS, with the results tending to show that individual differences, particularly cognitive style (most often examined in terms of Field Dependence/Field Independence), domain knowledge and computer experience, influence learners’ levels of disorientation, learning performance and attitudes in HLS. Though widely studied individually or in pairs, the influence of, and relationships between, all of these individual differences in relation to disorientation, learning performance and attitude have not been looked in a single study.
It is argued that supporting individual differences in HLS to reduce learners’ disorientation would be helpful, as it can improve learning performance and increase learning satisfaction [51]. A common way to reduce disorientation is to provide instructional guidance, in the form of visual navigational aids (e.g., maps) and a set of visual cues (e.g., breadcrumbs, highlighting of context, pagination and so on), within the HLS. The relationship between individual differences and such instructional guidance has been explored in many studies, yet there are no studies that have integrated a map and these visual cues in a single HLS.
This paper reports a study that seeks to address these gaps in the research literature through by examining the effects of three individual differences (cognitive style, domain knowledge and computer experience) on disorientation, learning performance and attitudes in HLS with and without instructional guidance in the form of visual navigational aids and a set of visual cues. The paper begins by reviewing relevant literature related to individual difference and HLS use, in order to identify and to frame three research questions that will address these gaps. The paper then sets out the methodological approach to exploring the research questions, introducing the empirical study, its research design, sample, the range of materials and instruments used, and the detailed experimental procedure. The paper then presents the results of the study and analyses the resulting data in order to provide answers to the three research questions. The paper ends by framing a set of design guidelines from the analysis of the data and by setting out directions for future work.
- Background
As the World Wide Web (WWW) becomes ever more widely used as an educational platform [55], HLS are gaining increased attention from researchers [5, 31]. One of the major reasons for moving from traditional classroom-based learning to offering instruction through the use of HLS is that the latter can present learning material in a non-linear structure [10]. Such non-linearity affords learners greater flexibility in navigating the learning content and allows them to choose their own paths through it to meet their learning goals [4]. Additionally, non-linearity allows learners to access and sequence information in accordance with their individual needs [25]. Furthermore, allowing learners to have control over their learning may also make them motivated to learn, improving their learning performance and cognitive flexibility [53].
However, the flexibility offered by HLS may cause problems for some users. It has been argued that not all users can ‘develop’ their own learning paths effectively to achieve their learning goals when using HLS [27], and many studies have shown that users may experience disorientation – reflected as questions of ‘where am I?’, ‘where have I been?’ and ‘where can I go next?’ – when seeking to navigate through HLS [6].
Some studies have suggested that learners who encounter higher levels of disorientation may, in turn, perform less well in learning tasks [36]. Separately and in combination, disorientation and poorer performance in learning tasks may have an impact on learners’ attitudes towards HLS. Dringus [22], for example, argues that when learners experience disorientation in HLS, and are hindered in their learning performance, there have an increased chance of showing negative attitudes towards the non-linear learning environment. As a result, such learners may feel less motivation to learn using HLS.
The range of reactions to being given freedom in terms of navigation may be explained by the different characteristics that learners possess, meaning that the individual differences that these characteristics represent are critical for effective HLS’ design. It has been suggested that individual differences such as cognitive style [15], domain knowledge [8], and computer experience [55] are the most commonly studied in research related to student learning and HLS use. Each of these individual differences will be briefly introduced in the context of research into HLS.
The ways in which an individual thinks, memorizes, perceives, organizes, processes and presents information is often referred to as cognitive style [47]. Among the different dimensions of cognitive style that have been studied to date, Field Dependence (FD) and Field Independence (FI) are often argued to be of interest, especially with respect to research that is related to HLS [12]. FI learners tend to rely on internal references, adopt an active approach to learning and process information using an analytical approach. Conversely, FD learners tend to rely on external references, adopt a passive approach to learning and accept information in exactly the way it is presented to them [17, 58]. Witkins et al. [58] Group Embedded Figure Test (GEFT) and Riding’s [46] Cognitive Style Analysis (CSA) are two common instruments used to identify a learner’s cognitive style.
The research related to cognitive style and HLS suggests that FI learners: prefer being given high levels of freedom of navigation; experience less disorientation in HLS; perform well in learning tasks; and have a positive attitude towards HLS compared to FD learners[16, 32, 54]. One explanation of FI learners’ performance in HLS is that they are able to follow a restructuring approach more easily because they are internally directed and tend to adopt an active approach to learning; and they can extract relevant items from within the complex context offered within complex content, such as that offered in HLS, because they are more analytical. In contrast, FD learners prefer to follow the structure of the learning material because they are externally-directed and tend to adopt a passive approach to learning; they also have difficulties extracting the relevant items within the complex context because they are less analytical [17, 59].
Research into domain knowledge, mostly suggests that domain knowledge influences the degree of disorientation and learning performance in HLS, with low domain knowledge (novice) learners experiencing higher levels of disorientation and performing less well in HLS than high domain knowledge (expert) learners [7, 23, 35, 57]. A general conclusion drawn in studies in this area to explain this finding is that, novices are unfamiliar with the subject content, which makes it difficult for them to impose a meaningful conceptual structure on the content compared to experts[13].
In terms of the research related to computer experience, studies suggest that compared with those with lower levels of computer experience, learners with high levels of computer experience: prefer the non-linear pathways that are normally offered in HLS; navigate effectively; take fewer steps to reach the information they need in the tutorial; browse more pages and are able to reach more detailed levels of the subject content; enhanced their time efficacy; and overall, perform well in learning tasks in the HLS [30, 37, 52, 55]. It is argued that this is because a well-developed understanding of different computer applications enables these learners to better navigate through the HLS to achieve their learning goals. In contrast, the lack of skills related to the use of computers and their applications makes it difficult for those with low levels of computer experience to successfully navigate through HLS to find the information that they need[40, 52].
It is suggested that reducing the levels of disorientation experienced by learners with different characteristics may improve their learning performance and, in turn, may show more interest in learning using HLS [12, 13]. To reduce learners’ disorientation, the use of instructional aids, such as maps which support visual navigational, has been suggested [5, 8, 13, 14], and a number of studies have examined the effects of both maps and individual differences (such as those considered in this paper) on learners’ levels of disorientation, learning performance and attitudes in HLS [3, 8, 33, 34, 45]. The results from these studies suggest that when maps are provided in HLS, learners’ levels of disorientation are decreased and learning performance and positive attitude are increased. However, these studies have not considered the effects of all of the three individual differences – cognitive style, domain knowledge and computer experience – in combination with maps in HLS. Rather, they have tended to examine the effects of only one or two of the individual differences presented in this study. Studies which examine the effect of maps in relation to a wider set of individual differences may help to determine whether maps reduce disorientation and increase learning performance and learning satisfaction for all users with different combinations of the individual differences, or whether maps create problems for specific groups.
In addition to maps, previous studies suggest that visual cues – such as breadcrumbs [1, 44], graphic visualizations[39, 42], history-based mechanisms [24], context highlighting [20], page labels [18], different colors link [42] and link annotations [9] – can also reduce disorientation and enhance efficiency of learning performance and satisfaction in HLS [19, 49].
Despite the fact that these studies have explored the same set of individual differences considered in this paper and have considered a range of these visual cues, an important gap remains. There are no studies where cognitive style, domain knowledge and computer experience have been considered together in relation to all of the visual cues identified earlier within a single HLS. This implies that the combined effects of these three individual differences and all of these visual cues on learners’ disorientation, learning performance and attitudes in HLS have not been fully examined. Though methodologically complex in terms of study design and subsequent analysis, addressing this gap may provide results which can help designers and developers to gain a better understanding of the relationships between HLS instructional aids (in the form of visual cues), individual differences and learner disorientation, performance and attitude. This leads to the framing of the three research questions addressed in the study reported in the remainder of this paper:
- Research Question 1:
What are the effects of and between cognitive style (FD/FI), domain knowledge, and computer experience on learners’ levels of disorientation when using a HLS that includes a map and the defined set of visual cues and when using a HLS without any instructional aids?
- Research Question 2:
What are the effects of and between cognitive style (FD/FI), domain knowledge, and computer experience on learners’ learning performance when using a HLS that includes a map and the defined set of visual cues and when using a HLS without any instructional aids?
- Research Question 3:
What are the effects or and between cognitive style (FD/FI), domain knowledge, and computer experience on learners’ attitudes when using a HLS that includes a map and the defined set of visual cues and when using a HLS without any instructional aids?
- Methodology
This section describes the research methodology that was used to address the three research questions. An explanation of the research design is given, followed by a description of the sample, the materials and instruments used in the study, the experimental procedure and the approach to data analysis.
3.1 Research Design
To answer the three research questions that were proposed, this study adopted an experimental research approach [48],in which a set of independent variables and dependent variables were identified and used. The independent variables were the HLS and the users’ individual differences (cognitive style, domain knowledge and computer experience). With respect to the HLS, two versions, one without instructional aids and one with visual instructional aids (in the form of a map and a set of visual cues) were required. The dependent variables in this experimental study were learning performance, disorientation and attitudes towards the HLS. A between-subjects design was used in this study, with one half of the sample using the HLS that provided no instructional aids and the other half of the sample using the HLS containing the set of visual instructional aids defined above. It is acknowledged that this research design is complex and has a wider range of variables than have been used in other studies in the field. This brings significant issues in terms of analysis of the data and the implications that can be drawn from them. These issues are discussed, and reflected on, in later sections of the paper.
This study also aimed to gather detailed user information on individual differences, learners’ attitudes, feelings and preferences, their experience of disorientation, and their interaction behavior with respect to the HLS that they used. To support the experimental study, a descriptive study was also employed (using a qualitative approach), in which learners were observed, surveyed and interviewed [41, 48]. However, for reasons of space, this paper will consider only the experimental study and its results.
3.2 Sample
The sample was drawn from university students across London, UK. University students were considered to be suitable participants because, as mentioned in sections 1 and 2, this study was concerned with research related to students and HLS in Higher Education. Recruitment of the sample was also supported by the existence of channels through which we were able to contact a large number of students at universities in the London region. Though the choice of university students as participants can be seen as part of a purposive sampling strategy, the restriction of the recruitment of participants to those from the London region introduce a ‘convenience’ characteristic to the sample which is important to acknowledge, and which will be returned to in section 7 when the implications of the findings with respect to the study’s limitations, are discussed.