Research Minor Thesis
Submitted for undergraduate honours degree
Bachelor of Information Technology (Honours)
School of Information Technology and Mathematical Sciences
The Role of Virtual Internet Routing
Lab in Network Training
Email: herbm001(at)mymail.unisa.edu.au / Email: grant.wigley(at)unisa.edu.au
Author / Supervisor
2 December 2015
Keywords: Cisco, VIRL, education, practical learning, remote, laboratories
Abstract
Learning computer network concepts requires more than theoretical knowledge alone. The learner also needs to develop a contextual understanding of how real world networks operate. According to the literature, this is best achieved through a hands-on practical approach using physical equipment. However, construction of a remote lab is often not feasible due to costs and maintenance. In this work, we reviewed a number of potential platforms that could be used for education and we attempt to demonstrate, from the literature, why these platforms may not be ideal for network education. We investigated Virtual Internet Routing Lab (VIRL) as a possible education platform for learning advanced network concepts. We did this by undertaking a user study on a number of people at the University of South Australia by measuring their perceptions, their performance and their confidence and knowledge both before and after using Virtual Internet Routing Lab. We conclude that Virtual Internet Routing Lab is a more suitable platform than software education platforms when learning troubleshooting, debugging, spanning-tree port fast concepts and the Hot Standby Routing Protocol (HSRP).
Declaration
I declare that this thesis is, to the best of knowledge, original work and has not been submitted for any other assessment at any education institution, except where clear acknowledgement of sources is made.
Bradley Mark Herbert
Acknowledgements
We acknowledge for the following persons for their assistance in helping to prepare this thesis. This list is not exhaustive.
Dr. Grant WigleyUniversity of South Australia
Program Director
Grant was the supervisor for the project, issuing advice on a weekly basis
/ Mr. Michael Head
University of South Australia
Learner Advisor
Provided useful advice on the structure and formatting of the thesis. / Dr. Ross Smith
University of South Australia
Researcher
Ross provided useful advice on how to conduct studies and the processes to ensure ethics approval.
Robert J. Mislevy
University of Maryland,
EDMS, Benjamin 1230-C, College Park, MD 20742
Robert J. Mislevy was a co-author, who kindly sent me a copy of his paper, which otherwise I would not have been able to access. The material was useful.
Contents
1.Introduction
1.1Problem Statement
1.2Research Outline
1.3Thesis Structure
2.Literature Review
2.1 Review of Education Research
2.1.1Current Network Education Methods
2.1.2The role of Kolb’s Learning Model in network training
2.1.3The Impact of network education in industry
2.1.4Learning Patterns of Students
2.2Network Platforms in network education
2.2.1Realism
2.2.2User Interface and Visualisation in network education
2.2.3Management of Platforms
2.2.4Perceptions
2.2.5Collaboration
2.2.6Design and Complexity of Platforms
2.3Conclusion
3.Methodology
3.1Research Questions and Hypothesis
3.2Experimental Design
3.2.1Experiment Overview
3.2.2 Detailed Process
3.2.3Recruitment of Participants
3.2.4Justification
3.3Data Analysis
3.4Alternative Approaches
4.The Case for Investigating Cisco VIRL
4.1Overview of Virtual Internet Routing Lab
4.2Benefits of this research
4.1.1Gaps in the literature
4.1.2 Uncertain speculation due to lack of research
4.3Conclusion
5.Experiment and Results
5.1Experiment
5.1.1Design
5.1.2Setup
5.1.3Implementation
5.2Results
5.2.1Knowledge Statistics
5.2.2Confidence Data
5.2.3Analysis of Perceptions
5.2.4Analysis of Performance
5.3Summary
6.Discussion
6.1User Perceptions and Feedback
6.1.1System Performance
6.1.2User Interface and Topology
6.1.3Comparison to other platforms
6.2Influencing Factors and Other Considerations
6.2.1Human Factors
6.2.2Limited Scope
6.2.3Less than ideal sample size
6.2.4Selection Bias
6.3Research Question Analysis
6.3.1 Are the visualisation tools, lacking from Cisco VIRL necessary for understanding of computer networking?
6.3.2What are the perceptions of students who use Cisco VIRL?
6.3.4Does the additional features in Cisco VIRL help with understanding?
6.3.5Leading Research Question
6.4Key Findings
7.Conclusion
References
Appendices
Appendix 1 – Raw Data Results
Knowledge Questionnaires
Participant Self-Rated Confidence Data
Participants’ Perceptions of VIRL
Participant’s Self-Rated Learning Patterns
Practical Exercise Performance and Perceptions
Data Variation and Analysis
Appendix 2 – Pre-Lab Questionnaire
Appendix 2 – Practical Booklet
Appendix 3 – Post Lab Questionnaire
List of Figures
Figure 1 Packet Tracer Incorrect Notification
Figure 2 VIRL Correct Notification
Figure 3 Packet Tracer User Interface 1
Figure 4 Sample Packet Tracer Topology
Figure 5 Packet Tracer Sample Log
Figure 6 CORE Sample Topology
Figure 7 CORE Sidebar
Figure 8 CORE full UI
Figure 9 VIRL Main Pane
Figure 10 Sample Link between two nodes
Figure 11 NetLab UI - Example Lab Reservation System
Figure 12 Example Booking System for NetLab
Figure 13 NetLab UI - Fixed Lab Topology
Figure 14 NetLab UI - Example Router Console
Figure 15 VIRL Topology with External Network
Figure 16 NetLab Chat UI
Figure 17 Cisco VIRL Pane
Figure 18 Architecture Diagram
Figure 19 Network Topology For Experiment
Figure 20 Experiment Console Setup
Figure 21 Results - Pre-Knowledge Questionnaire Answers
Figure 22 Comparison of Knowledge Averages
Figure 23 Results - Post Knowledge Questionnaire Answers
Figure 24 STP Scores for Both Questionnaires
Figure 25 Results - Confidence Prior to the Lab Activity
Figure 26 Results - Confidence in Networking Averages
Figure 27 Results - Confidence After the Lab Activity
Figure 28 The Average Confidence For Each Participant
Figure 29 Results: VIRL Perceptions By Category
Figure 30 Results - Average Perceptions Data
Figure 31 Results: Exercise Data
Figure 32 Rated Performance Averages
Figure 33 Most Recurring Values for Performance
Figure 34 Results - Pre-Knowledge Questionnaire Answers
Figure 35 Results - Post Knowledge Questionnaire Answers
Figure 36 Comparison of Knowledge Averages
Figure 37 STP Scores for Both Questionnaires
Figure 38 Results - Confidence Prior to the Lab Activity
Figure 39 Results - Confidence After the Lab Activity
Figure 40 Results - Confidence in Networking Averages
Figure 41 The Average Confidence For Each Participant
Figure 42 Results: VIRL Perceptions By Category
Figure 43 Results - Average Perceptions Data
Figure 44 Results: Learning Patterns
Figure 45 Self-Rated Learning Patterns of Participants
Figure 34 Results: Exercise Data
Figure 47 Results: Practical Exercise Completion Percentages
Figure 48 Results: Percentage of Staff
Figure 49 Results: Learners Lab Completion Percentages
Figure 50 Results: Performance in Debugging Bar Graph
Figure 51 Rated Performance in various concepts
Figure 52 Observed Personality During Lab Bar Graph
Figure 53 Rated Performance Averages
Figure 54 Results: Number of Problems Troubleshooted
Figure 55 Most Recurring Values for Performance
Figure 56 Variation of Performance Bar Graph
Figure 57 Most Recurring Knowledge Scores
Figure 26 Left Computer Screen
Figure 27 Right Computer Screen
List of Tables
Table 1 Summary of Learning Styles
Table 2 Comparison of User Interfaces
Table 3 Comparison of Platforms (Management)
Table 4 Comparison of Simulator Features
Acronyms
Acronym / WordASA / Adaptive Security Appliance
BGP / Border Gateway Protocol
CORE / Common Open Research Emulator
CPU / Central Processing Unit
HSRP / Hot Standby Routing Protocol
IP / Internet Protocol
IT / Information Technology
LTS / Long Term Support
OSI / Open Systems Interconnection (Model)
Ospf / Open Shortest Path First
RAM / Random Access Memory
SSH / Secure Shell Host
STP / Spanning Tree Protocol
svn / Apache Subversion
TCP / Transmission Control Protocol
unisa / University of South Australia
VIRL / Cisco Virtual Internet Routing Lab
VLAN / Virtual Local Area Network
VM / Virtual Machine
Glossary
Word / MeaningAbstract concepts / Concepts that cannot be demonstrated in a practical way. Usually, mathematical problems, or in the case of networking, packet movement and understanding.
ABSTRACT CONCEPTUALISATION / A stage of Kolb’s learning model that involves the memorisation and interpretation of facts.
Active experimentation / A stage of Kolb’s learning model that generates new knowledge by applying existing knowledge to new problems; design, planning, troubleshooting.
Cloud Computing / Virtual computers that are started as needed, and which may be stored on a server whose hardware setup is not known.
Concrete experience / A stage of Kolb’s learning model that involves stimulating new knowledge through exposure to a new concept.
Emulation / Software that converts instructions designed on one type of CPU to that of the host system, enabling the program to run.
hOT STANDBY ROUTING PROTOCOL / An advanced Cisco networking service that runs on routers and switches that has its own MAC address. Provides redundant gateways by allowing a single router to represent a group of routers. Automated fail over.
KOLB’S LEARNING MODEL / A learning model often used in technical subjects like networking to ensure sound learning. Consists of four stages; Concrete Experience, Reflective Observation, Active Experimentation and Abstract Conceptualisation.
Network/networking / The interconnection of hardware (or virtualised) devices together to facilitate the sharing of information. The configuration of such networking and the knowledge needed to build and maintain such infrastructure.
NODE / A node is a single unit that represents a network device, usually a router, switch or ASA in a topology.
Physical equipment / Actual hardware that can be touched and experienced by the learner, such as real routers and switches cabled together.
Reflective observation / Receiving feedback about the learner’s progress. Facilitated by teacher providing feedback and a practical experience where mistakes are recognised. A stage of Kolb’s learning model.
Routing / A router making decisions where to send IP packets based on source/destination IP addresses. Rewrites MAC addresses.
sPANNING-TREE PROTOCOL / A network protocol that shutdowns links on a switch intended to be used as backups to prevent loops from occurring.
Switch / A device specially designed to forward frames on a computer network. A smart hub.
Switching / A switch making decisions on where to send traffic based on source/destination MAC addresses.
Troubleshooting / A process undertaken by a person to fault-find problems in the network, identify poor network configuration, possible security flaws and may include the steps to mitigate any problems.
Virtualisation / Virtualisation is the allocation of computing resources to so-called guest operating systems to allow multiple operating systems to run on a single machine.
Visualisation / Understanding a concept through seeing different colours and imagining in your mind how it worked if it was drawn on paper.
1
1. Introduction
Education institutions aim to deliver high quality technical education to prepare the student for transition into industry [1][2][3][4]. It is argued that training students in computer networking requires more than knowledge alone [5][6][7]. It also requires a hands-on experience with managing and configuring numerous network equipment. However, in remote locations including; prisons, mobile buses or in third-world countries, there is often no infrastructure available to provide a practical networking environment [4][7]. Even in western countries, for instance, Australia, education institutions often cannot justify the costs of establishing a network-training environment, especially when there are low enrolments[3]. Even when this is possible, the training labs are often small, unrealistic and waste resources, providing a limited and unrealistic worldview of how real networks operate [4][8][9]. Physical networks are often hardwired, so students may not have access to change or modify the topologies in use, preventing them with experimenting with alternative designs, limiting their experience and understanding in a negative fashion [6][8].
The use of software-driven emulators and simulators such as GNS3 and Packet Tracer are now common in education institutions worldwide due to their low cost, ease of use and flexibility [3][10][11][12]. User studies and surveys undertaken on students and staff who use Packet Tracer suggest low exam grades, despite use of the tool [12]. In a study by Ceil Goldstein [13], Packet Tracer is seldom used to facilitate active learning.
1.1Problem Statement
In this work, we reviewed a number education platforms, both hardware and software driven, that could potentially be used in a network training environment to facilitate the teaching and learning of advanced network concepts. We argue that these platforms, including Packet Tracer, GNS3 and other less common platforms such as Bosten NetSims, iNetSims, ns-3 and CORE are suboptimal for network education. We argue that realism of the network environment is an essential aspect to facilitate learning based on experience. We derive this from several papers [13][14][15]. In our view, Packet Tracer is a very good tool for teaching computer networks but it cannot provide the full realistic experience. For example, Packet Tracer lacks features to interface with other real-world network diagnostic tools such as Wireshark and does not generate real traffic, potentially resulting in a distorted view of networking [3][9]. Moreover, if visualisation of network concepts is so important to understand computer networks, it puts students with vision impairments at a disadvantage [16]. Real equipment, on the other hand, is expensive and not necessarily accessible to external students [3][8]. GNS3, a free network emulator is intensive on resources and emulation of switches is suboptimal, if not, impossible [11]. This limits the realism offered by GNS3.
As studies found in the literature seem to suggest, existing platforms may not be realistic enough to provide a realistic experience for students learning computer networking [8][9]. Moreover, real equipment is costly and, unless explicitly configured, not accessible to external students [3]. Emulators, such as GN3 greatly depend on being able to emulate full embedded operating systems, but propriety restrictions may make this infeasible, if not, illegal [9]. While software tools such as Packet Tracer can help visualise certain network concepts, this may not help all kinds of learners, especially students with vision-impairments [10][16][17]. Since 2007, cloud computing and virtualisation has been used in education to address some of these issues, in particular, to address the issue of lack of realism in software solutions [14][18]. Virtualisation can also reduce maintenance by provisioning resources on demand, eliminating the requirement to have dedicated training networks setup. In impoverished nations, training environments can be provisioned in a cloud environment to accommodate the lack of infrastructure that is available. Now here is the problem, virtualisation is typically suited to allow one operating system to run on another operating system. It is typically not designed to allow vendor-specific embedded operating systems, for instance, the Cisco IOS to run in a virtual machine. A review of the tools in the literature found that, while GNS3 can emulate some network devices, it is not used in a cloud environment [11]. Recently, in 2014, Cisco released Virtual Internet Routing Lab that has the ability to run full network topologies in a virtualised (or real) computing environment.
Despite this, there is no data available on VIRL from the literature so it is yet unclear as to how potential students would perceive it, or how well, it impacts learning in training environments. Its impact on education, whether positive or negative, has not been empirically tested, according to the literature found. Most of the literature dates back to 2014 or earlier, prior to the release of VIRL, opening up a potential research gap in the literature. The assumptions and limitations of software solutions as noted in the literature may not be valid for VIRL. Any attempt to argue that VIRL is a good or bad tool for education is, at best, speculation due to no research having been done.
1.2Research Outline
Having ascertained a possible research gap in the existing body of knowledge due to the lack of data available on VIRL, this thesis aims to contribute to this research gap by recruiting a group of people with a networking background to evaluate the learning effectiveness of the new platform called Virtual Internet Routing Lab in a modern classroom. We propose an overall research question, which we aim to test.
"What role does Cisco Virtual Internet Routing Lab play in network education environments to help students and trainees understand advanced network concepts?”
First, the participants will be evaluated on their knowledge and confidence in computer networking, prior to undertaking a practical-driven activity using Virtual Internet Routing Lab. Second, their performance will be qualitatively assessed by the investigator, while the participant works their way through a series of structured activities using VIRL. Third, the participant’s knowledge and confidence will once again, be tested and compared with the first evaluation to see how VIRL has (potentially) affected their confidence and knowledge. Fourth, the participants will be asked to rate the tool and provide their opinions on using it.
It is speculated that use of advanced debugging features may help greatly improve troubleshooting and/or improve learning of network concepts. In many cases, these debugging commands are unavailable on software platforms such as Packet Tracer. The data on the effectiveness of the debugging commands is limited, so we propose to integrate the use of the debugging commands into an exercise to be done on VIRL to see how well the debugging commands enhance learning.
It is important to note that the purpose of the research is not to evaluate VIRL in terms of its usability, performance or deployment in a cloud environment, but rather, the purpose is to evaluate, whether or not, this tool facilitates active learning of students. Unlike Packet Tracer, which is designed by Cisco for education purposes, VIRL was not initially designed with education in mind [4][19].It was originally designed as a testing platform for network designers to test a network design, prior to rolling it out on a production network[1].
1.3Thesis Structure
First, we present and review the existing research on the use of various platforms for education and compare each of the platforms in terms of features we perceive as useful such as realism, user interfaces, visualisation, management, perceptions, collaborative and design and complexity, arguing that each platform, negatively impacts education (based on the literature). A review of the literature found no existing work done on the role of VIRL in network education. Second, having found no existing work on VIRL for use in education, we devise a methodology, proposing how we are going to evaluate a group of academic staff and students using VIRL and how their learning will be measured. Third, we outline why it is necessary to undertake research on VIRL, notably because unlike other existing tools, it uses new virtualisation features and in our view, this provides a more realistic network environment. VIRL does not contain the visualisation features available in Packet Tracer, so it remains uncertain as to how VIRL will aid visual learners, who depend on visual cues to understand network concepts [13][20]. It is also uncertain as to how VIRL will be perceived by students and instructors alike. The fact that the student does not physically interact with real equipment might deter students from wanting to use it, or perceive it as being unrealistic. Fourth, we present the results of our study and the implications it has on network education. Fifth, we discuss the results and the outcome of the study, in particular, how VIRL might be used in network training environments. Seventh, we draw conclusions and outline any research gaps and possibilities for future work.