Project Consortium

Done-IT

This document may not be copied, reproduced, or modified in whole or in part for any purpose without written permission from the Done-IT consortium. In addition to such written permission to copy, reproduce, or modify this document in whole or part, an acknowledgement of the authors of the document and all applicable portions of the copyright notice must be clearly referenced. All rights reserved.

This document may change without notice, but consortium members should be informed, and number of version, stage and date should be given.

Project consortium

• Sør-Trøndelag University College, Faculty of Technology, Trondheim, Norway

• Centrum for Flexible Learning, Söderhamn, Sweden
• Petru Maior University of Targu-Mures, Tirgu Mures, Romania
• Magyar Hegesztéstechnikai és Anyagvizsgálati Egyesülés (MHtE), Budapest,Hungaria
• Institut za Varilstvo (IzV), Ljubljana, Slovenia
• HiST Contract Research, Trondheim, Norway

Table of Contents

Done-IT

Introduction

Basic data model

Data formats used in communication

Offline operation - Mobile PeLe Service Unit

Typical scenario

Preparation

Assessment time

Post-assessment time

Post class processing

Testing and evaluation

Design

Use cases

Register to system

Login as teacher

Create new assessment

Edit assessment

(UCT4) Start assessment

(UCT5) Monitor assessment

(UCT6) Close assessment

UCT7 Post assessment activity

(UCS1) Register to the system

(UCS2) Login to system

(UCS3) Start assessment

(UCS4) Respond to assessment

(UCS5) Retrieve assessment feedback

UCS6 Participate in Post Assessment activity

Messages – communication workflow

General Requests

Before Lecture

Assessment Phase

Interactive (SRS) Phase

After Lecture

Scoring Model

Item Level

Section Level

Assessment Level

Normalized Scores

PeLe Service Documentation

Design

RESTful Interface

Authentication

Access Rights

HTTP Status Codes

HTTP Headers

REST Resources

REST Resource Details

Implementation

Package rs.sol.Hist.TestDefinition

Package rs.sol.Hist.Testing

Package rs.sol.Hist.UserManagement

Teacher client documentation

Design

Use cases

UI and user experience

Code architecture - packages

The context

Communication infrastructure

Implementation

The context

hist.net - Communication with the service

hist.model - Data model

The main application DoneiTC.mxml

hist.doneit.datamodels

hist.doneit.events

SessionControlEvent

hist.doneit.gui

hist.doneit.gui.about

hist.doneit.gui.admin

hist.doneit.gui.create

hist.doneit.gui.create.simulate

hist.doneit.gui.end

hist.doneit.gui.general

hist.doneit.gui.itemRenderers

hist.doneit.gui.itemRenderes.adg

hist.doneit.gui.menu

hist.doneit.gui.monitor

hist.doneit.gui.postasess

hist.doneit.gui.start

Student app documentation

Use cases

UI and user experience

Code architecture

Project file structure

Implementation

Data processing

Mobile PeLe Service Unit Documentation

Why

What

How

Index of Tables

Table 1: definitions for identifiers and names...... 40

Table 2: User roles...... 42

Table 3: Access rights matrix...... 44

Table 4: HTTP status codes...... 46

Table 5: Mime-types...... 48

Table 6: Rights for ASI resources based on the role...... 49

Table 7: Custom metadata fields for ASI resources...... 49

Table 8: Custom metadata fields...... 50

Table 9: QTI-XML resources...... 51

Table 10: Service folders...... 52

Table 11: REST resource details...... 52

Table 12: Assessment parameters...... 58

Table 13: Completion state values...... 58

Table 14: Resource state values...... 59

Table 15: Assessment type values...... 60

Table 16: Section type values...... 60

Table 17: User account properties...... 64

Table 18: Attributes of the ASIEntity class...... 66

Table 19: Assoc. ends of the ASIEntity class...... 67

Table 20: Attributes of the ArbitraryData class...... 67

Table 21: Attributes of the DefaultSection class...... 67

Table 22: Assoc. ends of the DefaultSection class...... 67

Table 23: Attributes of the PossibleResponse class...... 68

Table 24: Assoc. ends of the PossibleResponse class...... 68

Table 25: Attributes of the Question class...... 68

Table 26: Assoc. ends of the Question class...... 70

Table 27: Attributes of the Test class...... 70

Table 28: Assoc. ends of the Test class...... 71

Table 29: Enumeration literals of the AsessmentType enumeration...... 71

Table 30: Enumeration literals of the ContentType enumeration...... 72

Table 31: Enumeration literals of the ResponseMultiplicity enumeration...... 72

Table 32: Enumeration literals of the ResponseType enumeration...... 72

Table 33: Attributes of the DefaultSectionCompletionState class...... 74

Table 34: Assoc. ends of the DefaultSectionCompletionState class...... 75

Table 35: Attributes of the DSLifeCycleState class...... 75

Table 36: Assoc. ends of the DSLifeCycleState class...... 75

Table 37: Attributes of the QuestionState class...... 75

Table 38: Assoc. ends of the QuestionState class...... 76

Table 39: Attributes of the SRSSection class...... 76

Table 40: Assoc. ends of the SRSSection class...... 77

Table 41: Attributes of the StudentResponse class...... 77

Table 42: Attributes of the StudentTestRecord class...... 78

Table 43: Assoc. ends of the StudentTestRecord class...... 78

Table 44: Attributes of the Testing class...... 79

Table 45: Assoc. ends of the Testing class...... 79

Table 46: Attributes of the TestingListener class...... 80

Table 47: Assoc. ends of the TestingListener class...... 80

Table 48: Enumeration literals of the ASILifeCycle enumeration...... 80

Table 49: Enumeration literals of the CompletionState enumeration...... 81

Table 50: Attributes of the User class...... 83

Table 51: Assoc. ends of the User class...... 83

Table 52: Enumeration literals of the Role enumeration...... 83

Table 53: Attributes and pins of the CmdCreateUser command...... 84

Table 54: Association ends and pins of the CmdCreateUser command...... 84

Table 55: Attributes and pins of the CmdCreateUsersBatch command...... 85

Table 56: Association ends and pins of the CmdDeleteUser command...... 85

Table 57: Attributes and pins of the CmdEditProfile command...... 86

Table 58: Association ends and pins of the CmdEditProfile command...... 86

Table 59: Association ends and pins of the CmdFetchCurrentUser command...... 86

Table 60: Classes in the hist.doneit.gui.create package...... 113

Table 61: Classess in package hist.doneit.gui.create.simulate...... 118

Figure Index

Figure 1: Post assessment dilemma...... 13

Figure 2: PeLe System overview...... 15

Figure 3: System overview when using Mobile PeLe Service Unit...... 18

Figure 4: Main use cases of the system...... 22

Figure 5: Main communication workflow between system components...... 32

Figure 6: ASI Basic...... 73

Figure 7: ASI Definition...... 74

Figure 8: Testing...... 82

Figure 9: Listener...... 82

Figure 10: User Management...... 87

Figure 11: The current context minimizes the need for complicated dependencies between the components 94

Figure 12: Communication infrastructure...... 95

Figure 13: Context and communication related classes...... 98

Figure 14: Request flows...... 100

Figure 15: Server information...... 101

Figure 16: Login and zero configuration related classes...... 104

Figure 17: Resource definition data model...... 105

Figure 18: Results data model...... 108

Figure 19: Resource state related classes...... 109

Figure 20: About box...... 111

Figure 21: adminCanvas class and related classes...... 112

Figure 22: adminCanvas user interface (design view from IDE) - shows the adminCanvas with the export function and the datagrid displayed. 113

Figure 23: CreateAssessmentWizard classes and their relations...... 114

Figure 24: shows the editAssessmentComponent as seen in design mode in the development environment. 116

Figure 25: shows the EditAssessmentComponent that is connected to the yellow model objects and the EditQuestionItemRenderer and AlternativesItemRenderer and how they are wrapped inside Air components (blue). 117

Figure 26: shows the QuestionItemRenderer displayed in design mode in the developer environment 117

Figure 27: shows the ViewAlternativeComponent and how an hist.model.Alternative is rendered. 118

Figure 28: shows how the ControlComponent, MonkeycageClass, monkeyResponseClass and the HistogramForMonkeycage are related. Communication and control is done through events up to the main application. 119

Figure 29: shows the simulation control dialog...... 121

Figure 30: shows the end interface in “sectionClosed” state where results are displayed from the assessment. 123

Figure 31: shows the class diagram of the ResultsOverview and how that use hist.doneit.gui.monitore.SummaryViewComponent to display results from the model. 123

Figure 32: shows the SessionInformationPanel, that is used to display more detailed information about sessions from a list of sessions. 125

Figure 33: shows the menu with the menu elements and the selector triangle to amplify the current position in the “user flow” of the program 128

Figure 34: shows the MonitorComponent, the two related views, SummaryViewComponent and MatrixViewComponent, related model (yellow) and related item renderers. 129

Figure 35: shows the MonitorComponent when it displays the default MatrixViewComponent. 130

Figure 36: shows the MatrixViewComponent that is inside MonitorComponent...130

Figure 37 shows the SummaryViewComponent with two sections. The default section is the main assessment, and the second chance section is not really used in the application as it is today. To the right, a histogram displays the selected question in more detail. 131

Figure 38: shows the ItemSummaryResponseItemRenderer2 displaying results as percentage bars. 131

Figure 39: shows the component in NoDocument state...... 133

Figure 40: shows the component in DocumentNotReady state...... 133

Figure 41: shows the assessment in DocumentReady state. The Start button is made available 134

Figure 42: shows the component in DocumentUploaded. Normally the user will be redirected to monitoring when the assessment has been started and the user has to deliberately enter the start stated to get this state. 134

Figure 43: relation between the views...... 141

Figure 44: System overview - Use of MPSU with the internal AP enabled...... 145

Figure 45: MPSU architecture overview...... 146

Figure 46: Use of MPSU with one AP/Wireless Router added...... 150

Figure 47: Use of MPSU with multiple AP/Wireless Routers added for a combined wireless network capacity 150

Introduction

The PeLe (Peer Learning Assessment Services) system developed within the frame of the Done-IT project aims to provide the teachers and the students with a set of services that allow information exchange and information aggregation in order to support evaluation activities that promote easy and fast verification and elaboration based learning processes immediately after a test has been done. The main targets are higher education and VET considering both contexts of inside and outside of the classroom.

The PeLe system (PeLe) allows on one hand the students to use their own smart-phones, tablets or portable computers to provide the responses for assessments and post assessment activities and on the other hand allows the teacher to monitor the activity and give verification or elaborative feedback to individual students/groups of students immediately after a test or activity.

This is a key factor helping students to improve their skills by the use of active collaborative supported learning. Students will, when they still remember the questions in the test, learn why the correct answer is correct and why the other ones are incorrect. Thus, mobile technology provides new evaluation and testing criteria within education and training.

The traditional focus when introducing electronic assessment is to save time for correction answers and give immediate feedback on learning. The aspect of this project is turned another way around and focus on the learning perspective. So you have performed an assessment, the students has been working on a problem for a time and given you as a teacher a digitally delivery of the results, you can see the results immediately, what to do next?

If the feedback to you as a teacher is good, dependent upon the results, you can do things in many different ways. With the Done-IT assessment solution our target is to provide the teacher with instant feedback on the status of the assessment. What questions did the students solve correctly and what questions caused more problems. Now the teacher can do the following for each question:

a)Continue as usual and only give the results.

b)Give the students verifying feedback and explain what has been misunderstood.

c)Give the students a hint of what might be the problem for this question, but not the actual solution.

d)Give the students the results, “this is what you voted” and allow the students the possibility to discuss the problem.

e)Allow the teacher to pick one question, as set up in the assessment, and send it out prepared for an SRS session on the question.

f)Allow the teacher to collect several of the questions after a pedagogical treatment, and allow the students to resubmit their answer to parts of the assessment.

In case of c) and d) should the students be allowed to take part of the test again, or should the students after a group / peer discussion be able to renegotiate their response? There are several possibilities that we can allow the teacher to do.

The main advantage of entering the test subject just after the test is first of all, the problem is fresh in mind since the student has just been working with it. The student might have spent time on parts of the assessment but have just not found the right answer. Dependent upon the nature of the subject being thought, there might be just a hint from a peer student or a teacher that might solve the problem.

The system has been developed using an iterative prototyping process. In the design process have been involved the main system stakeholders: teachers and students. The very context of use dictates that a special emphasis must be placed on the human computer interaction aspects. The educational activity must be supported in a transparent and usable way. The system has been evaluated through a number of usability tests that included both expert based evaluation and user testing.

The system is based on a client-server architecture and the communication between the components is over the HTTP protocol. The main components of the system (figure 1) are: a server application, a student client application and a teacher control application. The responsibilities of each component are defined as follows:

•The server component includes the PeLe webservice, the back end database (that in our case is collocated with the webservice, but by no means restricted to be collocated) and portal pages. The server also hosts the students' web app. The web service manages the central data model and keeps the entire system in a consistent state at all times. It provides an interface to remotely modify and read the productive resources following the "Representational State Transfer" (REST) principles. The web service is the main communication node and its data represents the "truth", if synchronization issues occur.

•The teacher software is responsible for modelling the workflow for a given use-case. This is achieved by modifying the data inside the web service according to the required work-flow. The teacher role is granted write access to various parameters which allow for shaping a custom work-flow.

•The student software usually reacts to the actions of the teacher. Again, the user interface has to implement the workflow for the respective use-case. A central aspect of the student client is the ability of being remotely controlled. According to the state of the data maintained in the web service, the student client may entirely block input or provide restricted read/write access to a set of resources.

Apart from the main basic components the system includes few other components that aim at improving of the user experience and facilitating the system maintenance:

•The authentication service allows integration with other authentication methods and also allows the scenario when one user is accessing several servers.

•The autoupdate service allows the teacher client to update over the internet. When a new build is available it is published on the autoupdate location. The teacher clients will check that location periodically and will notify the user to update.

•The zero configuration service allows the teacher client to automatically retrieve the required information to connect to a certain server.

The PeLe tool is a web service in combination with use-case specific clients. During lectures, the system is controlled by the teacher and the student clients react according to the situation, thus generating an interactive environment with immediate feedback between teacher and students.

The ability of reacting to the teacher's actions poses certain challenges in the implementation, as the HTTP protocol does not specify a reliable back-channel for browsers yet. Possible approaches to address this issue are polling, long-polling or web sockets where each of which undeniably has specific drawbacks.
In this interactive environment, both the teacher’s as well as the student’s tools are designed to be streamlined. The amount of available options is kept small and the important functionality for the targeted workflow can be reached with a minimum amount of interactions. A quick interaction with the service is necessary because of the time pressure created in the interactive environment. Neither teacher nor students should get distracted or overwhelmed by special functionality. Any delay in usage of the tool directly causes the other users to wait for the intended change.
Outside the lecture, other clients (e.g. a dynamic web page) are used to work on the resources that has been collected inside the classroom. These tools can offer a rather complex user interface as the time pressure of a live system does not apply here. Typically, the teacher may create new or modify the existing content. Further, both teachers and students have read access to the collected data. Naturally, the teacher is allowed to see all data related to his/her lectures. Students usually only have access to unlocked data, further filtered for personal content. The data of other students remains entirely invisible.

Basic data model

The system data model includes data structures mainly for assessment definition, assessment results and users. In the data model referring to the assessments we considered three levels: Assessment, Section, Item (ASI). An assessment is considered to be a single working unit that is created by an instructor and used for a specified task, e.g. one single lesson during a course. Each assessment can have one or more sections which in their turn can contain items. An assessment item corresponds to a single question in a test.