contract no 031517 / Co-funded by the European Commission
TRAIN-ALL
Integrated System for driver Training and Assessment using Interactive education tools and New training curricula for ALL modes of road transport
Contract no. 031517
Demonstration pilot plans
Deliverable No.: / D6.1Dissemination Level / Public
Workpackage No. / WP6 / Workpackage Title / Demonstration pilot evaluation
Activity No. / A6.1 / Activity Title: / Demonstration Pilot Plans
Activity Leader: / ICCS
Workpackage Leader / VTI
Authors (per company, if more than one company provide it together) / B Peters (VTI), I Mantzouranis (ICCS), S Poshadel (IFADO)
Status (Final; Draft; Revised Draft): / Draft
File Name: / TRAIN-ALL Deliverable 6.1_updated.doc
Project start date and duration: / 01 November 2006, 38 Months
Submission date: / July ‘09
Version Number: / Version 2
Pages Number: / 110
Distribution / All partners, EC
V1- Page 3 of 79
TRAIN-ALL Deliverable 6.1contract no 031517 / Co-funded by the European Commission
Version history
Version / Date / ModificationsV0 / 20/05/2008 / First version available.
V0 / 21/05/2008 / Some minor update in Chapter 2.
V0 / 16/06/2008 / Completed chapter 4 and a lot of work on questionnaires in the Annexes by VTI.
V1 / 20/06/2008 / Completed chapter 2 and 5, overall correction.
Final version of the Deliverable.
V1.1 / 02/07/2009 / Update of Chapter 6 by IFADO
V2 / 22/07/2009 / Overall update incorporating Outcomes of the D3.X Deliverables
Executive Summary
This Deliverable describes the objectives, design and main features of the WP6 pilot tests, which will be performed in the TRAIN-ALL project. It aims to define the pilot plan design by detailing the respective objectives and main features. The WP6 demonstrator pilots will follow the WP5 verification pilots, thus, this is the updated version of the deliverable featuring all new information available about the modules and simulators that has been presented in the WP3 Deliverables, as well as input from the verification pilots.
Specifically, the present document outlines the methodological framework of all tests and experiments that will be conducted in the pilots, including evaluation criteria and methodology, along with the new technological concept to be validated in a series of on-site pilots. During the pilots, a detailed evaluation of the usability, the effectiveness and the impact of the modules produced within TRAIN-ALL will be conducted. In addition, a series of subjective measures will be conducted in order to evaluate user acceptance and workload when using the proposed solutions.
The Pilot plans are constructed and presented using the CONVERGE (TR1101) project methodology, which has been successfully applied to many EU projects.
Figure 1: Model for Intelligent Transport System (ITS) Architecture (based on the CONVERGE methodology)
In Figure 1, the demonstration tasks are supposed to be acquired for the existing simulators. This plan focuses on the whole validation of the Modules functionalities throughout the implemented Tools, which represent the TRAIN-ALL demonstrators based upon the partners’ existing simulators. Partners should be aware that modules have to be testable in all different TRAIN-ALL configurations; the testability remains a dominant feature of the TRAIN-ALL system architecture.
The current document is an updated version, with revisions in Chapter 6 and generic updates regarding the WP3 modules that have been developed.
Table of Contents
List of Figures 6
List of Tables 7
Abbreviations List 8
TRAIN-ALL applicable documents 9
1 Introduction 10
2 Overview of Pilots and Modules 11
2.1 A3.1 Ambient intelligence in driving simulation (HIT) 14
2.2 A3.2 Co-driving Group Training (BPP) 15
2.3 A3.3 Immersive simulation for multi-vehicle simulator platforms (USTUTT) 17
2.4 A3.4 P2P Internet technologies for CBT (UP) 21
2.5 A3.5 Virtual instructor and debriefing module (INRETS) 21
2.6 A3.6 Dynamic scenario management module (GreenDino) 23
2.7 A3.7 ADAS/ IVICS simulation module (HIT) 24
2.8 A3.8 Simulator sickness aversion (COAT) 26
2.9 A3.9 Enhanced reality (IZVW) 26
2.10 A3.10 Adaptive training (IZVW) 29
3 Assessment criteria and methodology 32
3.1 The “dose-response” problem 33
3.2 Impact analysis without an on-road assessment 33
3.3 On-road test with the TRIP-protocol 36
4 A General Pilot Protocol (Procedure) 40
5 Pilot Protocol (Procedure) per Module 43
5.1 Ambient Intelligence (A3.1) 43
5.2 Co-driving and Group Training (A3.2) 43
5.3 Immersive Simulations for Multi-vehicle Simulator Platforms (A3.3) 45
5.4 P2P Internet Technologies (A3.4) 46
5.5 Virtual Instructor and Debriefing (A3.5) 46
5.6 Dynamic Scenario Management (A3.6) 47
5.7 ADAS/IVIS Simulations (A3.7) 47
5.8 Simulator Sickness (A3.8) 48
5.9 Enhanced Reality (A3.8) 49
5.10 Adaptive Training (A3.10) 49
5.11 Conclusion 50
6 Relating Pilots and Modules to the European Driving Licence Directive (2006/126/EC) 51
6.1 How to read the following tables? 52
6.2 Skills expected from all driver groups based on the EC “Driving Licence Directive” (2006/126/EC) 52
6.3 Skills expected from novice car drivers (Category B) based on the EC “Driving Licence Directive” (2006/126/EC) 56
6.4 Skills expected from motorcycle drivers (Categories A1, A2 and A) based on the EC “Driving Licence Directive” (2006/126/EC) 59
6.5 Skills expected from heavy truck drivers and bus / passenger transport drivers (categories C, CE, C1, C1E, D, DE, D1 and D1E) based on the EC “Driving Licence Directive” (2006/126/EC) 63
6.6 Skills expected from emergency drivers 68
6.7 Summary: Relating Pilots and Modules to the European Driving Licence Directive (2006/126/EC) 68
7 The ethical checklist 70
7.1 TRAIN-ALL Informed consent form template (preliminary) 72
8 Pilot Reporting 76
9 References 78
10 Annex 1 Pre-test Questionnaires and Forms 79
10.1 General reporting 79
10.2 Pre-test questionnaires (mandatory for all) 79
11 Annex 2 Questionnaires to be used during pilots 83
11.1 Scenario related questionnaires 83
11.2 Use Case related questionnaire 86
12 Annex 3 Questionnaires to be used after the pilot 88
12.1 Module related questionnaires 88
12.2 Trainer Checklists – [D3.8 p.54 (TRAINER)] 102
13 Annex 4: TRIP protocol of the AGILE project 104
List of Figures
Figure 1: Model for Intelligent Transport System (ITS) Architecture 3
Figure 2: Scenario of a box falling from a truck. 24
Figure 3: Pilot flow chart with respect to the administration of common questionnaires 41
Figure 4 The “User Acceptance Scale” of Van der Laan et al. (1997). 90
List of Tables
Table 1 Overview of WP6 Pilots 11
Table 2 Overview of simulators per module (pilot responsible/module developer) 12
Table 3 Pilots (conductor, simulator) modules and relevant UC 13
Table 4 Use Cases for the Ambient Intelligence Module 14
Table 5: Use Cases for the Co-driving Module 15
Table 6: Scenarios to test the immersive simulator module 18
Table 7 : Immersive Simulation module – scenarios and performance-based success criteria parameters. 18
Table 8 : Implementation Status for Immersive Simulation module 20
Table 9: Scenarios to be test the Virtual instructor and debriefing module 21
Table 10: Scenarios to test the dynamic scenario management module 23
Table 11: Scenarios to test the ADAS/IVICS Module 25
Table 12 : Scenarios to test the enhanced reality module 27
Table 13: Enhanced reality module – selected key driving parameters for novice car drivers, how they are visualized in the simulation and performance-based success criteria. 28
Table 14 : Scenarios to test the enhanced reality module 30
Table 15: Adaptive training module – selected training contents for emergency drivers, variation of task difficulty and performance-based success criteria. 30
Table 16: Implementation Status for Adaptive Training Module 31
Table 17: Comparison between the different TRIP Versions 36
Table 18 Specification of common questionnaires 42
Table 19: Ambient Intelligence Module Success Criteria 43
Table 20: Co-driving Module Success Criteria 43
Table 21: Immersive Simulation Module Success Criteria 45
Table 22: Virtual Instructor and Debriefing Module Success Criteria 46
Table 23: Dynamic Scenario Module Success Criteria 47
Table 24: ADAS/IVIS Module Success Criteria 47
Table 25: Simulator Sickness Module Success Criteria 48
Table 26: Enhanced Reality Module Success Criteria 49
Table 27: Adaptive Training Module Success Criteria 49
Table 28: Skills which must be shown (all driver groups) during the driving test /driver training in relation to the TRAI-ALL modules 52
Table 29: Skills and driving manoeuvres which must be shown by car drivers (category B) in relation to the TRAIN-ALL modules 56
Table 30: Skills and driving manoeuvres which must be shown by motorcycle drivers (Categories A1, A2 and A) in relation to the TRAIN-ALL modules 59
Table 31: Skills and driving manoeuvres which must be shown by heavy truck drivers and bus drivers (Categories A1, A2 and A) in relation to the TRAI-ALL modules 63
Abbreviations List
Abbreviation / Definition / Abbreviation / Definition /ADAS / Advanced Driver Assistance Systems / NASA-TLX / NASA Task Load Index
API / Application Interface / P2P / Peer to Peer
ASAP / As Soon As Possible / PC / Personal Computer
CAS / Collision Avoidance System / ReCoTo / Remote Control Tool
ASCII / American Standard Code for Information Interchange / RSME / Rating Scale of Mental Effort
CAVE / Cave Automated Virtual Environment / RT / Real Time
CBT / Computer Base Training / SE / Synthetic Environment
CGE / Computer Generated Entities / SMS / Short Message System
NA / Not Apply / SUS / System Usability Scale
DARe / Discovery and Analysis Resource / TLC / Time to Line Crossing
DDS / Distributed Data Store / TRIP / Test Ride for Investigating Practical Fitness to Drive
DBQ / Driver Behaviour Questionnaire / TTC / Time To Collision
ECG / Electro Cardiogram / UAS / User Acceptance Scale
UC / Use Case
EEG / Electro Encephalogram / UME / Usability Magnitude Estimation
EMG / Electro Myogram / UPI / Usability Problem Inspector
GADGET / Guarding Automobile Drivers through Guidance, Education and Technology / VR / Virtual Reality
HMI / Human Man Interaction(Interface) / WTH / Willingness To Have
IVICS / In-Vehicle Information and Communication Systems / WTP / Willingness To Pay
LDW / Lane Departure Warning
TRAIN-ALL applicable documents
[Ref 1] / Annex I - “Description of Work”, TRAIN-ALL project, 23, Feb.-06CN. 031517, Sixth Framework Programme.
[Ref 2] / TRAIN-ALL Del. 5.1 “Verification Pilot Plans Framework, Questionnaires and Template” I.Mantzouranis, K.Paglé, 2008
[Ref 3] / TRAIN-ALL Del.1.2 “Training Needs, Scenario and Curricula Definition and Specification of Tools and Curricula” S.Poschadel (editor, IFADO), J.Kapplusch, P.M.Lindberg Kristiansen (IFADO), Ph.Vanhulle (THALES), J.Pfaffenzeller, Ch. Kainz (BPP), S.Espié (INRETS), B.Peters (VTI), E.Bekiaris, M.Panou (HIT), 2008
[Ref 4] / TRAIN-ALL Del. 3.1 “Ambient Intelligence module” M. Panou, E. Bekiaris (HIT), Christian Mark (WIVW), Bjorn Peters (VTI), Kai Foerst (FOERST), 2008
[Ref 5] / TRAIN-ALL Del. 3.2 “Co-driving, cooperative and group training modules” H.Grattenthaler (IZVW), C.Mark (WIVW), A. Neukum (IZVW), J.Pfaffenzeller (BPP), R.Gassner (BPP), M.Ritz (BPP), M. Schmidt (BPP), 2008
[Ref 6] / TRAIN-ALL Del. 3.3 “Immersive Simulation for multi-vehicle simulator platforms”, Matthias Bues (USTUTT),Giovanni Turi (CRF)
[Ref 7] / TRAIN-ALL Del. 3.4 “i3 based tool for networked learning and remote control of simulators”, Richard Holzer (UP)
[Ref 8] / TRAIN-ALL Del. 3.5 “Virtual Instructor and debriefing module” V.Bodard (INRETS), H.Grattenthaler (IZVW), C.Mark (WIVW), K.Ritter (IZVW), J.Kuipers (GreenDino), K.Foerst (Foerst), 2008
[Ref 9] / TRAIN-ALL Del. 3.6 “Dynamic Scenario Management Module” Christian Mark (WIVW), Jorrit Kuipers (Green Dino), Ingo Wassink (Green Dino), 2009
[Ref 10] / TRAIN-ALL Del. 3.7 “ADAS/IVICS simulation module”, Maria Panou, Kostas Kalogirou, Evangelos Bekiaris (CERTH/HIT)
[Ref 11] / TRAIN-ALL Del. 3.8 “Simulation sickness aversion checklist” Marcel Delahaye (COAT), PD Dr. Ralph Mager (COAT), 2008
[Ref 12] / European Driving Licence Directive (2006/126/EC)
[Ref 13] / TRAIN-ALL Del. 3.10 “Enhanced Reality module” Heidi Grattenthaler & Alexandra Neukum (IZVW), Christian Mark (WIVW) Kai Foerst (FOERST), Jorrit Kuipers (Green Dino), 2009
[Ref 14] / TRAIN-ALL Del. 6.1 “Demonstration pilot plans and Ethics Manual” B Peters (VTI), I Mantzouranis (ICCS), S Poshadel (IFADO), G Turi (CRF), J Kuipers (Green Dino), draft, 2008
1 Introduction
The objective of this report is to define the structure of the demonstration pilot tests, which will be carried out during the TRAIN-ALL project. The prospective results are the output of the WPs: WP1 “Training Needs and Scenarios Definition”, WP2 “Towards a single training and assessment platform”, WP3 “Enabling Technologies” WP4 “Tools development” and WP5 “Verification Pilot Plans”.
The Pilot framework and tests that are described in this document, aim to test the developed modules with respect to their value for training. The Modules produced in WP3 and integrated within WP4 should be technically tested and verified form a user’s point of view in the WP5 pilots. The validation of simulator fidelity (closeness to reality) is not a major concern here. However, simulator fidelity can have an impact on the training effectiveness (Allen et al., 2007). Once the technical coherence is reached and proven the effects on training shall be examined and reported through the Demonstration pilots. During these pilots, drivers from specific target groups (as presented in the Technical Annex I, DoW; WP6, A6.2 Table, p.66 [Ref 1]) will evaluate the modules through the demonstrators both developed under TRAIN-ALL. An extensive comparative evaluation including usability of TRAIN-ALL modules and training impact will take place.
The overview of Pilots and Modules as these will be tested within WP6 Pilots are presented in Chapter 1. A case-control approach is used in most pilots even if there can be big differences between individual pilots depending on e.g. the nature of the modules. A more detailed discussion on the general methodology approach for WP6 can be found in Chapter 3. Chapter 4 and 5 provide descriptions of general and individual procedures for the actual conduct of pilots per module. Chapter 6 relates modules and pilots to existing training curricula, e.g. EU directives. The Ethics Manual has been already incorporated in Deliverable D5.1 “Verification Pilot Plans” [Ref 2] and thus an ethical checklist that can be used directly in each site is incorporated in this deliverable in Chapter 7. In Chapter 8 guidelines for the Pilot reporting are provided in order for all sites to provide a coherent – as possible - group of data that can easily be further processed. In the Annexes that follow three different sets of pilot questionnaires are presented:
o Pre – test questionnaires and Forms