Workpackage 04 - Implementation

Final Report

Version 0 - 20/07/2016

INTRODUCTION

Referring to the Work Package “WP4 – Implementation” of the Project CompAssess, the aims of this document are:

  • Reporting the executed implementations of the open framework developed in this project;
  • Reporting the feedbacks
  • Definition of the improvements

INPUT FOR THE IMPLEMENTATION

The goal of the work package was a first testing step of a new open framework for the transversal competences assessment in the VET centres at European level.

The “-test” model has been developed in the WP3 and then imported and adapted in an excel file with the purpose of facilitating his use under an operative point of view.

Taking into consideration that the framework is “open” and can be applied to different situations,in diverse contextsand ways, and, at the same time, it seems complex on first glance, the model contents a list of specifications, general information of the topic and operative instructions for the user.

The “excel” open framework could be introduced and/or supported by a presentation which explains purposes and the whole process for the application.

The implementation activities, made by the partners involved, have been defined and scheduled through a shared “Implementation Plan” defined in the meeting in Bolzano (Italy) and updated in case of need.

Input tools and docs:

WP4 Implementation Flow.pptx

Open Framework Implementation 2016 03 07.xlsx

Implementation plan.doc

1

IMPLEMENTATION PLAN

See Implementation Plan – Table for details

PICTURES OF THE EXPERIENCES


Italy /
Italy

Spain /
Spain

Spain /
Denmark

FEEDBACKS

Input:

6 complete responses to questionnaires

8 VET centres

4 countries

Country / VET Centre names / Nr. of teachers involved / Nr. Of students involved
Spain / (5) / 4 / 20???
Italy / (1) IIS Galileo Galilei Bolzano / 2 / 21
Northern Ireland / (1) NRC / 2 / 5
Denmark / (1) EUC-SYD / ? (17??) / ????

Considerations summary:

  • Too many fields

Considerations summary:

  • Useful the setting specifications

Considerations summary:

  • Some parameters are suited to higher level apprentices (University)
  • Different question for the autoevaluation/assessment

Considerations summary:

  • A negative rating is missing
  • Explanations of the parameters could be clearer and easier

Considerations summary:

  • The individual perspective added the teamworking should be considered

Considerations summary:

  • Simplifying and making easier the “rubrics”
  • Adapting the language for the self-assessment

Considerations summary:

  • “As is” is time-consuming

Considerations summary:

  • -

Considerations summary:

  • Social and civic competences are too difficult to assess
  • The majority of the implementation concentrate on 2 transversal competences

Considerations summary:

  • -

Considerations summary:

  • -
  • Simplifying and adapting the language to the communication target
  • The rating should start from “0” and/or a more negative value
  • Elaborating automatically the results and in a “visual” way (graphs, shapes, etc.)
  • Complexity and experience with the tool
  • Similar to an used existing model
  • Support by the guidelines
  • Visual and automatic colours for the ratings

EUC SYD (DK) has reported the results of its implementation test with a different questionnaire (the file report is attached). The reflections based on the feedback collected are reported:

  • We need to define the level of competence that the individual student should acchieve at the current state of his/her training. The desired level is not necessarily Expert level for a student at a beginners level.(NOTE: actually the goal field is defined in the framework for each parameter)
  • The language used in the tool should match the target group.
  • Some of the transversal competences eg. Citizenship is difficult to understand in a Danish VET context.
  • We are surprised (positively)that the students are so reflected on the value and importance of transversal competences.
  • It could be interesting to question a bit deeper as to see if they really understand the meaning of these competences and if they can use this in a professional context (work or education)
  • Evaluation of the transversal competences is very difficult and it is always a subjective matter.
  • The transversal competences is difficult to define and describe, so that we have a common understanding.
  • Grading – is performing on expert level better than performing on routine level?

CONSIDERATIONS AND SUGGESTED IMPROVEMENTS

Taking all the feedbacks into consideration the first version of the open framework has given a positive first impression, even though some improvements are needed.

A briefanalysis is reported in the table below.

POSITIVE ASPECTS / NEGATIVE ASPECTS
A new and innovative tool
Flexible
The content can be adapted to the context
With the guideline, easy to understand
Good visual progression of results / Inadequate language in some points
Complexity and time-consuming
Rates and criteria can be developed
An initial agreement with the students is needed

Since the CompAssess Open Framework is an “open” tool, it needs to be “general”, although it has to be “precise” enough to be adapted to different context and specification. These properties together determined “complexity”. This complexity can be managed and reduced, but not completely removed.

In addition, the content and the definitions in the first version of the tool have not be reviewed before the implementation, since the application itself may show the weak points. Thus, they may sound “theoretic”.

Language, visual presentation of all results, rates definitions are the expected responses from a completely new model for the competence assessment in VET centres.

On the other hand, the most important merging aspects are:

  • That the open framework has been agreed positively for the transversal competences assessment by the executors of the implementations and it shows a significant potential;
  • That the teachers and the students should agree the framework and train with the model in order to make it more effective;
  • That, as an “open” tool, it needs a “custom design” by the VET - users. This design should countthe identification of the relevant competence and parameters in the rubrics, the translation of these in an adequate language to the specific VET environment and the target (for example: students and the self-assessment)
  • That the guidelines as supporting tool can make clearer the complexity of the open framework.

ANNEX

WP4 Implementation Flow.pptx

Open Framework Implementation.xlsx

Implementation Plan.docx

Implementation and Feedback Questionnaire.pdf

EUC syd– Porto.pptx

NRC Map Enterprise to CompAssess.pdf

1