Report of WP7 (Quality management)
Partner Responsible: University of Tartu (P4)
Activities related to the quality management plan (QMP)
Introduction
During the first few months of this project we developed the details of QMP – we defined how we will evaluate products and processes. After 1st transnational meeting we decided to evaluate products (PR) as in project plan (internal evaluation, external evaluation and Local Committee evaluation) and agreed upon the rules which partner will work out evaluation criteria and who will conduct the evaluation.
Additionally to product evaluation the team agreed up on-going process evaluation in terms on evaluation of the effectiveness of transnational meetings.
In order to have a clear overview and to track the workflow and results we made a product and evaluation matrix that will be filled out respectively to the results achieved. This matrix with all used tools and summaries is published on website
We decided to conduct the three type evaluation (internal, external and Local Commitee evaluation) of all the products concerned in the project (WP1 – WP6, exepted PR5, what is working version of PR7). Before that the partner responsible for creating evaluation criteria will make validation forms that Local Committee can fill in. After three types of validation are done for each WP, University of Tartu will combine the evaluation reports and make a quality dossier of each WP.
The products will be evaluated at Local committees as product bundles (different products of several WPs will be evaluated at the same time). In this way will get more integral and comprehensive results regarding the evaluation and also it will save time.
Conducted product evaluation
1. Evaluation of product 1 (State of the Art Report on Distance Learning
Quality for SMEs) and product 2 (Criteria for Evaluating Quality in e-
Learning) as one product
The internal evaluation was conducted by CCC in cooperation with CECOA. The filled in evaluation questionnaire is enclosed ( The evaluation was initially done by CCC on 12 of May 2006, after the promoting organization CECOA gave its contribution on this.
The external evaluation of productswas Anders I. Mørch from InterMedia, University of Oslo. The results are added to PR1 as epilogue ( pages 157 – 158).
The Local Commitee evaluationwas done by 7 persons. The results are available on
2. Evaluation of product 3 (Guide for Training Consultants: e-Learning
Quality and ROI Evaluation)
The internal evaluation was conducted by ProfitWise in cooperation with CECOA. The filled in evaluation is enclosed
The external evaluation of productwas Ruben Eirasfrom CECOA. The results are available on
The Local Commitee evaluation was done by 6 persons. The results are available on
3. Evaluation of product 4 (Training Profile: Training Consultants
Performance Improvement Programme)
The internal evaluation was conducted by ProfitWise in cooperation with CECOA. The filled in evaluation is enclosed
The external evaluation of productwas Ruben Eirasfrom CECOA. The results are available on
The Local Commitee evaluation was done by 6 persons. The results are available on
4. Evaluation of product 5 (ROI Methodology for e-Learning Courses: working version)
The internal evaluation was conducted by BFI in cooperation with CECOA. The filled in evaluation is enclosed
The external evaluation of productwas Ed Fennema - CEO of SkillThing BV and De Volgende Verdieping BV (Netherlands). The results are available on
We did not use the Local Commitee evaluationfor working vesrion.
5. Evaluation of product 6 (Report on Application of the ROI Methodology for e-Learning Courses)
The internal evaluation was conducted by f-BB in cooperation with CECOA. The filled in evaluation is enclosed
The external evaluation of productwas Ed Fennema - CEO of SkillThing BV and De Volgende Verdieping BV (Netherlands). The results are available on
The Local Commitee evaluation was done by 3 persons. The results are available on
6. Evaluation of product 7 (Final version of the ROI Methodology for e-Learning Courses)
The internal evaluation was conducted by ProfitWise in cooperation with CECOA. The filled in evaluation is enclosed
The external evaluation of productwas Wolfgang Schaffer - authorized officer
and sales manager for Austria of BIT Schulungscenter NfG GmbH & Co KG. The results are available on
The Local Commitee evaluation was done by 6 persons. The results are available on
7. Evaluation of product 8 and 9 (Guide to Conceive and Evaluate E-Learning Courses for SMEs Entrepreneurs and Training Providers: Implementing E-Learning in SMEs)
The internal evaluation was conducted by NKI in cooperation with CECOA. The filled in evaluation is enclosed
The external evaluation of productwas Wolfgang Schaffer - authorized officer
and sales manager for Austria of BIT Schulungscenter NfG GmbH & Co KG. The results are available on
The Local Commitee evaluation was done by 2 persons. The results are available on
The evaluatios were very helpful in regard of completing and improving the product. Additionally all the partners were involved in evaluation and improving the products via correspondence at mailing list and audio conferences.
Conducted on-going processes evaluation
- TRANSNATIONAL MEETINGS
For evaluation of transnational meeting we used the web-based tool like
The filled in evaluation questionnaires is enclosed
1ST TRANSNATIONAL MEETING (IN BARCELONA 27.01.2006)
The total grade of the evaluated meeting was 3,5 on the scale from 1-4.
All the items had a at least good evaluation. Some of the topics had very high evaluation as expected: "Organisation of the transnational meeting" and „Quality and appropriateness of the domestic arrangements and the comfort factor ".
Item with the highest evaluation is "Suitability of working venue".Itmes with very high evaluation include: "Realistic timetables", "Achievement of aims and objectives planned for the meeting", „Relevance and quality of materials for the meeting".
The lowest grade was given for the itens of the topic "Input into meeting by the project partner's" (3,07).
2ND TRANSNATIONAL MEETING (IN OSLO 20-21.06.2006)
The total grade of the evaluated meeting was 3,7 on the scale from 1-4 that is better than the total grade of the previous meeting (the 1st transnational meeting, held in Barcelona).
All the items had a at least good evaluation. Some of the topics had very high evaluation as expected: "Quality and appropriateness of the domestic arrangements and the comfort factor" and "Partners contacts".
Item with the highest evaluation is "Suitability of working venue" and "Attention to practical details and catering arrangements". Itmes with very high evaluation include: "Opportunities for development of positive attitudes towards Europe and transnational activities", "Evidence of clear planning", "Efective mix of activities", "Appropriateness of social program", "Appropriate information distributed to partners prior to event", "Partners contacts are correct and updated", "Means of communication are flexible and simple", "Sensibility to any spacial requirements of partners".
The lowest grade was given for the itens of the topic "Each partner's contribution to the event" (3,0).
3RD TRANSNATIONAL MEETING EVALUATION (IN AMSTERDAM 18-19.01.2007)
The total grade of the evaluated meeting was: 3,9 on the scale from 1-4 that is better that the total grade of the previos meetings (the 1st transnational meeting, held in Barcelona and the 2nd meeting in Oslo).
All the items had a at least good evaluation. Some of the topics had very high evaluation: „Materials, Resources, Equipmnet" and "Quality of the Local Arrangements".
The lowest grade was given for the items of the topic "Quality of Transnational elements" - "Each partner`s contribution to the event" (3.3) and "Evidence of partners sharing roles and responsibilities during the event or as part of the overall project" (3.5).
4TH TRANSNATIONAL MEETING EVALUATION (IN TARTU 14-15.05.2007)
The total grade of the evaluated meeting was: 3,8 on the scale from 1-4 that is better than the total grade of the 1st transnational meeting, held in Barcelona and the 2nd meeting in Oslo, but less than 3rd meeting in Amsterdam.
All the items had a at least good evaluation. Some of the topics had very high evaluation: "Effectiveness of content and appropriate variety of activities", "Effectiveness of monitoring and evaluation processes" and "Partners contacts".
The lowest grade was given for the items of the topic "Input into the meeting by the project partners" - "Each partner`s contribution to the event" (3.2) and "Effectiveness of delivery by the coordinator" - "Communication and language skills" (3.4).
5TH TRANSNATIONAL MEETING EVALUATION (IN LISBON 12-13.11.2007)
The total grade of the evaluated meeting was: 3,7 on the scale from 1-4 that is the same than the total grade of the 2nd transnational meeting, held in Oslo.
All the items had a at least good evaluation. Some of the topics had very high evaluation: "Quality and appropriateness of the domestic arrangements and the comfort factor" (3,9), "Effectiveness of share ownership of the meeting" (3,8), "Effectiveness of monitoring and evaluation processes" (3,8), "Partners contacts", "Definition and achivement of aims and objectives of the meeting" (3,8), "Organisation of the transnational meeting" (3,8) and "Link between the aims of the meeting and the overall objectives of the European ELQ - SME Project" (3,8).
The lowest grade was given for the items of the topic "Input into the meeting by the project partners" (3.4).
- THE EFFICIENCY OF TELEPHONE CONFERENCES
During the project the partners conducted 14 transnational telephone conferences. The conferences were powerful tool to get instant feedback and contribution on the issues of the project.
This also report includes the minutes of the meetings as they were tools to help partners to follow the project activities and execution and also the project timetable (also included to the report:
The telephone conferences took place on:
- 1st of March 2006
- 3rd of April
- 24th of July 2006
- 21st of August 2006
- 27th of September 2006
- 31st of October 2006
- 23rd of February 2007
- 30th of March 2007
- 9th of May 2007
- 11ht of June 2007
- 20th of July 2007
- 31st of August 2007
- 26th of September 2007
- 26th of October 2007
Audio-conferences / 1 / 2 / 3 / 4 / 5 / 6
Dates / 1.03.2006 / 3.04.2006 / 24.06.2006 / 21.08.2006 / 27.09.2006 / 31.10.2006
Organizer / NKI / NKI / CECOA / CECOA / ProfitWise / ProfitWise
Participants
NKI / NKI / NKI / NKI / NKI
CECOA / CECOA / CECOA / CECOA / CECOA / CECOA
BFI / BFI
f-BB / f-BB / f-BB / f-BB
CCC / CCC / CCC / CCC / CCC
UT / UT / UT / UT / UT / UT
ProfitWise / ProfitWise
Number of partners / 6 / 4 / 5 / 3 / 6 / 6
2nd year of the project
Audio-conferences / 7 / 8 / 9 / 10 / 11 / 12 / 13 / 14
Dates / 23.02.2007 / 30.03.2007 / 9.05.2007 / 11.06.2007 / 20.07.2007 / 31.08.2007 / 26.09.2007 / 26.10.2007
Organizer / CCC / CCC / CECOA / CECOA / CECOA / NKI / f-BB / f-BB
Participants
NKI / NKI / NKI / NKI / NKI / NKI
CECOA / CECOA / CECOA / CECOA / CECOA / CECOA / CECOA / CECOA
BFI / BFI
f-BB / f-BB / f-BB / f-BB / f-BB / f-BB / f-BB
CCC / CCC / CCC / CCC / CCC / CCC / CCC
UT / UT / UT
ProfitWise / ProfitWise / ProfitWise / ProfitWise / ProfitWise / ProfitWise
Number of partners / 4 / 7 / 4 / 6 / 5 / 3 / 4 / 6
- THE EFFICIENCY OF MAILING LIST
The project has mailng list with aadress of and with following members:
During the two year of the project there has been 467 emails sent to the list my different project members. The emailing list has proved to be efficient tool in order to keep all the project partners updated on latest issues and contributions.