Self-evaluation inventory for e-Learning producersInventar zur Selbstevaluation für Produzenten und Designer von e-Learning

Especially for the providers and designers of e-learning it is very important to consider the target audience, i.e. the learners. In traditional learning environments the teachers or trainers can easily adjust their style in the interaction with the learners. In e-learning settings the trainer or tutor has to communicate with learners whom he/she will perhaps never see or whom he/she will not see while they are learning. That is one of the main reasons why it is necessary to know the learners very well or why designers of e-learning should do some research or systematic reflections about the target group and criteria of good instructional designs and products when planning and producing a programme.

"Who are my learners and what are their needs?" therefore is the first important question when designing an e-learning programme (Lewis & Whitlock 2003, p.3).

The second part of this self-test is concerned with questions about the design of instructional programmes.

,T and the third part leads to criteria that may be decisive for the effectiveness of a programme.

This inventory was compiled by eL3-project partners and is free for your adoption and usage.

Diese Version ist noch eine Probefassung. Personen, die an der Erstellung von Blended-Learning oder e-Learning Maßnahmen beteiligt waren oder sind, werden gebeten, das Inventar unter dem Gesichtspunkt eines praktischen Projekts auszufüllen und jeweils im Hinblick auf die Nützlichkeit und Verständlichkeit der Fragen zu kommentieren.

A. Do I know my learners and their needsKenne ich meine Lerner und ihre Bedürfnisse?

(According to: Lewis & Whitlock 2003, p.4-5 und CEWID).

Be aware that these questions or aspects are to be regarded in their relation to e-learning and specialities of the programme in question.

Learner characteristics

A1. Analyse der Lernermerkmale
 well known  to be enquired
Allgemeine Merkmale:
1) Age (mean, variance)
2) Gender (one-sided or even?)
3) Physical characteristics
4) Acquaintances (learners with each other, with the teaching persons)
Bildungs- und Arbeitshintergrund:
5) Qualifications they already have
6) Occupation
7) Interests
8) Likely level of skill in handling words, numbers diagrams, equipment [bw1]
9) Professional/Experiences
Kultureller Hintergrund und Lernsozialisation:
10) Language and place of origin
11) Likely confidence in learning new things
12) Learning methods with which they are familiar
13) Likely attitude to electronic learning (devices)
Lernerfahrungen und Erwartungen:
14) Learning experiences so far (different methods of learning/competence)
15) Individual learning styles and learning strategies
16) Prerequisites in knowledge (previous knowledge/qualification)
17) Preferences in special learning arrangements
18) Motives for learning
19) Expected results

I would divide this part into sub-sections.

1.

Physical General features:
Age
Gender (It is now acknowledged that there exist gender-related differences in performance and interaction style in e-learning environments.)
Physical characteristics
Acquaintances Are you thinking of disability here?
Education and employment background:
Qualifications they already have
Occupation
Interests
Likely level of skill in handling words, numbers diagrams, equipment [bw2]
Professional/Experiences
Cultural background and attitudes esp. to learning:
Language and place of origin
Likely confidence in learning new things
Learning methods with which they are familiar
Likely attitude to electronic learning (devices)
Learning experiences and eExpectations:
Learning experiences so far (different methods of learning/competence)
Individual learning styles and learning strategies
Prerequisites in knowledge (previous knowledge/qualification)
Preferences in special learning arrangements
Motives for learning
Expected results
(?)
Note: as for the question “Number of learners”, I think this question should not be here as this is the learner’s description.


A2. Analyse des Lernkontextes  well known  to be enquired
1) Where will they learn (at home? at work? at learning centers?)
2) How much time will they be able to spend for learning?
3) What equipment will they have (hardware? software? network connection? other resources?)
4) What kind of access will they have/need?
5) When will they learn? (at night? during working hours? with short or long during intervals?)
6) What are their family circumstances?
7) What difficulties may they experience (for example in gaining access to the necessary technology)?
8) Do they need some training to get started with the technology?
9) What kind of support will the learners have? (tutors/trainers or others: who? how? qualifications of the tutors? when are they available)

2.

Learners’ contexts

B. QB. Questions to think about the design of your e-learning products /processes/offers

The didactic design of your e-learning programme is the basis on which your target group will be able to learn. As a producer of e-learning it is you to plan how to make learning possible. You will think about the contents and required competencies and the ways to offer your knowledge to the learners or manage this. Of course there are many ways of teaching and each subject can be presented in various ways. Below we give you seven criteria of didactic designs, i.e. plans and products within an e-learning programme. You can use them to reflect your didactic design and to check if you thought about these questions when planning or offering your e-learning programme.

B1. Vielfaltariety
der Inhalte, der Lernerpersönlichkeiten, der Methodenof contents, learner personality, methods

My/our programme ...

/ poornot at all
1--- / just a little
2-- / quite a bit
3- / rather much
4+ / great
much5
++


 / very much
++








+
1) …addresses to several senses (vision, hearing, touch, smelling, equilibrium) /  /  /  /  /  / 
2) …uses several didactic methods/models /  /  /  /  /  / 
3) …mMatches content to delivery methods /  /  /  /  /  / 
4) …addresses to reason and emotion /  /  /  /  /  / 
5) …changes between phases of tension and relaxation /  /  /  /  /  / 
6) …includes individual and collaborative activities /  /  /  /  /  / 
7) …motivates learners with different ways of learning /  /  /  /  /  / 
8) …has opportunities to learn in other situations than sitting on a chair /  /  /  /  /  / 
9) …changes between guided and self-directed learning /  /  /  /  /  / 
10) …allows for different learning locations /  /  /  /  /  / 
*1 / *2 / *3 / *4 / *5 / *6
B2. Ökonomie
im Sinne einer wirkungsvollen Nutzung der Ressourcen

My/our programme ...

/ not at all
--- / just a little
-- / quite a bit
- / rather much
+ / much
++ / very much
+++
1) …avoids counterproductive effects like a negative attitude towards learning /  /  /  /  /  / 
2) …uses simple and cheap means and media which are little susceptible and that are available anyway /  /  /  /  /  / 
3) …converts much “teaching time” into effective learning time /  /  /  /  /  / 
4) …leads to learning outcomes that are close to the aims /  /  /  /  /  / 
5) …imparts additional knowledge without greater efforts by the learners /  /  /  /  /  / 
6) …avoids learning disorders /  /  /  /  /  / 
7) …uses means which are trouble-free /  /  /  /  /  / 
8) …does not teach things as new that are already known to most learners /  /  /  /  /  / 
9) …builds up on the given knowledge and abilities of the learners /  /  /  /  /  / 
10) …has undergone a risk assessment for potential failure /  /  /  /  /  / 
*1 / *2 / *3 / *4 / *5 / *6
B3. Anpassung
an die gegebene/erwartete Situation

My/our programme ...

/ not at all
--- / just a little
-- / quite a bit
- / rather much
+ / much
++ / very much
+++
1) …avoids counterproductive effects like a negative attitude towards learning /  /  /  /  /  / 
2) …uses simple and cheap means and media which are little susceptible and that are available anyway /  /  /  /  /  / 
3) …converts much “teaching time” into effective learning time /  /  /  /  /  / 
4) …leads to learning outcomes that are close to the aims /  /  /  /  /  / 
5) …imparts additional knowledge without any further effort /  /  /  /  /  / 
6) …avoids learning disorders /  /  /  /  /  / 
7) …uses means which are trouble-free /  /  /  /  /  / 
8) …does not teach things that are already known to most learners /  /  /  /  /  / 
9) …builds up on the given knowledge and abilities of the learners /  /  /  /  /  / 
10) …enhances the interests of the learners and excites motivation /  /  /  /  /  / 
*1 / *2 / *3 / *4 / *5 / *6
B4. Produktivität
und Nützlichkeit

My/our programme ...

/ not at all
--- / just a little
-- / quite a bit
- / rather much
+ / much
++ / very much
+++

1) …supplies cognitive schemes (e.g. methods that can be generalised)

/  /  /  /  /  / 
2) …helps the learners to gain more routine with the use of reference works /  /  /  /  /  / 
3) …offers a line of orientation /  /  /  /  /  / 
4) …imparts key qualifications /  /  /  /  /  / 
5) …creates and improves contacts (e.g. to other learners, practitioners) /  /  /  /  /  / 
6) …conforms to appropriate regulatory frameworks /  /  /  /  /  / 
7) …is fit for purpose /  /  /  /  /  / 
*1 / *2 / *3 / *4 / *5 / *6


B5. Selbmotivation
für Lehrende und Lernende

My/our programme ...

/ not at all
--- / just a little
-- / quite a bit
- / rather much
+ / much
++ / very much
+++
1) …shows the learner especially the positive effects of what he/she has already learnt /  /  /  /  /  / 
2) …takes the joy in physical exercise during the learning process into account /  /  /  /  /  / 
3) …is a suitable basis for fruitful questions /  /  /  /  /  / 
4) …means learning in a playful way /  /  /  /  /  / 
5) …means learning in a social way /  /  /  /  /  / 
6) …bears recreational features /  /  /  /  /  / 
7) …is carried out without glancing at a certification /  /  /  /  /  / 
8) …is carried out without fearing to fail /  /  /  /  /  / 
9) …promotes the learning rather in competition with oneself and not to others /  /  /  /  /  / 
10) …takes the joy in intellectual/artistic games into account /  /  /  /  /  / 
*1 / *2 / *3 / *4 / *5 / *6


B6. Entwicklung
der Lernerfähigkeiten

My/our programme ...

/ not at all
--- / just a little
-- / quite a bit
- / rather much
+ / much
++ / very much
+++
1) …increases the moral sensibility of the learner (in regard to the subject, qualification) /  /  /  /  /  / 
2) …increases the intelligence of the learners /  /  /  /  /  / 
3) …promotes the creativity of the learners /  /  /  /  /  / 
4) …helps to diminish fears of the learners /  /  /  /  /  / 
5) …strengthens the conflict abilities of the learners /  /  /  /  /  / 
6) …increases the communication abilities of the learners /  /  /  /  /  / 
7) …gives the learners a better self-esteem (self confidence) /  /  /  /  /  / 
8) …promotes the decision-making ability of the learners /  /  /  /  /  / 
9) …increases the sense of responsibility of the learners /  /  /  /  /  / 
10) …increases the intelligence of the learners /  /  /  /  /  / 
*1 / *2 / *3 / *4 / *5 / *6
B7. Konsistenz

In my/our programme ...

/ not at all
--- / just a little
-- / quite a bit
- / rather much
+ / much
++ / very much
+++
1) …is a close distinction/distribution between the various modi of blended learning on the basis of their different functionalities and possibilities /  /  /  /  /  / 
2) …the teaching people themselves are experienced in handling the electronic devices /  /  /  /  /  / 
3) …there are no serious discrepancies between what is said and what is done /  /  /  /  /  / 
4) …the teaching people “live” what they are imparting /  /  /  /  /  / 
5) …quality of verbal/audiovisual material is high /  /  /  /  /  / 
6) …the tasks agree with the practical requirements /  /  /  /  /  / 
7) …the learners do have as much scope as in professional practice /  /  /  /  /  / 
8) …equally distributes the learning difficulties, not clustering them only to the beginning or to the end /  /  /  /  /  / 
*1 / *2 / *3 / *4 / *5 / *6


C. Criteria to think about the efficacy of your e-learning programme

The following criteria reflect how efficacious an e-learning programme is expected to be. Effects will depend on many circumstances, and it is quite sure that only the practical try-out will prove them in last instance. But as we are here in an inspective evaluation the first judgment can be done by yourself.

C1. Klarheit

reliable and comprehensible performances

My/our programme has clarity ...

/ not at all
--- / just a little
-- / quite a bit
- / rather much
+ / much
++ / very much
+++
1) …of knowledge /  /  /  /  /  / 
2) …of learning steps/process /  /  /  /  /  / 
3) …of expected results /  /  /  /  /  / 
3) of classification /  /  /  /  /  / 
4) …of questions and assignments to the learners /  /  /  /  /  / 
5) …of aims and criteria for judgments /  /  /  /  /  / 
6) …of positions (authenticity) /  /  /  /  /  / 
*1 / *2 / *3 / *4 / *5 / *6


C2. Relevanz und Bezug
für gültiges Wissen

My/our programme is relevant for and refers to ...

/ not at all
--- / just a little
-- / quite a bit
- / rather much
+ / much
++ / very much
+++
1) …the subject/underlying discipline /  /  /  /  /  / 
2) …the research level (state of the art) /  /  /  /  /  / 
3) …to the further learning process /  /  /  /  /  / 
4) …to the forthcoming examinations /  /  /  /  /  / 
5) …the learners’ situations /  /  /  /  /  / 
6) …the learners’ future /  /  /  /  /  / 
7) …the society and subsystems (professions e.g.) /  /  /  /  /  / 
*1 / *2 / *3 / *4 / *5 / *6


C3. Struktur und Sequenz
Verteilung der Wissensvermittlung

My/our programme has structure...

/ not at all
--- / just a little
-- / quite a bit
- / rather much
+ / much
++ / very much
+++
1) …by resuming what is already known /  /  /  /  /  / 
2) …by accentuation of main points /  /  /  /  /  / 
3) …by time management /  /  /  /  /  / 
4) …by space management (learning environment) /  /  /  /  /  / 
5) …by continuity /  /  /  /  /  / 
*1 / *2 / *3 / *4 / *5 / *6


C4. Anschaulichkeit

durch Lern- und Denkhilfen

My/our programme is representational through …

/ not at all
--- / just a little
-- / quite a bit
- / rather much
+ / much
++ / very much
+++
1) …visualization (images, transparencies e.g.) /  /  /  /  /  / 
2) …examples /  /  /  /  /  / 
3) …analogies /  /  /  /  /  / 
4) …concrete experiences for learners /  /  /  /  /  / 
5) …reflection by the learner /  /  /  /  /  / 
*1 / *2 / *3 / *4 / *5 / *6

C5. Anwendung

des Wissens

My/our programme ...

/ not at all
--- / just a little
-- / quite a bit
- / rather much
+ / much
++ / very much
+++
1) …is proving knowledge in practice /  /  /  /  /  / 
2) …has well defined assignments /  /  /  /  /  / 
3) …has opportunities to train /  /  /  /  /  / 
4) …offers insight into the practical application and usefulness /  /  /  /  /  / 
*1 / *2 / *3 / *4 / *5 / *6

C6. Rückmeldung

zum Lernprozess, zur Entwicklung und zum Ertrag

My/our programme gives feedback ...

/ not at all
--- / just a little
-- / quite a bit
- / rather much
+ / much
++ / very much
+++
1) …through praise and reproach /  /  /  /  /  / 
2) ...by products /  /  /  /  /  / 
3) …by certifications /  /  /  /  /  / 
4) …by the natural environment („natura docet“) /  /  /  /  /  / 
*1 / *2 / *3 / *4 / *5 / *6
C7. Weitere Anreize

My/our programme gives stimulus ...

/ not at all
--- / just a little
-- / quite a bit
- / rather much
+ / much
++ / very much
+++
1) …by consideration of learners’ interests and needs /  /  /  /  /  / 
2) …by teachers’ enthusiasm /  /  /  /  /  / 
3) …also with a certain irony and self-criticism /  /  /  /  /  / 
*1 / *2 / *3 / *4 / *5 / *6

Variety of contents, learner personality, methods

:

My/our programme....

addresses to several senses (vision, hearing, sense of touch, smelling, equilibrium sense)

uses several didactic methods

matches content to delivery methods

addresses to reason and emotion

changes between phases of tension and relaxation

includes individual and collaborative activities

motivates learners with different ways of learning

has opportunities to learn in other situations than sitting on a chair

changes between guided and self-directed learning

allows for different learning locations

equally distributes the learning difficulties[bw3]

2. Economy in the sense of efficacious use of resources:

My/our programme....

covernvertts much “teaching time” into effective learning time

leads to learning outcomes that are close to the aims

imparts additional knowledge without any further effort[bw4]

avoids learning disorders

avoids counterproductive effects like a negative attitude towards learning

uses simple and cheap means and media which are little susceptible and that are available anyway

uses means which are trouble-free

does not teach thingsks that are already known to most learners[bw5]

builds up on the given knowledge and abilities of the learners

has undergone a risk assessment for potential failurenot too much risc for failing

Adjustment to the given situation:

My/our programme is adjusted....

to the language habits of the learners

to the previous knowledge of the learners

to the ways of learning predominant among the learners

to the environment and the way of living (live styles) of the learners

to the resilience of the learners

to resources that learners are used to

to those tasks which result from the framework, that are asked in practice

to those ways of communication which are preferred of the learners

to the time possibilities of the learners and of the framework

to the interests of the learners and excites motivation

Productivity:

My/our programme....

offers a line of orientation

imparts key qualifications

creates and improves contacts (e.g. to other learners, practitioners)

supplies cognitive schemes (e.g. methods that can be generalised)

develops and promotes (learning) strategies

helps the learners to gain more routine with the use of reference works

Self motivation for teaching and learning persons:

My/our programme....

is a suitable basis for fruitful questions

means learning in a playful way

means learning in a social way

bears recreational features

is carried out without glancing at a certificate

is carries out without fearing to fail

promotes the learning in competition with oneself and not to others

takes the joy in physical exercise during the learning process into account

takes the joy in intellectual games into account

takes the joy in an artistic expression into account

shows the learner especially the positive effects of what he/she has already learnt

Development of learners’ personality:abilities

My/our programme....

increases the intelligence of the learners

promotes the creativity of the learners

helps to diminish fears of the learners

strengthens the conflict abilities of the learners

increases the communication abilities of the learners

gives the learners a better self-esteem (self confidence)

promotes the decision-making ability of the learners

increases the sense of responsibility of the learners

increases the moral sensibility of the learner (in regard to the subject, qualification)

Consistency:

In my/our programme....

the teaching people “live” what they are imparting

the teaching people themselves are experienced in handling the electronic devices

quality of verbal/audiovisual material is high

the tasks agree with the practical requirements

the learners do have as much scope as in professional practice

there are no serious discrepancies between what is said and what is done

is a close distinction/distribution between the two sites of blended learning on the basis of their different functionalities and possibilities

7.Utility

Is fit for purpose

Conforms to appropriate regulatory frameworksC. Criteria to think about the efficacy of your e-learning programme

Clarity:

(reliable and comprehensible performances)

of knowledge

of learning steps-process

of concepts

of classification

of clauses[bw6]

of questions (challenge)

of assignments („what shall the learner do?“)

of aims and criteria for judgments

of positions (authenticity, teacher as model)

Relevance and reference:

(imparting valid knowledge)

for/to the subject/underlying discipline

for/to the research level (state of the art)

for/to the further learning process

for/to examinations

for/to the learners’ situations

for/to the learners’ future

for/to the society and subsystems (professions e.g.)

Structure and sequential order:

(imparting knowledge “in measured doses”)

by resuming what is already known

by accentuation of main points

by time management

by space management (learning environment)

by continuity

Representational forms of instruction

(using aids for learning and thinking)

through visualization (images, transparencies e.g.)

through examples

through analogies

through concrete experiences for learners

Reflection by the learner

Practising:

(application of knowledge)

proving knowledge in practice