No.26 Benchmarking with Other Institutions

No.26 – Benchmarking with other Institutions

Benchmarking for Quality Assurance in vocational, education and training (BEQUAL) can be found on-line at . It is available in 11 languages: English, Greek, German, French, Italian, Spanish, Bulgarian, Romanian, Lithuanian, Polish and Turkish; and is based on a self assessment questionnaire that the provider fills in online. The replies are stored in a database and the user can access the overall results in a public page, or they can register and see filtered results by country and by type of institution.

The BEQUAL questionnaire is based on the EQAVET quality cycle and is structured in four steps to quality assurance, namely:

1. Planning
2. Implementation
3. Evaluation
4. Feedback and improvement

Each step is divided into detailed processes. Statements under each process are indicative of the ways an organisation may handle the specific processes.

The user may enter “My Statistics” on the site in order to see results as well as the strongest and weakest points compared to other VET organisations in the same country or organisations of the same type throughout Europe. The tool is dynamic, so results may be different for these two types of comparison and of course may change over time.

The user may edit (resubmit a new) questionnaire at 12 month intervals. Data is stored anonymously.

Example of one quality statement.

Planning: Definition of needs in Human Resources

Once we set our objectives, we draw up a human resources strategy or an equivalent plan to support their achievement.

For example:

• We systematically determine competences which are necessary to achieve objectives
• We annually evaluate the competences of our staff
• We have an annual training plan in order to achieve our objectives
• We evaluate the efficiency of the human resources management policy
• We continuously evaluate the competences of our trainers, including subcontractor

No.27 – Defining indicators for quality management

Performance indicators provide, at a glance, a picture of the performance of the vocational, education and training organisation in a given dimension of performance.

When designing a quality assurance framework, it is important to carefully choose the dimensions of performance that you want to monitor and evaluate, in line with the main objectives of the organisation, within available resources.

Other important criteria for the selection of dimensions of performance are:

• Is it feasible to monitor and evaluate the selected dimensions of performance?
• Is there capacity for improvement in the selected dimensions of performance?
• Do the people involved have a shared understanding of what is included in the dimensions of performance to be evaluated?

For each dimension of performance define the quality criteria and list the key evaluation questions.

The performance indicators serve as a measurement of the satisfaction of the quality criteria that have been set.

The performance indicators should be carefully selected to be in line with the organisation’s quality policy and objectives, and be meaningful and measurable. They can refer to the process, the outcome or the impact. For training organisations, process and outcomes indicators are more relevant, while impact indicators are relevant at policy level.

Indicators can be expressed in many different forms and can be qualitative or quantitative. Quantitative indicators are easier to measure and to understand. They can express explicitly measurable units (e.g. number of enrolments, number of graduates) or qualitative information expressed in numerical scale. A typical example is learners’ grades or marks that are expressed in 0-10 or 0-100 scales. Qualitative indicators may express levels of performance (good, ..., bad), views and opinions (satisfied, ..., dissatisfied) and other subjective information. For qualitative information, we often use levels defined with descriptors (level 1-5, A-C, etc), rubrics, checklists and other instruments, so that the data are scientifically sound.

Hints and Tips:

• This instrument is best used with staff at their induction or in quality staff training.

Worked Examples

Performance dimension: Qualifications of teachers

Defining indicators

Performance dimension: Provision of information and induction services

Defining indicators

No.28 – Data collection planning

This instrument will support an organisation to plan its data collection activities for the performance indicators that are specified in its quality framework.

For each performance indicator the user fills in one row in the table, as follows:

Column one: performance indicator

Column two: the specific data elements that will be used for measurement of the indicator. For example if the indicator is ‘drop out rates’, the data elements are: number of learners enrolled and number of learners completing their studies.

Column three: the data sources are mentioned here. Data sources can already exist in the organisation (eg registration records, records of students etc), in external sources (eg statistics, external reports) or can be obtained from other sources, mainly learners, teachers, staff but also stakeholders, parents etc.

Column four: the data collection method should be written (eg aggregation of existing records, surveys, questionnaires, interviews and other methods that have to be planned, according to the needs of data collection).

Column five: write down who is responsible for the data collection. It can be a single person or a group of people. For example, if you plan a large scale survey of learners, you may need to identify one person as having responsibility overall for the whole survey and one teacher in each class to get responses from learners.

Columns six and seven: write down the start and end dates of the data collection.

When you have completed the table for all performance indicators, you will need to review the planning schedule overall. Some indicators may have the same data elements, or you may be able to adjust your indicators slightly to use the same data elements. Some data may be difficult or impossible to obtain for ethical, practical, budgetary or time reasons. Some data collection methods are easier to administer, while some others may require specific expertise. A careful review is needed, bearing in mind the available resources.

Hints and Tips:

• This instrument is best used with staff at their induction or in quality staff training.

Data collection planning

No.29 – Ideas for Effective Communication and Transparency

This is a guide to support reflection and to generate new ideas on how the institution tackles the issues of effective communication and transparency. If you have further ideas or suggestions, please contact a project partner so that they can be added to this list.

Ideas for effective communication and transparency:

• Often information is considered to be clear when it is part of the institution for a while. The management should be aware of the fact that new staff might not know much about the institution. A personal introduction into the institution can be a very good strategy to ensure that all new staff get to know the institution in depth.

• Information is often published on the website, the intranet or provided via email to staff. Publishing information has the benefit that everyone can access consistent information when it is needed. At the same time teachers and trainers experience a lack of time. This can lead to the problem that they do not read information carefully or don’t even look for it.

• Staff briefings with groups can be a very straightforward and useful way of providing information directly to staff. Information can be presented succinctly and clearly, perhaps with a handout, or guidance of where the information is held on the intranet, which staff can access later. The benefit of this approach is that all staff have been given the information in an efficient way; best used for non-controversial, straightforward messages. Unfortunately it is not necessarily known how well staff have listened!

Effective communication between management and staff is essential to support the development of quality within the institution. By informing and involving staff in the developments of your institution, staff feel more integrated into the institution which can raise their commitment and the effectiveness of their contribution.

• Personal conversations with staff members can support the spread of important information, especially where it is ciritical that staff ‘buy into’ what they are being told. Direct contact with staff will enhance the atmosphere in general and the information is more likely to be valued and remembered. Personal conversations give staff the opportunity to ask questions if they are unclear about anything or to discuss the themes. This is more likely to increase understanding and ownership of the information. Conversations can be held during breaks, during the working time or in team meetings. Although it takes more time to have conversations with staff instead of sending an email, it is much more effective.

• Communication routines are an important tool in the communication strategy. Develop a number of ways to communicate with staff that become part of the institutions’ culture. For example, there may be an annual address by the CEO or Principal; a termly update on the intranet; monthly newletters; and weekly staff meetings in each area. Staff will expect to receive information through these different media and will know where to look for it and where / how to question or discuss it.

• Overall, consider What? Who? When? How? Did it work? Whenever there is information to communicate ask the following questions and prepare carefully:

 What information needs to be communicated? Be clear on the message and prepare background or support materials if appropriate.

 Who needs to know it? Is it all staff or just some?

 When do staff need to know? Does it matter if staff hear this message at different times? Is it important to let staff know together or perhaps in a specific order?

 How shall it be communicated? Do you need some response from staff or do they need to be given the opportunity to discuss it or ask questions? A controverisal topic, or one that will greatly affect the way staff will work, is best discussed with staff individually or in small groups. A piece of general information about changes to the stationary supplier can be communciated in a newsletter or on the intranet.

 Did it work? Immediately and over time consider whether the communication worked. However it was delivered, was the message listened to and really heard? Are staff now acting on the information? Repeat and refresh messages and use different media to share them as necessary until you are confident that ‘it worked’. Remember that even if the information was communicated effectively once, staff may still need reminding and new staff won’t know!

No.30 – How can we avoid early learner drop out? Ideas and small successes

This is a guide to prompt reflection and to generate new ideas on the theme of avoiding early learner drop out. If you have further ideas or suggestions, please contact a project partner so that your ideas can be added to this list.

• Know your learners really well

 Get to know their strengths so that you can help these to shine through

 Get to know their weaknesses so that you can help them overcome these

 Get to know their personal circumstances so that you can refer them to the correct services. They can then receive the most appropriate emotional, financial and academic support they need

 Get to know their personal and professional aspirations so that you can steer them towards the most appropriate agencies, experiences and opportunities. Design assessments which enable learners to pursue and explore their own interests rather than yours

• Put yourself in their shoes - consider what you would like to experience as a learner and make sure that is what they get. For example:

 A good induction / welcome to the institution and the course

 A clean and well resourced environment that is fit for purpose

 Thorough processes that identify needs and ensure that these needs are met

 Caring teachers and tutors who are interested in learners as individuals

 Positive working relationships with peers and teachers

 Interesting and challenging lessons and assessments

 Swift and detailed feedback on assessments which praise achievements and support individual improvement

 Opportunities to give feedback, to influence the learning experience and to have suggestions acted upon

• Provide support mechanisms

 Teachers and tutors need to meet with their learners individually on a regular basis to provide the support they may need. If the teacher or tutor cannot provide the necessary support themselves, they need to refer the learner onto the correct source of help.

 The 'buddy project' is very successful in Landstede. Volunteers aged 25+, although many are pensioners, are linked to and guide a learner. This may be support with calculating or language learning, and can also be coaching a learner on how to schedule their homework, helping a learner out with personal problems, getting them out of bed to class.... Sometimes the buddies stay in contact with their learners throughout their time studying.

 At Westminster Kingsway College peer buddies are also used. Within a class learners can be ‘buddied up’ making the learners responsible for their peer. This may involve phoning them up if they tend to sleep late or collecting handouts for them if they miss a lesson. Making the learners responsible for each other has had a very positive impact on keeping learners on programme.

• Measure - by checking on figures for the last three years – and identify when learners are most likely to drop out.

 If learners are most likely to drop out in the first year – put more / the best resources into the first year.

 If learners are most likely to drop out when the first assessment is due – put in more individualised support at this stage.

 If learners are most likely to drop out when the first assessment is returned to learners – provide supportive and personalised feedback. Ensure that the learners who are not performing well get the individual support and help they need.

 If learners are most likely to drop out after a longer holiday – schedule in something exciting after the holiday, such as a visit or trip or a social occasion. Make sure the learners know before the holiday that it will happen after. Do not set an assessment deadline for after the holidays. Learners may not have finished their work, perhaps because they have not had support from their teachers over the holiday period, and may not feel able to return without it completed.

• Get all the help you can

 Work closely with parents, carers, and employers to make sure learners stay on programme and are on track to be successful. Discuss progress regularly with the people who are interested in the learner, and alert them to any concerns.

 Work closely with community groups and employers to ensure learners are getting all the support they need including a range of positive experiences and opportunities to support their learning and their future aspirations.

 At Landstede a representative from the local government - involved in reducing drop out – attended a team meeting. Working together increases the understanding of everyone involved and this may help reduce early drop outs and develop strategies to keep learners on programme.