Abstract
This article is based on research and development carried out at the Centre for Outcomes-Based Education at the Open University (OU). It explores the way in which higher education (HE) assessment could meet specific criteria in order to assess the unique qualities that underpin the relationship between practitioner-researchers and the workplace. These unique qualities relate to the new knowledge and understanding created from finding solutions to real issues. Evaluation of curriculum developments assessing learning in the workplace at the higher education level suggests that being an experienced and successful practitioner requires both the ability to seek out relevant knowledge and the skills to apply that knowledge in the appropriate way to real situations. HE therefore needs to assess not only cognitive knowledge, but also the social and cultural knowledge that is an essential characteristic of a successful practitioner-researcher. This article sets out the criteria for developing this type of assessment. Social and cultural knowledge has been seen as 'intangible', but it is the position of the authors of this article that developing appropriate assessment must help to reveal this type of knowledge by teasing out how individuals apply them to their working practices. This could fulfil the aim of helping to make these skills more explicit for the individual practitioner and the workplace, while also ensuring that the assessment criteria is appropriate for the purpose of assessing the relationship between the practitioner's knowledge and behaviour in the workplace. Experienced practitioners rely on their experiential learning from past experiences within the culture of their workplace in order to develop new ideas that are successful and relevant to that workplace or sector. Practitioner-researchers therefore display a range of social, cultural and interpersonal skills, as well as a tacit knowledge and understanding of their particular workplace and sector. The workplace sets the scene in which the practitioner needs to operate, and the practitioner-researcher uses a wide range of behaviours to create practical solutions to real problems. Some areas of HE assessment, such as health and social care, have long required articulation with the workplace and cultural norms, and these have evolved to facilitate this interaction of research and practice. It is the spread of this articulation into new areas and across new cultures that presents issues for the development of effective assessment practices that are able to elucidate the full range of knowledge and skills used by practitioner-researchers in the workplace.
Introduction
This article is based on research and development carried out in the Centre for Outcomes-Based Education (COBE) at the Open University (OU). The relationship of practitioner-researchers and the workplace has been a focus of work carried out within COBE over the last six years and forms an important strand of the Centre's involvement in curriculum development across all faculties and disciplines. In the context of this development, 'practice-based learning' has been defined as an active process involving behavioural and emotional engagement with learning as well as cognitive engagement (Hansen, 2000).
Our findings suggest that the unique qualities of the new knowledge and understanding created from finding solutions to real issues are important considerations when designing experiential learning opportunities across the higher education (HE) sector. These unique qualities include the skills that relate to the social, emotional and cultural aspects of successfully operating in different work contexts and situations (Jarvis, 1999). When developing assessment strategies that can be applied to the experiences of practitioners, developments within COBE has found that the type of written work traditionally associated with HE assessment may fall short of capturing much of the learning that has occurred through such experiences and, in consequence, may be perceived as being one-dimensional rather than holistic assessment. While our innovative approaches to curriculum design in this area have been taken up successfully by OU faculties within their practice-based learning courses, our evaluation has begun to identify how more effective assessment criteria, directly related to the way practitioner-researchers operate in the workplace, may help HE deliver more holistic learning opportunities.
Evaluating practice-based research activities within the workplace
In order to fully understand the impact on OU students of assessing practice-based activities and experiences within the workplace, COBE commissioned an evaluation of one of our new curriculum designs for workplace learning. We define 'workplace learning' as learning that is based on students' real practices in the workplace. The workers are practitioner-researchers and students because they are carrying out research based on their practice in their own work situations. Our teaching and learning strategies for workplace learning support students through a range of reflective accounts of their experiences of undertaking practitioner-research activities, such as finding out how policies and procedures affect the way tasks are carried out. Furthermore, our distance learning pedagogical framework enables students to identify, within their own workplace, a person as their workplace facilitator who provides support in the workplace while the university provides academic tutors to assess students' assignments.
The evaluation was carried out with a cohort of 77 students who registered for the first presentation of an information and communication technology (ICT) focused work-based learning course based on COBE's approach to workplace learning. The aim of the evaluation was to measure the success of this curriculum design in terms of the students' satisfaction with their learning experiences. Student feedback was collected through informal discussions via e-conferencing, a formal student survey and follow-up interviews. The informal discussions with students were undertaken to encourage individuals to express their thoughts freely, along the lines of an ethnographic enquiry. Although only a small number of students participated, a high proportion described having difficulties with the reflective aspects of the learning tasks.
The survey was sent to the 68 students who completed their end-of-course assessment, and 19 responses were received. The survey comprised 17 statements and was designed to collect quantitative data using a Likert 5-point scale (Likert, 1932) ranging from 'strongly agree' to 'strongly disagree'. Each statement referred to an aspect of the students' learning experiences on the course. There was also an opportunity for students to include any feedback that they were not able to give using the formal survey responses. The number of participants who indicated that they had had problems with the reflective aspects of the course was 37% - a significant number. The follow-up interviews with a small number of the participants were used to verify the data and clarify the feedback received in the formal survey. In considering the viewpoints of the students, the key findings of the evaluation highlighted the overall positive feelings expressed by the majority of participants on this type of course. However, it was the few dissenting voices that gave the most valuable feedback because they highlighted the difficulty that practitioners can have in demonstrating, for assessment purposes, the full range of knowledge, behaviours and skills used within their workplace.
In our analysis of the evaluation findings, it became clear that the process of reflection on experience was less than straightforward in that those individuals who struggled with the process of being assessed through written assignments on their workplace experiences expressed viewpoints that ranged from feeling that it was 'too easy' to those who felt it was 'too difficult'. These comments led us to question whether it was those individuals who did not engage strongly enough with the essential process of reflection on practice who felt it was 'too easy', and those individuals who engaged too strongly with this process considered it 'too difficult'. By referring to the engagement of students with the reflection process, we were able to define 'reflection' as more than an academic activity of producing a written account of a given experience. Furthermore, the written reflection did not include the essential qualities of engagement that our students, as practitioner-researchers, had with the workplace. This led us to consider whether using written reflective accounts to capture the whole range of knowledge, skills and behaviours applied by practitioner-researchers in the workplace was the most appropriate method of assessment. We questioned whether assessing reflection by written accounts was more about the ability to write in an acceptable HE style rather than the actual workplace knowledge used by the practitioner-researcher. Assessment needs to be able to recognise the unique qualities of knowledge created by practitioner-researchers and the behaviours and skills that they use in their relationship with their workplaces.
This finding also had an affect on our own professional practice and highlights a particular benefit of practice-based inquiry in that this type of research enables new insights into what needs to be changed and how change should be made. Having carried out an evaluation into the appropriateness of the methodology used to assess the practitioner-researcher, we are aware of the benefits of developing new approaches based on practice-based research. We have concentrated our further considerations of these issues by exploring more fully how assessment strategies need to recognise the full breadth of knowledge used by practitioner-researchers to create solutions successfully in a range of different contexts and thereby validate that experiential-based knowledge.
The unique qualities of knowledge created by practitioners
Schn (1987) put forward the view that experienced practitioners in the workplace have particular skills that enable them to apply their knowledge in a way that leads to the best use of that knowledge. We have found that experienced practitioners are likely to have been undertaking a range of tasks over a number of years based on a sound level of knowledge and understanding. Such knowledge and understanding is used as the basis of seeking out new knowledge by applying practitioners' skills in a relevant way in an actual event or scenario. It should be recognised that these experienced workplace practitioners are already engaged in a learning process through practice by finding out what works best in any given situation. This enables them to develop resolutions for problems that do not necessarily mean they will need to read a book or attend a course (Gibbs & Angelides, 2004). Those engaged in practical professions (e.g., electricians and plumbers) regularly need to use their existing knowledge to research into ways of dealing with problems by developing new solutions while carrying out a task (e.g., finding new ways of laying wires or fitting pipes). Similarly, new surgical techniques have been discovered during surgery where experiential learning as well as more academic learning has provided a solution. Furthermore, law is created during practice in the courtroom where decisions are based upon previous, more academic learning and the facts of a particular situation. Likewise those designing new curriculum and assessment methodology may develop new approaches based on evaluation of student feedback. All practitioner-researchers, ranging from educationalists to plumbers, develop a unique type of knowledge that would be difficult to assess using the traditional written, reflective account.
Thus practitioner-researchers may acquire and create new knowledge and understanding from their need to address a real and usually immediate issue. As part of this knowledge creation process, they need to use and extend their tacit knowledge of their workplace thereby enabling the knowledge to be embedded into the workplace (Lindley & Wheeler, 2001). The tacit knowledge can be construed as knowing how things actually work and happen rather than how any textbook might say that they should happen. Tacit knowledge is often what makes one worker more effective than another and is built up through experience (Gourlay, 2006). Understanding the culture of the work- or life-setting and realising that if one moves to another setting the culture will be different is again something that is learnt through experience. With increasing globalisation, businesses are beginning to realise that even within the same sector of work (e.g., engineering) different cultures develop different social and cultural skills and ways of learning in the workplace (Hampden-Turner & Trompenaars, 2003). Although workplaces may superficially look the same and be engaged in similar activities, there may be a difference in culture, even within the same country, that requires social skills to negotiate, learn about and come to understand.
Identifying the knowledge used by practitioner-researchers in the workplace
Tacit knowledge includes social knowledge, emotional intelligence, interpersonal skills and behaviours, and cultural skills. Practitioner-researchers who are successful in the workplace need to display the full range of these behaviours, knowledge and skills in order to progress their ideas and achieve the most effective solution for the problem they are facing (Sternberg & Horvath, 1999). They need to know how to work effectively with others and how to communicate their ideas. Each workplace has its own culture and way of doing things and so those creating new knowledge to meet the needs of practical situations must know how to operate in a range of different contexts and scenarios. Gaining support and backing for innovation is very important because otherwise new ideas will not be taken forward. Thus knowing how to behave and communicate in a range of different contexts are key criteria for those who wish to carry out research and development as part of their practice.
Evidence for the relevance of such knowledge and skills also comes from the British Chambers of Commerce that represents many employers in the United Kingdom. This organisation has commented on the importance of the workforce having softer skills such as communication, teamwork and time-keeping in order for employers to be successful and profitable. Furthermore, Barrie (2004) refers to the necessity for graduates to be able to communicate, have social and professional understanding as well as a personal and intellectual autonomy, which might be described as being able to work independently in other contexts. These skills have been given many collective labels, but are often referred to as 'generic and transferable skills'. Although generic and transferable skills can be viewed as being crucially important in an ever-changing world of work, if HE is to be successful in providing meaningful opportunities, then the assessment of practitioner-researchers in the workplace must recognise the significance of social, cultural and interpersonal skills that are specific to a workplace industry or sector. For example, the Institute of Engineering and Technology (see http://www.theiet.org/) requires all members to have qualifications in the field of engineering that, particularly City and Guilds qualifications (see http://www.city-and-guilds.co.uk/cps/rde/xchg/cgonline), require workplace assessment of some of the social and interpersonal behaviours, knowledge and skills highlighted in this article. In seeking to find a more holistic approach to assessment so that the process of applying knowledge can be considered and evaluated (Biggs, 2003), we have first explored current practices.