Title: The development of a rubric for peer assessment of individual teamwork skills in undergraduate midwifery students
Carolyn Hastie1, Kathleen Fahy2, Jenny Parratt3
Background: Poor teamwork is cited as one of the major root causes of adverse events in healthcare. Bullying, resulting in illness for staff, is an expression of poor teamwork skills. Despite this knowledge, poor teamwork persists in healthcare and teamwork skills are rarely the focus of teaching and assessment in undergraduate health courses.
Aim: To develop and implement an assessment tool for use in facilitating midwifery students’ learning of teamwork skills.
Methods: This paper describes how the TeamUP rubric tool was developed. No research reports on how to teach and assess health students’ teamwork skills in standing teams were found. The literature, however, gives guidance about how university educators should evaluate individual students using peer assessment. The developmental processes of the rubric were grounded in the theoretical literature and feminist collaborative conversations. The rubric incorporates five domains of teamwork skills: Fostering a Team Climate; Project Planning; Facilitating Teams; Managing Conflict and Quality Individual Contribution. The process and outcomes of student and academic content validation are described.
Conclusion: The TeamUP rubric is useful for articulating, teaching and assessing teamwork skills for health professional students. . The TeamUP rubric is a robust, theoretically grounded model that defines and details effective teamwork skills and related behaviours. If these skills are mastered, we predict that graduates will be more effective in teams. Our assumption is that graduates, empowered by having these skills, are more likely to manage conflict effectively and less likely to engage in bullying behaviours.
Keywords: teamwork; teamwork skills; collaboration; midwifery students; assessment; peer-assessment; graduate attributes; social and emotional intelligence; maternity services; safety
Introduction
This paper reports on the process of development, implementation and initial evaluation of the teamwork skills assessment tool: the TeamUP Rubric. See table 1 for definition of key terms. Teamwork skills are based on the assumption that health professionals should relate respectfully to each other as peers instead of a model of domination and submission (1). The development of the rubric is nested within a larger action research study that is concerned with the processes of teaching and assessing teamwork skills in undergraduate health courses. There are many examples of tools to use to teach teamwork with emergency ‘drill teams’ (2, 3). In contrast, the focus of TeamUP is on ‘standing teams’’(4). A search of the literature showed that there are no research reports on how to teach and assess health students’ in standing teams.
INSERT TABLE ONE HERE
Poor teamwork is expensive; in both human and financial terms. Research and government enquiries consistently identify poor teamwork as one of the top modifiable causes of adverse health outcomes and avoidable suffering; not just for patients, but also for staff (5-10). Many of these adverse events result in death; in maternity care that most often means avoidable baby deaths (11-15). Bullying, a major contributor to adverse health outcomes for workers, is an expression of poor teamwork skills (16-21) and a major cause of high staff turnover and absenteeism (6, 17).
The National Competency Standards for the Midwife (22) the Code of Professional Conduct for Midwives (23) and the Code of Ethics for Midwives (24) refer to the midwife’s duty to collaborate with colleagues in respectful ways. A review (25) found no examples of any Australian university who assures that their Bachelor’s degree graduates meet the Australian Quality Framework (26) requirement to be able to communicate and collaborate: that is, teamwork skills.
Our first attempt to assess individual student teamwork skills involved students working together on a team project and then giving anonymised (that is, not anonymous to the lecturer) peer feedback. In this first attempt, students used a set of criteria (without definitions) to provide numerical marks and comments to each other. We did not provide any education about what constituted teamwork skills neither did we provide information on how to give and receive such feedback. Although the students’ evaluations of this peer making process were primarily positive, they did say that they wanted “more guidance, more specific teaching about teamwork, more teacher involvement, mid-session peer feedback” and “an improved ability to meet face to face” in their teams (27). Based on this feedback, we decided that a systematic, whole of curriculum approach was needed – this decision led directly to the action research study and the development of the TeamUP rubric.
Review of Related Literature
As discussed above, there are no published individual teamwork assessment tools for ‘standing teams’ in the field of health. Broadening the focus to include any university discipline, only one teamwork assessment tool for use with students was found: the Comprehensive Assessment of Team Member Effectiveness (CATME) (see http://www.catme.org). The CATME tool has been widely adopted for use in Universities. CATME was developed from 180 items extracted from a literature search and tested on university students to identify what students thought were critical team-member behaviours (28). In its most recent form, CATME is a five-item tool which is centrally administered from the CATME website.
Following our own review of the literature and our analysis of the CATME tool, we had a Skype meeting with Matthew Ohland, one of the chief authors of CATME to discuss how the CATME tool could be refined to suit our purposes. We eventually decided not to use the CATME tool because of the limitations we found that include;
1. The assessment process is all under the control of the CATME team in the USA allowing no opportunity for change or the addition of qualitative feedback,
2. The CATME items are ill-defined which reduces their usefulness in helping students to know exactly what is required and,
3. The lack of definition undermines validity of the students’ ratings because different words mean different things to different raters (29).
The Association of American Colleges and Universities (AACU) (30) developed a generic rubric for use at the institutional level (available from http://www.aacu.org/value/rubrics/pdf/teamwork.pdf). The AACU rubric, like the CATME tool, did not have the detail required to be useful as a guide to students and teachers about what teamwork skills actually are.
Hugh and Jones (31), in their critical review of the teamwork teaching and assessment literature, highlighted the paucity of existing tools for use by university educators and made the following recommendations:
1) Modify the AACU teamwork rubricand use the rubric to give students formative and summative feedback from their peers within subjects.
2) Commit to the Development of Teamwork; this means teaching students how to be effective teammates and giving them opportunities to practise and get feedback on their developing teamwork skills.
3) Design Assignments that Elicit Teamwork
4) Focus on the Teamwork Process; not just the end product
5) Provide Meaningful Feedbackby giving students the opportunity to practise, receive feedback, and then try again before any marks are assigned to teamwork skills.
We have integrated these recommendations into the development of the TeamUP rubric as discussed below.
Educational Theory Informing the TeamUP Rubric
An assumption guiding the development of the TeamUP rubric is that teamwork skills and associated behaviours can be taught, practised and assessed. This methodology is in contrast to a ‘personality’ or ‘strengths’ approach to teamwork based on traits like ‘openness’, ‘kindness’ or strengths like ‘completer/finisher’ (32, 33). These types of models assume that people’s current way of behaving in teams defines how they will be in future teams, which we, as educators, reject. In this section we outline the educational theoretical foundations of the rubric.
We used a situated learning perspective to guide educational decisions. Situated learning is based on the belief that knowledge and skill development are contextually situated and vitally influenced by the activities and culture in which it is used (34-37). Relevant key attributes of situated learning integrated into this project are:
1) Encouragement of student reflection on experience;
2) Scaffolding, including apprenticeship and coaching (38) e.g. the development and refinement of a detailed rubric and self-paced lessons;
3) Interactivity, which means designing learning activities where students and academics are interactive with others;
4) The engagement of multiple senses and emotions to enhance the learning process; and,
5) Multiple opportunities for practice over time, particularly for practical and complex skills (39).
Aspects of behaviourist theory were also relevant to guiding the development of the TeamUP rubric. Behaviourist theory underpinned the provision of clear behavioural descriptors and outcome measures, together with the use of assessment feedback to reinforce or extinguish certain behaviours; all of which are valuable educational practices (39).
Assessment in higher education has been mainly focused on the grades achieved by students; not how they communicate and collaborate with others. David Boud (2010) a highly respected expert in adult education argues for assessment tasks that are centered on meaningful, long-term learning, not simply used to make judgments about grades (40). Leading educationalists accuse academics of holding onto old models of teaching and assessment that focus on short term and relatively superficial knowledge, understanding and technical skills (41-43). For health care undergraduate students, this means (in part) that insufficient attention is currently given to teaching and assessing teamwork skills. The lack of attention to these crucial aspects of professional practice may be because teamwork is considered difficult to assess, partly because it is ill defined.
This lack of assessment of individual teamwork skills is in spite of the fact that many health curricula include large amounts of teamwork for students: the most well-known method being team-based learning (TBL) (42). Student involvement in TBL does not specifically involve teaching or assessing individual teamwork skills. The key goal of TBL is to facilitate learning of course content through teamwork activities instead of traditional lectures and tutorials. We rejected a TBL approach because we wanted to implement team-based assessments in addition to lectures and tutorials. Our goal was for students to learn pre-specified teamwork skills, which would be self and peer-assessed, along with content. When students work on team assignments, we generally assign a weighting of 50% for the team’s assignment (all students get the same mark) and 50% for individual teamwork skills (students get different marks).
Methods used to develop the TeamUP Rubric
We transformed the AACU rubric into the TeamUP rubric by: 1) defining each of the domains and clarifying the related teamwork skills, 2) writing the skills in positive behavioural terms, and 3) considering our own knowledge and experiences as these related to what we considered to be key teamwork skills. We used two methods during this development phase: Feminist Collaborative Conversations and Content Validation via pilot testing. These methods are explained below.
Feminist Collaborative Conversations
Feminism is defined as “the theory, research and practice of identifying, understanding and changing the intrapersonal and social factors that sustain women’s disempowerment” (44). In this case we wanted to ensure a framework within which midwifery students would learn teamwork skills and so free themselves from the inhibitions that come with habitual and gendered ways of relating. Habitual and gendered behaviours that we were keen to change include those associated with the victim-persecutor-rescuer dynamic of bullying (45) and the ‘doctor-nurse game’ (46, 47). Feminist collaborative conversations work particularly well as methodology for action research studies where the conversations go beyond ‘chatting’ to become an intensive site for practice transformation (48, 49). This model guided us in critically discussing teamwork skills as they might relate to the TeamUP rubric. In these conversations, we valued our own and each other’s’ lived experiences of working in and leading teams; both at work and in the community. These experiences included clinical and educational leadership in the health and higher education bureaucracies. Community-based experiences involved leading and participating in women’s meditation and spirituality groups; participating in the management of women’s services including setting up and managing a domestic violence service. We challenged ourselves, and each other, to draw learning from our experiences; valuing similarities and differences. We also critically discussed the theoretical and philosophical basis for our beliefs about what teamwork skills are important and how they should be worded and justified (48, 49). We made explicit the assumptions underpinning our decisions e.g. that as health professionals, midwifery students need to know and are guided by the relevant Codes of Conduct and Ethics; that as university students, they are subject to the Codes of Conduct of the university; including no bullying; that all health students learn basic communication and counselling skills (so these do not have to be repeated) and; that midwifery students learn, and regularly practice reflection to learn from experience (50).
The outcome of these feminist collaborative conversations was the decision to keep, and slightly modify the five teamwork domains of the AACU rubric. Also arising from these conversations was the definition of ‘teamwork skills’ as shown in Table 1. We debated and consensually decided upon the initial definitions of each of the teamwork domains and the behavioural descriptors. Decisions about the theoretical foundations arose from critical conversations between all authors. The process of this decision-making included a review of the literature; mainly the grey literature (in particular: (51-56). Our aim was to ground each definition and teamwork skill description in the extant theoretical literature whilst ensuring that it also resonated with our own lived teamwork experiences.
Content validation of the TeamUP rubric
Once the initial draft of the TeamUP rubric was complete, we engaged in a process of pilot testing. To improve the content validity of the rubric prior to implementation, we conducted one focus group with midwifery students (n=8) and another with health and business academics (n=14) experienced in teamwork assessments. The academics were members of the larger, multidisciplinary, action research team. Both groups used a paper version of the rubric to anonymously assess their own and each other’s performance during the focus group process. The midwifery students’ focus group also undertook a practice of self and peer assessment using an online version of survey tool. The outputs from both paper and online versions of the assessment process were not shared with their peers, although the experience and learning was explored within each group.