Title: Using Responsive Evaluation Techniques to ‘Mirror’ the Teacher Professional Development for ICT Landscape in a Development Context
Paper Type: Postgraduate
Abstract:
The professional development landscape for ICT integration in Africa is complex.Fundamental to the complexity is the myriad of initiatives and schemes for new technology integration that have emerged over the last decade. Many such ICT initiatives in Education and Development have been externally driven and have failed to live up to the ambitious aspirations of their proponents because they have not been demand-led historically and because they give insufficient attention to the involvement of stakeholders in defining the need and purpose of the development process. An absence of conceptual clarity on the objectives of teacher professional development for ICT initiatives in the region adds to the confusion and undermines the quality of provision.
In this research the author seeks to explore the potential of responsive evaluation techniques incorporating a hybrid of dialogical and story-based frameworks and tools to ‘mirror’ the TPD for ICT landscape in a partner country in the Sub-Saharan Africa (SSA) region. The study will have an action research orientation. The aim will be to encourage reflection on the appropriateness of a responsive evaluation technique utilizing an evolving toolbox of hybrid problem-solving frameworks and tools to address stakeholder needs in the partner country, and whether principles and elements of this blended technique could be applied to other developing country contexts in the region.
Taking on a responsive evaluator role of facilitator, interpreter, educator and Socratic guide, the author will test the potential of the hybrid tools and frameworks to facilitate participatory processes of analysis and reflection of TPD using ICT. The author will initiate the responsive evaluation process by conducting in-depth interviews using an ‘activity theory’ interview protocol for probing issues and exploring the perspectives of different stakeholder groups - managers, teacher educators, teachers - affiliated to national and local TPD for ICT programmes and initiatives. The author will integrate a ‘Most Significant Change’ narrative technique into the interview protocol to engage participants in telling their stories of significant change since the integration of technology in their practices.
The author’s role throughout the responsive evaluation process will be to concentrate on the emerging contradictions and controversies and to create the conditions for the interaction between stakeholders on the issues identified. While unveiling contradictions can be problematic, if they are handled constructively they can invoke development through expansive learning. The collaborative reflection on the emerging issues can provide stakeholders with a ‘mirror’ of the work of implementation at different levels. As a facilitator to the dialogue process the author’s research report should constitute a vehicle – a ‘working document’ for the next level of stakeholder dialogue – that of ‘negotiating’ solutions that are appropriate to local needs and context. (440 words)
Keywords: Responsive Evaluation; ICT; Teacher Professional Development; Activity Theory; Most Significant Change
1. Introduction
In this paper I seek to explore the potential of responsive evaluation techniques incorporating a hybrid of dialogical and story-based frameworks and tools to facilitate a participatory process of analysis of ICT integration practices in Teacher Professional Development in a partner country engagement in the sub-Saharan Africa (SSA) region. The results of the joint analysis can contribute towards the conceptualization of a national framework for ICT in Teacher Professional Development.
Isaacs (2006) defines Teacher Professional Development (TPD) as a systematized, initial and continuous, coherent and modular process of professional development of educators in accordance with professional competency standards and frameworks.
Abma (2006) describes responsive evaluation ‘as an approach which aims to reframe evaluation from assessment of programmes on the basis of policy maker’s goals to engagement with and among stakeholders about the value and meaning of their practice’. Stakeholders would include policymakers, managers, practitioners, community and target groups (Stake, 1975; Guba & Lincoln, 1989; Stake & Abma, 2005 cited in Abma, 2005).
2. An Overview of the Use of ICT in Teacher Professional Development in Africa
2.1 A complex landscape of provision
Isaacs (2006) identifies the professional development landscape for ICT integration in Africa as complex. Fundamental to the complexity is the myriad of national and international initiatives and schemes for new technology integration that have emerged over the last decade.Research conducted by SchoolNet Africa, the Commonwealth of Learning and the International Institute for Communication and Development (2004) identified an estimated sixty ICT-related TPD programmes underway in Africa. According to Farrell and Isaacs (2007) many of the programmes often involve ‘one-off, topic-led, short-term training programmes that aim to develop specific skills of teachers, but which do not necessarily comply with professional standards of competency development’ (ibid.:20).
Isaacs (ibid) reports an absence of conceptual clarity on the objectives of teacher professional development for ICT initiatives in the region. Mandinach (2005) suggests that the lack of clarity is pervasive in education systems globally, noting that while educational institutions seem to be aware they should be joining the ICT integration movement, they are not clear as to the purpose or the gains.
2.2 A definition of purpose
Kennedy (2005) proposes that defining whether the fundamental purpose of a TPD intervention is to achieve transmission or to facilitate transformative practice can provide a powerful tool for conceptual analysis. Drawing from the literature, she developed a framework for categorizing different models of TPD along a broad spectrum of delivery inherent in their purposes of provision - moving from transmission through transitional to transformative purposes which develop increasing capacity for teacher autonomy.
The UNESCO (2008) ICT competency framework for teachers envisages a similarly broad spectrum of delivery inherent in the purposes of modular TPD provision for ICT integration - moving from technology literacy, through to knowledge deepening to knowledge creation purposes which develop increasing capacity for teacher autonomy in the utilization of ICT to enhance practice (Figure 1).
Institutions and schools in collaborationwith the community (local, national, regional)
Given /
‘Transition’
‘Knowledge deepening’
Teacher professional development focus on the use of ICT to guide students through complex problems and manage dynamic learning environment
3 / ‘Transformation’
‘Knowledge creation’
Teachers are themselves master learners and knowledge producers who are constantly engaged in educational experimentation and innovation to produce new knowledge about learning and teaching practice
4 / Experimentation in context
Programmes / 1
‘Traditional’
‘Technology add-on’
Teacher training focus the use of ICT as an add-on to the traditional curricula and standardized test systems / 2
‘Transmission’
‘Technology literacy’
Teacher training focus on the development of digital literacy and the use of ICT for professional improvement / with varying solutions
Institutions and schools
as relatively isolated from the community
Figure 1: Two dimensions of TPD transformation.
Adapted: Kennedy, 2005; Hakkarainen et al. 2008; UNESCO, 2008b
The two dimensional trajectory for TPD transformation represents a conceptual framework in which to situateevaluationprocesses to engage stakeholders in debate and reflection on issues of both the purposeand power relationsinherent in the trajectory; aspractitioners and institutions seek to move from isolated, passive consumers and implementers of externally defined education programmes for ICT knowledge and skills acquisition towards more open communities of active learners and learning organizations that generate new knowledge on the use of ICT to enhance educational practice(Butler, 2004; Kennedy 2005; Hakkarainen et al. 2008)
3. Responsive Evaluation Techniques to ‘Mirror’ Practice
3.1 A hybrid of evaluation frameworks and tools
Dialogue fostered through in depth-interviewsand sharing of stories with stakeholders, captures the essence of responsive evaluation. This is an approach that would seem appropriate for the creation of a multi-voiced dialogue involving programme managers, teacher educators and teachers in a joint analysis of the prevailing practices of TPD using ICT in the partner country.I will use responsive evaluation techniques incorporating a hybrid of dialogical and story-based frameworks and tools to create the necessary conditions for poly-vocal ‘dialogue’ or more specifically for what Senge (2006) terms ‘learning conversations’ that balance inquiry and advocacy – where stakeholders can turn the ‘mirror inward’ and “expose their own thinking effectively and make that thinking open to the influence of others” (ibid.: 8-9).
3.2 In-depth interviews using an ‘Activity Theory’ framework
Activity Theory (AT) (Engestrom, 1987) is currently widely applied to study technology-based learning and working situations (Issroff and Scanlon, 2001). The use of an AT framework can both generate clarity of the environment and make more explicit the assumptions, values and beliefs that underpin organizational, technological and pedagogical perspectives of ICT integration processes (Lim and Hang, 2003; Demiraslan and Usluel, 2008; Robertson, 2008).
While it is beyond the scope of this paper to provide a full description of AT, a key feature of the theory for the inquiry process is the extended model of Activity Systems developed by Engestrom (2001) that conceptualizes all human activity as the interaction of six inseparable and mutually constitutive elements: subjects, tools, object and outcome, rules, community and division of labour (Figure 2).
ToolsSubject / /
Object Outcome
Rules / Community / Division of Labour
Figure 2: The six elements of an activity system
Source: Engestrom, 2001
The common lexicon defined by the six elements of an activity structure can provide a framework for developing an in-depth interview protocol to track the purpose of ICT integration across the different system models. Table 1 presents an AT interview protocol with examples of questions for probing issues and exploring the perspectives of the different groups of stakeholders (managers, teacher educators, teachers) affiliated to the activity systems of national and local models for TPD using ICT. The exploration of the complex pedagogical, social and technological issues inherent in the ICT integration processes marks the start of a responsive evaluation analysis.
The interviews with stakeholders provide an opportunity to identify the tensions and contradictions at play within and between model systems. While unveiling contradictions can be problematic, if they are handled constructively they can invoke development through expansive learning (Engestrom, 2001).
Table 1: In-depth interviews with stakeholders affiliated to different ‘Activity Systems’Managers / Teacher Educators / Teachers
Subject:Managers experience, administrational use of ICT, the place of ICT in daily life, training needs and skills related to ICT
Object:
- What are the objectives of the institution/school with respect to TPD programme for ICT integration? Are the objectives directly/ indirectly linked to the objectives of national vision/ policies for ICT integration? If so, how are they related?
- Are there implicit objectives in the programme that may be a ‘contradiction’ to national objectives of ICT integration?
- How have the objectives of TPD for ICT programme changed over the years (before the programme was in the respective phase of ICT integration)?
- Are there different objectives for different individuals in the institution/ school (Directors/Principals, ICT coordinators, Heads of departments, ICT department)? Who dominates the objectives? Why?
ICT and non-ICT Tools used to mediate the institution’s/school’s object to implement TPD programmes for ICT integration:
- TPD Model type – training, award bearing, deficit, cascade, school-centred, standards based, coaching/mentoring, community of practice, inquiry/action research, transformative?
- ICT tools - Administration, Learning and Teaching tools, e-mail communications with teachers, websites
- Non-ICT tools – in terms of curriculum, thinking, assessment, creativity, project work and other administrative, system management (reports/correspondence) and lecturing functions
Procedures and policy that mediate TPDfor ICT design and implementation
- ICT policy (do institutions and schools have a voice in policy formulation?)
- Standard: who sets the standard? – National TPD Institutions? MoE? International Bodies?
- Administration of facilities, acquisitions and budgets
- Percentage of time and curriculum integration
- Curriculum changes
- National examinations
- Implicit rules and explicit beliefs about learning (self-directed, co-operative, collaborative) knowledge (constructing and/or memorizing) and performance (ranking)
Roles and responsibilities of different actors working together in order to achieve the object of ICT integration in institutional/school TPD programmes
- ICT co-ordinator
- teacher educators, teachers
- technical support staff
- administrators
- leaders
- Ministry of Education
- National Curriculum Development Centre
- Other TPD institutions
- Other Schools
- Suppliers of gds (hardware/ software) and services (contractors, solution providers)
- Society at large
Object:
- What are the most important factors which encourage teacher educators to use ICT in instructional activities? Do they plan to go on using ICT?
- What are the implicit and explicit beliefs teacher educators (and student teachers) have on learning and teaching? How have these objectives changed over the years (before the institution was in the respective phase of ICT integration)?
- Are there different objectives for different individuals in the institution (teacher educators, student teachers, teachers)?
- Who dominates the objectives? Why? Is there shared ownership?
ICT and non-ICT tools directly related to: curriculum content, classroom/online discourse, communication and management, and assessment
- ICT tools (Admin, Learning and Teaching tools, Computer Mediated Communications, and the Internet) – What are the ICT tools that teacher educators use in the teaching-learning process? Have the ICT tools become the object of teaching and learning or are ICT tools used to enhance teaching and learning?
- Non-ICT tools - Administration, Learning and teaching Tools, reports, correspondence
- Implicit and explicit beliefs – what should the classroom/online environment be to make the most effective use of ICT?
- Standard setting –what are the criteria that teacher educators use when evaluating learning of student teachers/ teachers? Do criteria change in the programmes using ICT? Or are new criteria developed for these courses? Are there rules set by the institution about using ICT in courses? What is the impact of institutional/ national/ international examinations/ awards?
Roles and responsibilities –
- What kind of different roles and responsibilities do teacher educators assume in programmes using ICT?
- Does the institution administration adopt a supportive attitude about the use of ICT in teaching and learning processes?
- How does the supporting system function?
- Communities of practice – what kind of communication and collaboration is there among teacher educators in the institution (with other institutions) about the use of ICT in the teaching-learning process?
Object:
- What are the most important factors which encourage teachers to use ICT in instructional activities? Do they plan to go on using ICT?
- What are the implicit and explicit beliefs teachers (and students) have on learning and teaching? How have these objectives changed over the years (before the school was in the respective phase of ICT integration)?
- Are there different objectives for different individuals in the institution (teachers and students)?
- Who dominates the objectives? Why? Is there shared ownership?
ICT and non-ICT tools directly related to: curriculum content, classroom discourse, communication and management, and assessment
- ICT tools (Admin, Learning and Teaching tools, Computer Mediated Communications, and the Internet) – What are the ICT tools that teachers use in the teaching-learning process? Have the ICT tools become the object of teaching and learning or are ICT tools used to enhance teaching and learning?
- Non-ICT tools - Administration, Learning and teaching Tools, reports, correspondence
- Implicit and explicit beliefs – what should the classroom environment be to make the most effective use of ICT?
- Standard setting –what are the criteria that teachers use when evaluating learning of students? Do criteria change in the classes using ICT? Or are new criteria developed for these classes? Are there rules set by the school about using ICT in classroom programmes? What is the impact of national examinations?
Roles and responsibilities –
- What kind of different roles and responsibilities do teacher educators assume in class programmes using ICT?
- Does the school administration adopt a supportive attitude about the use of ICT in teaching and learning processes?
- How does the supporting system function?
- Communities of practice – what kind of communication and collaboration is there among teachers in the school (with other schools) about the use of ICT in the teaching-learning process?
Adapted: Lim and Hang, 2003; Gaible and Burns, 2005; Kennedy, 2005; Demiraslan and Usluel, 2008
3.3 Story elicitation using a ‘Most Significant Change’ technique
The ‘Most Significant Change’ (MSC) technique involves processes for engaging participants in telling stories of significant change, followed by dialogue with different stakeholder groupsusing the stories in order to select those that are perceived to be most indicative of significant change (Le Corru et al. 2003).This process provides a simple means of making sense of a large amount of complex information collected from many participants across a range of settings (Davies and Dart, 2007).