Giving up on Success

Giving Up On Success:

A Case Study

Paper presented at the BERA Annual Conference 2003 (Heriot-Watt University, Edinburgh, 11-13 September 2003)

Chrysanthi Gkolia,

Faculty of Education,

University of Manchester,

e-mail:

Alan Jervis,

Faculty of Education,

University of Manchester,

e-mail:

Keywords: Integrated Learning Systems, computers in education, case study, implementation

Giving Up On Success

Abstract

This paper reports on a part of a case study of the implementation and use of an Integrated Learning System in a school.

The school could, and did, meet all of the apparently necessary criteria for the successful implementation of an ILS and yet, the ultimate experience was unsatisfactory. This study was designed to explore the reasons why the particular school chose to end its use of the ILS after two years of implementation and to examine the pupils’ attitudes towards their experience with the software as crystallised since its discontinuation. The majority of the school staff who used the software in their teaching was interviewed and the pupils’ attitudes were measured based using questionnaires.

The paper discusses the results of the staff interviews and tries to shed some light on the deeper reasons behind the cessation of ILS use. Some more general conclusions on the nature of difficulties involved in implementing an ILS are drawn.

Introduction

There has been a proliferation of educational software in recent years and software applications are now used in many diverse settings. They can be an alternative or supplementary learning medium which can be utilised to serve all kinds of educational goals (Dickson & Parsons, 1998).

Integrated Learning Systems made their appearance in the US in the late 1970’s and were primarily used for teaching of arithmetic and reading skills. Soon they became very popular and their use expanded very quickly. UK schools began purchasing and implementing ILS software in the early 1990’s and since then they have attracted a fair degree of controversy in the educational world (Cole, 1999).

Many researchers and commentators (e.g. Underwood, 1997; Bagley, 1996; NCET, 1994; Mills, 1994; Steeg, 1998; Becker, 1992) give an analytical description of an ILS and its main components. These are: curriculum or subject content, a pupil record system and a management system.

The published literature concerning the use of ILS is vast and extends to the three past decades (McFarlane, 1997). However, research in the field that dates from before 1990 has little relevance to today’s issues since the ILS courseware that is used today is very different from that used 20 years ago. Even research of the last decade is “of variable quality and relevance to present day implementations…” (McFarlane, 1997, pp.15).

The research questions and the methodology

This study attempts to explore the grounds on which a school that purchased an ILS, implemented it and used it actively for two years subsequently discontinued its use. The ILS was first used in 1999 and its use ceased two years later. The study attempted to find answers to the following questions:

1)Why did the school decide to invest in an ILS and why did they choose the particular package?

2)What were the experiences of the members of staff that used the ILS with their classes?

3)What were the conditions of implementation and use of the ILS in the school?

4)Why did the school abandon the ILS?

5)Did the school replace the ILS with other software and if yes what were the reasons they did so?

In order to answer the questions above, a case study was conducted in the school using various methods.

At first demographic data about the school were gathered based on the school’s official website and on performance data as reported in databases on the DfES website. Subsequently, a visit to the school was carried out in to collect quantitative and qualitative data.

Qualitative data consisted of semi-structured interviews with the five members of the school staff who had broad experience of the ILS during its years of use. Among them, the head of the Mathematics Department in the senior school was the person who initiated the purchase and choice of ILS for the school and the ICT Systems manager was the person who supervised the installation and technical implementation of the system. The rest of the interviewees came from the junior school where implementation was more intensive and rigorous. All had made use of the ILS with their classes for two years. The interviewees are coded in the results section as shown in the table below:

Teacher / Code
Head of mathematics in the senior school / A
Information Systems Manager for the whole campus / C
Teacher of mathematics and ICT in the junior school / B1
Teacher of Numeracy and Literacy in the junior school / B2
General teacher in the junior school / B3

During the visit more data about the school's plans for implementation were gathered from a school file kept by the Head of Mathematics which contained information on the weekly schedule and timetabling of the ILS at the time of its use.

Subsequently, the recorded interviews were transcribed, analysed and grouped with the purpose of shedding some light on the research questions. Afterwards the conditions of use of the ILS in the school were compared to the ones recommended by the suppliers of the software.

The school’s profile and planning

The school is mixed and consists of three parts; the pre-school nursery which caters for children from 3-7 years old, the junior school which has approximately 130 pupils, aged 7-13 and the senior school which has approximately 330 pupils and provides for children from 13-18 years old. The school has no students with assessed SEN but 14% of its pupil population is SEN without statements.

According to teacher A the school decided to purchase the particular type of ILS after a visit they made to another school in 1999 which had already implemented the software and used it consistently for two years. The school reported significant learning gains for its pupils, albeit informally and with no supporting numerical data.

Subsequently the study school bought 10 licences for the system and installed it in 1999. After the staff involved received training from the vendor, they started using it with their pupils in 2000. The junior school was given priority over the senior school and the planned level of use was to be much higher in the junior school.

The initial planning as laid out in the timetables designed at the beginning of the implementation in 1999 involved use of the system for support in English and Maths daily – for 40 minutes in total – for all of the pupils in the three oldest year groups of the junior school. Since priority was given to the junior school, the system was installed on workstations located in the main computer room of that part of the school and its use was restricted only to those PCs. The younger year group of the senior school was timetabled to use the software at the remaining times during the week when the computer room was available and it was convenient for the teachers. However, the senior school intended to use the system mostly with pupils who particular needs in mathematics and English (e.g. Dyslexics, pupils re-taking public examinations, etc.) in the pupils’ free time.

Results and Discussion

The purchasing decision

The school based its purchasing decision on the visit they made to another school where according to teacher A “…every pupil in school was on (the software) for half an hour a day… and they were getting great results…”. The biggest selling point as far as the school of the present study is concerned seemed, however, to be the differentiation that the system claimed to provide for its users. According to teacher A “…something that accelerates through the easy stuff for the able ones and doesn’t actually force them to go through every (task in sequence)…but shortcuts it or conversely slows down and just hammers the basic for the less able was certainly a very attractive sounding package…”

Other teachers felt the same way about the idea of purchasing an ILS,

“I thought it would help a class that I knew had got a wide range of ability, allow us to differentiate more easily, (allow) each child to carry on working at their own level and they enjoyed doing it” (Teacher B2)

Subsequent to the visit, teacher A, head of mathematics at the senior school initiated the purchasing decision and procedure.

The implementation

Teacher A reported that the school staff who received training from the manufacturers put at the beginning of the implementation a lot of effort and felt optimistic about the effect it would have on pupils’ performance. Describing his experience of the training the same teacher said: “Yeah … that (the training) was good. And we came out of that feeling very positive and very fired up that we were going to use it a lot and the rest of it…”

However, the staff started having technical problems with the system almost immediately. The main problem was that when users logged off the system, the central system server did not always register the logoff and pupils would still appear logged on. The result of that was that the number of licences available for use would appear reduced. As Teacher B2 described “It was frustrating particularly when you got into a situation where…two groups had rotated… and a third group arrived and they (the previous group) had logged off but there they were still logged on and some of them worked on what they thought was their session”.

The supplier’s service initially provided a software patch that allowed them to reset the number of licences to zero through a routine. This solution did not appear to be effective, however, since there was not always a skilled member of staff available to apply the routine. By that time most of the teachers were feeling very discouraged by the technical disruption and had started reducing the number of their pupils’ sessions.

“In the end we decided that it was just causing too much disruption…we would get to a stage where four people could log on but not the rest.” (Teacher B2)

As a response to the continuing problem the suppliers increased the number of licences available to a hundred so that even if the system did not register some of the logoffs, there was a sufficient number of licences remaining for use. But “…even when we got the larger number of licences…it still was a bit problematic.” (Teacher B2)

The ICT Systems Manager explained that there were other technical problems that disrupted the system’s use. Those had to do with certain files stored within the system that became corrupt. He also mentioned that “in terms of the level of PC resources…we are better now… at the time it was quite restricted so that’s possibly one of the reason it wasn’t as successful as it could have been.”

He explained that as a result of this situation, the use of the ILS did not take off even after the increase of the licences.

Pupils were registered on the system at the level the system’s initial placement assessment tests (IPM) put them. When teachers were asked whether the system’s judgement of pupils’ ability agreed with theirs they said that,

“The levels that I thought the pupils were at, (the system) put them. It (IPM) came out with the children in the order I thought they’d be in.” (Teacher B1)

Pupil’s attitudes

All of the teachers reported that pupils enjoyed using the system and appeared to be focused while on task on the system.

“Pupils did seem to be engaged with the software and were clearly focused on tasks.” (Teacher C)

“...they actually liked these lessons; they were always keen on those lessons.” (Teacher A)

However, when they were asked why they thought pupils enjoyed working with the software Teacher A mentioned that “they like the lessons because they like computers” and teacher B1 said that “…they were very keen on the scoring system.”

Educational value and effect on achievement

Teachers felt that the pupils learnt from the system although they did not have an opinion on how significant their gains were because they did not have either the means or the time to assess them. However, they felt that particular students who were either particularly able or very keen on the system made noticeable gains through the system’s material. As teacher B1 mentioned, “I think it would help all of them…and it helped the pupils that liked it the most. That would be my answer to that”.

Teachers also mentioned that some of the pupils often covered topics on the ILS that had still not been introduced in the classroom. When teachers were asked whether they felt that was an advantage or a disadvantage, their opinions were mixed. Some of them reported that it worked as a plus in the classroom because those pupils would recognise those topics when introduced for the first time in the classroom and were better equipped to understand them and deal with tasks. Other teachers, though, reported that it acted as a drawback for their classroom routine because those students who had worked on those concepts on the ILS were less interested in attending and participating in the classroom when their teacher introduced those same topics.

However, what all teachers considered a particularly positive feature of the ILS was the differentiation. They felt that the fact that the ILS could follow each pupil’s individual progress was the system’s most useful aspect.

“…I felt as a teacher it was good a program for the children...the differentiation was excellent…” (Teacher B3)

Teacher B2 felt the same way about the worksheets the systems offers offline for the pupils to practise, “…I used to print off the worksheets as well, the individual worksheets that went with it…we used them at (the junior) school about once a week and you knew then they were getting work at their own level, it was differentiated and the different reading programs that we used allowed us to be able to use it with a wide range of reading age”.

Teacher B1 mentioned that he would have preferred it if the system was more clearly and strongly linked to the National Curriculum and covered more comprehensively the subject content laid out by the latter. He said,

“…I would have liked it linked more to the National Curriculum, the reporting levels of the National Curriculum rather than the way it is but it was suggested that was happening when were starting…we were going to get an add-on to do that…If (the ILS) was thrown back working properly with the added bonus of the link to the National Curriculum I would go for it again.”

In previous work (Gkolia and Jervis, 2001) along with other studies (Underwood 1997)teachers and pupils have noted that some content has an American feel. When teachers’ opinions were sought concerning its presence in the software they said, “There were certain questions that they asked at certain stages which were in American English. I think in some of the maths…some things were American.” (Teacher C)

“You were supposed to able to set the units (from American to English) and I did go in and set them…it didn’t always do what I expected it to.” (Teacher B1)

The ILS has the capability to produce printed reports of progress and achievement and for teachers who persisted in learning how to translate them they proved to be useful; “… once you got into using it and kept on using it, it got quicker but each time you had to refer back to the book (manufacturers’ manual) but you got to know rapidly what scores you were looking for as being cause for concern”. (Teacher B2), in contrast to other teachers who found the interpretation of the reports’ data difficult.

“I was printing out some of the reports but by no means all of them. When I got the report I needed to do a translation job.” (Teacher B1)

“It wasn’t easy to decide what that (report) was telling us about the child’s progress.” (Teacher B3)

Reasons for abandonment

It became clear from all of the interviews that the main reason the teachers gave up using the ILS was the technical problems they were experiencing daily. However all of the teachers interviewed reported that they were generally happy with the system and would have been willing to keep on using it if it was not for those problems that disrupted the sessions and their lessons planning.

Another related problem was that as staff responsibilities changed and new teachers were appointed to the school, in the course of time there was not a sufficient number of teachers trained to use the system in the appropriate posts. As teacher A explained, “…we had certain members of that staff leave and we didn’t set up a kind of cascade system to train more people with the result that it gradually begun… because of the problems we were having… people were saying that “well there is no point you teaching me how to use this”.”