Articles Critique
Kathy Cobb
February 30, 2011
Introduction
In today’s technological savvy world one question is always on the mind of educators, “what new technologies and software programs should I adapt into my classroom?” Technology is advancing rapidly in today’s society and it is imperative for educators to evaluate new programs and choose carefully based on what impact the technologies will have on learners. In today’s society young people are constantly engaged in social networking and gaming, which supports and enhances learner engagement and motivation. Currently many educators are seeking ways to incorporate similar technologies into their classroom, but with so much out there this can be a very challenging endeavor. Educators must evaluate a variety of programs and decide what works best for their teaching style and their learners.
For my MEDT 8480 graduate class I have decided to evaluate the Classworks Program at Commerce Middle School, where I currently teach. This program was implemented four years ago to give individualized instruction to students and to potentially increase standardized test scores. Before beginning my own evaluation I reviewed seven articles depicting evaluations of educational programs. These articles gave me insight into how to implement and conduct an evaluation. As I read these peer-reviewed articles I gained valuable ideas on how to present my purpose for the evaluation and the importance of involving the stakeholders when developing the evaluation questions. I realize from reading the articles that the methods of data collection and analysis play a strong role in a successful evaluation. In evaluating educational programs I have learned that it is important to talk with the contractor concerning the purpose of the evaluation, who the stakeholders are, what questions will be answered, what methods will be used, how data will be collected and analyzed, and in conclusion report the results and recommendations.
It was very difficult to find articles on the evaluation of the Classworks software program so I chose articles on the evaluation of a variety of technologies to increase learner’s motivation, test scores, and promote collaboration in the classroom. In all of the articles the evaluation used the formative methods. Data collected was from student observation, student and teacher interviews and questionnaires. It is apparent that observations, interviews, and questionnaires are the most frequently used type of evaluation. Two of my articles used summative evaluation methods from data taken from test scores. All of my articles focused on evaluating the learners in the classroom except for one, “Technology Used for Teaching and Management in the Business Education Classroom”, which focused on the educators and what types of technologies they were implementing. The majority of the articles I critiqued were searching for answers to the same type of learner centered questions. The common evaluation questions were; “does this program enhance standardized test scores?, does this program promote individualized instruction?, does this program enhance collaboration and motivation in students?, do students find the program easy to understand and use?, and are teachers knowledgeable about the program?” The evaluation questions were very different on the article that was evaluating the teachers’ use of technology. The evaluation questions in this article reflected how the teachers were incorporating and using technology in their classrooms while the other articles concentrated more on what the evaluation program is doing for the learner.
The two articles that evaluated a program to increase student achievement on test scores and provided individualized instruction and motivation were “An Effective Way to Improve Mathematics Achievement in Urban Schools” and Using Web-Based Computer Games to Meet the Demands of Today’s High Stakes Testing: A Mixed Method Inquiry.” These two articles took a similar approach to the type of program I will be evaluating. The first article was evaluating a program to increase Math scores for underachievers in mathematics. The program was a Mathematics Enhancement Group (MEG) pull-out program, which was created to serve students with disorderly behavior but with a strong potential to improve their mathematics skills (Kim, p.60). I related this to the Classworks program that I will be evaluating, because it was initially geared to increase special education and at-risk student’s standardized test scores. The author’s main purpose in both of these evaluations was to raise individual student’s test scores. Even though the approach was different the evaluation of both programs was very similar. The second article used web-based computer games to review test materials. The results showed that students in the web-based program did have higher test scores, but not significantly so. However, more importantly, the games promoted higher order learning outcomes such as increased meaningful dialogue among students and the identification of student misconceptions, both of which contributed to deeper student understanding (McDonald & Hannafin, p. 459). The evaluation questions were not only geared to test scores, but also individualized instruction of the students, and the adequate training of the teachers to implement the program. If teachers are not properly trained the best educational program can become a failure. The evaluation questions used in these two articles are similar to the ones I would like to include in my evaluation. Both of these articles are well-organized, clear, and easy to read. Tables and charts are used in both articles to express the results of the study. The article on using games to motivate students validated my strong reaction on this subject. I have always felt that if you can incorporate something into your classroom that students love to do it will enhance their learning of the subject. I teach business education and I have never told a student that they could not play games in my classroom. Games provide motivation, eye-hand coordination, and critical thinking skills. The author stated that, students who played games were more motivated, had increased retention of information, and demonstrated improved reasoning skills and a greater level of higher-order thinking (McDonald & Hannafin, p. 461). I have found this to be true with my own son, who is now in college. Games were a very important part of his individuality and he learned so a lot of critical thinking skills from playing them. He even learned of diversity and cultural awareness, through friendships he formed all over the United States and Canada.
The two articles on evaluating the implementation of Wikis into the classroom were very concise and easy to read. The argument in both articles was logical and included sufficient evidence on how wikis promote collaboration among learners. The authors of the article, “The Good, the Bad, and the Wiki: Evaluating Student-generated Content for Collaborative Learning” presented their argument and point of view more precisely. The methods and data collected were appropriate for the evaluation. Four groups of education students in the authors’ own parent institution were evaluated, observed, and interviewed. Students used the wikis regularly during their classroom sessions as a space to store and edit the work from their research exercises, and as a forum for discussion (Wheeler, Yeomans, & Wheeler, p. 991). Positive and negative comments were included in the results of this evaluation. I feel that it is very important if you are including interviews from learners in your evaluation that both positive and negative comments need to be addressed. The second Wiki article, “Wikis in the Classroom: Opportunities and Challenges” was very accurate and precise, but I do not feel that their method of evaluation was as accurate as the first Wiki article. The author only evaluated one Marketing class and their approach to using the Wiki. It would have been a better evaluation to include different types of learners who use a Wiki for collaboration instead of only one Marketing class. The only type of data collected in this evaluation was an assessment questionnaire. A better evaluation would have included observation and individual interviews with the students. I do not plan to use students in my evaluation methods, but I still gained a lot of ideas and have a better understanding of program evaluations after reading these articles. I could relate to these two articles, because I have incorporated a Wiki into my eighth grade class this year. My students love the collaboration aspect and relate it to using social networking in education. They post all of their assignments on the Wiki, work in groups, post comments to each other and to me, and upload files from other programs. The students are excited about a different learning style and being able to relate their social networking skills to education.
The article on using interactive whiteboards was aimed to gather information regarding student’s views of the boards and the impact that the tool has on learning. Again this article did not address the impact the whiteboards have on the teachers or their views. I think a better evaluation would have been to gather data from both student and teacher. I chose this article, because my school system has just installed several whiteboards in selected teacher’s classrooms. I have noticed that some of the teachers are not using the boards at all and I feel that this is because there was not sufficient training. I found this article interesting because I wanted to get the pupils view on the boards. The method that was used consisted of a speech bubble template. The template was designed to initiate discussion, role-play in citizenship, and values in education (Higgins & Smith, p. 853). The student could interact with the template bubble by adding features. This method of evaluation seem to be very innovative, however it was hard for me to understand exactly how it worked. The results of the evaluation provided many positive comments from the students including the whiteboards provided creativity, motivation, and increased imagination. One of the negative comments was that the teachers frequently had technical difficulties with the boards. This is what I have found to be true in my school system. I think this evaluation would have improved if the teacher's view of the whiteboard had been included. All educators need to read this article so that they can become aware of the possibilities that the whiteboard can do to enhance student learning.
“Technology Used for Teaching and Management in the Business Education Classroom” is an evaluation article that I reviewed which gathered information from Business Educators on what type of technology they use in the classroom and how often it is used. This article evaluated several questions about the frequency of technology being used as a teaching tool for lesson presentation, remediation, attendance, grades, and lesson planning. Although many schools have embraced computer technology, little research has been done to determine specifically how this technology is being utilized by business educators (Bailey & Mountjoy, p.43). The method used for evaluating this study was an email survey, which was sent to teachers who were members of the state business education professional association from a Midwestern state. I feel that this type of data evaluation has several limitations and is not always accurate. A lot of times teachers may not read the questions thoroughly, they may rush through them; there can be technical difficulties, and bias problems. The article stated that one limitation to this study was the fact that the web-based survey link was an active server page and it was occasionally blocked by some of the respondents’ servers since it was viewed as a security risk (Bailey & Mountjoy, p.44). The article did provide detailed results and findings from the evaluation, using tables and charts from 216 usable responses for a return rate of 54.8%. I would like to see this type of evaluation study be conducted in a school including all the teachers instead of just one subject area. I believe as some of the recommendations state that a lot of times teachers just need to be educated in technology and be provided assistance and follow-up on how to use the tools. This article provided some very good information and made me really think about how I use technology as a business educator. I agree with the recommendation that this study should be replicated since technology changes rapidly.
The only evaluation article that I found concerning Classworks software was “Georgia Districts Implementing Classworks Instructional Software Report State Test Score Gains.” This article was not a very good evaluation, because it was done by Curriculum Advantage, Inc., which markets the software, making the article bias. The article evaluated three school systems currently using the software and looked at the gains made by these systems on the Georgia Criteria Reference Test, since the implementation of the software. Another method that was used to evaluate the program was interviews with administrators from all three systems. The results showed that all three systems have shown substantial increases in test scores since implementation of the program. I enjoyed reading this article, because it provided a better understanding of exactly what Classworks does and how each system uses the program. What I did not like about the study is that it did not give any teacher opinions of the program and what type of training if any that they have received. I plan to not only look at the school’s test scores when I do my evaluation of Classworks, but look at how the educators perceive the implementation of the program.
Conclusion
In reviewing these seven articles I gained a better appreciation of the importance evaluation studies play in the field of education. I read many different types of articles and methods of data collection. I now realize the importance of sitting down with the contractor and developing meaningful and appropriate questions to lead the evaluation. I feel that the best evaluations use both quantitative and qualitative methods to obtain the most accurate results. Administrators, teachers, parents, and learners need and want to know the success and/or failure rate of newly implemented programs. If data is not reflecting what the program intended to do then the program should be revamped or completely eliminated. Technologies are increasing rapidly on a daily basis and educators must become aware of the best tools to use in their classrooms and school systems. It is very important to evaluate programs and learn what works best for the individual system, school, teacher, and learner.
References
Bailey, G .A.,Mountjoy, K. J. (2009). Technology used for teaching and management in the business education classroom. Business Education Forum,63(4), 43-49.
Curriculum advantage inc.; Georgia districts implementing classworks instructional software report state test score gains.(2009,November). Education Letter,16. Retrieved January 30, 2011, from Career and Technical Education.
McDonald, K.K.,& Hannafin, R. D.(2003). Using web-based computer games to meet the demands of today's high-stakes testing: a mixed method inquiry.Journal of Research on Technology in Education,35(4),459-472.
Taik, K. (2010). An effective way to improve mathematics achievement in urban schools. Educational Research Quarterly, 34(2), 60-71.
Wall,K., Higgins,S., Smith, H.(2005). 'The visual helps me understand the complicated things': pupil views of teaching and learning with interactive whiteboards.British Journal of Educational Technology,36(5),851-867.
Wheeler,S.,Yeomans,P.,Wheeler,D.(2008). The good, the bad and the wiki: evaluating student-generated content for collaborative learning.British Journal of Educational Technology,39(6),987-995.
Workman, J. P. (2008). Wikis in the classroom: opportunities and challenges. Marketing Education Review, 18(1), 19-24.
1