Assignment #2: Critique
Kevin Miller
CECS 5610
February 23, 2006
Reference:Steven H. Shaha, Valerie K. Lewis, Tamara J. O’Donnell, and Diana H. Brown. (Fall 2005)“Evaluating Professional Development: An Approach to Verifying Program Impact on Teachers and Students”Journal of Research in Professional Learning.Retrieved February 21, 2006 from
Principle: A case study is reviewed that illustrates how well-designed and delivered professional development is validated by a thorough evaluation component, and the statistically significant impact a specific professional development program had on teachers and their students.
Type of Design: Two Pretest-Posttest Control group design studies were done.
R O X O
R O O
The focus of the evaluation was to verify the impact of a professional development program focused on reading. It was designed to contrast the comparative learning and attitudinal impacts of the teachers (and their respective students) participating in the new program — the Experimental Group — versus those teachers (and their respective students) that did not participate — the Control Group. Resource-related analyses were also conducted at the end of the study by contrasting the cost-effectiveness of this new program versus two other professional development programs previously undertaken for reading. Teachers and Resource personnel Data were gathered from 34 teachers and resource personnel from K through high school, 25 who participated in the reading-focused professional development (the Experimental Group) and nine who did not participate (Control Group). Data for 741 students from across grades K-12 were reflected in the research, including 479 taught by teachers that participated in the professional development (the Experimental Group) and 262 taught by teachers that did not participate (Control Group).
Factors Jeopardizing Internal Validity: More students could have been used had the school district been larger. Another factor could be the Hawthorne Effect. It occurs when participants modify their behavior when they know they are being evaluated.
Factors Jeopardizing External Validity: Pre-existing level advantages within the Experimental group could jeopardize external validity.
Adequacy of Statistical Procedures Used: All statistical analyses were conducted using SPSS and all differences were evaluated at the p<.05
Briefly Summarize Logic: Organizations cannot justify spending on professional development programs that do not represent investments. Program providers must prove with data that significant improvements in skills, knowledge and attitudes will result from expenditure and implementation
Design Improvements: This study would be much better with larger groups and conscious effort to target those teachers that could benefit the most from professional development. Including larger demographics might be more representative of large school districts.
Extension of Study: The study should be used on larger and more diverse school districts.