Summary of Research on Various Teacher Employment Screening Instruments

Summary of Research on Various Teacher Employment Screening Instruments

Howard Ebmeier

University of Kansas

8/18/2008

Citation / Sample / Independent measure / Dependent Measure / Correlation/Effect Size / Reliability / Comments
AARESTAD, JANNA SERINA (1980) ANALYSIS OF TEACHER SELECTION/RETENTIONAND STUDENT ACHIEVEMENT AS RELATED TO SELECTION RESEARCH INCORPORATED'STEACHER PERCEIVER INTERVIEW. Ph.D. dissertation, University of Minnesota. / N=34 grades 1-6 / TPI / Gain score in math achievement / No relationship / Dissertation not available, thus information derived from abstract
TPI scores dropped across grade levels
ARAMBURO, DOROTHY JAMES (1981) THE EFFICACY OF THE STRUCTURED INTERVIEWIN THE SELECTION OF SPECIAL EDUCATION TEACHERS. Ed.D., dissertation, University of New Orleans. / N=179 special education teachers / TPI / Multiple Teacher Factor Survey / 0.29 via canonical correlations among 12 TPI themes and 4 factors of MTFS / Dissertation not available, thus information derived from abstract
Not sure what the MTFS measured but apparently completed by the teacher
Antoline, Carlos Mitchell (2000) A validation study of the Teacher Perceiver Interview. Ph.D. dissertation, Texas Woman's University. / N=349 employed by district teachers / Teachers group into 4 categories via TPI (not continuous variable as hoped) / Principal ratings / Significant difference but very small F value. Only top vs. bottom teachers were significantly different in post hoc tests / Cited in study is Gallup 0.70 reliability estimate for TPI
Student evaluations were not different.
Ball, Jane Rowlett 1992) A validation study of students' and parents' perceptions of the talent of teachers selected using the SRI Teacher Perceiver interview. Ed.D. dissertation, KansasStateUniversity. / N=11 One school / TPI / Parent, student ratings / None reported / Very small qualitative study. Concludes that teachers selected with TPI were liked by parents and students—same could probably be said of all teachers however they were selected
Baskin, Maryan Kay (1994) The predictability of the Urban Teacher Selection Interview to the classroom success of elementary student teachers. Ed.D. dissertation, The University of Memphis. / N=68 student teachers / Star Teacher by Haberman / Practice Teaching Portfolio / The regression procedure found the Pass/Fail element of the Urban Teacher Selection Interview predicted success on the Communication aspect of the evaluation instrument. / “Overall, however, the UTSI did not differentiate student teachers who would be evaluated as successful in other areas of teaching which were part of the Practice Teaching Portfolio--areas like planning, classroom management, and using effective teaching strategies. Findings from this study indicate that Rank on the UTSI, which is emphasized by Haberman as the basis for selecting students, was not a significant predictor of the classroom success of student teachers.”
Bingham, Patrick Jerome (2000) The concurrent and predictive validity of elementary school teacher pre-employment success indicators. Ph.D. dissertation, The University of Texas at Austin. / N=138 teachers / TPI / Achievement on TAAS (Texas assessment)
Principal Rating / 0.04 (reading)
0.03 (math)
0.07 (writing)
-0.05 (principal rating) / Very small correlation that was significant but only because of large N-student (N=6600) unit of analysis
Brown, Karl Kevin (2004) The impact of the Gallup Teacher Perceiver Interview on hiring teachers as perceived by selected administrators in the Alamo Heights Independent School District, San Antonio, Texas. Ed.D. dissertation, TexasA&MUniversity. / Three groups Recommended by TPI (201), not recommended but hired (105), pre-TPI (602) / TPI / Retention rates
Administrator perceptions / ANOVA across 3 groups was significant with difference due to not recommended different than pre and recommended.
Retention rates of TPI recommended teachers were higher.
Buresh, Richard John (2003) The predictive validity of the Teacher Perceiver Interview in selecting effective elementary teachers in amid-sized Midwestern school district. Ed.D. dissertation, The University of North Dakota. / N=59 all grade levels / TPI / Panel of experts scored teacher evaluations of principals / 0.039 for total
Also reports each scale / Contains good review of TPI previous research
Reposts negative correlation with leave days
CHALKER, DONALD MERLE (1981) THE TEACHER PERCEIVER INTERVIEW AS ANINSTRUMENT FOR PREDICTING SUCCESSFUL TEACHING BEHAVIOR. Ed.D.. dissertation, WayneStateUniversity. / N=40 HS urban teachers in 3 districts / TPI / Quartile rankings by building principals
Ratings by sample of students / Significant correlations between admin ratings and TPI and student ratings and TPI / Dissertation not available, thus information derived from abstract
Correlation numbers absent from abstract
COOK, RONALD EUGENE (1981) CHIPPEWAVALLEY LOCAL VALIDATION STUDY OF THETEACHER PERCEIVER INTERVIEW PROCESS. Ed.D.. dissertation, WayneStateUniversity. / N=30 middle school teachers
High and low scoring scores were used for t-test / TPI / Administrator ratings, student ratings / Unclear, author concludes no significance but then states there was a positive t test and correlations / Dissertation not available, thus information derived from abstract
CORNINE, LARRY LEE (1980) A VALIDATION STUDY OF THE TEACHER PERCEIVER INTERVIEW USING THE ILLINOIS TEACHER EVALUATION QUESTIONNAIRE. Ed.D., dissertation, University of Kansas. / N=All Ottawa, KS secondary teachers
Experimental group=secondary teachers hired after TPI
Control=teachers before TPI / TPI / Student ratings of teacher competence / No relationship / Dissertation not available, thus information derived from abstract
Chesley, Daryl (2004) The validity of teacher employment success predictors. Ed.D. dissertation, The George Washington University. / N=587 teachers / UTPI
Praxis / Principal ratings / 0.144 (statistically significant but large N)
Praxis not significant / Restricted range problems since they were all employed district teachers
Davis, David / N=12 elementary teachers / Ventures / Tenn value added composite achievement scores / Reading was positive but rest were negative (rather strange since reading and math scores were positively correlated in study) / “there is little association between Ventures for Excellence scores and TVA scores”
Sample size too small to conclude much of anything
Delli, Dane Armand (2000) The predictive validity of the Teacher Perceiver Interview on the teaching performance of classroom teachers. Ph.D. dissertation, The Ohio State University. / N=72 teachers
N=124 / TPI / Principal ratings
Absence / 0.37 if all themes are added together in one regression equation
R**2 0.14 / Little or none internal reliability in the various themes / Capture in article with Young
Good review of the development and research behind TPI
Failed to composite score correlation.
Diemer, Janaan S. (1998) Identifying effective teachers of behavior disordered students: Is the Gallup Teacher Perceiver Interview an answer? Ed.D. dissertation, New MexicoStateUniversity. / N=33 teachers (special ed) Identified as top/bottom quadrant of TPI / TPI / Supervisor Ratings / 0.00 / Teachers deemed ineffective had higher average scores on the total and subscales that those deemed effective by TPI
Dunning, Christopher Michael (2005) Teacher Perceiver interview tool: A contemporary validation study within a Florida district. Ed.D. dissertation, University of SouthFlorida. / N=40 teachers
20 NBC certified 20 not
786 Respondents for factor analysis from college interviews / TPI / 0.31 for NBC/non-NBC teachers (restricted range problem likely) / Cronback alpha ranged from 0.0-0.30 on the 10 scales. / Factor analysis of first 24 questions on TPI (three factors suggested: interpersonal (0.35), intrapersonal (0.20), extrapersonal (0.45) Reliability of total instrument (0.63) Three factor model had good confirmatory factor fit (0.90) but one factor model was slightly better.
95% of board certified teachers had TPI highly recommend
Expert panel supported 22/24 questions as important.
RACHEL DAVIS (1984) THE RELATIONSHIP BETWEEN THE TEACHERPERCEIVER INTERVIEW AND INSTRUCTIONAL BEHAVIORS OF TEACHERS OF LEARNINGDISABLED STUDENTS. Ed.D.. dissertation, University of Virginia. / LD teachers N=? / TPI / The Classroom Planning and Management Assessment (CPMA), based upon research of effective teaching / A non-significant correlation between the TPI total score and the CPMA total mean score / From abstract
Some positive and significant correlations reported among various scales on the TPI and the dependent measure but overall no effect
Frey, Patricia (2003) Ability of the Urban Teacher Selection Interview to identify teachers who are likely to be retained in the Buffalo Public Schools. Ed.D. dissertation, SetonHallUniversity, College of Education and Human Services. / N=876 teachers at all grade levels / Star Teacher / Retention / 0.13 correlation (low but significant given the large N) / Star Teacher instrument used in initial employment decision thus possible range restriction issues
Gillies, Terry Kim (1988) The relationship between selection variables and subsequent performance ratings for teachers in an Oregon school district. Ph.D. dissertation, University of Oregon. / N=196 first year teachers employed by one district / TPI / Principal ratings / Average of 0.17 (great variation over the 4 years of data collection—one year was actually negative) / Overall correlation was significant but pretty low. Range restriction effect likely since were employed by district after results of TPI known.
Howard, Mildred Anthony (1998) The relationships among teacher selection predictors and teacher performance with principals' opinions about a teacher selection process. Ph.D. dissertation, GeorgiaStateUniversity. / N=143 teachers (relatively new)
30 elementary principals / TPI / Principal evaluation of each teacher via a form reflecting values of TPI / 0.47 for all 6 scales entered collectively / Individual subscales correlated from 0.03-0.38. Regression was run for 6 of the theme variables but not total.
JOHANNSEN, KENNETH LUDWIG (1981) THE RELATIONSHIP OF A UNIQUE TEACHERSELECTION METHOD TO STUDENT ACHIEVEMENT. Ed.D.. dissertation, PeabodyCollege for Teachers of Vanderbilt University. / N=40 20 teacher who interviewed using TPI and 20 who were previous hires and had not used the TPI
Teachers were balanced in terms of background / TPI / Math gain score / t-test indicated no difference between groups / Dissertation not available, thus information derived from abstract
JONES, DAVID EDWARD, JR. (1978) PREDICTING TEACHING PROCESSES WITH THETEACHER PERCEIVER INTERVIEW. Educat. D. dissertation, Virginia Polytechnic Institute and StateUniversity. / Dissertation not available from KU library
Kanipe, Mary Lou Smith (1996) A study of the predictive validity of the Urban Teacher Perceiver Interview in the Knox County School System. Ed.D. dissertation, The University of Tennessee. / N=233 teachers—all grade levels
68 Principals / Urban TPI / Principal Evaluations / 0.10 total
subscales ranged from 0.0-0.18 / Correlation with minority teachers (0.39) indicated definite bias
May be Range restriction effectsince all teachers were screened with TPI prior to employment
Koerner, Robert Jacob (2007) The relationship between the Teacher Insight score and student performance as measured by the Texas Growth Index. Ed.D. dissertation, University of NorthTexas. / N=123 3-11 grade
4500 students / Teacher Insight / TGI (Texas Growth Index) / 0.045 English and reading
0.042 Math / Ceiling effects are present since only teachers hired are used.
Kreifels, Robert G. (2003) A study of selected teacher interview characteristics and student achievement. Ed.D. dissertation, Saint LouisUniversity. / N=260 elementary teachers (N=17 at 4/5 grade ?) / TPI (reported 3 subscales) / 5th reading
4th math / Math—no significant relationships with correlations ranging from 0.20 to –0.03
Reading 0.43 to 0.25 with no significant results / The sample size reported for the correlations was 17, which I cannot figure out.
No relationship between ITBS and TPI found
LASHER, HUDSON FRED (1976) THE TEACHER PERCEIVER INTERVIEW AS ANINSTRUMENT FOR IDENTIFYING CHARACTERISTICS OF PROSPECTIVETEACHERS. Ed.D.. dissertation, University of Wyoming. / Students in education program / TPI / Of little comparative use.
Compared college student scores with average national TPI scores
LEAHY, RICHARD EDGAR (1984) TEACHER CHARACTERISTICS AND STUDENTACHIEVEMENT: AN EXPLORATION OF THE EMPLOYMENT INTERVIEW. Ph.D. dissertation, University of Minnesota. / Document could not be located at KU
Loyd, David Eugene (2004) The effect of a screening instrument to determine teacher effectiveness as measured by academic gain scores. Ed.D. dissertation, The University of Tennessee. / N=242 teachers in 3-8 grades / Urban TPI / Tenn. Value Added achievement in math, reading, language / No significant main effect ANOVA F values were pretty low thus the correlation probably was close to zero / Sample was divided into 6 categories based on UTPI score and ANOVA conducted.
MAUSER, PHILLIP J. (1986) THE PREDICTIVE VALIDITY OF THE TEACHERPERCEIVER INTERVIEW TO TEACHER EFFECTIVENESS. Ed.D.. dissertation, OklahomaStateUniversity. / N=77 all levels / TPI / Principal Evaluation / 0.02 / Excellent background review of the TPI development and older studies. Early studies indicated 0.7-0.8 KR internal reliability
MILLS, JONATHAN KENDALL (1986) THE TEACHER PERCEIVER INSTRUMENT AND ITSCORRELATION WITH OBSERVER RATINGS OF TEACHER-PUPIL RELATIONSHIPS (TEACHER INTERVIEWS, CALIFORNIA, TEACHER EVALUATION). Ed.D. dissertation, LomaLindaUniversity. / N=? Seventh-day Adventist schools / TPI / Principal evaluation using ORS McDaniel Observer Rating Scale / Concluded no relationship / Dissertation not available, thus information derived from abstract
MOODY, WALTER EUGENE (1973) A COMPARISON OF TEACHER PERCEIVER INTERVIEWGUIDE AND MINNESOTA TEACHER ATTITUDE INVENTORY SCORES WITH SUPERVISORRANKING OF SPECIAL EDUCATION TEACHERS. Ed.D.. dissertation, University of Kansas. / Document could not be located at KU
Martin, Linda J. (2008) Searching for effective teachers: A statistical analysis of the Ventures for Excellence Teacher Interview questionnaire. Ed.D. dissertation, SeattlePacificUniversity. / N=315 / VE Interview / 0.757 Cronback / “Construct validity of the instrument was not confirmed”
Three factors were discovered: interpersonal sensitivity, engagement beliefs and strategies, and flexible thinking
Mills, Scott A. (2007) An investigation of the predictive validity of a structured teacher-interview instrument. Ph.D. dissertation, PurdueUniversity. / N=61 grades 3-5 sample was used for other purposes
N=38 teachers had VE scores / VE interview / Northwest Evaluation Association’s Measure of Academic Progress in reading, LA, and math / 0.36 (Reading)
0.33 (LA)
0.27 (math)
7%-13% of variance / Unclear if the teachers having VE scores were younger than those with no scores. Appears to be a lot of data lost as many correlations were based on only 100 or so students.
Neal, Stephen O. (1997) A comparison of principals' perceptions of teacher candidates and selected talent profiles. Ed.D. dissertation, NorthernIllinoisUniversity. / TPI / Compared principal and teacher self-ratings on the 12 TPI themes. Did not report a great deal of overlap
OVERMAN, WILLIAM F. (1981) THE IMPORTANCE OF THE TEACHER PERCEIVERINTERVIEW AS A TEACHER SELECTION CRITERION TO ADMINISTRATORS INRELATIONSHIP TO THEIR LEADERSHIP STYLE. Ed.D.. dissertation, University of NorthernColorado. / Dissertation not available, thus information derived from abstract
Qualitative study of how administrators perceived the value of the TPI and how they used the results
Ostlund, Charles N. (2006) The Teacher Perceiver Interview as apredictor of teacher retention in special education. Ed.D. dissertation, University of Virginia. / N=339 special education teachers / TPI Interview / Rates of retention of teachers / TPI explained almost no variance. (full Dissertation not available, thus information derived from abstract)
Papiernik, Henry Joseph (1988) An analysis of rating proficiency in the use of the teacher perceiver interview in the selection of teachers. Ed.D. dissertation, TempleUniversity. / N=127 principals up for recertification on the TPI / TPI / Full Dissertation not available, thus information derived from abstract
Original proficiency gained from the TPI training is maintained through recertification program.
So what was learned? Isn’t that obvious?
Potthoff, Dennis Eugene (1991) A qualitative case study of the relationship between the scores of selected students on the SRI/UNL Teacher Perceiver Interview and their subsequent student teaching success. Ph.D. dissertation, The University of Nebraska - Lincoln,. / Qualitative study of 5 elementary education majors
Interview scores and classroom behavior were different
Ricker, Ann Marie McNeela (1999) Teacher selection: Chicago Region One elementary neighborhood school administrators since school reform. Ed.D. dissertation, NorthernIllinoisUniversity. / Single building / Qualitative study of what principals deem important in a candidate
Robison, Scott E. (2003) An exploratory study of the utility of a structured interview in the employment of classroom teachers. Ph.D. dissertation, PurdueUniversity. / N=14 (half scored well the other half did not) 2-4 grade teachers. N=263 students / VE Interview / Northwest testing data, student surveys, supervisor’s assessment / 5/14 teachers moved to the other group after one year and re-interviewing using the VE / Comparison of high and low groups was significant in LA and Math but not reading
Comparison groups was not significant for student surveys. Principal evaluations were handled in a qualitative manner
SIMMONS, BERTIE AUSTIN (1984) AN ANALYTICAL AND COMPARATIVE STUDY OF THESELECTION RESEARCH INTERVIEW AS AN INSTRUMENT FOR SELECTING TEACHERS FORA LARGE, URBAN SCHOOL DISTRICT. Educat. D. dissertation, Texas Southern University. / N=164 teachers / TPI / Principal rating
Employment stability / No significant relationship via ANOVA
0.372 correlation that was not significant / Divided the teachers into 5 groups based on supervisor’s rating followed by ANOVA
Also did a correlation which was NS
SIMMONS, JAMES ELWOOD (1976) A STUDY TO TEST THE TEACHER PERCEIVERINTERVIEW AS AN INSTRUMENT THAT WOULD SELECT VOCATIONAL AGRICULTUREINSTRUCTORS THAT DEVELOP POSITIVE RAPPORT WITH THEIR STUDENTS. Ph.D. dissertation, The University of Nebraska - Lincoln. / N=42 / TPI / Student evaluations--rapport
Administrator ratings / 0.40 with student ratings
0.37 with administrator ratings / Not all the TPI scales correlated but the total did
Sawyer, Gayle Buckheister (2005) A study using the Star Teacher Selection Interview to predict the successful performance of teachers in South Carolina's Program of Alternative Certification for Educators. Ph.D. dissertation, University of South Carolina. / N=33 first year teachers from alternative certification program / Star Teacher / Principals evaluation / No predictive value / Dissertation not available, thus information derived from abstract
Schmidt, Wayne D. (1993) The Teacher Perceiver Interview as a predictor of teacher performance. Ph.D. dissertation, University of Missouri -Columbia. / N=154 hired and 144 not hired / TPI / PBES-Principal evaluation / 0.015 for TPI and Principal Evaluation N=155 / TPI was a good predictor if candidate was hired (TPI use as screening device thus results were obvious)
Simmons, Anne Carol Brown (1996) A validation study of the use of the Teacher Perceiver Interview (TPI) in a selected southeastern urban high school. Ph.D. dissertation, GeorgiaStateUniversity. / N=55 teachers / TPI / TPAQ-Principal evaluation—measure
GTOI-Principal evaluation / unknown / GTOI was not usable since little if any variation
TPAQ study was qualitative in nature
Study of little use
THOMPSON, ALBERT RAY (1982) THE TEACHER PERCEIVER INTERVIEW AS RELATEDTO STUDENT ACHIEVEMENT. Ed.D.. dissertation, University of NorthernColorado. / N=27 4th and 5th grade teachers / TPI / Student scores on math, reading, and la on the CTBS / TPI total score not related to achievement / Dissertation not available, thus information derived from abstract
Watson, Robert Lewis (1991) A study comparing the profile of selected University of Wyoming student teachers with the SRI Gallup Teacher Perceiver Interview. Ed.D. dissertation, University of Wyoming. / 40 student teachers / TPI / Compared to TPI national sample / Of little value in assessing the TPI