HOWARD WAINER
EDUCATION
Princeton University 1968 Ph. D. Psychometrics
Princeton University 1967 A. M. Psychometrics
Rensselaer Polytechnic 1965 B. S. Mathematics
Institute
EXPERIENCE
2002-Professor (adjunct), Department of Statistics, The Wharton School, University of Pennsylvania, Philadelphia, Pennsylvania.
2001-Distinguished Research Scientist. National Board of Medical Examiners, Philadelphia, Pennsylvania.
2001-2004Governing Board of the Association of Princeton Graduate Alumni.
1998-2004Elected member, Princeton Board of Education (two terms).
1990-1994 Professor (adjunct), Department of Civil Engineering, Princeton University, Princeton, NJ
1989-1991Professor (adjunct), Department of Psychology, Princeton University, Princeton, NJ
1987- 2001Principal Research Scientist, Division of Statistical and Psychometric Research and Services, Educational Testing Service, Princeton, NJ
1981-1987 Senior Research Scientist, Division of Statistical and Psychometric Research and Services, Educational Testing Service, Princeton, NJ
1980-1981 Research Scientist, Educational Testing Service, Princeton, NJ
1977-1980 Senior Research Associate, Bureau of Social Science Research, Washington, DC
1970-1977 Assistant Professor in the Departments of Psychology, Behavioral Science and in The College, The University of Chicago, Chicago, IL
1968-1970 Assistant Professor of Psychology, Temple University, Philadelphia, PA
Summer 1968 Research Associate, Princeton University, Princeton, NJ
1965-1968 Educational Testing Service Psychometric Fellow, Educational Testing Service, Princeton, NJ
HONORS
The Samuel J. Messick Award for Distinguished Scientific Contributions Award from Division 5 of the American Psychological Association, 2009.
Who’s Who in America, 2009.
Fellow, American Educational Research Association, 2009
Career AchievementAward for Contributions to Educational Measurement. National Council on Measurement in Education, April 2007.
Award for Scientific Contribution to a Field of Educational Measurement for the development of Testlet Response Theory, National Council on Measurement in Education, April 2006.
Graphic Discovery: A Trout in the Milk and Other Visual Adventures (Princeton University Press) was named by Choice as the “Best Math book of 2005”.
Distinguished Visiting Lecturer University of Twente, Enschede, The Netherlands. October 1-4, 2001.
Distinguished Visiting Lecturer at the Hebrew University in Jerusalem, under the sponsorship of Israel's National Institute for Testing & Evaluation, February 1990
Senior Scientist Award, Educational Testing Service, 1990-1992
Fellow, American Statistical Association, 1985
PROJECTS DIRECTED
2004-2005Director and Principal Investigator, William Playfair’s Atlas and Breviary (Kellogg Foundation).
2001Director and Principal Investigator, Who submits SAT scores when they are not required? (College Entrance Examination Board).
1999-2001Director and Principal Investigator, Assessing the efficacy of AP exams: Developing adjustment methodology (Advanced Placement Research Committee, College Entrance Examination Board)
1998-2001Director and Principal Investigator, Testlet Response Theory (Graduate Record Exam Research Committee, Joint Staff Research & Development Committee)
1998-1999Director and Principal Investigator, Testlet Response Theory (TOEFL Research Committee)
1997-1998Director and Principal Investigator, Pennsylvania's Performance on NAEP (Pennsylvania State Education Association).
1996-1998Director and Principal Investigator, Revising display methods for NAEP (National Center for Education Statistics).
1994-1996 Director and Principal Investigator, Measuring the statistical characteristics of the TOEFL while incorporating its inherent cluster structure: I. Reliability (TOEFL Policy Council).
1994-1995 Director and Principal Investigator, A study of display methods for NAEP results(National Center for Education Statistics).
1993-1994Director and Principal Investigator, On the Performance of New Jersey Public Schools (New Jersey Education Association).
1992-1993 Director and Principal Investigator, Measuring statistical characteristics of the LSAT while incorporating its inherent cluster structure: Reliability and DIF (Law School Admissions Council)
1992-1993 Director and Principal Investigator, How well can we equate test forms that are constructed by examinees? (Graduate Record Exam Research Committee)
1988-1989 Director and Principal Investigator, Differential Item Functioning (Air Force Human Resources Laboratory Contract)
1986-1987Director and Co-Principal Investigator, Test Validity for the 1990's and Beyond (Air Force Human Resources Laboratory Contract)
1985-1986 Director and Principal Investigator, Drawing Inferences from Self-Selected Samples (National Science Foundation Grant)
1984-1988 Director and Principal Investigator, Development of a Computerized Adaptive Test version of the ASVAB (Navy Personnel Research and Development Center Contract)
1982-1984 Director and Principal Investigator, Robust Estimation for Item Response Theory (Air Force Human Resources Laboratory Contract)
1980-1981 Director and Principal Investigator, Robust Estimation of Victim Vulnerability (National Institute of Justice Grant)
1979-1980 Director and Principal Investigator, Dynamic Data Display (National Science Foundation Grant)
1978-1979 Director and Principal Investigator, Robust Estimation in Latent Trait Analysis, Department of Justice (Law Enforcement Assistance Administration Grant)
1976-1978 Director and Co-Principal Investigator, The Graphics Social Reporting Project (National Science Foundation Grant)
OTHER PROJECT EXPERIENCE
2009Expert Witness, McCleary, Venema, and Network for Excellence in Washington Schools vs. State, King County Superior Court, Seattle, WA.
2008Expert Witness, Pennsylvania State Education Association vs. Downingtown School Board, Downingtown, PA.
2007Consultant, on general issues of data display and exploratory data analysis for SIG Corporation, Bala Cynwd, Pennsylvania
2007Consultant, on general issues in testing and evaluation for New York State United Teachers, Albany, NY
2007 Member, NCES Task Force on CAT for Longitudinal Study. Washington, D.C.
2007 -Consultant, on general issues in testing and evaluation for Pennsylvania State Education Association, Harrisburg, PA.
2006 Consultant on Survey and Analysis Methodology. The Philadelphia Eagle’s Eye Mobile Project. Philadelphia, PA
2006Expert Witness, Russo vs. NCS Pearson and the College Entrance Examination Board.
2005Expert Witness, Springfield Board of Education vs. Springfield Education Association.
2005Member, Task Force on Benchmarking. NCATE. Washington DC
2004 –Member, Advisory Board of Operation Public Education. Philadelphia, PA.
2004Consultant, on general issues in testing and evaluation for New Jersey Education Association, Trenton, NJ.
2000-2003Member, Assessment Advisory Committee for the South African Department of Education, Pretoria, South Africa.
2000-2004Consultant, Statistical methodology in the revision of the Addiction Severity Index. Department of Psychiatry, University of Pennsylvania, Philadelphia, PA.
1996 –1999Consultant, IRT for medical indicators. The Health Institute of the New England Medical Center, Boston, MA.
1996Consultant, Predicting sales calls for pest and termite extermination from location, date and weather. J. Walter Thompson, Atlanta, GA.
1993Consultant, Linking English and Spanish forms of the GED, American Council on Education, Washington, D. C.
1989 Consultant, Adaptive Strategies for Survey Questionnaires, Bell Communications Research, San Francisco, CA
1987 Consultant, U. S. Department of Energy, Graphic Display of Energy Data, Washington, D. C.
1985-1987 Consultant, the Cost of Capital Punishment, National Center for State Courts, Williamsburg, VA
1985-1986 Consultant, Effective Analysis and Display of Statistical Information, NBC, New York, NY.
1981 Consultant, Project on Standard Errors in Logistic Response Models, McFann-Gray and Associates, San Antonio, TX
1980 Consultant, Project on Evaluation of Drug Abuse Programs, Institute for Survey Research, Philadelphia
1979-1980 Consultant, National State Courts Project, Law Enforcement Assistance Administration, Washington, DC
1979 Consultant, Analysis of the National Survey of Children, Child Trends, Washington, DC
1977-1978 Consultant, Assessment of Uniform Parole Data Program, Law Enforcement Assistance Administration, Washington, DC
1976-1980 Consultant, Problems in the Intercultural Study of Attitudes, University of Texas, El Paso
1976 Consultant, Problems in the Analysis of Criminological Data, Social Science Research Council, Washington, DC
Consultant, Problems of Assessing the Efficacy of Visual Displays, Bureau of the Census, Washington, DC
1975-1986 Consultant, Assessment of Literacy, Touchstone Applied Science Associates, New York
1973-1975 Consultant, Methodology for the Development of a Measure for Effectiveness of Reading, The Riverside Research Institute, New York
1973 Consultant, data analysis, The Field Museum, Chicago, IL
1972-1980 Consultant, Prediction of Stature, The Fels Research Institute, Yellow Springs, OH
1970-1972 Consultant, Methodological Development of Prediction Models, The University City Science Center, Philadelphia, PA
1970 Consultant, scaling and experimental design, National Analysts, Incorporated, Philadelphia, PA
1969 Consultant, scaling, experimental design and data analysis, Arthur D. Little, Incorporated, Boston, MA.
PROFESSIONAL AFFILIATIONS
2007-2008Member, Committee to revise the APA publication manual.
2003-2004COPSS Committee on John W. Tukey Award, Chairman
2002-2004Journal of Educational and Behavioral Statistics, Editor
2001-2003Psychological Science, Member of theEditorial Board.
1997-1999Member, Task Force on Statistical Inference. American Psychological Association.
1996-1998Psychological Methods, Member of theEditorial Board.
1994 -2001 Journal of Educational and Behavioral Statistics, The Teacher's Corner Editor
1989-1992 The Psychometric Society, Trustee
1987- 1994Journal of Educational Statistics, Associate Editor
1981-1983 Management Committee of the Journal of Educational Statistics (Vice Chairman)
1977-1980 Ad Hoc Committee on Statistical Graphics, the American Statistical Association
1977-1997Applied Psychological Measurement, Associate Editor
1977-1983 Journal of the American Statistical Association, Associate Editor
1973-1981 The Psychometric Society, Treasurer, Trustee
BOOKS
Picturing the Uncertain World: How to Understand, Communicate and Control Uncertainty through Graphical Display . Princeton, NJ: Princeton University Press, 2009.
Testlet Response Theory and its Applications (with E. Bradlow & X. Wang).New York: Cambridge University Press, 2007.
The Commercial and Political Atlas, Representing, by means of Stained Copper-Plate Charts, The Progress of the Commerce, Revenues, Expenditure, and Debts of England, during the whole of the Eighteenth Century, by William Playfair. Edited and introduced by Howard Wainer and Ian Spence. New York: Cambridge University Press, 2005.
The Statistical Breviary; Shewing on a Principle entirely new, the resources of every state and kingdom in Europe; illustrated with Stained Copper-Plate Charts, representing the physical powers of each distinct nation with ease and perspicuity by William Playfair. Edited and introduced by Howard Wainer and Ian Spence. New York: Cambridge University Press, 2005.
Graphic Discovery: A Trout in the Milk and Other Visual Adventures. Princeton, N.J.: Princeton University Press, 2005. (Named by Choice “Best Math book of 2005”).
Test Scoring. (with D. Thissen) . Hillsdale, NJ: Lawrence Erlbaum Associates, 2001.
Visual Revelations: Graphical Tales of Fate and Deception from Napoleon Bonaparte to Ross Perot. (2nd edition) Hillsdale, N. J.: Lawrence Erlbaum Associates, 2000.
Computerized Adaptive Testing: A Primer, 2nd edition (with N. Dorans, D. Eignor, R. Flaugher, B. Green, R. Mislevy, L. Steinberg & D. Thissen) Hillsdale, N. J. : Lawrence Erlbaum Associates, 2000.
Drawing inferences from self-selected samples (2nd printing). Hillsdale, N. J. : Lawrence Erlbaum Associates, 2000.
Visual Revelations: Graphical Tales of Fate and Deception from Napoleon Bonaparte to Ross Perot. New York: Copernicus Books, 1997.
Differential Item Functioning (with P. W. Holland). Hillsdale, NJ: Lawrence Erlbaum Associates, 1993.
Computerized Adaptive Testing: A Primer (with N. Dorans, R. Flaugher, B. Green, R. Mislevy, L. Steinberg & D. Thissen) Hillsdale, N. J. : Lawrence Erlbaum Associates, 1990.
Test Validity (with H. Braun) Hillsdale, N. J. : Lawrence Erlbaum Associates, 1988.
Drawing inferences from self-selected samples. New York: Springer-Verlag, 1986.
Principals of modern psychological measurement. (with S. Messick) Hillsdale, N.J. : Lawrence Erlbaum Associates, 1983.
Semiology of Graphics by Jacques Bertin (translated by William Berg; Howard Wainer, Technical Editor)Madison, Wisconsin: University of Wisconsin Press, 1983.
Graphics and Graphic Information Processing by Jacques Bertin (Translated by William Berg; Howard Wainer, Technical Editor) Elmsford, N. Y. : Walter de Gruyter, 1981.
Predicting adult stature for individuals. (with A. F. Roche & D. Thissen) Basel: Karger, 1975.
Skeletal maturity: The knee joint as a biological indicator. (with A. F. Roche & D. Thissen) New York: Plenum, 1975.
ARTICLES AND CHAPTERS
2009
What happens if the SAT is made optional? Bowdoin College as an example. The American Scientist, 97, xxx-xxx, 2009.
An Eye-Witness to the Principles and Principals of 20th Century Statistics -- A Review of Erich L. Lehmann’sReminiscences of a Statistician: The Company I Kept. Journal of Educational Measurement, 46(2), xx-xx, 2009.
A centenary celebration for Will Burtin: A pioneer of scientific visualization. Chance, 22(1), 51-55, 2009.
2008
Why is a raven like a writing desk? Musing on the power of convention. The American Scientist,96, 446-449, 2008.
Looking at blood sugar (with P. Velleman). Chance, 21(4), 56-61, 2008.
Giving the finger to dating services (with G. Lee & P. Velleman). Chance, 21(3), 59-61, 2008.
Computerized Adaptive Testing. In Corsisi’s Encyclopedia of Psychology, 4th edition (P. Rossi ed.), John Wiley & Son: New York., 2008, pages xxx-xxx.
14 conversations about 3 things. Journal of Educational and Behavioral Statistics, 33, xxx-xxx, 2008.
Improving graphic displays by controlling creativity (with discussion).Chance, 21(2), 46-53, 2008.
Until Proven Guilty: False Positives and the War on Terror.(With S. Savage), Chance, 21(1), 55-58, 2008.
Introduction to Linda S. Gottfredson. Journal of Educational and Behavioral Statistics, 33, xxx-xxx, 2008.
A Profile of Linda S. Gottfredson. (w/ D. Robinson). Journal of Educational and Behavioral Statistics, 33, xxx-xxx, 2008.
A Bayesian method for studying DIF: A cautionary tale filled with surprises and delights (with E. Bradlow, X. Wang, & E. Muller). Journal of Educational and Behavioral Statistics, 33, 363-384, 2008.
2007
Science and the SAT. Princeton Alumni Weekly, 4-5. November 7, 2007.
L’Equazione piu Pericolosa, Le Science, 470, 80-87. October, 2007.
A Psychometric Cicada: Educational Measurement returns. Educational Researcher, 36(8), 485-486, 2007.
A Profile of Susan E. Embretson. (w/ D. Robinson). Journal of Educational and Behavioral Statistics, 32, 431-439, 2007.
Taking a Chance: An interview with William F. Eddy and Stephen E. Fienberg. Chance, 20(4), 33-39, 2007.
A Catch-22 in assigning primary delegates (With A. Gelman), Chance, 20(4), 6-7, 2007
Improving data displays: ours and the media’s,Chance, 20(3), 8-16, 2007
The most dangerous equation. American Scientist, 95(3), 249-256, 2007.
Galton’s normal is too platykurtic. Chance, 20(2), 57-58, 2007
Insignificant is not zero: Rescoring the SAT as an example. Chance, 20(1), 55-58, 2007.
Value-Added Assessment (with Henry Braun). In Handbook of Statistics (Volume 27) Psychometrics (Eds C. R. Rao and S. Sinharay). Elsevier Science, Amsterdam. 2007, pages 867-892.
Three statistical paradoxes in the interpretation of group differences: Illustrated with medical school admission and licensing data (with Lisa Brown). Chapter 26 in the Handbook of Statistics (Volume 27) Psychometrics (Eds C. R. Rao and S. Sinharay). Elsevier Science, Amsterdam. 2007, pages 893-918.
A Profile of Roderick McDonald. (w. D. Robinson). Journal of Educational and Behavioral Statistics, 32, 315-332, 2007.
A Profile of Fumiko Samejima. (w. D. Robinson). Journal of Educational and Behavioral Statistics, 32, 206-222, 2007.
2006
Chance Conversation with Judith Tanur. Chance, 19(4), 52-57.
A modern classic expands and improves: A review of the 2nd edition of Leland Wilkinson’s The Grammar of Graphics, Psychometrika, 71, 603, 2006.
Logical and empirical evidence that smaller schools do not improve student achievement (with Harris Zwerling). The Phi Delta Kappan, 87, 300-303, 2006.
Finding what is not there through the unfortunate binning of results: The Mendel effect (with M. Gessaroli & M. Verdi) , Chance, 19(1), 49-52, 2006.
How’s your wife? A review of Defending standardized testing by Richard Phelps. Journal of Educational Measurement, 43(1), 77-84, 2006.
Using graphs to make the complex simple: The Medicare drug plan as an example, Chance, 19(2), 55-56, 2006.
On model-based inferences: A fitting tribute to a giant. Chapter 4 in D. Hantula (ed.) Methodological & Theoretical Advances in Social & Organizational Psychology, Hillsdale, NJ: Lawrence Erlbaum Associates, 2006, pps. 61-73.
An application of Testlet Response Theory in the scoring of a complex certification exam (with L. M. Brown , E. T. Bradlow, X. Wang, W, P. Skorupski, & J. Boulet). In D. M. Williamson, R. J. Mislevy, & I. I. Bejar(eds.)Automated Scoring of Complex Tasks in Computer Based Testing. Hillsdale, NJ: Lawrence Erlbaum Associates, 2006, Chapter 6, pps. 169-200.
Statistical Graphics: A guidepost for scientific discovery. (with Paul F. Velleman). In Complementary methods for research in education (3rd edition). (edited by Green, J. L., Camilli, G., & Elmore, P.B.) Washington, D.C.: American Educational Research Association, 2006, pps. 605-621..
2005
Chance Conversation with Martha Farnsworth Riche. Chance, 18(4), 48-51.
William Playfair and his graphical inventions (with I. Spence). The American Statistician, 59(3), 224-229, 2005.
Old Mother Hubbard and the United Nations: An adventure in exploratory data analysis.. (with discussion) (with D. Vasilescu). Chance,18(3), 38-45, 2005.
Truth is slower than fiction: Francis Galton as an illustration (with B. Clauser). Chance,18(4), 52-54, 2005.
Shopping for colleges when what we know ain’t. Journal of Consumer Research, 32(3), 337-342, 2005.
Non Random Sampling. In B. Everitt & D. Howell (eds.). Encyclopedia of Statistics in Behavioral Sciences (Volume 3). London: Wiley, 2005, pps. 1430-1433.
Graphical presentation of longitudinal data (with I. Spence). In B. S. Everitt & D. C. Howell (eds.). Encyclopedia of Statistics in Behavioral Sciences. Volume 2, Chichester: Wiley, 2005, pps. 762-772.
A Bayesian method for evaluating passing scores: The PPoP curve (with Xiaohui Wang, William P. Skorupski, and Eric Bradlow). Journal of Educational Measurement, 42(3), 271-281, 2005.
Was it ethnic and social-class bias or was it statistical artifact? Logical and empirical evidence against Freedle's method for reestimating SAT scores(with W. P. Skorupski), Chance, 18(2), 17-24, 2005..
Stumbling on the path toward the visual communication of complexity. Chance,18(2), 53-54, 2005.
Item and Test Bias. (with Stephen Sireci). Encyclopedia of Social Measurement (volume 2) (K. Kempf-Leonard, editor) . San Diego, CA: Elsevier Academic Press. 365-371, 2005.
William Playfair and the Invention of Statistical Graphs. (with Ian Spence). Encyclopedia of Social Measurement(volume 3) (K. Kempf-Leonard, editor) . San Diego, CA: Elsevier Academic Press. 71-80, 2005.
Resizing triathalons for fairness (with R. de Veaux) pages 317-321 in Anthology of Statistics in Sports. J. H. Albert, J. Bennett, & J. J. Cochran (Eds.) Philadelphia, PA: Society for Industrial and Applied Mathematics, 2005.
2004
An Editor’s Gratitude: Reviewer Acknowledgement. Journal of Educational and Behavioral Statistics, 29(4), 489-490, 2004.
User’s Guide for SCORIGHT (Version 3.0): A computer program for scoring tests built of testlets including a module for covariate analysis (with Xiaohui Wang & Eric Bradlow). ETS Technical Report RR-04-49. Princeton, NJ: Educational Testing Service.
On Assessing the Quality of Physician’s Clinical Judgment: The Search for Dependent Variables (with Janet Mee), Evaluation & the Health Professions, 27(4), 369-382, 2004.
Two statistical paradoxes in the interpretation of group differences: Illustrated with medical school admission and licensing data (with L. Brown). The American Statistician, 58, 117-123, 2004
Value Added Assessment: An introduction and history, Journal of Educational and Behavioral Statistics, 29(1), 1-4, 2004.
When form violates function. Chance,17(3), 49-52, 2004.
Curbstoning IQ and the 2000 presidential election. Chance,17(4), 43-46, 2004.
Nobody’s perfect. (with Michael Friendly). Chance,17(2), 48-51, 2004.
Numbers and the remembrance of things past. (with Henry Braun),Chance,17(1), 44-48, 2004.
How much does extra time on the SAT help?(with B. Bridgeman, M. Najarian and C. Trapani). Chance, 17(2), 19-24, 2004.
The promises and pitfalls of making national educational assessments adaptive: America’s assessment as an example. Metodología de las Ciencias del Comportamiento (Methodology of Behavioural Sciences), 5(2), 211-224, 2004.
2003
John Wilder Tukey: Statistical inventor, discoverer and revolutionary. Statistical Science, 18(3), 285-286, 2003.
Editor’s Forward to: “Comparing harm done by mobility and class absence: missing students and missing data,” by Michelle Dunn, Joseph Kadane and John Garrow, Journal of Educational and Behavioral Statistics, 28(3), 267-268, 2003.
One cheer for null hypothesis significance testing. In Methodological Issues & Strategies in Clinical Research (3rd Edition). A. E. Kazdin (ed.). Washington, DC: American Psychological Association, 461-464, 2003.
Shaping Up the Practice of Null Hypothesis Significance Testing. (with D. H. Robinson). Educational Researcher,32(7), 22-30, 2003.
A political statistic. (with Daniel Koretz)Chance,16(4), 45-47, 2003.
La diffusion de quelques idées: A master’s voice. Chance,16(3), 58-61, 2003.
How long is short? Chance,16(2), 55-57, 2003.
A graphical legacy of Charles Joseph Minard: Two jewels from the past. Chance,16(1), 56-60, 2003.
2002
A Small Hurrah for the Black Death (with S. Zabell). Chance,15(4), 58-60, 2002.
The BK-Plot: Making Simpson’s Paradox clear to the masses. Chance,15(3), 60-62, 2002.
On the Past and Future of Null Hypothesis Significance Testing (with D. H. Robinson). Journal of Wildlife Management, 66, 263-271, 2002.