Objectives and Scope

Sections

· Home

· Objectives and scope

· Tutorial materials

· Contributors

· Books and software

Home

This web site presents materials developed under the ESRC Researcher Development Initiative grant “Advancing Quantitative Methods in Psychological Assessment”[1]. The research outputs from the grant were series of training workshops and summer schools in Applied Psychometrics, developed and held at the University of Cambridge between June 2010 and April 2012.

From this page, you can access all of the materials from five short courses and two summer schools. These training courses take learners through the theory and applications of established and state-of-the-art psychometric methods in the social sciences. Each course comprises a combination of lectures, research examples, software demonstrations and practical computing sessions, using commercial and open source software packages (Mplus, R, and other freeware).

The tutorials can be used for distant learning. Guidance provided includes an introduction to each tutorial that covers learning objectives and the minimum entry level of knowledge for learners.

Objectives and scope

Recent developments in psychometrics such as item response theory are having an increasing impact on analysis in all the social sciences. At the same time, the demand for psychometric expertise in education, medicine, business and forensics is increasing considerably. As a result of these developments a serious skills shortage is emerging. To address this shortage, an ESRC RDI award initiated a training programme in applied psychometrics and computing.

The aim of this award was to develop short training events (2-day and 3-day training workshops) and longer, 5-day summer schools. Another objective of this award was to sustain these learning resources as e-learning materials in a perpetual e-resource on psychometric methods.

The lectures and hands-on practical sessions aimed to exemplify existing and recently developed methods for analysing and modelling psychological assessment data, using multiple software packages, both commercial such as Mplus, and free software such as R.

Considering the growing impact of psychometrics on research and the wider society, the training was focused on topical issues surrounding the use of psychometrics in a diverse society, particularly where decisions can have an adverse impact on specific groups defined by social, gender, ethnic, cultural, religious, or disability factors.

The target audience for training was early career researchers from various disciplines within the social sciences, and more mature researchers who wished to familiarise themselves with new and emerging psychometric methods. The courses were attended by graduate students, post-doctoral researchers, academics, and researchers/analysts based in commercial organisations.

Tutorial materials

2 and 3-day short courses

Introduction to Mplus: latent variables, traits and classes.

Introduction to the Mplus modelling environment covering Exploratory Factor Analysis (EFA) with different rotations, Confirmatory Factor Analysis (CFA), regression and path analysis, multiple-group analysis and Latent Class Analysis (LCA).

Confirmatory Factor Analysis in Mplus

Introduction to the analysis of psychometric data emerging from ability tests and personality questionnaires, covering best practice for performing Exploratory Factor Analysis (EFA) and Confirmatory Factor Analysis (CFA)with scale and item-level test data in realistic conditions.

Measurement invariance and Differential Item Functioning (DIF)

Introduction to concepts of measurement invariance, covering the cases of categorical item response data (DIF studies) and continuous data (factorial invariance studies). Software packages used for this course are DIFAS, R and Mplus.

Introduction to Longitudinal Modelling

Introduction to most popular longitudinal designs and models, and issues of measurement invariance across time. Models include autoregressive, latent dynamic models, and latent growth models, and the growth mixture modelling approach to exploring typical development trajectories in longitudinal data. All modelling is carried out using Mplus.

Structural Equation Modelling (SEM)

Introduction to Structural Equation Modelling using Mplus. Simple and advanced methods including measurement models, path analysis, and full structural models with continuous and categorical variables.

Summer Schools

Introduction to Item Response Theory (IRT)

Introduction to the latent trait approach to modelling test items. Simple IRT models for binary and ordinal test items and their application to measurement, including studies of measurement invariance and bias. Software packages used for this course are R, Mplus, and DIFAS.

Psychometric modelling of Patient-Reported Outcome Measures (PROMs)

Introduction to advanced psychometric modelling of self-reported questionnaires, specifically concentrating on Patient-Reported Outcome Measures (PROMs). Classical and modern views and conceptions of reliability, validity and sensitivity of clinical measures using multiple approaches and methods, including advanced multidimensional modelling with Item Response Theory (IRT). Software packages used for this course are R, Mplus, IRTscore and DIFAS.

Contributors

The grant was awarded to Tim Croudace (Principal Investigator: Department of Psychiatry, University of Cambridge) and John Rust (co-Investigator: The Psychometrics Centre, University of Cambridge).

Anna Brown developed and delivered a suite of high quality learning materials, in collaboration with two visiting scientists, Jon Heron (University of Bristol) and Jan Boehnke (University of Trier), and Jan Stochl (University of Cambridge) who contributed to several course sessions. The courses were chaired and introduced by Tim Croudace.

All courses were hosted by John Rust at the Psychometrics Centre, University of Cambridge between June 2010 and April 2012.

Books and software

Software

Mplus (Muthen & Muthen) http://www.statmodel.com/

R statistical computing (R Development Core Team) http://www.r-project.org/

DIFAS (Randall Penfield) http://www.education.miami.edu/facultysites/penfield/

IRTscore (Flora & Thissen) [download here]

Recommended books

Bartholomew, D. J., Knott, M., & Moustaki, I. (2011). Latent variable models and factor analysis: A unified approach. Wiley.

Bollen, K. A. (1989). Structural equations with latent variables. Wiley.

De Ayala, R. J. (2009). The theory and practice of item response theory. Guilford Press.

Embretson, S. E., & Reise, S. P. (2000). Item response theory for psychologists. Lawrence Erlbaum Associates.

Hambleton, R. K., Swaminathan, H., & Rogers, H. (1991). Fundamentals of item response theory. Sage Publications.

Kaplan, D. (2000). Structural equation modeling: Foundations and extensions. Sage Publications.

Mellenbergh, G. J. (2011). A conceptual introduction to psychometrics: development, analysis and application of psychological and educational tests. Eleven International Publishing.

McDonald, R. P. (1999). Test theory: A unified treatment. Lawrence Erlbaum Associates.

Van der Linden, W. & Hambleton, R. eds. (1997). Handbook of modern Item Response Theory. Springer.

Van de Vijver, F. J., & Leung, K. (1997). Methods and data analysis for cross-cultural research. Sage Publications.

COURSE PAGES

1. Introduction to Mplus: latent variables, traits and classes.

Level: beginner to Mplus.

Session 1 introduces Mplus modelling environment and shows how to describe your data and variables. It then explores Mplus's modelling capabilities, covering Exploratory Factor Analysis (EFA) with different rotations, Confirmatory Factor Analysis (CFA), regression and path analysis. Continuous and categorical observed variables are covered. Download slides here.

Session 2 is devoted to models with multiple groups, exploring issues of measurement invariance with continuous and categorical variables. It starts with logistic regression to test for the presence of Differential Item Functioning (DIF). It then introduces the group-covariate approach and the multi-group approach with equivalence constraints. Finally, it explores the presence of unobserved homogeneous groups in the data with the Latent Class Analysis (LCA). Download slides here.

Practical sessions. Datasets and scripts used in practical sessions can be downloaded here.

2. Confirmatory Factor Analysis in Mplus

Level: beginner to Mplus.

Session 1. “From Exploratory to Confirmatory Factor Analysis (CFA)”. The session introduces the common factor model and discusses differences between Principal Component Analysis (PCA), Exploratory Factor Analysis (EFA) and Confirmatory Factor Analysis (CFA). It also discusses the goodness of fit in factor analytic models. Download slides here.

Session 2. “Using CFA to analyse structure in tests and questionnaires”. The session outlines EFA and CFA common rules and best practice. It then shows how to analyse test scales and item-level test data, discussing some common problems with fitting factor models. It discusses several alternatives for incorporating multidimensional features of test data (hierarchical factor models, bifactor models etc.). Download slides here.

Practical sessions. Datasets and scripts used in practical sessions can be downloaded here.

3. Measurement invariance and Differential Item Functioning (DIF)

The tutorial materials can be used for learning psychometric principles involved in assessments of measurement invariance, including longitudinal invariance. We recommend this course for those familiar with confirmatory factor analysis and regression (understanding of principles of Item Response Theory is desirable). Software packages used for this course are: DIFAS (free software from Randall Penfield), R (packages ‘difR’ and ‘lordif’) and Mplus.

The lecture slides can be downloaded here.

All lectures are supported by practical exercises. Data for the exercises, necessary instructions and software scripts can be downloaded here. The download includes free software DIFAS, and short instructions slides for this software. It also includes short reference slides for Mplus.

4. Introduction to Longitudinal Modelling

Session 1. The course starts with a discussion of basic longitudinal designs. The session introduces models for change with observed and latent variables, suitable for measures taken at just two time points. Next, autoregressive and latent dynamic models are discussed. Download slides here.

Session 2. The second session is devoted to growth modelling with measures taken at three or more time points, including the sequential (accelerated) cohort design. Download slides here.

Session 3 introduces the growth mixture modelling approach to exploring typical development trajectories in longitudinal data. The course ends with a discussion of issues of measurement invariance across time. Download slides here.

Archives containing descriptions of practical exercises, data and Mplus scripts we used in practical sessions can be downloaded here.

5. Structural Equation Modelling (SEM)

The tutorial materials below can be used for learning concepts of SEM. We recommend this course for those wishing to make a start with SEM; or advance their knowledge of SEM with categorical variables; or learn how to model in Mplus.

Session 1. Regression in the SEM framework; logit and probit. Download slides here.

Session 2. Confirmatory and Exploratory Factor Analysis; continuous and categorical variables. Download slides here.

Session 3. Path analysis; continuous and categorical variables. Download slides here.

All lectures are supported by practical exercises. Data for the exercises and software scripts can be downloaded here. Full instructions for the practical exercises are here.

Short reference slides for Mplus can be downloaded here.

6. Introduction to Item Response Theory (IRT)

Session 1 introduces the latent trait theory and basic IRT models (binary). Download slides here.

Session 2 explores two- and three-parameter IRT models in detail, and then introduces models for polytomous data. It introduces Test information in IRT and reliability, and shows how to test IRT assumptions and assess model fit. Data are analysed using R (package ‘ltm’). Download slides here.

Session 3 introduces the Rasch model for binary and polytomous data, and discusses properties of Rasch measurement and scaling. Rash scaling is done using R (package ‘eRm’). Download slides here.

Session 4 introduces the concept of Differential Item Functioning (DIF), and shows how to test for DIF using various approaches (Mantel-Haenszel method, CFA with covariates and multi-group approach). Mantel-Haenszel method is demonstrated using DIFAS package, and the model-testing approach using Mplus. Download slides here.

Session 5 is devoted to doing further work with DIF, this time using statistical software R (packages ‘difR’ and ‘lordif’). Download slides here.

Archives containing descriptions of instruments, data and software scripts we used in practical sessions can be downloaded here.

7. Psychometric modelling of Patient-Reported Outcome Measures (PROMs)

The tutorial materials below can be used for learning psychometric principles involved in development, assessment, analysis and interpretation of data arising from self-report questionnaires. Each topic includes a brief statement of level / knowledge /software requirements, workshop slides, and materials for practical exercises.

Session 1. Introduction to health-related Quality of Life measures. Level: beginner. Download slides here.

Session 2. Concepts of validity, reliability, sensitivity and generalizability. Level: beginner. Download slides and materials for a practical exercise. Software for practical: R (packages ‘psych’, ‘corrgram’, ‘CTT’ and ‘psy’).

Session 3. Principles of measurement scales. Level: beginner. Download slides here.

Session 4. Developing a questionnaire. Level: beginner. Download slides and materials for a practical exercise. No software is required for the exercise.

Session 5. Structure and quality of scales. Level: intermediate. Download slides.

Session 6. Factor analysis and basics of structural equation modelling. Level: intermediate. Download slides and materials for a practical exercise. Software for practicals: Mplus (download simple reference for Mplus).