Chester CBS Lab

Quality Checklist for Experimental and Component Studies

Item / Check
Blinding
conditions /
  1. Blinding condition allocation to the experimenter for each participant (e.g. by automating the randomisation).
/ ☐ /
  1. Make sure you’re not familiar with participants (if impossible, balance between conditions).
/ ☐ /
Balancing
conditions /
  1. Randomising participants to a condition based on relevant variables (e.g. gender), so that all conditions are balanced.
/ ☐ /
  1. Check that all conditions are as equal as possible, except for the differences you are manipulating (e.g. word and time length, the protocol, how the intervention is delivered, or levels of engagement with the material).
/ ☐ /
  1. Automate the entire procedure (pre-, during- and post-intervention) as much as possible. If it is not possible, make use of recordings/video clips, which can be checked by an independent rater.
/ ☐ /
Measures /
  1. Can you include a baseline ability or personality measure of capacity to do/understand the task? (e.g. ability to understand analogy).
/ ☐ /
  1. If testing brief interventions, are there suitable state and trait measures available?
/ ☐ /
  1. If the intervention aims to improve a set of behaviours usually captured as a mid-level construct, (e.g. flexibility, fusion), measure that construct at baseline. Anticipate that participants already scoring high on the construct will not improve with a brief intervention (or perhaps any intervention).
/ ☐ /
  1. Make sure you collect enough demographic details to summarise the sample. Look at the journals you might target to assess what they expect.
/ ☐ /
Comprehension
and perceived applicability /
  1. Remind participants of what they are to do prior to the intervention.
Can this be checked by an independent rater? / ☐ /
  1. Throughout the intervention, let the participant explain the intervention in their own words.
/ ☐ /
  1. Make use of standardised self-report measures which check participant understanding and applicability of the task involved.
/ ☐ /
  1. If 12. is not possible, ask participants to summarise what they were asked to do upon completion of the study, and whether they could use this outside of a laboratory environment.
Can this be checked by an independent rater?
Does understanding differ between groups? / ☐ /
Power /
  1. Assure the sample sizes are large enough to adequately power the key hypotheses, any interaction effects, and designs that use mediational analyses.
/ ☐ /
Other /
  1. If null results are predicted, make sure the actual measurement characteristics, outliers, and similar issues do not undermine the calculated power.
/ ☐ /
  1. At the start of the study, ask participants if they have knowledge of the relevant techniques/processes that will be used (e.g. within RFT/ACT).
/ ☐ /
  1. Ask relevant questions at the end of the procedure that might have affected the results, such as likeability of the experimenter.
/ ☐ /
  1. Make use of multiple active and experiential elements when testing specific ACT components. Providing a rationale generally doesn’t work.
/ ☐ /
  1. If testing multiple ACT components, consider how to assess for changes in multiple ACT processes and whether comparison conditions should tease apart the impact of individual components.
/ ☐ /
  1. Make sure the intervention is as connected to the experimental task as possible, e.g. if pain is the outcome, use a pain-specific intervention.
/ ☐ /
  1. Protocols of ACT/RFT studies should be tied to RFT concepts. Can you theoretically predict how the task will influence behaviour in the study?
/ ☐ /

Adapted by Rosina Pendrous and Lee Hulbert-Williams from the work of Steve Hayes and Dermot Barnes-Holmes as found at: