Qualitative Studies (featuring COREQ)
Andrew Booth, Karin Hannes, Angela Harden, Jane Noyes, Janet Harris for the Cochrane Collaboration Qualitative Research Methods Group, and Allison Tong for the Consolidated Criteria for Reporting Qualitative Studies (COREQ) initiative.
1Timetable
Name of reporting guideline initiative / Notes / Consensus meeting date / Reporting guideline publicationCOREQ / --- / NA / 2007
2Name of guideline
Many qualitative research methods are used to understand complex meanings and interpretations of health, disease or interventions. These may involve such techniques as discourse analysis, ethnography, grounded theory, biography, field studies using observation, action research or process evaluation(1). Clearly development of a single reporting standard for all qualitative research presents a considerable challenge. This chapter features the checklist known as the Consolidated Criteria for Reporting Qualitative Studies (COREQ) (2). COREQ does not intend to set standards for conducting qualitative research. Rather, it establishes a set of criteria, based on a systematic literature review, for assessing the comprehensiveness of reporting with a focus on studies reporting interviews and focus groups. It features as the only reporting guidance for qualitative research to have received other than isolated endorsement. Tong and colleagues (2) developed the 32-item checklist by reviewing and identifying items published within other tools and checklists for qualitative studies. Notwithstanding its circumscribed ambitionsCOREQ can be used by peer reviewers and editors reviewing many primary qualitative research submissions. COREQ can also be used by readers wishing to gauge the validity and transferability of the results of a published report of primary qualitative research and by researchers undertaking systematic reviews and qualitative evidence synthesis.
3History/Development
The qualitative research community has been slow in developing and adopting reporting standards for both ideological and practical reasons. First the plethora of research methodologies, each with considerable variation in data collection methods, makes development of a single reporting standard problematic. Second, researchers may perceive a prescriptive,overarching framework as limitingtheir ability to capture rich and important contextual data (3). Third, descriptions of methods and presentation of narrative and graphical data (which may include field notes, documents, transcriptions of interviews and interactions, and artefacts) vary considerably between articles with different underlying purposes. Fourth, journals often impose word limits that are not conducive to detailed reporting of qualitative research. Such tensions are evidenced in the stylised format of qualitative research in high profile medical journals, which could be seen to favour presentation over content and context.
Recent years have, however, seen wider recognition that the systematic and careful documentation of all procedures—an “account of practice”— is located securely within the tradition of qualitative inquiry (4). Description, and corresponding documentation of procedure, is acknowledged as critical in driving forward methodological innovation and in advocating the value of a published research study. Although resisting reporting standards that are determined by the needs of quantitative research, qualitative researchers increasingly appreciate the value of communicating in a form that is more readily understood, evidenced in trends towards use of structured abstracts (5) and publication of research protocols in open access journals.
4When to use this guideline (what types of studies it covers)
The COREQ checklist was developed to promote explicit and comprehensive reporting of interviews and focus groups (2).
Basic definitions
Qualitative studies use “non-quantitative methods to contribute new knowledge and to provide new perspectives in health care” (2). Although other methods, such as grounded theory, action research and case study approaches, are becoming more plentiful, qualitative research publications in health care most frequently describe use of interviews and focus groups. These two methods are particularly useful for eliciting patient and consumer experiences, opinions, priorities, barriers, expectations and needs to improve the quality of health care (6)
In-depth interviews, with degrees of structure varying according to the approach used, explore the experiences of participants and the meanings they attribute to them. Researchers encourage participants to talk, usually on a one-to-one basis, about issues pertinent to the research question by asking open-ended questions. In-depth interviews are often used to study the experiences and meanings of disease, and to explore personal and sensitive themes.
Focus groupscapitalise on semi-structured discussions between groups of 4–12 research participants to explore a specific set of issues (7). The focus group often begins with a moderator asking broad questions about the topic of interest, before proceeding to askthe focal questions. Although participants individually answer the facilitator’s questions, they are encouraged to talk and interact, thereby highlighting cultural values or group norms (7).
5Previous initiatives
Systematic reviews of qualitative research almost always show that key aspects of study design are not reported, and so there is a clear need for a CONSORT-equivalent for qualitative research (8). They sometimes reveal that methods used are incongruent with the specified methodology (9). Although checklists for critical appraisal are available for qualitative research, and indeed are plentiful, there is no one style for reporting the findings from qualitative research(10) or indeed for synthesis of qualitative evidence.
At the time of its publication the developers of COREQ reported that only the British Medical Journal (BMJ) of the mainstream biomedical journals had criteria for reviewing qualitative research (2). However, this checklist was not routinely used,with the COREQ developers concluding that the BMJ checklist is neither comprehensive nor provides specific guidance for key aspects of study reporting.Within nursing the Journal of Advanced Nursing has both a reporting framework to structure a paper and supplementary guidance on appropriate use of methods.
6Current version
The COREQ checklist for reporting of qualitativestudies consists of 32 criteria, with a descriptor to supplement each item. It is organised into three domains (Table 1):
Table 1 Domains and items covered by the COREQ checklist
Domain / Total number of items / Details of ItemsDomain 1: research team and reflexivity / 8 items / Personal Characteristics [5 items]
- Interviewer, Credentials, Occupation, Gender, Experience & Training
- Prior Relationship; Participant knowledge of interviewer; Interviewer characteristics
Domain 2: study design / 15 items / Theoretical framework [1 item]
Participant selection [4 items],
- Sampling; Method of approach; Sample size and Non-participation
- Setting of data collection; Presence of non-participants; Description of sample
- Interview guide; Repeat interviews; Audio/visual recording; Field notes; Duration; Data saturation; Transcripts returned
Domain 3: analysis and findings / 9 items / Data analysis – 5 items]
- Number of data coders; Description of coding tree; Derivation of themes; Software; Participant checking
- Quotations presented; Data and findings consistent; Clarity of major themes; Clarity of minor themes
The COREQ statement with full explanation and supporting detail is published in full in only one journal article (2). Several editorials have reproduced abbreviated variants such as the COREQ checklist (11).
7Extensions and/or implementations
No extensions or implementations of COREQ have been published to date.One potential reason is a lack of international consensus on reporting criteria for qualitative research. The COREQ statement differs from such statements as CONSORT and STROBE (Cross Reference to CONSORT and STROBE chapters) as it does not result from extensive exploration or sharing of opinions from experts in the field. It is primarily the result of individual academic effort with supporting literature searches. Delphi studies or consensus meetings might optimise the COREQ statement increasing uptake amongst key stakeholders.
Consideration of some extensions might benefit authors and readers. For example, natural developmentswould be guidelines for different qualitative research methodologies(e.g. ethnography, studies using a grounded theory approach), mixed methods studies which use statistical and qualitative methodsand for systematic reviews of qualitative research (qualitative evidence syntheses). Methodological reviews of published qualitative syntheses have found that many papers lack explicitness about methods for searching, appraisal, and synthesis (12), with little evidence of emerging consensus. In particular a disparity exists between what people claim as a synthesis approach and what is actually done in practice with an associated need for explicit agreement on the procedures for particular methods of synthesis (13).Of twenty seven elements of the PRISMA statement, nineteen could translate directly into reporting requirements for qualitative evidence synthesis, four (relating to the method of synthesis) would require minor modification and a further four (relating to quantitative analysis) may be inappropriate. Clearly more commonality exists than difference. It is also worth noting that the STARLITE proposed standards for reporting literature searching derive from a survey of qualitative systematic reviews and yet apply to all types of systematic review (14)(Cross reference to PRISMA Chapter).
Some authors suggest that principles for reporting qualitative research could start from evaluation principles for quasi-experimental research or natural experiments. TheTREND (Transparent Reporting ofEvaluations with Nonrandomized Designs) standard (15) was designed specifically forstudies where the randomisation principle was somehow restricted. Thecriterion of transparency, central to the TREND system, emphasises detailed description of all steps and procedures and a detailed justification of thechoice and manner of application of the individual methods and theoretical background. These principles could be expanded with additional criteria, specific to qualitative research (16, 17). However receptiveness to such a suggestion will likely depend upon whether qualitative researchers favour a realist or pragmatic paradigm or are situated at the more interpretative, constructivist end of the spectrum.
8Related activities
The Robert Wood Johnson Foundation has sponsored the Qualitative Research Guidelines Project to develop a website for people developing, evaluating and engaging in qualitative research projects in healthcare settings ( Entitled Using Qualitative Methods in Healthcare Research: A Comprehensive Guide for Designing, Writing, Reviewing and Reporting Qualitative Research the Project has identified two sets of guidelines for those publishing qualitative research, those byMalterud (18) and those created by Miller & Crabtree (19)for the Journal of Family Practice.
In addition to the COREQ statement and the publication by Malterud (18) the EQUATOR Network documents three further sources of guidance for reporting qualitative research ( Those by Blignault and Ritchie (20) and Clark (21) are generic, the latter known as “the RATS guidelines” are specifically intended for the purpose of peer review. Earlier guidelines by Elliott (22) are specific to qualitative research studies in psychology and related fields.
9How best to use the guideline
aAuthors
It can be argued that a COREQ-type statement should start from the methodology, rather than the data-collection techniques, in order to situate its reporting criteria. In the absence of methodology-based reporting standards, authors can follow the COREQ checklist quite closely if they are reporting interviews or focus groups. In such cases the COREQ checklist may need to be supplementedby reference to critical features of the methodology used (e.g. ethnography, grounded theory, etcetera) and by looking at a good published example of that methodology, preferably from the target journal. The checklist can be adapted where authors use data collection methods different from interviews and focus groups.
bPeer reviewers
Peer reviewers will likely find it useful to use the COREQ checklist as a series of general prompts rather than a prescriptive structure. Journal editors could encourage reviewers to use the checklist to encourage completeness of reporting within the prescribed word limit and overall framework required by each journal’s Instructions to Authors.
cEditors
The COREQ checklist can help journal editors in two main ways. Several journals encourage authors to complete the checklist prior to submission of their manuscript. If the checklist is included with the submitted manuscript it can function as an additional quality assurance mechanism prior to peer review. Alternatively editors can draw the contents of the checklist to the attention of peer reviewers for their journal, while emphasising that the emphasis is on completion and not on standardisation of the format. In this way the COREQ checklist will contribute to the clarity, accuracy and transparency without this being at the expense of the richness of content and context.
dReviewers
Completeness of reporting becomes increasingly important with the increase in numbers of qualitative evidence syntheses (13). Reviewers need good quality reports to enable them to appraise,and then take into account, the quality of included studies or, more controversially, even to exclude low quality studies from their reviews.
10Development process
Search strategy.The developers of COREQ performed a comprehensive search for published checklists used to assess or review qualitative studies, and guidelines for reporting qualitative studies. Sources included Medline and CINAHL from the respective start dates of each database (1966 and 1982) to April 2006, Cochrane and Campbell protocols, systematic reviews of qualitative studies, author or reviewer guidelines of major medical journals and reference lists of relevant publications (2). They used citation pearl-growing (i.e. indexterms from relevant articles were used to conduct a broad search). The electronic databases were searched using terms and text words for research (standards), health services research (standards) and qualitative studies (evaluation). Duplicate checklists and instructions focusing on the conduct and analysis of qualitative research, rather than the appraisal or reporting, of qualitative studies were excluded.
Data extraction. From each included publication, the COREQ developersextracted all criteria for assessing or reporting qualitativestudies. They compiled seventy-six items from 22 checklists into a comprehensive list (2). They recorded the frequency of eachitem across all the publications. Items most frequentlyincluded in the checklists related to sampling method, settingfor data collection, method of data collection, respondentvalidation of findings, method of recording data, descriptionof the derivation of themes and inclusion of supportingquotations. They grouped all items into three domains: (i)research team and reflexivity, (ii) study design and (iii) dataanalysis and reporting.
Duplicate items, and those that were ambiguous, too broadly defined, not specific to qualitative research, or impractical to assess were removed. Where necessary, the remaining items were rephrased for clarity. Based upon consensus among the authors, two new items that were considered relevant for reporting qualitative research using interviews or focus groups were added. The two new items were:
(1) identifying those who conducted the interview or focus group, and
(2) reporting the presence of non-participantsduring the interview or focus group.
11Evidence of the effectiveness of guideline
In introducing COREQ its developers acknowledged that “there is no empiric basis that shows that the introduction of COREQ will improve the quality of reporting of qualitative research” (2). However they state that this was also the case for other reporting checklists such as CONSORT and QUOROM (now PRISMA). They also point out that subsequent researchhas shown that “such checklists have improved the quality ofreporting of study types addressed by each checklist”affirming that “the effect of COREQ is likely to besimilar” (2).
Many qualitative researchers would agree that the “underlying aim oftransparency in research methods” and “the theoreticalpossibility of the reader being able to duplicate thestudy methods”are common to quantitative and qualitative approaches (2). In providing comparable reporting guidelines the developers seek to encourage improvement in the quality ofreporting of qualitative studies and to promote understanding of the rigour inherent in good qualitative research studies across researchers and across disciplines.
12Endorsement and adherence
Thus far COREQ is endorsed by the Croatian Medical Journal, Headache, International Journal of Nursing Studies, Journal of Pediatric Psychiatry, Journal of Sexual Medicine, Palliative Care, Physiotherapy, The Radiographer, Scandinavian Journal of Work, Environment & Health. The authorial team does not appear to have actively sought endorsement by other journals and editorial groups. There does not appear to be any data on journal adherence to the COREQ statement. It remains to be seen whether COREQ sees increased uptake, expansion to other types of qualitative research or is superseded by a more generic statement. Engagement with experts in the field in evaluating and improving the COREQ statement could be an important step to increase its use. Until these issues are resolved journals and editorial groups could consider endorsingCOREQ, including reference to the statement and its checklist in their “Instructions to Authors”. Journals often itemize COREQ as one of a battery of reporting standards, as a means to ensure editorial consistency across multiple study types, rather than specifically targeting qualitative research.
13Cautions and limitations (including scope)
The COREQ checklist tool is appropriate for studies that use focus groups and interviews. Its value is more limited when applied to studies that use different data collection methods, and it may be more or less appropriate for different qualitative methodologies. In this context debates regarding reporting standards mirror issues related to the use of checklists for assessment of qualitative research. Some researchers prefer to use quality criteria as prompts rather than prescriptive guidelines (23, 24). Quality in qualitative researchmay be viewed as holistic in nature and not constructedover a scaffold of individual criteria. Unlike a randomised controlled trial where problems with internal validity can completely compromise the results, a limited qualitative study can still yield useful insights (23). However none of the above is an argument for poor reporting of the research.
The COREQ checklist claims to promotecomplete and transparent reporting among researchers and,indirectly, to improve the rigor, comprehensiveness and credibilityof interview and focus-group studies. The checklist consists of items specific to reporting qualitative studies and therefore presents a level of detail not captured effectively by generic criteria intended for all types of research reports. COREQ is a comprehensive checklist that covers necessary components of study design. Criteria included in the checklist can elicit details of importance for all research reports such as aspects of the research team, study methods, context of the study, findings, analysis and interpretations.