The Use of Cognitive Interviews to Revise the Quality of Trauma Care Patient-Reported Experience Measure (QTAC-PREM)

Online Resource 2

Quality of Life Research Journal

Authors:NiklasBobrovitz, MSc1,2

Maria J Santana, MPharm, PhD2, 3

Theresa Kline, PhD4

John Kortbeek, MD5

Henry T. Stelfox, MD, PhD2, 6,7

  1. Nuffield Department of Primary Care Health Sciences, University of Oxford, Oxford, United Kingdom
  2. Department of Community Health Sciences, University of Calgary, Calgary, Alberta, Canada
  3. W21C Research and Innovation Centre, University of Calgary, Calgary, Alberta, Canada
  4. Department of Psychology, University of Calgary, Calgary, Alberta, Canada
  5. Department of Surgery, University of Calgary, Calgary, Alberta, Canada
  6. Department of Critical Care Medicine, University of Calgary, Calgary, Alberta, Canada
  7. Department of Medicine, University of Calgary, Calgary, Alberta, Canada

E-mail:

Original Survey/Item # / Specific Issue/Interpretation / Action Taken
Acute 1, 2, 3: All three items consisted of one core question with five sub-components addressing: before hospital care, emergency department, intensive care unit, hospital unit, in-hospital rehabilitation. /
  • Difficulty distinguishing phases due to recall issues (pre-hospital, emergency, ICU) and distinctions too fine to make (in-hospital rehab, hospital ward)
/ Revised. All three items were collapsed and subsequently underwent further cognitive testing.
Post 1, 2, 3: All three items consisted of one core question with two sub-components addressing: out-patient rehabilitation and follow-up. /
  • Difficulty distinguishing sub-components
/ Revised. All three items were collapsed and subsequently underwent further cognitive testing.
Acute 2, Post 2: During your care for this injury, how often were your healthcare providers (doctors, nurses, therapists, etc) skilled and competent in helping you recover from your injury? /
  • Participants not able to identify specific components of medical skill to evaluate
  • Providers trusted to be skilled and competent, aspects of communication and interpersonal characteristics of providers evaluated instead
/ Deleted.
Acute 11: How often were you treated fairly by all healthcare providers and treated the same as other injured people around you? /
  • Participants could not assess how other patients were treated in order to compare
  • Prompts necessary to direct evaluation of discrimination
/ Revised.
Acute 30: How often were you treated unfairly because of your age, ethnicity, gender, or personal characteristics?

Table 3.Items that participants did not have the information to answer

Table 4. Items that had ambiguous terminology or were inconsistently interpreted

Original Survey/Item # / Specific Issue/Interpretation / Action Taken
Acute 1: How well-timed was the care you received (did you receive care when it was needed?) /
  • "Well-timed" too ambiguous
  • Multiple interpretations: responsiveness to care request; waiting times in the emergency department, for diagnostic imaging, for surgery, and for transfers
/ Deleted. Item on responsiveness effectively covered on the HCAHPS. Unable to develop an item on wait times for different aspects of care.
Acute 3: How often were you told what you needed to know in a way you could understand? /
  • "What you needed to know" too confusing
  • Participants evaluated clarity of all the information they received regardless of perceived need
/ Revised.
Acute 9: How often did your healthcare providers (e.g. doctors, nurses, therapists, etc.) explain things in a way you could understand?
Acute 5: How often was your care well organized? (all of your tests, treatments, and visits from different healthcare providers) /
  • Unclear what participants considered "organized" care
  • Multiple interpretations: the clarity of information during transfers (where and why); prompt responses to care requests; consistency of information from providers
/ Deleted. Content covered by QTAC-PREM and HCAHPS items.
Acute 6: People with injuries are often moved from one unit or hospital to another during their injury care. If this happened to you, how often did the moves go well? /
  • Multiple interpretations: being told where they were being transferred to; being kept comfortable during transfers; oriented to new hospital units upon arrival
/ Revised. Item split into the three components participants identified. List for acute item 17 were key landmarks described by participants.
Acute 15: During your transfers did the hospital staff or healthcare providers clearly explain where you were being transferred to?
Acute 16: During your transfers, how often were you kept comfortable?
Acute 17: When you arrived to a new hospital unit did a healthcare provider explain where important landmarks were in the unit? (e.g. call button, bathroom, nurse's station, water/ice machine)
Acute 8b: How often did members of your care team (e.g., nurses, doctors, other healthcare providers) provide kind and friendly care? /
  • Multiple interpretations: whether providers introduced themselves; explained their role in care; provided a lot of information; were respectful
/ Deleted. Content covered by survey or HCAHPS items.
Acute 8c: How often did the care team deal well with your concerns or frustrations? /
  • "Deal well" too ambiguous
  • "Taking action" to deal with concerns proved better as it was more consistently interpreted as provider acknowledgment of and responsiveness to requests
/ Revised.
Acute 25: When you expressed concerns or frustrations about your care how often did your healthcare provider take action to deal with them?
Acute 10e: How often was your personal hygiene treated well? /
  • Hygiene being "treated well" nonsensical and ambiguous
  • Answers given even though family (not providers) helped with hygiene
  • Explored overlap with HCAHPS item on help getting to the bathroom and using a bedpan
  • Getting to the bathroom not the only aspect of hygiene
  • "Hygiene" included: frequency of showers/sponge baths; cleaning defecation and excoriation
/ Revised. Providers that help to maintain hygiene listed to direct participants to think of the care provided by staff, not family.
Acute 23: How often did your nurses or other hospital staff help you maintain your personal hygiene?
Acute 10h: How often was your dignity treated well? /
  • Dignity being "treated well" nonsensical and ambiguous
  • Dignity distinct from being treated kindly
  • Being physically covered up and not being exposed a key part of dignity
/ Revised.
Acute 27: How often was your dignity considered by the healthcare providers?
Acute 10j: How often were your cultural, religious, and spiritual needs treated well? /
  • Needs being "treated well" nonsensical and ambiguous
  • Key issue was respecting preferences rather than considering or accommodating needs
  • Respecting preferences prompted participants to think about providing access to spiritual staff and not discriminating based on personal preferences
/ Revised.
Acute 29: How often were your cultural, religious, or spiritual preferences respected by the healthcare staff and religious or spiritual staff?
Post 1: How well-timed was the care you received (did you receive care when it was needed?) /
  • "Well-timed" too ambiguous
  • Key aspect of care timing post-discharge was promptness and availability of follow-up appointments
/ Revised.
Post 8a-8c: Did you have difficulty getting follow-up appointments when you wanted them with....
A. a trauma doctor, surgeon, or specialist?
B. A family doctor or general practitioner?
C. A physio, rehabilitation, or occupational therapist?
Post 3: How often were you told what you needed to know in a way you could understand? /
  • "What you needed to know" too confusing
  • Evaluations were of the clarity of information about specific injuries in follow-up appointments
/ Revised.
Post 12: During your follow-up appointments, how often did your healthcare providers explain things about your injuries in a way you could understand?
Post 5: How often was your care well organized? (all of your tests, treatments, and visits from different healthcare providers) /
  • Unclear what participants considered "organized" care
  • Interpretations included: ease of getting appointments; family doctors receiving information from the hospital
/ Deleted. Content covered by other items on survey.
Post 9c: When you first met the care team how often did they deal well with any concerns or frustrations? /
  • "Deal well" too ambiguous
  • "Taking action" to deal with concerns proved better as it was more consistently interpreted as provider acknowledgment of and responsiveness to requests
/ Revised.
Post 13: During your follow-up appointments, when you expressed concerns or frustrations how often did your healthcare providers take action to deal with them?

Table 5. Items not measuring intended constructs

Original Survey/Item # / Specific Issue/Interpretation / Action Taken
Acute 8a: How often did the care team introduce themselves and explain what they do? /
  • "What they do" limited to describing only their specific medical responsibilities
  • The phrase "their role in your care" was preferred because it prompted patients to assess if a provider explained all aspects of their involvement with the patient (i.e. treatments they will deliver, how often they will be seeing the patient)
/ Revised.
Acute 24: When meeting a new healthcare provider for the first time how often did they introduce themselves and clearly explain their role in your care?
Acute 9: How often were you included as part of your care team? /
  • The intended construct was the degree of inclusion in decision making however, participants thought of whether their questions were answered and whether treatment and care options were explained, which are aspects of communication and do not reflect if patient preferences were incorporated into the treatment plan
/ Deleted. Content covered by other items.
Acute 10a: How often was your shortness of breath treated well? /
  • Shortness of breath interpreted as the type of breathing experienced during panic attacks, not as difficulty breathing due to injured chest/abdomen
  • Participants identified situations when providers were not clinically successful at treating the symptom
  • Participants suggested evaluation of the effort to treat, not treatment outcome
/ Revised.
Acute 20: How often did the healthcare providers do everything they could to help you with your difficulty breathing?
Acute 10b: How often were your feelings of agitation treated well? /
  • Agitation sometimes confused with pain
  • After adding "irritability" to the item, participants could differentiate from pain
  • Participants identified situations when providers were not clinically successful at treating the symptom
  • Participants suggested we evaluate the effort to treat, not treatment outcome
/ Revised.
Acute 21: How often did the healthcare providers do everything they could to help you with your agitation or irritability?
Acute 10f: How often were your emotional needs considered? /
  • Most would recall whether emotional support was offered although those not requiring emotional support would answer no even if support was offered
  • Many thought consideration of emotional needs was only necessary for critically injured patients or those subjected to violence
  • Important to know if emotional support was available regardless of need
  • Support staff and nurses are the ones to provide this support, not physicians
/ Revised. Reworded to identify whether the support was offered, regardless of need.
Acute 26: Did a healthcare staff member (e.g. psychologist, social worker, nurse) offer to speak with you about your emotional needs.
Post 7: Was your regular doctor well informed about your injuries and treatments? /
  • Intended to measure coordination between hospital and family doctor but there was confusion about who should be informing the physician: some rated their own explanations to their physicians, rather than the transfer of information from the hospital or attending acute care physician
/ Revised.
Post 16: In your opinion, how much information about your hospital stay was communicated to your family physician or general practitioner on a scale of zero to ten, zero being no information and ten being all the information?

Table 6. Items that included assumptions about healthcare processes

Original Survey/Item # / Specific Issue/Interpretation / Action Taken
Acute 7b: Did you get the information you needed in a way you could understand about how the injuries might affect your life? /
  • Providers often do not know how a patient's injuries could affect them after they leave the hospital but participants said providers would discuss it even if they couldn't give exact details
  • Item too long
/ Revised.
Acute 11: Did your healthcare discuss how the injuries might affect you after you leave the hospital?
Acute 7c: Did you get the information you needed in a way you could understand about how long it would take you to feel better and what would be involved in recovery? /
  • Providers often cannot give an exact timeframe for recovery or know exactly what will be involved in recovery
  • Participants valued an approximate timeframe and some discussion about how to care for injuries post-discharge
  • Item too long and double barreled: addressing recovery timeline and activities
/ Revised. Item split into two components.
Acute 12: Did your healthcare providers provide instructions on how you should care for your injuries after you leave the hospital?
Acute 13: Did your healthcare providers discuss how long it might take you to recover from your injuries?
Post 7b: Did you get the information you needed in a way you could understand about how long it would take you to feel better and what would be involved in recovery? /
  • At the time of follow-up providers still may not be able to give an exact timeframe for recovery
  • Approximate timeline still important
  • "What would be involved in recovery" often interpreted solely as physical restrictions on activities
  • We wanted a broader item assessing information about recovery
  • Item too long and double barreled: addressing recovery timeline and activities
/ Revised. Item split into two components.
Post 9: During your follow-up appointments, did your healthcare providers explain the next steps in your recovery from injury for example, activities you should or should not do, necessary medications, tests and treatments, or other follow-up appointments?
Post 10: During your follow-up appointments, did your healthcare providers explain approximately how long it would take you to recover?
Post 8b: Did you get the information you needed in a way you could understand about discharge and homecare? /
  • Assumptions about the services patients receive
  • All should get discharge information but not all necessarily get homecare information
  • Many had difficulty recalling if verbal information was given prior to discharge
/ Revised.
Post 2: Before leaving the hospital, did your hospital healthcare providers give you written instructions on how to care for your injuries after being discharged?

Table 7. Items measuring non-priority aspects of care

Original Survey/Item # / Specific Issue/Interpretation / Action Taken
Post 9a: When you first met the care team how often did they introduce themselves and explain what they do? /
  • Not relevant because patients have already met most of the follow-up providers (i.e. family doctor, surgeons from acute care centre)
/ Deleted.
Post 9b: When you first met the care team how often did they provide kind and friendly care? /
  • Not a crucial element of follow-up care
  • Support through information provision and encouragement more relevant
/ Deleted.
Post 10a-e: How often were the following treated well...
A. emotional needs?
B. respect?
C. dignity?
D. values and personal preferences for care?
E. cultural, religious, and spiritual needs? /
  • Not priority aspects of follow-up care
  • Participants more concerned with activities needed to recover
  • Not grammatically correct and nonsensical to "treat" respect, dignity, values and personal preference, cultural needs
/ Deleted.
Post 11: Were you treated fairly by all healthcare providers and treated the same as the other injured people around you? /
  • Participants suggested this was a low-priority issue in the post-acute setting
  • Additionally, participants do not observe interactions between providers and other patients in the post-acute setting and therefore had difficulty making comparisons necessary to provide an evaluation
/ Deleted.

Table 8. Items with redundant content and those that overlapped with content on the HCAHPS survey

Original Survey, Item # / Specific Issue/Interpretation / Action Taken
Acute 7a: Did you get the information you needed in a way you could understand about your injuries? /
  • Overlap with item on clarity of information
  • Unique interpretation of this item was whether providers explained all of the patients' injuries
  • Participants evaluated whether providers explained what was injured and anatomically how it occurred
/ Revised. Refocused on the unique part of participants' interpretations.
Acute 10: Did your healthcare providers clearly explain all your injuries to you in a way you could understand?
Acute 10c: How often were your feelings of anxiety treated well? /
  • Participants interpreted anxiety the same as agitation/irritability
  • Overlap with item on agitation/irritability
/ Deleted.
Acute 10d: How often was your pain treated well? /
  • Overlap with pain items on the HCAHPS survey
  • Items were compared and participants preferred the HCAHPS items
  • Pain management one of the most important components of the measure
/ Revised. Two well-validated pain items from the HCAHPS survey were added to replace our single pain item.
Acute 18: How often was your pain well controlled?
Acute 19: How often did the healthcare providers do everything they could to help you with your pain?
Acute 10g: How often were you treated with respect? /
  • Overlap with HCAHPS item about courtesy and respect
/ Deleted.
Acute 10i: How often were your values and personal preferences for care considered by the healthcare providers? /
  • Overlap with two items: “concerns and frustrations” and "cultural, religious, spiritual needs"
/ Deleted.
Acute 12: How often was the hospital kept clean, comfortable, and maintained? /
  • Evaluations were made of room cleanliness not overall hospital cleanliness, suggesting overlap with HCAHPS item on clean room and bathroom
/ Deleted.

Table 9. Additional content identified as relevant for injury patients