APPENDICES

Appendix A: Framework of VA-TAMMCS Improvement

Appendix B: Summary of the Primary and Supporting PACT Measures

Appendix C: PACT Collaborative Change Package

Appendix D: Institute for Healthcare Improvement Breakthrough Series Collaborative Model

Appendix E: PACT Staffing Model

Appendix F: Comparison between Institute for Healthcare Improvement Breakthrough Series Assessment Scale and PACT Collaborative Assessment Scale

Appendix G: PACT Compass Metrics

Appendix H: Data Collection Plan

Appendix I: PACT Collaborative Participant Follow-up Survey

Appendix J: Eighty percent (80%) of the implemented PDSA (across all regions) focused on the Nine Main Principles (from change package) to achieve the aims in the PACT collaborative

Appendix K-V: PACT Collaborative Participant Follow-up Survey Results

Appendix K: Unique Participant Number by Learning Session (LS) through LS 6

Appendix L: Follow-up Survey Results (Respondent Demographics)

Appendix M: Participants’ assessment of the PACT Collaborative

Appendix N: Participants’ assessment based on regions

Appendix O: Impact Measures based on PACT Collaborative

Appendix P: Follow-up Survey Results (Overall Evaluations of PACT Collaborative)

Appendix Q: Follow-up Survey Results (Evaluation of Participant Roles)

Appendix R: Follow-up Survey Results (Usefulness of Regional PACT Collaborative)

Appendix S: Follow-up Survey Results (Usefulness of PACT Collaborative Resources)

Appendix T: Follow-up Survey Results (Teams’ Barriers and Enablers)

Appendix U: Follow-up Survey Results (PACT Collaborative Success Factors)

Appendix V: Follow-up Survey Results (PACT Collaborative Participants’ Perspectives on Change)

Appendix A: Framework of VA-TAMMCS Improvement

  • Vision: Leaders identify potential areas (Veteran-centeredness, quality, effectiveness, equity, and efficiency of healthcare delivery and services) for improvement reflecting an integration of the strategies, resources, and performance goals identified at the VA, VHA, Network, and Medical Center levels.
  • Analysis: Narrowing the focus from the vision, leadership mines the broad opportunities into specific priorities (e.g.: improving the phone system) most amenable to action and change.
  • Team: Form a team to plan and implement improvement plans/projects in the transformation process.
  • Aim: Team set an aim (or aims) aligned to access, care coordination/management, practice redesign
  • Map: Flow map depicts the existing process from start to end. The experience of flow mapping allows teams to agree on the process and discover value to the patient or customer, non-standard and unreliable processes, and the re-work that is involved.
  • Measure: To recognize improvement and manage by fact, not feeling, we need to measure. The measure step makes the capability of the process more objective.
  • Changes: Two critical issues affect this area of change: Focus on change principles, not necessarily “best practices” or strategies; Use “small tests of change” as embodied in PDSA or an equivalent approach, before attempting large, system-wide change.
  • Sustain: “Sustain” strategies require team to first think about previous successful changes. Then team think about the process and task changes that resulted in the success. Next, we must “memorialize” those changes by assigning responsibility, changing job descriptions, evaluations, reward systems, policies,and new employee training. Ongoing measurement of the key processes sustains change.
  • Spread: Many good ideas can be applied in more than one VA facility. "Spreading" new developments to other units and work areas can multiply the value of the improvement.

Appendix B: Summary of the Primary and Supporting PACT Measures

Pillar / 7 Required Aim / Primary Measure
Access /
  • A1:Offer appointments on “today”
  • A2:Increase non face-to-face care
/
  • Third next available appointment OR
Difference between desired date and actual appointment date
  • Percent of care provided outside of single provider appointment venues
  • Continuity
  • Percent of calls answered within 30 seconds
  • Percent of patient-generated e-mails responded to within 24 hours
  • Percent of care provided in group visits

Care Coordination and Management /
  • CCM1:Identify and actively manage high risk patients
  • CCM2:Improve handovers with inpatient
  • CCM3:Improve handovers with specialty
/
  • Percent of high risk patients being actively managed
  • Percent of patients with contact or visit within 48 hours/7 days of transition from ED or Hospital
  • Percent adherence to PC portion of service agreement (right patient with correct work-up)
  • Percent increase in 2-way pre-discharge patient handoff communication

Practice
Redesign /
  • PR1:Start appointments within 10 minutes of scheduled time
  • PR2: Improve team communication
/
  • Cycle time (and subsets)
  • Minutes behind
  • Ratio of red zone to total cycle time
  • Percent increase in teamlet huddles/week
  • Percent increase in team meetings /month
  • Pre- and post-team communication assessment

Appendix C: PACT CollaborativeChange Package

Appendix D: Institute for Healthcare Improvement Breakthrough Series Collaborative Model

Source: Institute for Healthcare Improvement. The Breakthrough Series: IHI’s Collaborative Model for Achieving Breakthrough Improvement. 2003.

Appendix E: PACT Staffing Model

Note: This is an ideal model fora non-special population PACT, and there is variability from site-to-siteon the ratio of discipline-specific PACT staff (social worker, clinician,pharmacist, dietitian).

Appendix F: Comparison between Institute for Healthcare ImprovementBreakthrough Series Assessment Scale and PACT Collaborative Assessment Scale

IHI Assessment Scale for Collaborative* / PACT Coach Assessment Scale
Assessment / Definition / Assessment / Definition
0 / No effort in this area
1.0 / Forming team / 1.0 / Discussion has started and likely to move into action
1.5 / Planning for the Project has begun / 1.5 / Plans for action begun
2.0 / Activity, but no changes / 2.0 / A few tests of change have occurred
2.5 / Change tested, but no improvement / 2.5 / Tests of change begun, no measurable progress
3.0 / Modest improvement / 3.0 / Several successful tests of change with outcomes
3.5 / Improvement / 3.5 / Testing multiple changes; measurable progress over > 3 data points
4.0 / Significant improvement / 4.0 / Achieved aims(s)
4.5 / Sustainable improvement / 4.5 / Surpassed team's aims; spread to wider area
5.0 / Outstanding sustainable results / 5.0 / Spreading work into other areas, coaching others

*Source: Institute for Healthcare Improvement. The Breakthrough Series: IHI’s Assessment Scale for Collaborative. 2004.

Appendix G: PACT Compass Metrics

Category
/ Metric
Panel Management / PCP Panel Assignments
Panel Uniques
PC Patients Enrolled in CCHT
Percent PC PT Enrolled in CCHT
DCG Average
PCP Panel Size Average (PCP/AP Adjusted)
PCP Capacity Average (PCP/AP Adjusted)
PCP Capacity Total
PCP FTE (PCP/AP Adjusted)
PCP FTE Total
PCP/AP FTE Total
PC Staffing Ratio**
Access / Ratio of PC Telephone Encounters to All PC Encounters-PACT 9
Ratio of Non-Tradtional Encounters
Primary Care Group Encounters
ALL PC PT% Seen in Primary Care on Same Day of DD (Excluding C&P)
ALL PC Pt% Seen in Primary Care Within 1 Day of DD (Excluding C&P)
ALL PC Pt% Seen in Primary Care Within 7 Day of DD (Excluding C&P)-PACT 6
ALL PC Pt% Seen in Primary Care Within 14 Days of DD (Excluding C&P)
Average 3rd Next Available in PC Clinics (322,323,350)
Same Day Appts w/PC Provider Ratio-PACT 7
Established Primary Care Prospective Wait Times 14 Days
Established Primary Care Prospective Wait Times 7 Days
Total PC Patients Authenticated In Person
Total PC Patients Opted In to Secure Messaging
Total Inbound PC Secure Messages
Total Inbound PC Secure Messages Escalated
Total Outbound PC Secure Messages
Continuity / Continuity PCP (Fee ER Excluded)-PACT 8
Continuity PCP (Fee ER Included)
Ratio of ER Encounters While on Panel to PC Encounters While on Panel (FEE ER Excluded)
Ratio of ER Encounters While on Panel to PC Encounters While on Panel (FEE ER Included)
Coordination of Care / ER/Urgent Care Utilization Rate by Panel
Inpatient Admission Rate
2day Contact Post Discharge Ratio-VHA D/C-PACT 11
2day Contact Post Discharge Ratio
7day Contact Post Discharge Ratio
Medical Home Builder Survey
Click to View
VISN Detail / Overall Medical Home Builder Survey Response
Module 1: Patient-Centered Care & Communication
Module 2: Access & Scheduling
Module 3: Organization of Practice
Module 4: Care Coordination & Transitions of Care
Module 5: Use of Technology
Module 6: Population Management
Module 7: Quality Improvement & Performance Improvement
All Employee Survey / Satisfaction - Primary Care Avg
Satisfaction - Organization Avg
Satisfaction-2yrs - Primary Care Avg
Satisfaction-2yrs - Organization Avg

Appendix H: Data Collection Plan

Category / Objective / Measure / Survey / Timing
Satisfaction & Reaction /
  • Participants perceive the PACT Collaborative as a valuable initiative for the VA
  • PACT Collaborative objectives are relevant to participant’s work
  • PACT Collaborative participants find the support components effective (coaching, regional calls/reports, etc.)
  • PACT Collaborative Model and implementation structure is effective (Collaboratives, regional structure, teaching methods, etc.)
  • Coaching component is an effective support strategy to facilitate successful implementation.
/
  • EES Standard program evaluation
  • Post LS4 Questionnaire
/
  • At the conclusion of each Collaborative
  • Post LS4 (June 2011)

Learning & Skill Acquisition /
  • Collaborative participants gain knowledge and skills in the principles, processes and competencies of PACT.
  • PACT Collaborative participants accomplish intended session learning objectives.
/
  • EES Standard program evaluation survey
  • Post LS4 Questionnaire
/
  • At conclusion of each Collaborative
  • Post LS4 (June 2011)

Job Impact and Performance /
  • PACT Collaborative participants plan, implement and sustain actions/behaviors upon returning from collaborative training sessions.
  • PACT Collaborative team members comply with national and regional reporting & documentation requirements for projects.
  • Barriers to application
  • Enablers to application
/
  • Post LS4 Questionnaire
  • Team Reports, if applicable
/
  • Post LS4 (June 2012)
  • Post LS6 (Jan 2012)

Business Impact /
  • Primary and Support Measures
/
  • Follow Up Questionnaire
/
  • Post LS4 (June 2011)
  • Post LS6 (Jan 2012)

Appendix I: PACT Collaborative Participant Follow-up Survey

Introduction Section:

As a member of the PACT Collaborative Traveling Team or Teamlet who has attended at least one Collaborative Learning Session, please provide us with feedback on your progress with applying and implementing the concepts and skills you have acquired. This survey will take approximately 15 - 20 minutes to complete and all responses will remain confidential.

Instructions

  1. To provide responses, you will need to reflect on the Regional PACT Collaborative and think about specific ways in which you have applied what you learned from the experience. It may be helpful to review the materials provided during the variousLearning Sessions.
  2. Please take your time as you provide responses. Accurate and complete responses are critical. Your responses will help determine the impact of this program as well as future changes necessary to the program and process. By completing this survey, you will ensure your input is included along with the other participants of the Regional PACTCollaboratives.
  3. Please be objective in providing your responses. In no way will your name be linked to your input. Your responses will be consolidated with others.

Thank you for your time and valuable feedback!!!

Questionnaire Content:

Reaction/Learning Questions:

  1. I attended the following PACT Collaborative Learning Sessions (select all that apply)

Learning Session 1
 / Learning Session 2
 / Learning Session 3
 / Learning Session 4
 / None

  1. Please rate your level of agreement with the following statements.

Strongly Disagree / Disagree / Neutral / Agree / Strongly Agree
The PACT Collaborative was necessary to implement the Patient-Aligned Care Team initiative within the VA. /  /  /  /  / 
The format of PACT Collaborative Learning Sessions was appropriate for implementing the PACT change strategy. /  /  /  /  / 
The Regional PACT Collaborative provided me and my team with opportunities to learn from other teams and share lessons learned. /  /  /  /  / 
The Regional PACT Collaborative provided me and my team with opportunities to share successful strategies. /  /  /  /  / 
The Regional PACT Collaborative provided me and my team with opportunities to network with other teams. /  /  /  /  / 
I completely understand my role in implementing PACT aims. /  /  /  /  / 
Regarding the PACT model and the changes needed, I have the necessary influence to make the improvements expected of me /  /  /  /  / 
I completely understand the roles of my team members in implementing PACT. /  /  /  /  / 
  1. One of the expectations of the Learning Session attendees was to share the information with individuals who were not in attendance (teach back). With this in mind, please rate your level of agreement with the following statements (note: select N/A if this statement does not apply to you):

Strongly Disagree / Disagree / Neutral / Agree / Strongly Agree / Not Applicable
As an attendee of the Learning Session(s), I was provided with adequate resources and tools to share information with the team back at my facility /  /  /  /  /  / 
As an attendee of the Learning Session(s), the resources and tools provided enabled me to successfully inform the team back at my facility /  /  /  /  /  / 
  1. As a result of your participation in the Regional PACTCollaboratives, please indicate (in percentage) the increase in your knowledge and understanding of how to implement PACT.

0%
 / 10%
 / 20%
 / 30%
 / 40%
 / 50%
 / 60%
 / 70%
 / 80%
 / 90%
 / 100%

  1. In addition to the face-to-face Learning Sessions, you may have utilized additional PACT Collaborative resources. Please rate the level of usefulness you feel each of the following resources contributed to the PACT initiative.

Not Useful at All

Slightly Useful

Moderately Useful

Very Useful

Extremely Useful

Not Applicable

Not Useful at all / Slightly useful / Moderately useful / Very useful / Extremely Useful / Not Applicable
Coaching (formal support or coaching offered to assist with PACT-related efforts; provide feedback on team reports regularly) /  /  /  /  /  / 
Regional LiveMeeting/Conference Calls (venue to present work, receive coaching, and learn what others are doing) /  /  /  /  /  / 
Reports (monthly reports to update Collaborative Directors, Coordinators, and Coaches on work and aims progress) /  /  /  /  /  / 
Prework Guide (guide of PACT principles and applicable content teams can use as a resource) /  /  /  /  /  / 
PACT Collaborative Change Package (key principles and strategies organized by Pillar to assist teams in their work in the Collaborative)
SharePoint Library (venue for sharing resources, materials, team reports, and presentations) /  /  /  /  /  / 
VERC Industrial Engineer (Veterans Engineering Resource Center IE aligned with your region to support your Coach and team and assist with measurement)
Collaborative Coordinators (coordinate Collaborative events, collect team information, update SharePoint Library, and develop learning content) /  /  /  /  /  / 
Collaborative Directors (provide over-arching direction, goals, overall progress for Collaborative, share expertise with teams, and lead new learning content development efforts) /  /  /  /  /  / 
Measurement Tools (Excel spreadsheet developed for each Pillar to capture data and produce graphs)
  1. Please share any comments or suggestions you have regarding the resources mentioned above:

______

Application/Implementation Questions

  1. In my facility:

I have participated in team work for the following number of PACT aims: ____ (enter #)

Of the aims in which I have participated, the number of projects completed (aim has been attained) is: ______(enter #)

If you have not participated in or completed a PACT aim in your facility, please explain why:

______

  1. As a result of participating in the PACT Collaborative, please rate your level of success in completing the following activities:

4= Very successful (e.g., guidance is not needed to complete this task; able to teach peers)

3 = Successful (e.g., only occasional guidance is needed to complete this task)

2 = Somewhatsuccessful (e.g., frequent guidance is needed to complete this task)

1 = Not successful (e.g., not able to complete this task even with guidance)

Not applicable (e.g., haven’t had any experience with it)

(4)
Very Successful / (3)
Successful / (2)
Somewhat Successful / (1)
Not Successful / Not
Applicable
Recruit the right people on your team for a successful improvement effort /  /  /  /  / 
Set clear and mutually agreeable aim statements /  /  /  /  / 
Establish a simple measure that will indicate if change leads to an improvement /  /  /  /  / 
Flow map clinic process(es) to identify bottlenecks and waste /  /  /  /  / 
Report team’s progress towards aims to regional coordinators /  /  /  /  / 
Conduct Rapid PDSA cycle tests of change /  /  /  /  / 
Obtain buy-in from others in the organization to successfully implement recommended changes /  /  /  /  / 
Spread PACT to other teamlets at Medical Center /  /  /  /  / 
  1. In participating in PACT improvements, please rate your level of success in completing the following work on Pillar principles and strategies:

4 = Very successful (e.g., guidance is not needed to complete this task; able to teach peers)

3 = Successful (e.g., only occasional guidance is needed to complete this task)

2 = Somewhatsuccessful (e.g., frequent guidance is needed to complete this task)

1 = Not successful (e.g., not able to complete this task even with guidance)

Not applicable (e.g., haven’t had any experience with it)

(4)
Very Successful / (3)
Successful / (2)
Somewhat Successful / (1)
Not Successful / Not
Applicable
Practice Redesign:
Balance supply and demand /  /  /  /  / 
Synchronize patient, information and staff at time of appointment /  /  /  /  / 
Predict and anticipate patient needs /  /  /  /  / 
Optimize the environment (e.g., standardize room, fully stocked room, etc…) /  /  /  /  / 
Manage constraint /  /  /  /  / 
Case Management and Coordination
Implement alternatives to face-to-face provider-patient visits /  /  /  /  / 
Create infrastructure to manage / coordinate care /  /  /  /  / 
Improve transitions between PACT and other venues of care /  /  /  /  / 
Access
Measure demand & supply /  /  /  /  / 
Implement strategies to balance supply and demand /  /  /  /  / 
Implement contingency plans /  /  /  /  / 
  1. Please share any comments you have regarding the success or difficulties with completing the tasks above (if not applicable, please leave blank):

______

  1. Please rate the extent to which each of the following factors is either an enabler or barrier in facilitating your application of PACT principles:

Factor / Definitely an Enabler / Somewhat of an Enabler / Neither an Enabler or a Barrier / Somewhat of a Barrier / Definitely a Barrier
Opportunities to practice and use the skills learned /  /  /  /  / 
Usefulness of Skills/Behaviors /  /  /  /  / 
Time allowed to implement PACT principles /  /  /  /  / 
Level of complexity in implementing PACT principles /  /  /  /  / 
Support of work environment /  /  /  /  / 
Senior leadership support /  /  /  /  / 
Supervisor support /  /  /  /  / 
Impact on my career development goals /  /  /  /  / 
Support of my local PACT team /  /  /  /  / 
Productivity of my local PACT team /  /  /  /  / 
Customer support /  /  /  /  / 
Recognitions / awards received for improvement efforts /  /  /  /  / 
VHA's commitment /  /  /  /  / 
Priority level within the VHA /  /  /  /  / 
Confidence level to use the process /  /  /  /  / 
Availability of resources /  /  /  /  / 
Availability of assistance (e.g., coaching) /  /  /  /  / 
  1. Please provide any other enablers or barriers you have experienced during the implementation of PACT principles:

______