ALBERTA MANAGEMENT COMMITTEE ON DRUG UTILIZATION / ALBERTA DRUG UTILIZATION PROGRAM

INTERNAL EVALUATION - ACADEMIC DETAILING (BEHAVIORAL CHANGE) INITIATIVE – OSTEOPOROSIS


Prepared by Harold Lopatka

DRAFT –Revised June 2005

INTERNAL EVALUATION- ACADEMIC DETAILING (BEHAVIORAL CHANGE) INITIATIVE – OSTEOPOROSIS

EXECUTIVE SUMMARY

An internal evaluation of the behavioral change (academic detailing) initiative was conducted based on the osteoporosis educational topic provided in David Thompson Health Region over the period October 2003 to April 2004. The evaluation consisted of four components (individual reviews): 1) review of physician feedback, 2) review of academic detailer feedback, 3) review of Alberta Blue Cross prescription claims data, and 4) review of lessons learned. The following were the results. Physician feedback was consistent with that received from other topics and very positive. Academic detailer feedback was congruent with physician comments and indicated continued high levels of physician interest. The review of prescription claims data revealed a small discernable impact on drug utilization. Also, there is an indication that the intervention may have had an impact on drug special authorizations for high risk osteoporosis patients with fractures. In terms of lessons learned, it takes time and effort for relationships to be established between the detailer and front line physicians. The initiative is still maturing with continued growth occurring and this is an important point in terms of the evaluation context. Further evaluation of the impact on drug utilization will occur with additional claims data requested from Alberta Health and Wellness.

INTRODUCTION

In Canada, approximately 1 in 4 women and 1 in 8 men have osteoporosis. Osteoporosis is presumed to cause 24 000 hip fractures in Canada each year, and these figures are projected to double by 2040 as the population at highest risk, women aged 75 years and older increases. The risk of death within one year of hip fracture is 17% in women, and 32% in men. As well, post-fracture ramifications for patients include 50% losing their independence, and 33% never returning to pre-fracture health.

It has been estimated that in 1993, the total acute care cost for osteoporosis (inpatient, outpatient care and drug therapy) was over $1.3 billion CAD. Without effective action on osteoporosis prevention and treatment strategies, it is estimated thatby 2018Canada will spend at least $32.5 billion treating osteoporotic fractures. Given the increasing proportion of older people in the population, these costs will likely rise.

In 2002 a national guideline was published, 2002 Clinical Practice Guidelines for the Diagnosis and Management of Osteoporosis in Canada.

There are a number of pharmacologic agents available for the prevention and treatment of osteoporosis: bisphosphonates (etidronate, alendronate, and risedronate); calcitonin, hormone replacement therapy (estrogen and/or progesterone), and selective estrogen-receptor modulators (raloxifene). Figure 1 indicates the increased number of persons receiving various osteoporosis therapy over seven years. Only etidronate and hormone replacement therapy have open listing on the Alberta Health and Wellness Drug Benefit List (AHWDBL). Appendix 1 contains the special authorization criteria for the other agents (based on the April 2004 AHWDBL).

Figure 1 - Number of Patients Taking Different Osteoporosis Therapies in Alberta *

Over the Past 7 Years Ending March 31, 2003

*Includes claimants covered under the Alberta Health & Wellness drug programs: Group 1 and Group 66/66A.

Literature reviews were conducted for studies related to the assessment and treatment of osteoporosis in patients with fragility fractures, and in patients on long term glucocorticoid use. Reported medication prescribing adherence to clinical practice guideline rates vary from 4-86%. The following table summarizes studies with documented ratesof clinical practice guideline adherence.

Table 1 – Summarizes literature reported Osteoporosis clinical practice guideline adherence rates

Osteoporosis / Guideline adherence levels
Diagnosis / Medications
Patients with fragility fractures / 49% women / 4% men received BMD screening or treatment / 71% women / 27% men received treatment
Patients on long term glucocorticoid use / NA / Study 1 - 4-86% received appropriate agents
Study 2 - 63-86% received treatment

A recent systematic review by Grimshaw et al. showed the effect of various educational interventions for guideline dissemination and implementation. It is important to note that where evidence exists, the interventions produced modest to moderate improvements. The evidence suggests that single-patient directed interventions and reminders are more effective than other single interventions or multifaceted interventions (with educational outreach).

Table 2 - Absolute Improvements from Interventions

(Guideline Dissemination and Implementation Strategies)

Single Intervention / Absolute Improvement (range)
Educational outreach
Local consensus processes
Local opinion leaders
Financial, structural or regulatory
Dissemination of educational materials
Educational meetings (e.g. CE)
Audit and feedback
Patient directed
Reminders

Multifaceted interventions consisting of at least 2 interventions (including educational outreach) / No comparisons
+8.1% (+3.6 to 17%)
Small effect
+7.0% (+1.3 to 16%)
+20.8% (+10 to 25.4%)
+14.1% (-1 to 34%)
+7.3% (-5.6 to 17.4%)

Reported in review by Grimshaw et al. from RCTs of a single intervention vs. no intervention controls

One study was located about an educational intervention to improve osteoporosis clinical practice guideline adherence. Majumdar et al. reported on a multifaceted intervention designed to improve primary care physician adoption of evidence based guidelines in patients with wrist fractures. The intervention consisted of physician reminders (personalized, patient-specific and faxed), treatment guidelines generated and endorsed by opinion leaders and patient education (provision of written materials and telephone counselling). The intervention resulted in 30% higher treatment rates in treatment compared to control patients.

PURPOSE

The purpose of this report is to evaluate the impact of the Osteoporosis academic detailing (behavioral change) intervention in the David Thompson Health Region.

The following were anticipated apriori outputs or outcomes from the intervention.

  • Equivalent levels of physician satisfaction and interest as compared to earlier topics
  • Equivalent or increased usage of academic detailer compared to earlier topics
  • Increased utilization of first choice agents (and reduced use of editronate)
  • Increased requests for the newer special authorization agents
  • Increased claim costs for physicians receiving intervention

METHODS

Description of intervention

The multifaceted educational intervention consisted of passive clinical practice guideline dissemination, multi-disciplinary continuing education, academic detailing (educational outreach), printed educational materials (see appendix 2), opinion leader consultation and comparative prescribing feedback reports. The intervention was conducted in the David Thompson Health Region (RHA #4) over the period October- December 2003 and January-March 2004.

Figure 2 – Sequence of Individual Components of David Thompson Health Region Osteoporosis Educational Intervention

CME Academic detailing visits andComparative Prescribing

Session¹ opinion leader clinic visits² Feedback Report³

¹Dr. Don Morrish (endocrinologist) plenary speaker. Telehealthed to 10 sites in David Thompson Health Region – 61 (25 physicians) attended.

²51 detailing visits completed (detailing plus opinion leader sessions).

³Two reports sent – April and June 2004 (see appendix 3 for template).

Four evaluation strategies were employed; review of physician evaluations, review of academic detailer evaluations, review of Alberta Blue Cross prescription claims data and documentation of lessons learned. A description of the specific methods used for each evaluation strategy follows.

Review of physician evaluations

Physician feedback was collected from feedback from continuing medical education and post visit questionnaires. Copies of the data collection forms are attached in the appendix (see appendix 4 – CME evaluation and appendix 5 – post visit). Mean scores were determined and tabulated.

Physician feedback about the continuing education was collected after the initial continuing education session and evaluated the presentation and materials provided. Physician feedback about the educational visit and printed materials was collected after the academic detailing visit and evaluated the visit and the printed materials provided.

Review of academic detailer evaluations

Academic detailer written reports from visits documenting perceived success and accomplishments were examined. A copy of the data collection form is attached in the appendix (see appendix 6). Key data reviewed included wait length (minutes), visit length (minutes), key messages discussed (number), written material left (number of documents), follow-up items (number), perceived physician interest in visit (rating out of 5), perceived physician interest in written materials (rating out of 5), perceived physician interest in CPG (rating out of 5), and overall impressions about the guideline topic and physicians’ reactions. Information was tabulated.

Review of Alberta Blue Cross prescription claims data

Data was requested from Alberta Blue Cross (ABC) for the government sponsored plans (Groups 1, 66, 66A) for special authorization and drug claim data for the period April 1, 2003 to September 30, 2004 (see appendix 7). This provided data for 6 months preceding the intervention and 6 months post intervention. Data requested is specific to the requests received for special authorization of etidronate, alendronate, raloxifene, risedronate and synthetic calcitonin salmon. The data requested is for two health regions: David Thompson Health Region (#4), test group, and Peace Country Region (#8) for a control group. A number of physicians in David Thompson Health Region participated in individual components of the intervention but not the full intervention. They were not included in the test group. Agents were grouped into first and second choice agents for treatment. When there are no vasomotor symptoms or when fragility fractures occur, the clinical practice guideline lists alendronate, risedronate and raloxifence as first choice agents and calcitonin and etidronate as second choice agents. An inter- and intra- health region analysis was conducted with the above data. Tabulations and cross tabulations were performed.

Lessons learned

Academic detailer and management correspondence and reports were reviewed to determine lessons learned. Common implementation and operational themes were identified.

RESULTS

Physician evaluation

Table 3 – Comparison of evaluation scores from continuing medical education on two topics (Upper respiratory tract infections – URTI and osteoporosis – OP)

Questions (scale used was 1 unacceptable, 2 poor, 3 satisfactory, 4 good, and 5 excellent) / URTI / OP
1. The overall quality of the presentation was / 4.7 / 4.4
2. The session outline or brochure provided enough information / 4.7 / 4.5
3. The session package contained all need information / 4.8 / 4.6
4. The session content was presented in a well organized manner / 5 / 4.6
5. The session content was communicated with enthusiasm / 4.9 / 4.7
6. Learner questions and comments were responded to appropriately / 4.9 / 4.6
7. Learners were treated respectfully / 4.9 / 4.7
8. The support material (readings, AV materials, equipment, software) used in this session really helped me / 4.4 / 4.4
9. I learned a lot at this session / 4.6 / 4.5
10. Overall, I was satisfied with the session / 4.9 / 4.5

Evaluation scores were 4.4 or higher. Evaluation scores relating to the osteoporosis continuing medical education event ranged from 4.4-4.7. Scores for questions relating to the URTI event ranged from 4.4 – 5.

Table 4 – Comparison of physician evaluation scores from academic detailing visits on three topics (Upper respiratory infections – URTI, gastro-intestinal – GI, and osteoporosis – OP)

Questions / Topic mean scores
URTI / GI / OP
Academic detailer acted in polite and professional manner (out of 5) / 4.8 / 4.7 / 4.8
Academic detailer was knowledgeable (out of 5) / 4.8 / 4.7 / 4.7
Information received was useful to my practice (out of 5) / 4.5 / 4.7 / 4.6
Educational visit was valuable use of time (out of 5) / 4.4 / 4.3 / 4.6
Visit meet my expectations (out of 5) / NA / NA / 4.5
Value of educational visit compared to CME (out of five) / 4.2 / 4.0 / 4.2
Time spent with detailer (minutes) / 19.4 / 23.1 / 38.5
Was the amount of time (too long, short or just right) / Just right / Just right / Just right

Table 4 summarizes physician evaluations and compares the osteoporosis visits to data from previous topics. The mean scores for physician’s evaluations of the osteoporosis detailing visit was 4.5 (out of 5) or higher except for the one rating for the value of the visit compared to traditional CME (mean score was 4.2 out of 5). Mean scores are similar to scores attained for other topics. Visit times were the highest for the osteoporosis topicwhile physicians indicated the time was just right.

Table 5 – Physician evaluation scores to specific behavioral change questions as a direct result of the visit

Questions - As a result of this educational visit I will be more likely to: / Mean score (out of 5)
  1. Assess postmenopausal women and men over 50 for risk factors for osteoporosis
  2. Conduct BMD tests on all men and work over age of 65 years
  3. Consider bisphosphonate treatment for patients on long term moderate to high dose steroids (>7.5 mg prednisone for >3 months)
/ 4.2
4.2
4.3

Table 5 shows that the mean score of physicians was 4.2 out of 5 in agreement to the three major behavioral messages.

Academic detailer evaluation

Table 6– Academic detailer scores and measures of visits on three topics – Upper respiratory tract infections, gastro-intestinal conditions and osteoporosis

URTI
(66 responses) / GI
(42 responses) / OSTEOPOROSIS
(50 responses)
Wait Length (min) / 6 / not available / 9.8
Visit length (min) / 21.4 / 26.3 / 29.6
# Key messages discussed / 3.5 / 3 / 2.6
# Written material left / 2.4 / 2.8 / 3.8
# Follow up items / 0.6 / 1.4 / 0.7
Overall interest
(out of a score of 5) / 4.375 / 4.375 / 4.5
Interest in CPG
(out of 5) / 4.25 / 4.375 / 4.5
Interest in written info
(out of 5) / 4.125 / 4.25 / 4.5

Table 6 summarizes data and compares the osteoporosis visits to data from previous topics (upper respiratory infections, gastro-intestinal conditions). The average visit time was the highest at 29.6 minutes. An average score of 4.5 was rated for the three measures; perceived physician interest in the visit, written materials and clinical practice guidelines. This score was similar to that from the previous topic GI and slightly higher than the average scores for URTI. The average number of follow-up items was 0.7, which was similar to the URTI, but less than the GI. Overall, the detailer’s impression was that the osteoporosis clinical practice guidelines were frustrating to physicians because provincial payment guidelines do not cover all bisphosphonates for first line use and many were not aware of situations where Fosomax® and Actonel® could be used as first line agents. Physicians found summary treatment charts (printed educational materials) very helpful.

Drug utilization evaluation

Prescription claims

Table 7 – Distribution of prescription claims by RHA for the combined periods (Q 2 2003, Q3 2003, Q2 2004, Q 3 2004)

Prescription claims Number (and percent) / RHA #4 / RHA #8
(control)
Overall / Test
Total
Individual agents:
  • Alendronate
  • Calcitonin
  • Etidronate
  • Raloxifene
  • Risedronate
Clinical practice guideline:
  • 1st choice agents
  • 2nd choice agents
/ 19727
5347 (27.1)
635 (3.2)
11915 (60.4)
389 (2)
1441 (7.3)
7177 (36.4)
12550 (63.6) / 3193
887 (27.8)
82 (2.6)
1833 (57.4)
39 (1.2)
352 (11)
1278 (40)
1915 (60) / 3802
1091 (28.7)
149 (3.9)
2331 (61.3)
168 (4.4)
63 (1.7)
1322 (34.8)
2480 (65.2)

Table 7 summarizes the distribution of prescription claims (total, individual agents, first and second choice) forRHA #4 overall, RHA #4 test group physicians, and RHA #8 physicians. Overall, RHA #4 had approximately 5 times the claims of RHA #8. In terms of individual agents, Risedronate claims were more common in RHA #4 compared to RHA #8 (7.3 vs 1.7%). Use of 1st choice agents was slightly greater in RHA #4 compared to RHA #8. RHA #4 test group physicians had more risedronate claims and fewer etidronate claims.

Table 8 – Pre and post intervention comparison of clinical practice guideline adherence between RHA #4 test physicians and RHA #8 control physicians

Percent of Prescription Claims
RHA #4
(Test Group) / RHA #8
(Control)
Pre intervention (Q2 & Q3 2003)
  • 1st choice agents
  • 2nd choice agents
/ 34.8%
65.2% / 32.4%
67.6%
Post intervention(Q2 & Q3 2004)
  • 1st choice agents
  • 2nd choice agents
/ 44.3%
55.7% / 36.7%
63.3%

Figure 3 – Pre and post intervention comparison of clinical guideline adherence between RHA #4 test physicians and RHA #8 control physicians

Table 8 and Figure3 show the results frompre (Q2 & Q3 2003) andpost (Q2 & Q3 2004) intervention comparisons of agent selection. In the comparison between RHA #4 test physicians and RHA #8 control physicians, guideline adherence increased in both regions, but the increase was higher in RHA #4 test physicians (9.5% compared to 4.3%).

Table 9 – Pre and post intervention comparison of average claim costs between RHA #4 and RHA #8 physician groups

Period / RHA #4 / RHA #8 (control)
Overall / Test
Pre intervention (Q2 & 3 – 2003)
  • All agents
  • 1st choice agents
  • 2nd choice agents
/ $83.99
$103.44
$54.81 / $74.87
$89.40
$53.06 / $89.77
$112.32
$55.95
Post intervention (Q2 & 3 – 2004)
  • all agents
  • 1st choice agents
  • 2nd choice agents
/ $85.58
$106.05
$54.88 / $79.14
$98.03
$50.80 / $90.97
$114.39
$55.83

Table 9 shows the results from two pre and post intervention comparisons of average claim costs. In comparing within RHA #4, test physicians average claim costs increased but were lower than control physicians before and after the intervention. In comparing between RHAs, RHA #4 overall average costs were lower than RHA #8 physicians. RHA #4 test group physicians average costs were lower than RHA #8 control physicians.

Special authorization requests

Table 10 – Frequency of special authorizations, approvals and denials (July 2002 onward)

Special authorizations / Number (percent)
Test (RHA #4) / Combined
(RHA #4) / Control (RHA #8)
Requests:
Approvals:
Indications
OP + fracture
OP + intolerance / failure (didrocal)
OP + contraindicated (didrocal)
Denials / 522
496 (95)
128 (24.5)
365 (69.9)
3 (0.6)
26 (5) / 2197
2074 (94.4)
366 (16.7)
1681 (76.5)
24 (1.1)
123 (5.6) / 600
547 (91.2)
84 (14)
460 (76.7)
3 (0.5)
84 (8.8)

Table 10 summarizes total special authorization requests, approval and denial information for the test group physicians, RHA #4 overall and RHA #8 physicians (number and percent special authorization approvals for the agents alendronate, raloxifene, residronate and synthetic calcitonin salmon). Approvals for the indication osteoporosis and fracture are nearlytwo times higher in the test group (24.5%) compared to the control group (14%).

Lessons learned

The osteoporosis topic was the third educational topic offered to physicians in the region over a two year period. Physician participation changed from 10 physicians for the first topic to over 50 for the osteoporosis topic and the centers (cities and towns) covered increased from 2 to 15. Approximately 30% of eligible family physicians have participated in programs.

Overall, we have achieved successes because of the attention given to building and maintaining relationships at the “system” and “front line” levels. At the “systems” level relationships were established with our key provincial organizations to assist us in a number of support and production processes. As general operating principles, we work on the premise that activities should focus on front line delivery of services and that we should not duplicate activities performed by other organizations. The following illustrates some of the specific “system” relationships.