Curriculum renewal in musculoskeletal medicine:

Assessment of a National Framework within the curriculum of one institution

$5,000

Associate Professor Helen Keen, Assistant Professor Lisa Caputo, Winthrop Professor Fiona Lake

1.1. Project Outcomes

  1. To develop and implement a strategy to audit a local medical curriculum to determine its level of alignment to a national Musculoskeletal Education Framework (AMSEC), including development and testing of an auditing tool

1.2 Background and Rationale for project:

The frequency of chronic musculoskeletal (MSK) related disease or traumatic injury is increasing, with a major consequence being an elevated burden on medical and allied medical health resources worldwide (Woolf & Akesson, 2001; Brooks, 2006). There is however, significant evidence both nationally and internationally, that undergraduate education in MSK science is not adequate to meet these increasing health impacts (McColl, 2005; Freedman & Bernstein, 1998; Mulhall & Masterson, 2005). In Australia, the Australian MusculoSkeletal Education Collaboration (AMSEC) project began in 2005, with the aim of developing nationally agreed MSK core competencies based on the Global Core Recommendations for a Musculoskeletal Undergraduate Medical Curriculum, for implementation in all Australian medical schools by 2010 (Chehade & Bachorski, 2008). The Framework details a comprehensive basic and clinical science knowledge base which supports MSK competencies developed in physical examination, red flag emergencies, procedural skills, disease self management, patient education and rehabilitation.

At the University of Western Australia (UWA), challenges in providing quality education in MSK medicine to students have been recognised, with a number of developments having impacted on the delivery of this unit in the past 5 years, including the movement of patient care for MSK conditions from the traditional teaching hospital setting into smaller, more geographically dispersed locations and increasing student numbers, both of which have changed the nature of the teaching and learning environment with fewer clinical opportunities than were originally available for students. In addition, the medical curriculum at the University of Western Australia is being renewed into a new 4-year postgraduate format, with greater opportunities for vertical and horizontal integration necessary. With the confluence of both the UWA curriculum rewrite and the AMSEC competencies it was an ideal time to examine the MSK component of UWA’s medical program, providing an excellent opportunity to improve the quality of education and optimise the student experience. Although the AMSEC competencies could not be mapped to the future curriculum at this point (as content is still being designed as we write), it has provided invaluable insights as to what material can be taken forward, needs modification or needs to be added to the new curriculum.

2.0 Aims and Objectives

The objectives of this project were to develop an audit tool that would allow the investigators to map the current UWA MBBS curriculum onto the AMSEC competencies, and to use the resulting information to inform current curriculum discussion. In addition, it was hoped that this process would provide a template that would be of use to other institutions contemplating their MSK curriculum.

3.0 Methods

Design methodology of the curriculum audit tool

Using the AMSEC document “Summary/Overview of key MSK competencies” the investigators drafted an audit tool to attempt to capture the extent to which the current MBBS curriculum aligned with the recommendations of AMSCE regarding teaching of MSK material.

The AMSEC document consists of 3 main areas: (a) Basic Science and Supporting Knowledge Competencies, (b) Clinical Sciences and Skills Competencies, and (c) Priority Ambulatory Conditions, which is a list of medical conditions arranged according to body regions or other categories (such as rheumatology). The first 2 categories are general competencies addressing both broad areas of knowledge (such as professionalism), as well as areas that address MSK concerns specifically. The third category, Priority Ambulatory Conditions, is a list of conditions that AMSEC felt should be covered in an MSK curriculum. Within the Priority Ambulatory Conditions, there are also 2 special sub-categories, called “National Priority and High Burden Conditions” (those conditions that have been identified as having high personal and societal costs involved in their care) and “MSK/Red Flag Conditions” (conditions that are considered medical emergencies), which are deemed to be of special importance. The audit tool was also arranged in the same format with the same categories.

The first concern in the development of the checklist was the conversion of the competency statements in Categories 1 and 2 from generalized statements into more specific points that could be more easily itemized. However, there was also a need not to make the instrument more detailed that it absolutely needed to be. Finally, there were some areas where the document had broader competency statements listed at the top level of a category, then more specific areas listed beneath (e.g assessment vs. history, examination and investigations). Ultimately, decisions were made by investigators regarding which statements to include, and which to leave out to limit repetition in the document which would have created difficulty in the mapping exercise.

A second concern of the investigators was the best means to demonstrate how each competency “penetrated” the curriculum in terms of scope of coverage. It was felt that to have a meaningful analysis of the local UWA curriculum, that it was important to capture enough detail so that anyone looking at the audit tool could see what material was covered and in which course. It was also felt to be important to see whether competencies were fully reflected within the learning cycle (that is, the competency would be addressed in learning outcomes, learning experiences, and assessment strategies), or whether there was limited coverage (e.g. only a learning outcome is listed, without the related learning experience or assessment). Therefore, under each individual AMSEC competency 3 sections were added where auditors could record which local UWA learning outcomes addressed this competency (along with which courses these outcomes appeared in), what learning experiences addressed the competency, and which assessment methods are currently used to assess the competency.

The audit tool underwent several iterations with revisions taking place after each investigator independently utilised the instrument, and then met to review ease of use and whether all important information was being captured appropriately.

Once both investigators were confident with the final format of the audit tool, it was used to review the UWA MBBS course for MSK-relevant materials.

Curriculum Review methodology

Each investigator completed the audit tool independently, then met afterwards to review areas of discrepancy. When areas of discrepancy were noted, discussion between investigators was undertaken, often referring back to the original data sources, until there was consensus on how the UWA curriculum material was to be rated.

The main sources of data utilized for the curriculum audit were the MappEDout database (which captures all the learning outcomes for the Faculty of Medicine, Pharmacology and Health Sciences), and the learning materials for those courses which teaches information that appeared relevant to the MSK competencies, including guidebooks, lecture handouts, and assessments, where possible. All sources of data were matched to individual AMSEC competencies, and these were recorded in the audit tool.

After all materials had been reviewed and allocated to each competency, each AMSEC competency was rated according to 2 factors, level of coverage in the learning cycle (that is, whether it was addressed in learning outcomes, learning experiences, and assessment), and whether there was evidence of vertical integration (by presence of more than one year level covering the material in their course). Level of coverage was designated by a number from 0 (no coverage) to 3 (full coverage in outcomes, experiences and assessment), and the presence of vertical integration was designated by a “V”. This information was then recorded in a summary chart, looking at the category of competency (Basic Science, Clinical Sciences, Priority Ambulatory Conditions as well as the subcategories of National Priority and High Burden Categories) versus the level of coverage and integration.

4.0 Results

A completed audit tool for the MBBS course at UWA is included in Appendix A, including the summary chart of the level of integration at the end of the tool.

The vast majority of the AMSEC competency framework in the Basic and Clinical Sciences was addressed in the current MBBS MSK course as outcomes, and demonstrated a high level of coverage in an individual year level as well as in vertical integration across years. As would be expected, the Basic Sciences category demonstrated more vertical integration with earlier years of the MBBS program, and the Clinical Sciences demonstrating more vertical integration with the later years of the MBBS program.

The level of coverage of competencies in the Priority Ambulatory Conditions varied widely. Specifically, Rheumatological conditions and skeletal endocrine conditions generally demonstrated good coverage with a high level of coverage. Orthopaedic content tended to be based around body regions and joints, and there was large variation in the level of coverage, with some regions being very poorly covered, such as the foot and ankle.

Reassuringly, all the National Priority and High Burden conditions demonstrated good coverage (rating above 2) with 6 having full coverage as well as vertical integration. Of greater concern was the fact that there were many MSK/Red Flag conditions that appear to have limited or no coverage within the MBBS curriculum.

5.0 Budget Expenditure

$5,000 has been spent on Lisa Caputo’s salary.

6.0 Discussion

With the move towards outcomes-based education (OBE) within many health professional programs, as evidenced by the increasing number of competency documents at national and international level, has come an increased focus on moving the exercise from specification of desirable learning outcomes towards ensuring that the outcomes specified are indeed implemented. As has been noted by several authors (Harden 2007, Ellaway 2007), OBE can be a powerful tool for curriculum development, but only if the necessary steps are taken to ensure there is appropriate translation of those outcomes into the curriculum itself.

To this end, there are several methodologies that curriculum developers have suggested to assist in this process of ensuring appropriate implementation of competency documents. Harden (2007) advocates an “Outcome Based Implementation Profile” for a curriculum, an instrument that rates a curriculum along 9 dimensions relevant to curricula, and applies a 6-point Likert scale to each dimension to indicate how well a program has implemented OBE. This methodology is comprehensive, but some items such as “Student Selection” are well out of the realm of an individual course within the MBBS program, the application of such a tool at a course level appears to be somewhat limited. Further, the use of the Likert scale adds a level of subjectivity as there is limited information to guide scoring of each dimension, while at the same time removing essential information regarding the specific competencies and content covered.

Ellaway et al (2007) underwent an exercise cross-referencing two different national competency documents, the Scottish Doctor learning outcomes framework, and the Tomorrow’s Doctors guidance document. In her exercise, she found it necessary to break down larger prose statements into discrete, hierarchical structures that enabled a numerical identification system to be applied. These numerical identifiers were then used to create a relational database. This appears to be closer to the methodology undertaken in the current study, although Ellaway’s exercise was undertaken between 2 competency documents, as opposed to mapping a competency document onto a local curriculum.

The key areas that have arisen through the auditing process that will allow us to restructure the course to improve the quality and content of the course and it’s delivery within the structure of the MBBS at UWA and hopefully improve the student experience. These areas can be identified in the attached completed audit, but can be summarised as;

  • Better defining our course outcomes to reflect the AMSEC framework, or concepts and content that are currently taught and examined, but not currently well defined in the course outcomes.
  • History taking focusing on the MSK presentation
  • Imaging, laboratory tests and special tests
  • Defining “Major connective tissue diseases” to allow students to focus their learning on the common conditions such as Lupus, PMR, GCA
  • Changing the outcome regarding spondyloarthropathies to embrace seronegative inflammatory arthritis
  • Including Fibromylagia and other chronic pain syndromes, such as low back pain, CRPS
  • Duplication of some subject matter in other courses throughout the MBBS course, which can allow either vertical integration, or can become redundant
  • Osteoporosis (Geriatrics, pathology)
  • Back pain ( Foundations in Clinical Practice, MSK Medicine, GP)
  • Identification of curricula content considered core in the AMSEC framework, currently missing from the MBS MSK curriculum, that we also consider should be core curriculum
  • Fibromyalgia and other common pain syndromes such as complex regional pain syndrome.
  • Common foot and ankle conditions
  • Non traumatic back conditions
  • MSK/Red Flag conditions currently not taught (Cauda Equina Syndrome, Discitis, brachial plexus injuries)
  • Identification of concepts that are currently integrated into teaching, but could be developed as a focus through altering learning experiences and assignments to focus on these concepts
  • Presentation of MSK complaints, with a focus on history taking, diagnostic sieve, acute and chronic pain presentations, red flags and yellow flags and MSK mimickers
  • Alteration of in term assignments to address acute pain presentation, chronic pain presentation, trauma presentation and how this may influence the presentation, history taking and examination
  • Consideration of functional disability as a result of MSK complaints
  • Identification of course outcomes, that is appropriately delivered as a learning experience, but not examined
  • Common conditions of the hand
  • Connective tissue diseases
  • Compartment syndrome

Key issues that arose in the process of developing and implementing the audit tool include:

  • Ability to identify all areas where these topics are currently taught in the MBBS program. The MappEDout database was invaluable, but cannot provide enough granularity to truly assess whether we have captured all the content. This subsequently involved reverting back to the documentation available at each course level. It is also possible that some areas that have been marked as “not covered” are indeed covered in a small way, but that the investigators are unable to identify this information currently.
  • The ability of the individual auditors to agree on the interpretation of individual competencies and which learning outcomes match which competency. As Ellaway noted in her study, there is a certain amount of subjectivity within these exercises, and “it is possible, even likely, that different individuals would produce a different set of mappings” (p 634). As such, it is advisable to have a minimum of 2 auditors review materials in order to ensure accuracy of the information, and perhaps consider a longitudinal process involving a larger group that can validate the document.

7.0 Conclusion

This project has been valuable and timely, as the university develops a new curriculum for an MD course to begin in 2012. This has allowed us to objectively identify what areas we cover well, which areas need development, and which stakeholders within the MBBS course we should be working with to develop the new MD course with vertical and horizontal integration of the MSK course, but avoiding unnecessary repetition.

Additionally, we appreciate the opportunity to produce pilot data to assist us with further and larger grant proposals.

8.0 References

Brooks, P.M. (2006). The burden of musculoskeletal disease--a global perspective. Clin

Rheumatol., 25(6),778-81.

Chehade, M. & Bachorski, A. (2008). Development of the Australian Core Competencies in

Musculoskeletal basic and clinical science project – Phase I. MJA, 189(3): 162 – 165.

Ellaway, R., Evans, P., McKillop, J., Cameron, H., Morrison, J., McKenzie, H., et al. Cross-referencing the Scottish Doctor and Tomorrow’s Doctors learning outcome frameworks. Medical Teacher, 29, 630-635.

Harden, R.M. (2007). Outcome-based education – the ostrich, the peacock and the beaver.

Medical Teacher, 29, 666-671.

McColl, G. (2005). A survey of the musculoskeletal curricula of medical schools in the Asia-

Pacific region. APLAR Journal of Rheumatology, 8,84-89.

Mulhall, K. & Masterson, E. (2005). Relating undergraduate musculoskeletal medicine

curricula to the needs of modern practice. Ir J Med Sci, 174, 46-51.

Woolf, A.D., Walsh, N.E. & Akesson, K. (2004). Global core recommendations for a

musculoskeletal undergraduate curriculum. Annals of Rheumatic Disorders, 63, 517–524.