An Evaluation of the Effectiveness of a Medical School Musculoskeletal Curriculum at an Academic Medical Center

Background: Musculoskeletal disorders are commonly encountered by physicians and affected an estimated 126.6 million Americans in 2012. Nonetheless, musculoskeletal education has been inadequate in United States medical schools. Objective: To determine the musculoskeletal competency of third-year medical students. Method: A nationally validated cross-sectional, 25-question musculoskeletal competency exam was given to third-year medical students. A survey was given to secondand third-year medical students to assess students’ level of interest in musculoskeletal medicine and obtain their feedback regarding the curriculum. Results: The mean score of the competency exam was 69.0%. Forty-eight out of 107 students (44.9%) reached the minimum passing score of 70%. Free-response feedback from both classes featured themes of more hands-on learning, a longer clinical block, and more small-group learning sessions. Conclusions: Third-year medical students scored relatively well on the exam. Student feedback suggests the 2-week musculoskeletal block is useful and relevant to their future careers.

The high prevalence of musculoskeletal disorders underscores the need for adequate education in order to diagnose and treat patients in a timely manner (American Academy of Some progress has been made due to recommendations from AAMC and awareness of poor musculoskeletal test scores. Bernstein, Garcia, Guevara, & Mitchell (2011) published an update to their previous study and found that 83% of medical schools have a mandatory musculoskeletal curriculum, mainly consisting of preclinical coursework, compared with only 53% in 1998 (p. 895). Several individual schools have reformed their musculoskeletal curriculum with some success (Day et al., 2011;Saleh, Messner, Axtell, Harris, & Mahowald, 2004). However, performance in validated metrics continues to be poor and musculoskeletal education continues to be underrepresented in the clinical curriculums of medical schools nationwide (DiGiovanni, Sundem, Southgate, & Lambert, 2016). Recently Murphy, LaPorte, & Wadey (2014) highlighted the persistence and severity of the problem in a thorough review of the topic.
At our institution the musculoskeletal curriculum consisted of the following: (a) a required 2-week clinical block that combines orthopedics, rheumatology, radiology, and physical medicine and rehabilitation; (b) a 1.5-hour "Advanced Musculoskeletal Exam" session, where students practice hands-on exam skills in small groups guided by an attending, resident, or fourth-year medical student; (c) two 2-hour physical exam small group sessions that focused on teaching musculoskeletal exam of the upper extremity and one on the lower extremity; and (d) two simulated patient encounters in which students were assessed on how they conducted their shoulder and spine exams. The purpose of this study is to measure the competency of third-year medical students on musculoskeletal medicine knowledge and to examine attitudes and opinions that second-and third-year medical students have in regards to their musculoskeletal education.

Method
The program evaluation office, located at the School of Medicine of a public university in the western part of the United States, initiated an online competency exam that was administered to third-year medical students at the end of the school year. The purpose was to evaluate the effectiveness of the current curriculum prior to making changes. A 25-question exam nationally validated and published by Freedman and Bernstein (1998) was used in this study. The exam measures clinical knowledge of common musculoskeletal problems and was given to research participants according to the system described in a 2002 article. The authors surveyed directors of internal medicine residency programs across the country and used the responses to weigh each exam question and establish a minimum passing score (Freedman & Bernstein, 2002). This revised grading system is indicative of the expectations of training programs for generalist physicians and therefore provides results that are broadly applicable and relevant.
The exam was disseminated online through the Research Electronic Data Capture (REDCap) system during the end of the academic year. All students were required to complete the exam as part of the annual course feedback system. Twenty-five short answer questions were given to each student and then de-identified by the program evaluation office. The senior researcher of this study oversaw the grading of the exam and scores had no impact on student grades. The exam scores and responses were statistically analyzed using summary statistics, An Evaluation of the Effectiveness of a Medical School Musculoskeletal … 57 students' t-test, and ANOVA. No outside funding was provided for this program evaluation. IRB approval was obtained prior to submitting this work.
The free-response survey was also distributed through REDCap at the end of the academic year. Second-and third-year medical students were first asked to rate their confidence and attitudes in performing different musculoskeletal assessments on a 5-point Likert scale. Following his/her responses, each student then completed a short response question on how to improve musculoskeletal education. The first author (RC) aggregated all short answer responses and sorted them by frequency to identify common themes. The top ten themes were selected and compared to AAMC recommendations. The confidence and attitudes assessment was focused on identifying medical students' views before taking the musculoskeletal block (pre-course, second-year students) and after completing the block (post-course, third-year students). Secondyear students served as a control group compared to third-year medical students who had completed the musculoskeletal block and one year of clinical work.

Exam Results of Third-Year Students
The total number of exams completed by the students was 113. This analysis included 107 exams. Of the 113 exams, six assessments provided more than 50% blank responses and, therefore, were excluded from the analysis. The weighted mean score was 69.0%. Out of 107 students, 48 (44.9%) reached the minimum passing score of 70% set by internal medicine residency program directors (Freedman & Bernstein, 2002).
To compare these results with other studies, the unweighted scores of each exam were also calculated. The mean unweighted score was 67.5% ( Figure 1).

Exam Results Scores According to Perceived Importance of Musculoskeletal Medicine
All students who participated in the study were asked how important musculoskeletal education was to their future career, on a scale of 1-5, with one being no importance and five being critical importance. The exam scores distributed across the five scale groups were compared to determine if exam scores were correlated with perceived importance of musculoskeletal education. Scores were remarkably similar between groups (Figure 2), and ANOVA showed no statistical difference in scores between groups (p = 0.38).

Student Confidence Rating in Second-and Third-Year Medical Students
We compared the level of confidence of second-and third-year medical students in performing a musculoskeletal differential diagnosis exam (Figure 3). On a 5-point Likert scale (p = 0.000), third-year medical students scored significantly higher compared to second year students. Additionally, we compared the level of confidence among second-and third-year students in performing a musculoskeletal exam. Third-year students scored higher than secondyear students (p = 0.000). This suggests that completing the third-year musculoskeletal block improves the level of confidence in performing musculoskeletal (MSK) exams and differential diagnosis despite not having taken additional MSK electives available during the fourth-year of medical school.

Free-Response Feedback of Second-and Third-Year Students
Five strong themes emerged in the free-response feedback offered by second-and thirdyear students: (a) more hands-on practice with the musculoskeletal physical exam; (b) more time devoted to musculoskeletal clinical experience; (c) more small-group supervised practice; (d) a more interwoven (less fragmented) curriculum; and (e) more qualified teachers in practice sessions (such as residents and faculty in orthopedics and other musculoskeletal related specialties).

Discussion
It is imperative that medical schools prepare their students for diagnosing and treating musculoskeletal conditions. Musculoskeletal conditions are both common and costly. According to a 2016 report by the United States Bone and Joint Initiative, nearly half of all American adults are suffering from a musculoskeletal condition, and the cost of these conditions total 213 billion dollars annually (United States Bone and Joint Initiative, 2014). This is not only true for musculoskeletal specialties like orthopedic surgery, but also for family medicine, emergency medicine, pediatrics, and geriatrics. Although musculoskeletal education has been increasingly emphasized in medical school curriculums, national deficiencies continue to be widespread (Boyer, 2005;Murphy, LaPorte, & Wadey, 2014b).

Competency Exam
At our academic medical institution, third-year medical students achieved a 69.0% mean score with 44.9% of students passing the exam. These results compare favorably with other published performances on this metric (Al-Nammari et al., 2015;Comer, Liang, & Bishop, 2014;Day et al., 2007;Freedman & Bernstein, 1998;Freedman & Bernstein, 2002;Matzkin et al., 2005;Menon & Patro, 2009;Queally et al., 2008). Third-year medical students' exposure to musculoskeletal medicine at this point in their education largely consisted of a required 2-week musculoskeletal clinical block. Our institution is one of only 20 allopathic medical schools in the United States to have such a block (DiGiovanni et al., 2016). We hypothesize that the comparatively high scores on the competency exam is largely due to this block, which covers much of the basic material tested on the competency exam.

Exam Performance Between Students With Differing Levels of Interest in the Subject
Previous studies have correlated improved performance on the exam with taking musculoskeletal elective courses (Day et al., 2007;Matzkin et al., 2005). The reason for this www.hlrcjournal.com Open Access 60 A. Lalka, R. Caldwell, A. Black, and F. A. Scott difference is unclear, but it may be due to either the educational value of the elective courses themselves, or the increased level of interest in musculoskeletal medicine of the students who are enrolled in those courses. Students who have an increased level of interest may show more initiative and enthusiasm in the musculoskeletal components of their entire medical school curriculum. This may account for superior performance on the competence exam.
We administered the exam at the end of the third year when students have not had the opportunity to take elective courses. Students were asked how important they thought musculoskeletal education was to their future career as a proxy for their interest in musculoskeletal medicine, and we compared scores between groups. There were no significant differences in exam scores between groups. Students who perceive musculoskeletal medicine as critical to their future careers may take electives or pursue research in the subject, which may further improve their exam scores.

Student Confidence Before and After the Musculoskeletal Block
We evaluated the effect of the MSK block on students' level of confidence in performing both a musculoskeletal differential diagnosis and musculoskeletal exam. Third-year students obtained significantly higher confidence scores than second-year students. This observation was expected and used as a proxy since we can only survey cohorts at the end of the academic year.

Free-Response Feedback
The majority of the free-response feedback from both classes focused on a few major themes. Common themes identified included more hands-on learning opportunities with good supervision and feedback, a longer musculoskeletal clinical block, more workshop and/or smallgroup formats to demonstrate physical exam and diagnostic skills, less fragmentation in the curriculum, a teaching format focused on common regional complaints such as back pain, more emphasis formulating a differential diagnosis and performing a physical exam in didactic lectures, more training on imaging interpretation, and more expert instructors in fields such as orthopedics.
Students largely echoed the 2005 AAMC recommendations on how to improve musculoskeletal education (Association of American Medical Colleges, 2005). There are 19 of these recommendations, and ten of them captured 69% of the students' comments. Approximately 11% of the other comments requested that the musculoskeletal block be made longer. These findings suggest students recognize the disproportionately small portion of time spent on musculoskeletal education, in light of the massive cost and widespread prevalence of musculoskeletal conditions present in the population (American Academy of Orthopaedic Surgeons, 2016; Association of American Medical Colleges, 2005).

Limitations
There were several limitations to our study. The imperfect response rates on both the exam and survey may have introduced selection bias into our study population. Evaluating the musculoskeletal level of confidence, and interest levels between second-and third-year medical students was not ideal. The original program evaluation was cross-sectional in nature and we were not able to follow a cohort for this assessment. In the future, a comparison between the same class at the end of their second and third year would be a better assessment of the effect of the musculoskeletal block on their confidence and interest levels. Comparison of exam performances on many papers is subjective because of the different populations tested (many included residents of different specialties, some included attending physicians) and a lack of An Evaluation of the Effectiveness of a Medical School Musculoskeletal … 61 standardization of weighted scores. The most consistent measure given was the mean unweighted score. Four of the six previous studies utilized this score which was used for comparison in this study (Al-Nammari et al., 2015;Freedman & Bernstein, 2002;Matzkin et al., 2005;Menon & Patro, 2009).

Conclusion
Third-year medical students at an academic medical institution did fairly well on the competency exam compared with other studies using the Freedman and Bernstein exam. Exam results and free-response feedback suggest that the dedicated 2-week clinical musculoskeletal block is useful and effective. There is still much room for improvement in musculoskeletal education at our institution. It is far from adequate given the fact that only 44.9% of third-year students reached a passing score on a competency exam. Furthermore, the unweighted passing score of 73.1% was set by internal medicine residency program directors across the country as an indicator of expectations of an incoming resident. Therefore, this cutoff is not arbitrary and is relevant to a large portion of students who will enter into a generalist-type residency program (Freedman & Bernstein, 2002).
As we continue to revise and improve our musculoskeletal curriculum, we will aim for more hands-on practice. Future improvements may include a more continuous and interwoven curriculum and more supervised practice with experienced practitioners, in accordance with both AAMC recommendations and both second-and third-year students' feedback. This data serves as a baseline for program evaluation of the musculoskeletal curriculum at a major academic medical center.