Abstract
Context:
As graduate medical education evolves under the single accreditation system, osteopathic residency programs and consortia strive for sustainable ways to achieve and support the Osteopathic Recognition (OR) designation.
Objective:
To determine whether differences existed in perceived importance of OR from 3 cohorts of osteopathic stakeholders: students, residents, and faculty.
Methods:
A nonexperimental quantitative cross-sectional online survey was administered during February and March 2016 to osteopathic medical students at Michigan State University College of Osteopathic Medicine and residents and faculty from the affiliated Statewide Campus System. After examining final working dataset patterns, a series of Kruskal-Wallis tests were conducted to identify statistically significant differences in perceived OR importance response categories across sample subgroups, including program specialty and primary vs non–primary care specialty.
Results:
The final analytic sample comprised 278 osteopathic medical students, 359 residents, and 94 faculty members. Of 728 respondents, 497 (67.9%) indicated that OR was “somewhat important,” “important,” or “very important.” The overall perceived importance category patterns varied significantly across students, residents, and faculty cohort respondents (
, P<.001) and program specialty (
, P<.001), as well as between primary care and non–primary care residents and faculty (
, P<.001).
Conclusion:
Based on these initial results, OR is generally valued across osteopathic stakeholder groups, but significant differences may exist between different types of students, residents, and faculty. Pre- and postgraduate educational support structures designed to reduce barriers to OR implementation may help to sustain osteopathic principles and practice in the single accreditation system.
On February 26, 2014, the Accreditation Council for Graduate Medical Education (ACGME), the American Osteopathic Association (AOA), and the American Association of Colleges of Osteopathic Medicine (AACOM) announced their agreement to pursue a single accreditation system for graduate medical education (GME) in the United States.
1 Programs accredited by the AOA must gain pre-accreditation status by June 30, 2020—the date the AOA will cease residency accreditation activities.
2
Soon after the announcement of the single accreditation system, osteopathic physicians (DOs) representing the AOA joined with staff from the ACGME to form the Osteopathic Principles Committee. This committee developed the requirements for Osteopathic Recognition (OR) and milestones that went into effect July 1, 2015.
3,4 As defined by the ACGME, “recognition” is an additional acknowledgment, supplemental to accreditation, for identified elements or categories of a program. Osteopathic Recognition is overseen by the Osteopathic Recognition Committee, which is made up of volunteer experts and a representative of the public.
5
The ACGME has established that all residencies must achieve ACGME pre-accreditation or accreditation status before they apply for OR; however, OR applications will not be considered until they achieve initial accreditation.
6 The majority of AOA-accredited residency programs have been directing time and resources to meet ACGME program requirements for initial accreditation.
2,5 Although some DO educators may view these additional OR requirements as an opportunity to maintain the elements of osteopathic distinctiveness, some may question whether the additional administrative task is worth the effort.
In 2015, AACOM published a survey regarding student interest in OR.
7 This survey was directed toward third-year students at all colleges of osteopathic medicine, with 666 of 947 respondents (70.55%) indicating that an ACGME-accredited program with OR would be more appealing than an ACGME-accredited program without OR.
7 In preparation for the present study, we reviewed the existing literature concerning studies of the perceived importance of OR. Although a number of survey-based studies have been published regarding attitudes or beliefs toward osteopathic manipulative treatment (OMT),
8-16 we found few systematic investigations of the perceived importance of OR by key stakeholder groups such as osteopathic medical students, residents, and faculty.
17-19
As of July 1, 2015, Statewide Campus System (SCS), the Osteopathic Postgraduate Training Institution of the Michigan State University College Osteopathic Medicine (MSUCOM), served 1932 residents training in more than 200 AOA-accredited residency programs. In January 2016, the Osteopathic Principles and Practice Committee of the SCS assembled the Osteopathic Recognition Task Force (ORTF) to review the specific requirements of OR and make recommendations for implementation. The ORTF included 8 residents and 12 faculty members (including K.H. and J.R.) from diverse specialties including family medicine, internal medicine, osteopathic neuromusculoskeletal medicine, psychiatry, and physical medicine and rehabilitation.
The ORTF designed a survey for osteopathic medical students, residents, and faculty to examine (1) the perceived importance of having OR as an added option for respondents’ current/prospective specialty residency program; (2) what OR-related training opportunities stakeholders would want included during residency training; and (3) what primary barriers faculty members perceived to implementing OR for their respective specialty.
We conducted this nonexperimental quantitative exploratory study to determine whether differences existed in the perceived importance of OR from 3 cohorts of osteopathic stakeholders. We also investigated faculty members’ perceived barriers to OR but did not evaluate OR-related training opportunities. We believe the findings regarding OR interest and perceived barriers to OR could help educators develop an educational support structure for residency programs seeking OR. The authors’ primary null hypotheses were that they would find no significant differences in perceived OR importance scores across (1) osteopathic stakeholder groups or (2) residents and faculty members across different types of residency programs.
After receiving institutional review board approval, the ORTF cross-sectional survey, concerning various aspects of perceived OR importance, was e-mailed to 3 osteopathic cohorts of the SCS at MSUCOM (ie, medical students and residents and clinical faculty in SCS-sponsored residency programs) during February and March 2016. The e-mail, which contained a link to the online survey, was sent to 2937 stakeholders: 1281 MSUCOM students (first- through fourth-year students) and 1075 osteopathic residents and 581 faculty members (osteopathic and allopathic) in 47 member hospitals with more than 200 SCS residency programs. Respondents had to provide informed consent online before they were able to complete the survey. The survey link was active for 1 month, with reminder e-mails being sent to each stakeholder group twice during that timeframe. This instrument was designed for internal strategic planning and was therefore not validated. However, the survey was developed using questions from the previously disseminated AACOM survey
7 as well as additional items created by the ORTF that were vetted by content experts to establish initial construct validity.
Surveys contained 17 to 19 primary and sub-item questions, with items related to perceived importance of OR, using a standard 5-point, Likert-type ordinal scale, ranging from 0 (“not important”) to 4 (“very important”). Student surveys included 17 items, resident surveys included 18 items, and faculty surveys included 19 items. Survey items included the respondent's role in undergraduate or graduate medical education (ie, medical student, resident, or faculty) and questions related to perceived importance of OR. Respondents identifying as residents or faculty members were asked to provide their program specialty. Respondents identifying as faculty members (eg, program directors, directors of medical education, faculty/adjunct faculty) were asked to complete an additional item relating to perceived barriers to implementing and achieving OR (the ORTF concluded that osteopathic medical students and residents would not have the necessary administrative experience to provide relevant responses). We used the online tool SurveyMonkey, which allowed for anonymous survey submissions and extraction of raw response data.
Residents and faculty members were stratified into 2 program subcategories for sample subgroup analyses: primary care (eg, family medicine, internal medicine, obstetrics-gynecology, and pediatrics) and non–primary care (including “other specialty” and missing data). After generating a series of descriptive statistics and testing the dataset distributional patterns, a series of appropriate inferential multifactor statistical procedures were used to examine overall OR importance differences across cohorts. Additional procedures to test for differences in perceived barrier item scores within the smaller faculty cohort (n=90) with smaller cell frequencies were planned.
After creation of a working dataset, the distributional response patterns were examined using descriptive statistics, cross-tabulation charts, and graphics to guide selection of analytic procedures. Using the SPSS statistical software (version 22; IBM), data were first tested for overall sample normal or non-normal (ie, nonparametric) distributional patterns and homogeneity of variance across sample subgroups. Based on these findings, we used a series of separate (ie, 1 respondent characteristic at a time) nonparametric rank-based Kruskal-Wallis tests
20 to examine for differences across ordinal OR importance response categories from sample subgroups observing a 2-tailed α level of .05.
Faculty respondents were asked to complete an additional item regarding the 9 potential perceived barriers or challenges to implementing OR in their respective GME settings. Owing to smaller anticipated faculty cohort cell frequencies (ie, fewer survey scale responses from certain types of faculty), a series of Fisher exact tests
21 were used to examine faculty for subgroup differences observing a 2-tailed α level of .05.
The final working dataset had more than 99% complete data. The distributional response patterns were first examined to guide later selection of analytic procedures. Data were first tested for overall sample normality and homogeneity of variance across sample subgroups. It was clear that the distribution of OR importance responses across the sample cohorts were not normally distributed (ie, nonparametric) (Shapiro-Wilk test statistic = 0.77,
P<.001). The variability and shape of score category responses within subgroup categories (eg, type of GME role, program specialty) was, however, shown to be quite proportionately similar in distributional graphs. This distribution enabled the authors to compare overall ranked OR importance subgroup responses. Post-hoc power calculations later indicated that the final sample size of 728 respondents stratified into 3 GME role groups had afforded the authors with a high (0.99) level of 2-tailed statistical power to generally detect meaningful OR cohort response differences. This calculation was based on an overall moderate effect size
21 (f=0.25) and an α level of .05 (
F2,725=0.24). Further stratifying medical student and resident respondents into 4 discrete subgroups yielded a moderate effect size (f=0.30).
A total of 733 people responded to the survey. Of the 731 respondents who identified their GME role, 278 were students, 359 were residents, and 94 were GME faculty members affiliated with the SCS (
Table 1). These respondent cohorts represented approximately 22% of students, 34% of residents, and 16% of all faculty members who had been e-mailed the survey link, with an overall response rate of 25%. Data from several respondents were omitted in the remainder of the analyses because of missing data. Therefore, the total analytic sample was 728.
In overview, 497 respondents (68.2%) in the total analytic sample (N=728) indicated that OR was perceived as somewhat important (score of 2), important (score of 3), and very important (score of 4) on the 5-point scale for their respective medical school/residency program training or faculty positions. On this scale, 224 student respondents (80.8%) selected between somewhat important and very important (
Table 2).
In terms of specific differences across program specialty, respondents from some residency program types (eg, family medicine, mean rank of 287.0) were significantly more likely to select higher OR importance responses than others (eg, emergency medicine, mean rank of 145.7). Similar results were obtained when comparing primary care with non–primary care residents and faculty (Kruskal-Wallis test:
,
P<.001) (
Table 3 and
Figure).
The ranking differences of perceived barriers to OR implementation across faculty respondents are shown in
Table 4. The 3 most frequently selected challenges were related to perceived administrative time; availability of trained faculty; and number of requirements related to OR.
To assess the overall degree of perceived challenges to OR implementation, a composite score was calculated from the 9 barriers selected and the degree of challenges with each potential barrier (0 to 4, where 4 represented the greatest challenge), for a maximum possible composite score of 36. The mean (SD) score across all program specialties was 19.8 (8.4). While there was statistically significant variation across program specialties (Fisher exact test:
,
P=.007), the difference in composite barriers score between primary care and non–primary care program specialties was not statistically significant (Fisher exact test:
,
P=.166).
These findings further document the interest of osteopathic medical students throughout medical school training in pursuing GME programs with OR, as initially published in 2015 by AACOM.
7 While the AACOM study targeted third-year medical students, the present study demonstrated student interest in both the preclerkship and clerkship years.
In this report, nearly half of osteopathic medical student respondents (127 [46%]) reported that OR was important or very important; less than 2% described OR as not important at all. These findings may provide incentive to program directors and hospital systems as they consider the benefits of seeking OR for their ACGME-accredited residency programs. Resident respondents in this survey reported perceived importance of OR below that of the student and faculty cohorts. While the survey did not seek to explore why differences in perceived importance existed, several potential explanations could be considered.
First, the residents surveyed are currently enrolled in AOA-accredited programs and may not perceive how OR would alter their curriculum. Second, residents may regard the components of OR as not relevant to their specialty. Third, it is possible that some residents find OR less important because they may complete their program before SAS implementation. However, clear differences emerged between residents and faculty members in primary care specialties and those in non–primary care specialties, with the former reporting a greater perceived importance of OR.
Faculty respondents to this survey identified a number of barriers to implementing OR. Two most commonly identified challenges were the administrative burden and the number of required elements for OR. These data may help inform strategies to facilitate curriculum design and assist faculty in operationalizing OR requirements. Another potential barrier identified by faculty respondents was lack of faculty members skilled in OMT. This issue has been described in published reports on strategies to integrate osteopathic training into ACGME-accredited residencies beginning in the early 1990s through 2015.
22-25 In 3 of the articles,
23,25,26 OMT integration led by osteopathic residents was shown to be a successful strategy within ACMGE-accredited residencies. These articles described specific operating procedures, educational goals, objectives, and core competencies for summative evaluations of the integration of osteopathic principles and practice (OPP) and OMT.
23,25,26
After careful review of the OR requirements and milestones, the SCS ORTF designed a conceptual framework that categorized these requirements into 3 overarching OPP competency domains. These were “Biomechanical Competencies” (including OMT), “Health Promotion Competencies,” and “Physician Wellbeing Competencies.” The ORTF then created a list of educational modules, workshops, and resources to support each of these competency components.
Expanding on approaches described in these articles, and incorporating experiences of implementing the 6 published SCS OPP/OMT Integration Workshops,
27 an ORTF SCS subcommittee is currently developing an expanded educational model similar to the American Heart Association's
Advanced Cardiac Life Support Instructor Manual.
28 Following this blended learning model of online didactic instruction with onsite skills practice, participants must be able to successfully complete all requirements to receive an OMT provider completion card. Providing this type of educational model would offer faculty unfamiliar with OMT the assurance that the resident with the OMT completion card met all established requirements of the OMT provider course. It could also provide faculty members who have not recently used OMT an opportunity to brush up on their skills.
The results of this study should be reviewed within the context of several clear limitations. The results were derived from a diverse cross-sectional convenience sample of Michigan osteopathic stakeholders across the continuum of osteopathic medical education. Although the earlier-described post hoc sample size calculations indicated that the study had adequate statistical power to detect meaningful overall stakeholder subgroup differences, we acknowledge that they may still have been underpowered to detect significant differences across certain smaller-sized subgroups (eg, smaller surgical subspecialty programs). Ideally, the OR importance scale would have also had a neutral response option.
In hindsight, the results of the project may have proven further enlightening with additional survey items (eg, respondent sex, faculty degree). The results obtained were certainly subject to self-selection biases in that only data from those respondents who wished to participate were included in analyses.
Based on these initial results, OR within ACGME-accredited programs is generally valued by sizable proportions of major osteopathic stakeholder groups, although significant differences exist between different types of osteopathic medical students, residents, and faculty. More than 67% of the total respondents and 81% of osteopathic medical students expressed that OR is somewhat important to very important for their overall medical school or residency program training. Pre- and postgraduate educational support structures designed to reduce barriers to OR implementation may help to create more sustainable OR for specialty residency programs. In the future, we plan to examine student cohort and resident specialty differences as well as stakeholder interest in the types of OR educational support modules in greater depth.