Before the 2012 curriculum update, the structure of UNECOM's curriculum was based on the published American Association of Colleges of Osteopathic Medicine (AACOM) Core Competencies.¹¹ Yet, assessment of student performance was primarily knowledge-based rather than competency-based, and the course content was parallel rather than integrated. In the 2012 curriculum update, UNECOM faculty created a 4-semester course, Osteopathic Clinical Skills (OCS), by merging 3 anatomical science courses and 10 clinical courses (Figure 1). The OCS course goal was to prepare students for their first day of clinical clerkships regardless of discipline. Before implementing the new curriculum, faculty sought input from clerkship sites, residents, and medical students to identify 8 descriptors of work, such as tasks and responsibilities for every medical student on the first day of clinical rotations. These shared expectations became the course outcomes of the OCS course, and the 4 tenets² of osteopathic medicine were integrated into these UNECOM-developed EPAs (herein referred to as OCS EPAs). The final goals of the OCS course were set primarily by expert consensus because published guidelines for EPAs were not available at the time of development. Our implementation of entrustable behaviors was initiated in 2012, which was before the AAMC's 2014 publication of the Core EPAs for Entering Residency (CEPAER).⁵ Our final list of student expectations for the OCS EPAs included 8 of the 13 CEPAERs identified by the AAMC, but the OCS EPAs focused on osteopathic medicine (Figure 2). The OCS course has since adopted the language of AAMC's EPAs and the 2016 AACOM Osteopathic Considerations of CEPAER, which incorporated OPP into the EPA wording.¹² 

For each OCS EPA, the relevant core competencies (including OPP) were identified as well as the associated knowledge/skills/attitudes (KSA) expected at the start of clerkships were described.¹² Faculty then used a developmental framework and benchmarked the KSA associated with each OCS EPA to intervals in the education process, coinciding with the end of each semester of the first 2 years of the preclinical curriculum. At these points in time, faculty would expect students to meet specific benchmarks appropriate for that stage in their educational development. Performance indicators and associated competencies changed with subsequent semesters. For example, language introduced in the first semester was initially categorized as medical knowledge competency; in the second semester, that language was associated with interpersonal communication; by the third semester, the language was also associated with patient care and practice-based learning and improvement. Ultimately, the Osteopathic Core Competencies of patient care, medical knowledge, practice-based learning and improvement, interpersonal and communication skills, professionalism, OPP, and systems-based practice were all integrated. 

To best assign performance indicators to competencies, faculty made several consensus assumptions regarding the expression of competencies throughout preclinical training. For example, faculty decided to interpret the practice-based learning and improvement competency as the students’ ability to learn from their mistakes. As such, faculty designed assessments to be cumulative and developed a competency-assurance process to assist students who need help meeting benchmarks. 

After determining agreed-upon course goals and OCS EPAs, we created a theme of patient-physician interaction to guide the OCS course content. The faculty recognized that a thoughtful approach was needed to optimize integration and resources. Content was organized to mimic the progression of patient-physician interaction and began with physician KSA that required less intimacy or invasiveness and progressed to KSA that required more. 

For example, physical examination and gross anatomy have unique challenges related to medical students’ neurocognitive development and effective resource management. The physical examination requires that students develop cognitive and visual-motor skills yet maintain an empathetic perspective while touching intimate areas of another person's body. While students might be prepared to start medical school with discussions of cardiovascular disease and cardiac auscultation, most entering medical students are not comfortable with placing their hand near a stranger's breast and leaving it there long enough to count a heartbeat or auscultate a murmur. Asking a new medical student to touch the anterior chest or the bare abdomen of a peer-student is intimidating. 

Gross anatomy is unique because after a cadaver is dissected, it is no longer available for observation within the context of the whole patient. Like the physical examination, starting gross anatomy with the thorax can be daunting. Therefore, the OCS course was designed to begin with the upper extremities and then progress throughout the first year to the lower extremities, chest, abdomen, pelvis/perineum, and the head and neck. This sequence helps students develop dissection skills in less personal areas before moving into more sophisticated dissections and body regions with a higher possibility of emotional triggers. 

We determined that the OCS course would begin with communication, general observation, and noninvasive touch. The course would progress to more personal verbal and nonverbal information gathering. This sequence would allow the introduction of cultural humility, permission, patient autonomy, and shared decision-making as the students progressed. This structure created a continuum through which students developed more sophisticated skills. As the students gained confidence in professional, empathetic touch, the curriculum's content and expectations progressed into more personal and intimate realms. This sequencing and theme provided a mechanism by which the cadaver laboratory experiences were integrated into the patient-physician interaction and established a foundation for using the structure-function tenet of osteopathic medicine in the course.² 

The UNECOM faculty felt that the dissection experience was critical for the development of empathetic osteopathic physicians. The cadaver was presented as the student's first patient. All learning and teaching during the OCS course considered the cadaver as a patient to reinforce the concept of beginning with the patient and orienting data gathering to the patient rather than anchored to a diagnosis or laboratory value. The dissection process provided the student with a concrete, kinesthetic understanding of disease progression and presentation. This kinesthetic knowledge was carried with the students into the clinical setting, where they interacted with and examined peers and patients with the same structures and functions observed in the cadaver laboratory. Additionally, diseases that were found in various dissections provided a framework for faculty to use a Socratic approach to engage students in deductive reasoning. To guide this reasoning with an osteopathic lens, faculty developed open-ended prompts that illustrated the tenets of osteopathic medicine (eg, “What would have been this patient's complaint?”, “How would this have affected the patient's daily activities?”, “How would you have examined this patient to find this problem?”). 

Using a backward design that considered our 8 OCS EPAs, faculty established performance benchmarks throughout years 1 and 2. Third-year students starting a clinical clerkship were expected to gather a patient's history, perform a physical examination, and provide an oral presentation. To prepare students, benchmarks in the final month of year 2 included the ability to begin a respectful conversation with a patient, determine the appropriate questions to ask in the proper manner, determine the correct physical examination to perform and obtain consent to do so, and communicate their findings and impressions to the patient, attending physician, and peers. Faculty then identified specific KSA that would lead the student to this end point and categorized these into 3 tiers. The lowest tier was considered critical/essential, the middle tier was important, and the highest tier was considered aspirational. To continue or pass the course, students must have met all expectations in the critical/essential tier and most expectations in the important tier. The specific content assigned to the tiers increased throughout the semesters, thus creating benchmarks. 

Faculty determined the competencies associated with the benchmarks and used a modified Bloom taxonomy to identify the specific performance indicators needed to assess student progress. For example, recognizing an obvious abnormal vital sign would be a critical/essential skill. For the patient care competency, the third-year student should be able to take an accurate blood pressure (BP) measurement, identify an abnormal BP, relate its significance to the chief complaint, and form a differential diagnosis. From this goal, faculty created a progression in benchmarks for each competency. For example, expectations for the patient care competency for a BP reading in the first semester included identifying and describing relevant anatomy and clinical tools as the critical/essential tier. They progressed to interpreting findings in the context of the chief complaint as the aspirational tier (Figure 3A). The progressions for the interpersonal and communication skills competency associated with taking BP were detailed similarly but had a different focus (Figure 3B). 

Expectations increased for the second semester. This process was used for each competency and OCS EPA. It was also used to determine whether potential course content was essential, important, or aspirational at a given time. By the end of year 2, each student must have demonstrated minimum ability in each competency through multiple interactions with peers, cadavers, standardized and community patients, and simulators. There were many formative and summative steps created to reach this level of competency. Detailed course objectives and rubrics (Figure 4) were created to guide the students toward acquiring the expected KSA. Embedded in each objective and rubric were the core competency components that are unique to osteopathic medicine. 

Once the course sequence was determined, the content was woven throughout multiple learning environments and directed by the overarching theme, guiding principles, and course goals. We designed 9 distinct, but integrated, learning activities/environments during the first preclinical year and 13 distinct activities in the second preclinical year (Figure 5 and Figure 6). All learning activities had feedforward and feedback processes and were sequenced to build on each other. For example, being in the cadaver laboratory did not preclude a conversation about osteopathic manipulative treatment applied to a dissected muscle, and a session covering gland histology was not exempt from a conversation of structure and function interplay. Framing content in this way allows students to contextualize the tenets of osteopathic medicine² in multiple formats, further nurturing the students’ osteopathic identity. 

The transition from separate courses to an integrated, 4-semester course took place over a single academic year. In the summer of 2012, the faculty agreed on integration, outcomes, overarching principles/themes, and laid out the course progression. The course was implemented in the fall of 2012, and minor changes and corrections were made as the year progressed. Planning, faculty preparation, and communication were the most critical factors needed for success, including administrative coordinators and information technology support. Most of the discipline-specific content did not change with integration. However, the timing and presentation of models did. 

After creating online modules, faculty time was spent more on interactive teaching, course sequencing, and standardization of assessments. The student-faculty contact included a flipped-classroom approach, clinical skills practice, knowledge application, peer-to-peer teaching, clinical experiences, and face-to-face formative student assessments. 

Curriculum, like a living entity, constantly changes. The degree of change that now occurs is similar to when we had separate courses. Faculty workload associated with this change was hard to assess, as we simultaneously increased the class size in the fall of 2012. The new curriculum necessitated the hiring of 1 additional staff member to assist with scheduling and organization. Considering the increased class size, the additional faculty workload ascribed to this transition was less than 1 full-time equivalent of adjunct faculty. 

Approximately every 6 weeks, the students had a summative assessment in which each competency domain was assessed through multiple methods: clinical demonstration, clinical performance, oral presentation, documentation, short answer, and multiple-choice questions. During the first year, more than 80% of the KSA assessed through each method represented content woven throughout all learning environments. For example, during the “Donor Practical,” an anatomical structure was tagged on a cadaver with the prompt “identify an orthopedic test used to assess this structure.” In the “Live-Anatomy Practical,” students palpated landmarks on a peer or assess somatic dysfunction while describing what they were doing and why. During “Imaging Assessments,” students identified the structures or obvious disease process on a radiologic image. In “Clinical Skills Assessments,” students performed osteopathic physical examinations, osteopathic manipulative treatment, and orthopedic tests on a peer while describing the relevant anatomy, expected findings, and clinical reasoning for performing an examination or treatment. There were questions about specific diseases that asked about the associated anatomy, diagnostics, and interventions on written assessments. In the second year, faculty designed summative assessments to better resemble patient encounters by using standardized patients with appropriate laboratory and imaging studies. 

Students were benchmarked to the defined expectation throughout the course rather than to their peers, and students were required to meet the expected criterion for every performance indicator. The expectations were clearly described and provided to the students each semester. If a student failed to meet expectations for a performance indicator, they received either “needs improvement” or “unsatisfactory.” and were required to successfully complete a competency assurance process (CAP). Each CAP was customized to the indicator in which a challenge was identified. This customization allowed faculty and students to identify and address specific learning challenges and opportunities for improvement. The CAP was not designed as remediation, but as a way to precisely identify underlying causes of deficiency and determine how it can be corrected. Depending on the assessment, the CAP took about 0.25 to 0.5 faculty-hours per student to develop and deliver. Variability existed in the number of CAPs per assessment cycle because of student growth, material complexity, and increased expectations. 

After 2 years of implementing this course, a survey was sent to clinical faculty to assess the students’ performance during the first month of clinical rotations based on core EPAs. Clinical faculty rated UNECOM student performance compared with previous third-year medical students from UNECOM and other colleges on 12 parameters using a Likert scale. Forty-seven assistant deans and clerkship faculty (94%) returned surveys. The survey results showed that 87% of respondents rated student performance as “meets expectations” to “exceeds expectations” in 8 of 12 core EPAs, and by more than 83% of respondents in the remaining 4 areas. While the authors recognize that this survey was not validated and lacks statistical significance, the results reassured us that this undertaking was worthwhile. 

During the first 2 years of curriculum implementation, students demonstrated discomfort with the nonnumerical scoring system. Much of this discomfort was because of the perceived lack of interexaminer reliability. An annual standardized assessment training course was developed for faculty that involved scoring video-recorded student performances, group discussion, and analysis for outliers. Additionally, students were encouraged to view their performance videos and challenge an assessment score. 

UNECOM faculty are now in our seventh year with this integrated, competency-based course using core EPA benchmarks. Overall, there has been support and enthusiasm from students regarding the integration and methods of delivery. Throughout this curricular change, UNECOM maintained a first-time pass rate on the Comprehensive Osteopathic Medical Licensing Examination of the United States above the national average,¹³ although this examination does not capture the breadth of tasks encompassed by the core EPAs. We do not have enough information to determine whether the course has led students to advocate for ACGME programs with Osteopathic Recognition. However, the third-year clinical clerkship faculty reported making adjustments to their curricula because the students were entering with more mature, entrustable skills. We continue to enhance objective metrics to track our curriculum's success and demonstrate that our approach successfully engrains the osteopathic identity. 

The goal in creating the OCS course was to create a strong foundation on which a third-year osteopathic medical student entering into a clinical clerkship can develop into an osteopathic physician. We set benchmarks along our 4-semester, integrated preclinical course using backward design and clinical expertise. Our approach to creating EPAs with an osteopathic focus emerged as a logical way to assess whether learners were on track to reach the course targets. An integrated curriculum embeds concepts of osteopathic medicine into all aspects of the course to mimic the “osteopathic lens” through which osteopathic physician's practice. Thus, all of the learning environments and assessments were created with the osteopathic identity in mind. We believe that this is a logical, worthwhile approach to achieve the desired outcomes of an osteopathic medical student. 

We acknowledge the extensive work of Danielle St. Onge, Mary Spang, Nancy Goulet, Liz Havu, and Hank Wheat in creating materials, which support the operation of the Osteopathic Clinical Skills course. We also thank Oran Suta for making the figures.