Curricular Analysis of Competency-Based Osteopathic Medical Education: Application of a Matrix for Quality Enhancement to a Standardized Patient Encounter Example

Michael D. Lockwood; Stacy Tucker-Potter; Neil J. Sargentini

doi:10.7556/jaoa.2009.109.9.486

Free

Medical Education | September 2009

Curricular Analysis of Competency-Based Osteopathic Medical Education: Application of a Matrix for Quality Enhancement to a Standardized Patient Encounter Example

Michael D. Lockwood, DO; Stacy Tucker-Potter, MA; Neil J. Sargentini, PhD

Author Notes

Article Information

The Journal of the American Osteopathic Association, September 2009, Vol. 109, 486-500. doi:10.7556/jaoa.2009.109.9.486

- Email
- Facebook
- Linkedin
- Twitter
- CiteULike
- Get Permissions
- Get Citation
  Citation
  
  Lockwood MD, Tucker-Potter S, Sargentini NJ. Curricular Analysis of Competency-Based Osteopathic Medical Education: Application of a Matrix for Quality Enhancement to a Standardized Patient Encounter Example. J Am Osteopath Assoc 2009;109(9):486–500. doi: 10.7556/jaoa.2009.109.9.486.
  
  Download citation file:
  - Ris (Zotero)
  - EndNote
  - BibTex
  - Medlars
  - ProCite
  - RefWorks
  - Reference Manager
  © 2019
  ×
- Alerts
  
  User Alerts
  
  You are adding an alert for:
  
  Curricular Analysis of Competency-Based Osteopathic Medical Education: Application of a Matrix for Quality Enhancement to a Standardized Patient Encounter Example
  
  You will receive an email whenever this article is corrected, updated, or cited in the literature. You can manage this and all other alerts in My Account
  
  The alert will be sent to:
  
  Confirm
  
  ×
  
  This feature is available to Subscribers Only
  Sign In or Subscribe ×

Abstract

Context: With the formal adoption of the seven core competencies, the American Osteopathic Association's Commission on Osteopathic College Accreditation instructed osteopathic medical educators to guide curricular development with these goals in mind. Tools to facilitate and monitor these purposes have been under development separately at each of the nation's colleges of osteopathic medicine.

Objective: To demonstrate the utility of a checklist-based curriculum assessment tool, the Matrix for Quality Enhancement, as developed at Kirksville (Mo) College of Osteopathic Medicine-A.T. Still University.

Methods: Application of the Matrix is illustrated using examples selected from our analysis of a set of 16 standardized patient encounters provided as part of a first-year basic science course in medical microbiology. Encounters were developed to improve student understanding of infectious disease entities while also providing a variety of clinical experiences. Feedback on professionalism and humanistic behaviors was also provided. A novel aspect of the Matrix is the inclusion of a component dealing with patient safety.

Application: Adding standardized patient encounters to the medical microbiology teaching program at Kirksville College of Osteopathic Medicine was an effective means of integrating educational experiences with the seven core competencies, the requirements of Comprehensive Osteopathic Medical Licensing Examination-USA Level 2-PE (Performance Evaluation), and patient safety issues.

Conclusion: The Matrix is a valuable tool for evaluating or developing curricular components that maintain osteopathic integrity while working toward standards for medical education specified by the commission.

To guide osteopathic medical educators in preparing tomorrow's physicians for the many challenges ahead, the American Osteopathic Association's Commission on Osteopathic College Accreditation has adopted the seven core competencies for the osteopathic medical profession (Figure 1).^1,2 Although developed primarily for postgraduate educational objectives,^1,2 this goal has been extended to the development of the undergraduate curriculum at our institution.

Figure 1.

With the formal adoption of the seven core competencies, the American Osteopathic Association's Commission on Osteopathic College Accreditation instructed osteopathic medical educators to guide curricular development with these goals in mind. Source: American Osteopathic Association. Core Competency Compliance Program (CCCP). OPTI Clearinghouse Web site. https://www.do-online.org/index.cfm?PageID=lcl_opticcp. Accessed August 17, 2009.

View Original | Slide (.ppt)

Concurrently, hospitals,^3,4 industry groups,⁵ federal agencies,⁶ and physician groups^7,8 have put forth resources to deal with issues of patient safety and preventable secondary medical problems.

We suggest that if osteopathic medical educators align their curricula with the seven core competencies and with the expectations specified by patient-advocacy groups—as well as existing standards and regulations²—they will be able to improve the probability that their students will pass medical board examinations and become excellent osteopathic physicians.

Osteopathic medical educators have unique challenges in tracking whether their program is fully invested in the osteopathic paradigm.^9,10 Specifically, do collaborations between basic and clinical science educators satisfy the spirit of the seven core competencies? Does each graduate exhibit the cognitive knowledge, psychomotor skill sets, professionalism, and level of proficiency with osteopathic manipulative skills to demonstrate excellence in osteopathic medicine?

The evaluation of a curricular program has traditionally been inferred from student ability to perform measurable tasks.^11-15 Such evaluations have more commonly addressed allopathic graduate-level training rather than focusing on undergraduate osteopathic medical education.^16-18 Given the broad scope of current objectives for undergraduate osteopathic medical education,² we suggest that it may be difficult for administrators to determine the adequacy of student preparation in the core competency areas through curriculum review. Therefore, we describe a Matrix for Quality Enhancement (or Matrix) developed at Kirksville (Mo) College of Osteopathic Medicine-A.T. Still University (KCOM) for use by osteopathic medical educators as a curriculum-assessment tool. This Matrix allows educators to analyze a curriculum—or individual components within it—to evaluate how well educational requirements and public and industry expectations are met.

By way of example, we applied the proposed model to a set of 16 standardized patient (SP) encounters developed at our institution. These encounters were intended to integrate the subject content of medical microbiology with clinical skills (ie, history taking, physical examination, differential diagnosis, patient care, and osteopathic manipulative medicine) in a performance evaluation (PE) format similar to that used in Comprehensive Osteopathic Medical Licensing Examination-USA (COMLEX-USA) Level 2-PE.

Materials and Methods

The Matrix for Quality Enhancement

Our curriculum evaluation tool, the Matrix, consists of five submatrices. Submatrix 1 is based on the seven core competencies.¹⁰ Submatrices 2 and 3 are based on Dimensions I (The Clinical Presentation) and II (The Physician Task) of COMLEX-USA, respectively.¹ Submatrix 4 is derived from the physician behavior expectations found in COMLEX-USA Level 2-PE.¹

Accreditation standards² require osteopathic medical students to pass COMLEX-USA Levels 1, 2-CE, and 2-PE before graduation. This standardized test is a series of three examinations and one clinical PE examination. Level 1 is taken at the end of the second year; Level 2-PE, during the third year; Level 2-CE, at the end of the third year; and Level 3, is taken during residency training.

Finally, Submatrix 5 is formulated to evaluate student physician behavior as relevant to current patient safety concerns.^19-25

As constructed, the Matrix functions as a checklist to assist administrators in evaluating SP encounters or other curricular components. Methodologic details of all five submatrices are further described in “Application” using illustrative examples from our analysis of medical microbiology SP encounters.

Medical Microbiology Course: SP Encounters

The Matrix was applied to a set of 16 encounters required as part of a 44-hour medical microbiology course offered at KCOM during the spring quarter of 2006 to 168 first-year medical students. The broad objective of the course is to provide a basic understanding of bacteriology, mycology, parasitology, and virology. The course—which contextualizes and integrates principles from KCOM medical microbiology, family medicine, and osteopathic theories and methods (OTM) coursework into realistic patient-care scenarios—is a prerequisite for a 53-hour course in infectious diseases occurring in the second year of our curriculum.

Two SP encounters involving clinical situations based on infectious diseases were incorporated for each student. The objective was to allow students to improve their understanding of disease symptoms while also gaining experience in patient care and the integration of osteopathic principles. Students received feedback on professionalism and humanistic behaviors.

Overall, the encounters described here consist of 16 outpatient scenarios based on one of the following viral diseases: acute gastroenteritis, acute respiratory disease, aseptic meningitis, common cold, Epstein-Barr viral mononucleosis, genital warts, hepatitis type A, herpes genitalis, herpes simplex pharyngitis, human immunodeficiency viremia, influenza, keratoconjunctivitis, laryngitis, rubella, shingles, and West Nile encephalitis. Under the supervision of clinical faculty in KCOM's Department of Family Medicine, Preventive Medicine, and Community Health, the director of the KCOM Performance Assessment Center (PAC) trained SPs to portray disease symptoms and provide patient information for the clinical condition described. In addition, advanced undergraduate OMM fellows trained SPs on body and spinal areas that should be palpated by students in the context of the chief complaint. Each encounter involved presentation of a chief complaint and associated symptoms consistent with the specified viral disease—a lecture unit students had just completed.

The entire exercise required 8 hours (two 4-hour sessions) of faculty time during 2 days. All SP encounters (40 min per student) took place at the KCOM PAC, which consists of eight patient examination rooms with audio, digital video capture, and real-time video monitoring capabilities. Cohorts of 16 students arrived at the PAC in 50-minute intervals.

A 15-minute introduction to the exercise was included in a preceding lecture, and other logistical information was provided to students by e-mail 1 week in advance of the exercise. On arrival at the PAC, students received a 5-minute orientation, which further outlined logistical issues and gave them an opportunity to pose last-minute questions.

The 16 students were randomly divided into groups of two to complete the exercise. In each group, one student was randomly designated as the “physician,” while the other served as an “observer” for the first SP encounter. The observer was introduced to the “patient,” but had no formal role during the encounter other than to observe; note-taking was optional. The student observer was responsible for providing feedback to the student physician at the conclusion of the encounter. Two SP encounters were arranged for each student pair in the same 40-minute interval so that each student could perform once as physician and once as observer.

Together, each pair read the patient datasheet, which was posted on the examination room door. The datasheet listed the SP's pseudonym, chief complaint, and any abnormal values for body temperature, blood pressure, heart rate, and respiration rate.

The student pair entered the examination room together. The student designated as physician performed a history and focused physical examination. The student physician received up to two condition-appropriate “findings” cards from the SP (eg, picture of a lesion, text describing palpatory findings).

The physical examination always involved determining blood pressure, heart rate, and respiration rate, but if abnormal values were provided in the data sheet, these took precedence. Determination of chest and heart sounds were optional for the designated physician, and findings cards were given to the student physician, if he or she listened for sounds and as called for in the patient scenario.

Eleven minutes into the encounter, participants were given an audio signal that indicated they had 2 minutes to close the encounter. Paired students left the examination room before or at the 13-minute signal. They then reentered the examination room for 2 to 3 minutes to receive oral feedback from the SP who came “out of role.”

Just as students were paired in this exercise, pairing was similarly instituted for the excercises, the SPs performing these cases, and the examination rooms where the cases were performed. With each pair of SPs, one SP would be performing his or her case and the other SP (in a separate room) would be watching the encounter via real-time video and using an SP checklist to score the student physician's performance. Each of the 16 SPs always scored the same case within one afternoon session. Scoring relied on the SP checklist, which addressed technical issues (eg, standard history questions, physical examinations) and student attributes (eg, professionalism, interpersonal skills, compassion). The performing SP for the case used a short list of topics as a reminder of what to address in the short oral feedback given to the student physician immediately postencounter. This short list was designed to ensure that feedback consistently addressed certain topics (eg, professionalism).

After this feedback was received, the student team proceeded to a waiting area where they remained for approximately 4 minutes before being called to begin the second encounter. During that time, each student observer was asked to provide oral feedback to his or her partner (ie, designated physician). The second SP encounter followed the same format, except the students' (and SPs') roles were reversed and the case details presented by each SP were different.

At the end of the 40-minute session, after both students in the pair performed in the role of physician, the entire 16-student cohort proceeded to a computer laboratory to type their SOAP (Subjective, Objective, Assessment, Plan) Notes. Afterward, they returned to the Medical Microbiology laboratory to process “patient samples” for culture and identification.

After the SP encounter and completion of SOAP Notes, each student received by e-mail a set of laboratory or clinical procedure results appropriate to the case. This information was to be used by the student to provide an addendum to the previously submitted SOAP Note, which focused on updating the assessment and plan. Given the additional information, each student was expected to confirm a diagnosis and identify an evidenced-based therapy.

▪ Student Evaluations—The initial, on-site evaluation of student performance in the encounter was based on the SP checklist and resulted in a grade of pass or remediate. When remediation was indicated by the SP checklist score, the encounter video file was reviewed and followed up by a clinical faculty evaluator. Remediation necessitated a one-on-one discussion between the student and clinical faculty evaluator prior to the next SP encounter in our program.

The SOAP Notes were graded by a Family Medicine faculty member assisting in the program. The addendum, which confirmed the diagnosis and identified an evidence-based treatment was graded by Microbiology faculty (N.J.S.). The osteopathic structural component was graded and remediated by the OMM fellows.

All video files, SP checklists, and graded SOAP Notes were archived in a computer network folder that was available to participating students and faculty during student enrollment at KCOM.

It was expected that the SOAP Note (graded by the clinical faculty evaluator) would demonstrate student ability to record the patient encounter in an accurate, clear, and complete fashion. It was expected that the addendum (graded by the Microbiology faculty evaluator [N.J.S.]) would demonstrate critical thinking skills in evaluating laboratory and procedure data and in differential diagnosis based on entities addressed in medical microbiology coursework. It was expected that during the physical examination the student would palpate appropriate body areas based on the patient's chief complaint in an attempt to identify confirmatory sympathetic findings.

Figure 2.

Potential palpation targets in the sympathetic nervous system and methods of addressing the chief complaint as used in medical microbiology coursework standardized patient (SP) encounters. The approximate vertical distance of anticipated tissue texture change is indicated in parentheses. The authors analyze a set of SP encounters to illustrate their application of the Matrix for Quality Enhancement as developed at Kirksville (Mo) College of Osteopathic Medicine-A.T. Still University. As described, SPs were trained to expect palpation of at least one of the nine target sites listed. The student was evaluated for proper oral identification of the target region (or organ) and palpating that site. The student was then given a “findings card” that stated positive findings appropriate to the case depicted by the SP (eg, muscle hypertonicity, warmth, local edema, increased sweat gland activity, evidence of somatic dysfunction). For all nine target sites, motion testing was required in all three planes: rotation, left and right; sidebending, left and right; and flexion-extension. Abbreviations: GI, gastrointestinal tract; HEENT, head, eyes, ear, nose and throat; L, lumbar vertebrae; T, thoracic vertebrae.

View Original | Slide (.ppt)

During the physical examination portion of the SP encounter, students were expected to assess the functional anatomy and pathophysiologic process resulting in an increased sympathetic nervous system input to the spinal cord, which reflexively affects the somatic paraspinal musculature and adnexal tissue and embryologically related structures. (By way of example, readers may wish to refer to chapters 6, 8, and 48 in Foundations for Osteopathic Medicine,^26-28 which explain sympathetic innervation as it applies to the thoracic region.)

In addition, students were expected to evaluate lymphatic involvement in the disease process. Students were required to orally communicate to the SP (and audio recorder) what sites (ie, organs, body regions, vertebrae) they planned to palpate during the examination. This oral communication was used to assess students' abilities to obtain appropriate sympathetic (Figure 2) or lymphatic (Figure 3) findings.

After palpation was complete, students received text cards describing palpatory findings consistent with the condition portrayed by the SP. This information was to be used by students as they began constructing SOAP Notes for the encounter.

Because some of the case presentations had multiple “significant” influences, multiple correct answers were accepted by the faculty evaluator (M.D.L.). Some items—especially identified biomechanical or psychosocial issues—were to be addressed orally by the student or in the SOAP Note. The “Plan” portion of the SOAP Note Form includes areas for patient education, additional testing, public health concerns, lifestyle modification, and spiritual dimensions.

Application

Submatrix 1 evaluates how the SP encounters fulfill the seven core competencies (Table 1). Each competency and its required elements are itemized to facilitate the evaluation process. For example, the first core competency, osteopathic philosophy and manipulative medicine, appears as follows:

Table 1

Application of Submatrix 1 From Matrix for Quality Enhancement to Medical Microbiology Curriculum: SP Encounter With Acute Gastroenteritis Diagnosis

Core Competency Components and Required Elements	Score (1-5)*	Comment
OSTEOPATHIC PHILOSOPHY AND OSTEOPATHIC MANIPULATIVE MEDICINE:
*Demonstrate and apply knowledge of accepted standards in OMT.*	2	*No OMT was applied or expected during the SP encounter though it was an expected element of the SOAP Note “Plan.”*
Train a skilled and competent osteopathic practitioner who remains dedicated to life-long learning.	4	Part of the exercise required students to access an NIH database to use DNA sequence data to identify the agent of infection.
*Train a skilled and competent osteopathic practitioner who remains dedicated to practice habits in osteopathic philosophy and manipulative medicine.*	5	*The physical examination aspect of the exercise required students to use their palpatory skills to search for sympathetic and lymphatic involvement.*
Demonstrate competency in the understanding and application of OMT.	1	No OMT was applied or expected during the SP encounter though it was an expected element of the SOAP Note “Plan.”
Integrate osteopathic concepts and OMT into the medical care provided to patients as appropriate.	2	*No OMT was applied or expected during the SP encounter though it was an expected element of the SOAP Note “Plan.”*
Advance the understanding and integration of osteopathic principles and philosophy into all clinical and patient care activities.	5	The physical examination aspect of the exercise required students to use their palpatory skills to search for sympathetic and lymphatic involvement.
MEDICAL KNOWLEDGE:
Demonstrate and apply knowledge of accepted standards of clinical medicine.	5	Students learned to use The Sanford Guide to Antimicrobial Therapy to determine appropriate treatment. Students wrote a medical record including Assessment and Plan portions of SOAP Note, which was graded with comments and returned.
*Remain current with new developments in medicine.*	4	**Student knowledge was derived from annually updated lectures, class handouts, and the most recent edition of The Sanford Guide to Antimicrobial Therapy.**
Encourage participation in life-long learning activities.	4	Part of the exercise required students to access an NIH database to use DNA sequence data to identify the agent of infection. The course also includes a general, open-book medical informatics exercise, which benefits students in writing their SOAP Notes.
*Encourage development of research.*	3	Part of the exercise required students to access an NIH database to use DNA sequence data to identify the agent of infection. The course also includes a general, open-book medical informatics exercise, which benefits students in writing their SOAP Notes.
Demonstrate competency in the understanding of clinical medicine to patient care.	4	The SP encounters required students to develop an assessment with a minimum of three diagnoses.
Demonstrate competency in the application of clinical medicine to patient care.	4	*The SP encounters required students to develop an assessment with a minimum of three diagnoses and associated plans to verify and treat.*
Aid knowledge and applications of the foundations of clinical and behavioral medicine.	4	Many of the SP encounters involved patients with behavioral issues (eg, alcohol, drugs, smoking, unsafe sexual practice) that students were expected to recognize and address through counseling.
PATIENT CARE:
Demonstrate the ability to effectively treat patients.	5	After writing their SOAP Notes, students were provided additional information from which they were to confirm diagnosis. They were then to submit an evidence-based therapy for the patient via an addendum to the SOAP Note.
*Demonstrate the ability to provide medical care that incorporates the osteopathic philosophy.*	5	The physical examination aspect of the exercise required students to use their palpatory skills to search for sympathetic and lymphatic involvement. Many of the SP encounters involved patients with behavioral issues (eg, alcohol, drugs, smoking, unsafe sexual practice) that students were expected to recognize and address through counseling.
Demonstrate the ability to provide patient empathy.	5	Students were evaluated for a list of items categorized as patient-centered skills, which are based in empathy for the patient.
*Demonstrate the ability to provide awareness of behavioral issues.*	5	*Many of the SP encounters involved patients with behavioral issues (eg, alcohol, drugs, smoking, unsafe sexual practice) that students were expected to recognize and address through counseling.*
Demonstrate the ability to provide the incorporation of preventive medicine.	4	As all of these SP encounters are based on infectious disease, students were evaluated on hand-washing behaviors and wearing gloves at appropriate times during the examination. Patient counseling was to include recognition of contagiousness.
*Demonstrate the ability to provide health promotion.*	5	Many of the SP encounters involved patients with behavioral issues (eg, alcohol, drugs, smoking, unsafe sexual practice) that students were expected to recognize and address through counseling. Patient counseling was to include recognition of contagiousness.
Gather accurate, essential information from all sources.	5	In the open-book phase of writing the SOAP Notes, students were invited to refer to lecture notes, class handouts, and textbooks, as well as online sources of information.
Develop medical interview skills.	5	Students were evaluated three ways; orally immediately postencounter by the SP and peer observer and then in writing by faculty evaluator via SP encounter checklist. Any identified problems required remediation (ie, assessment and discussion) with faculty evaluator.
Develop physical examination skills.	5	Students were evaluated three ways; orally immediately postencounter by the SP and peer observer and then in writing by faculty evaluator via SP encounter checklist. Any identified problems required remediation (ie, assessment and discussion) with faculty evaluator.
Develop medical record-keeping skills.	4	Students were required to produce a SOAP Note with one addendum.
Develop diagnostic/therapeutic plans and treatments.	5	*Students were required to produce a SOAP Note with one addendum. Diagnostic, therapeutic, and treatment plans were required.*
Validate competency in the performance of diagnosis.	4	The SOAP Note is graded on the logic of the diagnoses.
Validate competency in the performance of treatment.	4	*The SOAP Note is graded on the logic of the treatment plan.*
Validate competency in the performance of procedures.	2	No clinical procedures were performed. Some medical microbiology procedures were performed and graded.
Demonstrate ability to provide healthcare services consistent with osteopathic philosophy.	5	Many of the SP encounters involved patients with behavioral issues (eg, alcohol, drugs, smoking, unsafe sexual practice) that students were expected to recognize and address through counseling. Patient counseling was to include recognition of contagiousness.
Demonstrate ability to provide preventive medicine and health promotion services that are based on current scientific evidence.	4	Comments regarding preventive medicine and health promotion were to be based on class handouts and reference materials.
INTERPERSONAL AND COMMUNICATION SKILLS:
Develop and demonstrate interpersonal and communication skills that enable students to establish and maintain professional relationships with patients.	5	Multiple videos have been used to demonstrate or model professional and effective communication skills. Students were assessed and remediated on their communication skills in the SP encounters.
*Develop and demonstrate interpersonal and communication skills that enable students to establish and maintain professional relationships with patients' families.*	1	Not addressed
Develop and demonstrate interpersonal and communication skills that enable them to establish and maintain professional relationships with other members of patient's healthcare team.	1	Not addressed
Demonstrate effectiveness in developing appropriate doctor-patient relationships.	5	*Teaching/assessment/remediation of patient-centered care was central to SP encounters.*
Develop effective listening skills in professional interactions with patients.	5	Teaching/assessment/remediation of listening skills was part of the SP encounters.
Develop effective written communication skills in professional interactions with patients.	4	*Teaching/assessment/remediation of writing skills was central to SP encounters.*
Develop effective oral communication skills in professional interactions with patients.	5	Teaching/assessment/remediation of communication skills was central to SP encounters.
Develop effective oral communication skills with patients' families.	1	*Not addressed*
Develop effective oral communication skills with other health professionals.	4	Students were required to discussed their performance immediately postencounter with the SP and peer observer. Any identified problems required remediation (ie, assessment and discussion) with faculty evaluator.
PROFESSIONALISM:
*Enhance expectations to uphold the osteopathic oath in the conduct of professional activities.*	2	*Although the osteopathic oath is not specifically addressed in the SP encounter model, behaviors addressed in the oath are part of expected behaviors for this exercise.*
Promote advocacy of patient welfare.	1	Not addressed
*Promote adherence to ethical principles.*	4	*Ethics and integrity were discussed and assessed during the SP encounter.*
Promote collaboration with health professionals.	4	The peer observer in the SP encounter may collaborate with the student “physician” in recording data gathered during the interview. The observer is required to provide the “physician” with constructive feedback designed to improve performance in future encounters.
*Promote life-long learning.*	4	*Students were referred to select sources and asked to use computer searches to answer clinical questions.*
Promote sensitivity to a diverse patient population.	4	After each encounter, students received feedback from their SPs, who represented a variety of demographic groups (eg, age, gender, economic diversity).
*Promote avenues of the student's physical and mental health in order to effectively care for patients.*	1	*Not addressed*
Demonstrate respect for patients.	5	Students were assessed directly and indirectly in terms of their respect for the patient.
Demonstrate respect for patients' families.	1	*Not addressed*
Advocate for the primacy of patient welfare and autonomy.	1	Not addressed
Adhere to ethical principles in the practice of medicine.	4	*We discussed and assessed the ethics of accurate SOAP Note records.*
Demonstrate awareness and proper attention to issues of culture, religion, age, gender, sexual orientation, and mental and physical disabilities.	4	Many of the issues listed were embedded in the SP cases, sometimes with direct application to understanding disease etiology.
PRACTICE-BASED LEARNING AND IMPROVEMENT:
Demonstrate the ability to critically evaluate their methods of clinical practice.	1	Not addressed
*Demonstrate the ability to integrate evidence-based medicine into patient care.*	4	*Students were required to seek data from NIH databases for their SOAP Note addenda.*
Demonstrate the ability to show an understanding of research methods.	4	Students were required to seek data from NIH databases for their SOAP Note addenda.
*Demonstrate the ability to improve patient care practices.*	1	*Not addressed*
Treat patients in a manner consistent with the most up-to-date information on diagnostic and therapeutic effectiveness.	5	Students were encouraged to seek data from current source of therapies for infectious disease (eg, The Sanford Guide to Antimicrobial Therapy, online resources).
Perform self-evaluations of clinical practice patterns and practice-based improvement activities using a systematic methodology.	4	*Students were required to write a self-reflection piece that focused on their performance and included any issues identified during peer (observer) assessment.*
Understand research methods as applied to the practice of medicine.	4	Students were given instructions to use an NIH database for genomic information and assessed on their performance through the graded SOAP Note and addendum.
Understand medical informatics as applied to the practice of medicine.	5	*One of the foundation experiences in the course is a medical informatics exercise, which the student is required to master.*
Understand technology as applied to the practice of medicine.	4	Both the medical informatics exercise and the NIH database search were designed around clinical questions.
SYSTEMS-BASED PRACTICE:
Provide an understanding of healthcare delivery systems.	1	Not Adressed
*Provide an understanding of effective and qualitative patient care within the healthcare system.*	1	*Not Adressed*
Provide an understanding of cost-effective medicine.	1	Not Adressed
Promote understanding of national and local healthcare delivery systems and how they impact patient care.	1	*Not Adressed*
Promote understanding of national and local healthcare delivery systems and how they impact professional practice.	1	Not Adressed
Advocate for quality healthcare on behalf of patients.	1	*Not Adressed*
Advance and assist patients in their interactions with the complexities of the medical system.	1	Not Adressed

Abbreviations: NIH, National Institutes of Health (Bethesda, Md); OMT, osteopathic manipulative treatment; SOAP, Subjective, Objective, Assessment, Plan.

*Submatrix 1 assists in the evaluation of standardized patient (SP) encounters so that medical educators may determine the extent to which those exercises fulfill the seven core competencies.¹⁰ For evaluation purposes, the components of each competency and its required elements, as indicated in boldface text, are extracted and itemized to facilitate curricular assessment. Therefore, though seven core competencies were defined with a total of 18 required elements in the original model as intended for osteopathic postgraduate medical education,¹⁰ readers will find a total of 66 curriculum evaluation items, as modified for undergraduate training, in submatrix 1. Competencies were adjusted for these preclinical encounters and were deemed appropriate for novice osteopathic medical students. In addition, within each competency, we determined a target for this experience based on a modification of Taxonomy of Educational Objectives: The Classification of Educational Goals.²⁹ How well the core competencies and required elements were satisfied was rated on a scale of 1 to 5: 1, element not included (none); 2, element included to minimal degree (“sort of”); 3, element included to moderate degree (satisfactory), 4, element included to strong degree (very good); 5, element completely satisfied (excellent).

View Large

Table 1

Application of Submatrix 1 From Matrix for Quality Enhancement to Medical Microbiology Curriculum: SP Encounter With Acute Gastroenteritis Diagnosis

Core Competency Components and Required Elements	Score (1-5)*	Comment
OSTEOPATHIC PHILOSOPHY AND OSTEOPATHIC MANIPULATIVE MEDICINE:
*Demonstrate and apply knowledge of accepted standards in OMT.*	2	*No OMT was applied or expected during the SP encounter though it was an expected element of the SOAP Note “Plan.”*
Train a skilled and competent osteopathic practitioner who remains dedicated to life-long learning.	4	Part of the exercise required students to access an NIH database to use DNA sequence data to identify the agent of infection.
*Train a skilled and competent osteopathic practitioner who remains dedicated to practice habits in osteopathic philosophy and manipulative medicine.*	5	*The physical examination aspect of the exercise required students to use their palpatory skills to search for sympathetic and lymphatic involvement.*
Demonstrate competency in the understanding and application of OMT.	1	No OMT was applied or expected during the SP encounter though it was an expected element of the SOAP Note “Plan.”
Integrate osteopathic concepts and OMT into the medical care provided to patients as appropriate.	2	*No OMT was applied or expected during the SP encounter though it was an expected element of the SOAP Note “Plan.”*
Advance the understanding and integration of osteopathic principles and philosophy into all clinical and patient care activities.	5	The physical examination aspect of the exercise required students to use their palpatory skills to search for sympathetic and lymphatic involvement.
MEDICAL KNOWLEDGE:
Demonstrate and apply knowledge of accepted standards of clinical medicine.	5	Students learned to use The Sanford Guide to Antimicrobial Therapy to determine appropriate treatment. Students wrote a medical record including Assessment and Plan portions of SOAP Note, which was graded with comments and returned.
*Remain current with new developments in medicine.*	4	**Student knowledge was derived from annually updated lectures, class handouts, and the most recent edition of The Sanford Guide to Antimicrobial Therapy.**
Encourage participation in life-long learning activities.	4	Part of the exercise required students to access an NIH database to use DNA sequence data to identify the agent of infection. The course also includes a general, open-book medical informatics exercise, which benefits students in writing their SOAP Notes.
*Encourage development of research.*	3	Part of the exercise required students to access an NIH database to use DNA sequence data to identify the agent of infection. The course also includes a general, open-book medical informatics exercise, which benefits students in writing their SOAP Notes.
Demonstrate competency in the understanding of clinical medicine to patient care.	4	The SP encounters required students to develop an assessment with a minimum of three diagnoses.
Demonstrate competency in the application of clinical medicine to patient care.	4	*The SP encounters required students to develop an assessment with a minimum of three diagnoses and associated plans to verify and treat.*
Aid knowledge and applications of the foundations of clinical and behavioral medicine.	4	Many of the SP encounters involved patients with behavioral issues (eg, alcohol, drugs, smoking, unsafe sexual practice) that students were expected to recognize and address through counseling.
PATIENT CARE:
Demonstrate the ability to effectively treat patients.	5	After writing their SOAP Notes, students were provided additional information from which they were to confirm diagnosis. They were then to submit an evidence-based therapy for the patient via an addendum to the SOAP Note.
*Demonstrate the ability to provide medical care that incorporates the osteopathic philosophy.*	5	The physical examination aspect of the exercise required students to use their palpatory skills to search for sympathetic and lymphatic involvement. Many of the SP encounters involved patients with behavioral issues (eg, alcohol, drugs, smoking, unsafe sexual practice) that students were expected to recognize and address through counseling.
Demonstrate the ability to provide patient empathy.	5	Students were evaluated for a list of items categorized as patient-centered skills, which are based in empathy for the patient.
*Demonstrate the ability to provide awareness of behavioral issues.*	5	*Many of the SP encounters involved patients with behavioral issues (eg, alcohol, drugs, smoking, unsafe sexual practice) that students were expected to recognize and address through counseling.*
Demonstrate the ability to provide the incorporation of preventive medicine.	4	As all of these SP encounters are based on infectious disease, students were evaluated on hand-washing behaviors and wearing gloves at appropriate times during the examination. Patient counseling was to include recognition of contagiousness.
*Demonstrate the ability to provide health promotion.*	5	Many of the SP encounters involved patients with behavioral issues (eg, alcohol, drugs, smoking, unsafe sexual practice) that students were expected to recognize and address through counseling. Patient counseling was to include recognition of contagiousness.
Gather accurate, essential information from all sources.	5	In the open-book phase of writing the SOAP Notes, students were invited to refer to lecture notes, class handouts, and textbooks, as well as online sources of information.
Develop medical interview skills.	5	Students were evaluated three ways; orally immediately postencounter by the SP and peer observer and then in writing by faculty evaluator via SP encounter checklist. Any identified problems required remediation (ie, assessment and discussion) with faculty evaluator.
Develop physical examination skills.	5	Students were evaluated three ways; orally immediately postencounter by the SP and peer observer and then in writing by faculty evaluator via SP encounter checklist. Any identified problems required remediation (ie, assessment and discussion) with faculty evaluator.
Develop medical record-keeping skills.	4	Students were required to produce a SOAP Note with one addendum.
Develop diagnostic/therapeutic plans and treatments.	5	*Students were required to produce a SOAP Note with one addendum. Diagnostic, therapeutic, and treatment plans were required.*
Validate competency in the performance of diagnosis.	4	The SOAP Note is graded on the logic of the diagnoses.
Validate competency in the performance of treatment.	4	*The SOAP Note is graded on the logic of the treatment plan.*
Validate competency in the performance of procedures.	2	No clinical procedures were performed. Some medical microbiology procedures were performed and graded.
Demonstrate ability to provide healthcare services consistent with osteopathic philosophy.	5	Many of the SP encounters involved patients with behavioral issues (eg, alcohol, drugs, smoking, unsafe sexual practice) that students were expected to recognize and address through counseling. Patient counseling was to include recognition of contagiousness.
Demonstrate ability to provide preventive medicine and health promotion services that are based on current scientific evidence.	4	Comments regarding preventive medicine and health promotion were to be based on class handouts and reference materials.
INTERPERSONAL AND COMMUNICATION SKILLS:
Develop and demonstrate interpersonal and communication skills that enable students to establish and maintain professional relationships with patients.	5	Multiple videos have been used to demonstrate or model professional and effective communication skills. Students were assessed and remediated on their communication skills in the SP encounters.
*Develop and demonstrate interpersonal and communication skills that enable students to establish and maintain professional relationships with patients' families.*	1	Not addressed
Develop and demonstrate interpersonal and communication skills that enable them to establish and maintain professional relationships with other members of patient's healthcare team.	1	Not addressed
Demonstrate effectiveness in developing appropriate doctor-patient relationships.	5	*Teaching/assessment/remediation of patient-centered care was central to SP encounters.*
Develop effective listening skills in professional interactions with patients.	5	Teaching/assessment/remediation of listening skills was part of the SP encounters.
Develop effective written communication skills in professional interactions with patients.	4	*Teaching/assessment/remediation of writing skills was central to SP encounters.*
Develop effective oral communication skills in professional interactions with patients.	5	Teaching/assessment/remediation of communication skills was central to SP encounters.
Develop effective oral communication skills with patients' families.	1	*Not addressed*
Develop effective oral communication skills with other health professionals.	4	Students were required to discussed their performance immediately postencounter with the SP and peer observer. Any identified problems required remediation (ie, assessment and discussion) with faculty evaluator.
PROFESSIONALISM:
*Enhance expectations to uphold the osteopathic oath in the conduct of professional activities.*	2	*Although the osteopathic oath is not specifically addressed in the SP encounter model, behaviors addressed in the oath are part of expected behaviors for this exercise.*
Promote advocacy of patient welfare.	1	Not addressed
*Promote adherence to ethical principles.*	4	*Ethics and integrity were discussed and assessed during the SP encounter.*
Promote collaboration with health professionals.	4	The peer observer in the SP encounter may collaborate with the student “physician” in recording data gathered during the interview. The observer is required to provide the “physician” with constructive feedback designed to improve performance in future encounters.
*Promote life-long learning.*	4	*Students were referred to select sources and asked to use computer searches to answer clinical questions.*
Promote sensitivity to a diverse patient population.	4	After each encounter, students received feedback from their SPs, who represented a variety of demographic groups (eg, age, gender, economic diversity).
*Promote avenues of the student's physical and mental health in order to effectively care for patients.*	1	*Not addressed*
Demonstrate respect for patients.	5	Students were assessed directly and indirectly in terms of their respect for the patient.
Demonstrate respect for patients' families.	1	*Not addressed*
Advocate for the primacy of patient welfare and autonomy.	1	Not addressed
Adhere to ethical principles in the practice of medicine.	4	*We discussed and assessed the ethics of accurate SOAP Note records.*
Demonstrate awareness and proper attention to issues of culture, religion, age, gender, sexual orientation, and mental and physical disabilities.	4	Many of the issues listed were embedded in the SP cases, sometimes with direct application to understanding disease etiology.
PRACTICE-BASED LEARNING AND IMPROVEMENT:
Demonstrate the ability to critically evaluate their methods of clinical practice.	1	Not addressed
*Demonstrate the ability to integrate evidence-based medicine into patient care.*	4	*Students were required to seek data from NIH databases for their SOAP Note addenda.*
Demonstrate the ability to show an understanding of research methods.	4	Students were required to seek data from NIH databases for their SOAP Note addenda.
*Demonstrate the ability to improve patient care practices.*	1	*Not addressed*
Treat patients in a manner consistent with the most up-to-date information on diagnostic and therapeutic effectiveness.	5	Students were encouraged to seek data from current source of therapies for infectious disease (eg, The Sanford Guide to Antimicrobial Therapy, online resources).
Perform self-evaluations of clinical practice patterns and practice-based improvement activities using a systematic methodology.	4	*Students were required to write a self-reflection piece that focused on their performance and included any issues identified during peer (observer) assessment.*
Understand research methods as applied to the practice of medicine.	4	Students were given instructions to use an NIH database for genomic information and assessed on their performance through the graded SOAP Note and addendum.
Understand medical informatics as applied to the practice of medicine.	5	*One of the foundation experiences in the course is a medical informatics exercise, which the student is required to master.*
Understand technology as applied to the practice of medicine.	4	Both the medical informatics exercise and the NIH database search were designed around clinical questions.
SYSTEMS-BASED PRACTICE:
Provide an understanding of healthcare delivery systems.	1	Not Adressed
*Provide an understanding of effective and qualitative patient care within the healthcare system.*	1	*Not Adressed*
Provide an understanding of cost-effective medicine.	1	Not Adressed
Promote understanding of national and local healthcare delivery systems and how they impact patient care.	1	*Not Adressed*
Promote understanding of national and local healthcare delivery systems and how they impact professional practice.	1	Not Adressed
Advocate for quality healthcare on behalf of patients.	1	*Not Adressed*
Advance and assist patients in their interactions with the complexities of the medical system.	1	Not Adressed

Abbreviations: NIH, National Institutes of Health (Bethesda, Md); OMT, osteopathic manipulative treatment; SOAP, Subjective, Objective, Assessment, Plan.

[Trainees] are expected to demonstrate and apply knowledge of accepted standards in OMT [osteopathic manipulative treatment] appropriate to their specialty. The educational goal is to train a skilled and competent osteopathic practitioner who remains dedicated to life-long learning and to practice habits in osteopathic philosophy and manipulative medicine.¹⁰

Figure 3.

Potential palpation targets in the lymphatic immune system and methods of addressing the chief complaint as used in medical microbiology coursework standardized patient (SP) encounters. The authors analyze a set of SP encounters to illustrate their application of the Matrix for Quality Enhancement as developed at Kirksville (Mo) College of Osteopathic Medicine-A.T. Still University. As described, SPs were trained to expect palpation of at least one of the nine target sites listed for enlarged lymph nodes, lymphatic congestion, or both. The student was evaluated for proper oral identification of the target region (or organ) and palpating that site. The student was then given a “findings card” that stated positive findings appropriate to the case depicted by the SP (eg, splenomegaly). For the nine target sites, identification of lymphadenopathy, tissue edema, altered tissue characteristics, or organ (spleen) enlargement was required. Abbreviations: C, cervical vertebrae; HEENT, head, eyes, ear, nose and throat; L, lumbar vertebrae; T, thoracic vertebrae.

View Original | Slide (.ppt)

Therefore, to streamline the process of curricular evaluation for osteopathic medical educators, the design of the Matrix reformats this statement into three focused statements:

Demonstrate and apply knowledge of accepted standards in OMT.
Train a skilled and competent osteopathic practitioner who remains dedicated to life-long learning.
Train a skilled and competent osteopathic practitioner who remains dedicated to practice habits in osteopathic philosophy and manipulative medicine.

In addition, within each competency, we determined a target for this experience based on a modification of Taxonomy of Educational Objectives: The Classification of Educational Goals.²⁹ By using the modified Bloom's Taxonomy²⁹ in assessing the design and expectations of SP encounters, osteopathic medical educators can ensure that each experience is consistent with students' abilities and that we are preparing our trainees for the rigor of osteopathic medical practice by appropriately escalating expectations.

Accordingly, the expectations set for the present preclinical encounter were appropriate for novice osteopathic medical students based on case review by the clinical faculty evaluator, who is also involved in our physician skills development course for first- and second-year students. As students progress into postgraduate work, performance expectations for clinical encounters necessarily increase.

As an example of how to use submatrix 1, an evaluation of one SP encounter for acute gastroenteritis is provided (Table 1). Although scoring is a subjective endeavor, scoring deficiencies do not necessarily mean the exercise under evaluation has failed in its educational objectives. Instead, the evaluation is intended as an indication of how well the exercise would fulfill the expectations delineated in the osteopathic medical profession's seven core competencies.¹⁰ Thus, scoring must be considered in the context of the level of student education.

For example, among first-year medical microbiology students, the application of OMT was not an objective of the SP encounter addressing acute gastroenteritis. Instead, the learning objective was for the student to focus on osteopathic philosophy and palpatory diagnosis. In addition, time limitations precluded the use of OMT at this early phase of medical education. Thus, the exercise was designed to evaluate deficiencies only for core competency 1—in particular, the first subelement, “Demonstrate and apply knowledge of accepted standards in OMT,” and two required subelements, “Demonstrate competency in the understanding and application of OMT” and “Integrate osteopathic concepts and OMT into the medical care provided to patients as appropriate.”¹⁰

In similar fashion, we scored the other 15 SP encounters that were developed for the medical microbiology curriculum. Analysis of all 16 encounters as related to core competency 3, patient care, is provided in Table 2. As shown, all 16 medical microbiology SP encounters were the same in their coverage of this core competency.

Table 2

Application of Submatrix 1 From Matrix for Quality Enhancement to Medical Microbiology Curriculum: Core Competency 3, Patient Care, as Addressed in SP Encounters *

	Medical Care That Incorporates Osteopathic Philosophy (n=6)	Gather Accurate, Essential Information From All Sources (n=5)	Competent Performance of Diagnosis, Treatment, and Procedures (n=3)	Deliver Healthcare Services Consistent With Osteopathic Philosophy and EBM (n=2)	Elements Addressed (n=16)
SP Encounter	No. of Required Elements Addressed						Expected Score	Actual Score
Acute gastroenteritis	6	5	1	2	14	14
*Acute respiratory disease*	6	5	1	2	14	14
Aseptic meningitis	6	5	1	2	14	14
*Common cold*	6	5	1	2	14	14
Epstein-Barr viral mononucleosis	6	5	1	2	14	14
*Genital warts*	6	5	1	2	14	14
Hepatitis A	6	5	1	2	14	14
*Herpes genitalis*	6	5	1	2	14	14
Herpes simplex pharyngitis	6	5	1	2	14	14
*Human immunodeficiency viremia*	6	5	1	2	14	14
Influenza	6	5	1	2	14	14
*Keratoconjunctivitis*	6	5	1	2	14	14
Laryngitis	6	5	1	2	14	14
*Rubella*	6	5	1	2	14	14
Shingles	6	5	1	2	14	14
*West Nile encephalitis*	6	5	1	2	14	14

Abbreviation: EBM, evidence-based medicine.

*n indicates the total number of required core competency components and required elements¹⁰ as delineated in submatrix 1 of the Matrix for Quality Enhancement (Table 1). For example, the 16 entries listed in Table 1 under core competency 3, patient care, are grouped here into four categories. Thus, the first six entries in Table 1 are represented in the column labeled “Medical Care That Incorporates Osteopathic Philosophy (n=6).” Data represent the number of elements addressed in each standardized patient (SP) encounter case. Expected scores were determined during the design phase of the SP exercises; actual scores indicate whether the final product met design expectations. Expectations were matched to first-year osteopathic medical students in medical microbiology.

View Large

Table 2

Application of Submatrix 1 From Matrix for Quality Enhancement to Medical Microbiology Curriculum: Core Competency 3, Patient Care, as Addressed in SP Encounters *

	Medical Care That Incorporates Osteopathic Philosophy (n=6)	Gather Accurate, Essential Information From All Sources (n=5)	Competent Performance of Diagnosis, Treatment, and Procedures (n=3)	Deliver Healthcare Services Consistent With Osteopathic Philosophy and EBM (n=2)	Elements Addressed (n=16)
SP Encounter	No. of Required Elements Addressed						Expected Score	Actual Score
Acute gastroenteritis	6	5	1	2	14	14
*Acute respiratory disease*	6	5	1	2	14	14
Aseptic meningitis	6	5	1	2	14	14
*Common cold*	6	5	1	2	14	14
Epstein-Barr viral mononucleosis	6	5	1	2	14	14
*Genital warts*	6	5	1	2	14	14
Hepatitis A	6	5	1	2	14	14
*Herpes genitalis*	6	5	1	2	14	14
Herpes simplex pharyngitis	6	5	1	2	14	14
*Human immunodeficiency viremia*	6	5	1	2	14	14
Influenza	6	5	1	2	14	14
*Keratoconjunctivitis*	6	5	1	2	14	14
Laryngitis	6	5	1	2	14	14
*Rubella*	6	5	1	2	14	14
Shingles	6	5	1	2	14	14
*West Nile encephalitis*	6	5	1	2	14	14

Abbreviation: EBM, evidence-based medicine.

The expected scores noted in Table 2 were determined during the design phase of the SP exercises; data in the Actual Score column indicate whether the final product met the design expectations. Our curriculum evaluation resulted in expectations being met for all SP encounters with two exceptions, both of which were in the “Competent performance of diagnosis, treatment, and procedures” category (Table 2). As discussed above, the design of the SP encounters precluded first-year students from performing “treatments” and “procedures.” Therefore, these latter two elements were not addressed by the exercise; scoring for this category was methodologically limited by class objectives to 1 of 3.

Submatrix 2 is based on COMLEX-USA Dimension I – The Clinical Presentation and consists of nine categories (Figure 4). All sixteen SP cases used included a primary clinical presentation or complaint and at least one secondary clinical presentation. All of the clinical presentation categories used in COMLEX-USA Level 1 were represented in the set of SP encounters with either a primary or secondary presentation with the exception of the tissues and trauma category (Figure 4).

Primary diagnoses clustered in digestion and metabolism, sensory alterations, human sexuality and urination, and respiration and circulation. The most common secondary diagnosis was found in the asymptomatic and general symptoms category with thermoregulation being involved as a secondary diagnosis in the majority of the SP cases presented.

As previously noted, COMLEX-USA Level 1 also contains Dimension II – The Physician Task.¹ Submatrix 3 is based on Dimension II and is divided into six categories: Health Promotion and Disease Prevention, History and Physical, Diagnostic Technologies, Management, Scientific Understanding of Mechanisms, and Healthcare Delivery. For submatrix 3, we used a checklist format to determine which physician tasks were satisfied by the medical microbiology SP encounters (Figure 5). All 16 cases satisfied the majority of evaluation elements in submatrix 3; however, no cases satisfied expectations related to healthcare delivery. Given that this experience occurred within the first year of undergraduate medical school, students would not be prepared to address the advanced issues related to this topic, which includes practice management and medical jurisprudence. Instruction in this area necessarily comes later in the course of medical education.

Similarly, submatrix 4 is based on PE criteria drawn from COMLEX-USA Level 2-PE, which contains eight biomedical/biomechanical domains and three humanistic domains. Using submatrix 4 to analyze our SP encounters, all sixteen cases satisfied seven of the eight biomedical/biomechanical factors, but none addressed OMT (Figure 5). While the SP encounter exercise did require incorporation of OMT into the “Plan” section of the SOAP Note, there was no expected OMT performance component. However, osteopathic diagnostic methods and palpatory skills were inherent in the exercise, reinforcing the osteopathic paradigm. The humanistic domain involving patient-physician relationships, interpersonal skills, and professionalism was assessed by the SPs, who completed the SP checklist, and by the peer observer, who provided oral feedback to the student physician as previously described.

Figure 4.

Application of submatrix 2 of the Matrix for Quality Enhancement to medical microbiology standardized patient (SP) encounters. The authors analyze a set of SP encounters to illustrate their application of the Matrix for Quality Enhancement as developed at Kirksville (Mo) College of Osteopathic Medicine-A.T. Still University. Evaluation elements from Comprehensive Osteopathic Medical Licensing Examination-USA (COMLEX-USA) Dimension I – The Clinical Presentation, are presented here. All sixteen cases included a primary (♦) clinical presentation or complaint and at least one secondary (♦) clinical presentation. With the exception of Tissues and Trauma, all clinical presentation categories addressed in COMLEX-USA Level 1 were represented.

View Original | Slide (.ppt)

The SP checklist evaluations as well as the composition of primary and secondary diagnoses in our 16 SP encounters for Dimension I – The Clinical Presentation (Figure 4) can be compared to the overall composition of COMLEX-USA (Table 3) to determine how this set of preclinical SP experiences compares with the content of COMLEX-USA. While the percentages outlined by the developers of COMLEX-USA should by no means be interpreted as prescriptive, the information provides valuable insight to osteopathic medical educators into the best ways to shape preclinical undergraduate experiences to conform with the national expectations as formalized through this standardized test.

Table 3

Application of Matrix for Quality Enhancement to Medical Microbiology SP Encounters: COMLEX-USA Evaluation Elements vs Curriculum

Clinical Presentation Categories	COMLEX-USA Design Parameters,%*	Primary SP Diagnosis, %^†
Respiration and circulation	8-16	25
*Human sexuality and urination*	*3-8*	*12.5*
Digestion and metabolism	4-10	12.5
*Sensory alterations*	*28-38*	*37.5*
Asymptomatic and general symptoms	8-16	12.5
*Thermoregulation*	*2-6*	*NA^†*
Human development	3-8	NA^†
*Motor alterations*	*6-12*	NA
Tissues and trauma	8-16	NA

*Design parameters for Comprehensive Osteopathic Medical Licensing Examination-USA (COMLEX-USA) are comprised of those evaluated in Level 1, Level 2-CE (Cognitive Examination), and Level 3 of that standardized test.

†Two clinical presentation categories that were not addressed (NA) in primary diagnoses for the standardized patient (SP) encounters used in the present investigation were addressed in secondary diagnoses: 75% of SP encounters addressed thermoregulation; 6%, human development.

View Large

Table 3

Application of Matrix for Quality Enhancement to Medical Microbiology SP Encounters: COMLEX-USA Evaluation Elements vs Curriculum

Clinical Presentation Categories	COMLEX-USA Design Parameters,%*	Primary SP Diagnosis, %^†
Respiration and circulation	8-16	25
*Human sexuality and urination*	*3-8*	*12.5*
Digestion and metabolism	4-10	12.5
*Sensory alterations*	*28-38*	*37.5*
Asymptomatic and general symptoms	8-16	12.5
*Thermoregulation*	*2-6*	*NA^†*
Human development	3-8	NA^†
*Motor alterations*	*6-12*	NA
Tissues and trauma	8-16	NA

In addition to using the core competencies and COMLEX-USA to evaluate the thoroughness of preclinical undergraduate experiences, the Matrix addresses patient safety issues in submatrix 5. This submatrix, which represents a novel approach to curricular assessment, helps medical educators in the efficient evaluation of seven patient safety issues: Official “Do Not Use List”,¹⁹ universal barrier precautions,²⁰ Health Insurance Portability and Accountability Act requirements,³⁰ recommendations regarding the legibility of medical records,²² radiation precautions,²³ workplace biohazard precautions,²⁴ and hand-washing guidelines.²⁵ These categories were added to the instrument with the goal of promoting positive professional behaviors that are intended to enhance awareness of patient safety issues and common medical errors. All but two patient safety categories were addressed in our SP encounters: recommendations regarding the legibility of medical records and radiation precautions (Figure 5). Because all SP encounters were ambulatory in nature and because typed SOAP Notes were required at the time of the investigation, these two items were not expected for this exercise. (Subsequently, we required written rather than typed SOAP Notes to allow for assessment of legibility in accord with formal recommendations.²²) Improved awareness of patient care safety issues involved with improper medical record keeping was emphasized and the Official “Do Not Use List”¹⁹ was incorporated in the curricular design for the first time at our institution in this particular class. These elements were applied to all SP case presentations. In addition, we subsequently dropped the “wet lab” component of the SP exercise and now supply students with microbiology findings (ie, written text or images).

Figure 5.

Application of submatrices 3-5 of the Matrix for Quality Enhancement to medical microbiology curriculum. A checklist format was used for standardized patient (SP) encounters with acute gastroenteritis diagnosed. Abbreviations: COMLEX-USA, Comprehensive Osteopathic Medical Licensing Examination-USA; NA, not addressed; PE, Performance Examination; SOAP, Subjective, Objective, Assessment, Plan.

View Original | Slide (.ppt)

A simplified overall assessment of our set of SP encounters is presented in Table 4, which includes scores from all five submatrices. Maximal scores and percentages mean the exercise fulfilled the expectations associated with a given element. According to our scoring for Table 1, a score of 3 indicated that an element was clearly included; 4, included to a strong degree; 5, completely satisfied. In general, submaximal values may indicate that an exercise could have matched expectations better, though values are limited by the nature of the exercise—which is itself limited by the preparation level of the student. We suggest that osteopathic medical educators first determine which of the subelements should be addressed at each stage of student education. The Matrix can then be used to assess whether a curriculum or its components are meeting those expectations.

Table 4

Application of Matrix for Quality Enhancement to Medical Microbiology SP Encounters: Summary of Curriculum Assessment *

Evaluation Element	Submatrix	Addressed, %
Core Competency 2, Medical Knowledge	1	100
*Humanistic Domain*	4	*100*
Core Competency 3, Patient Care	1	93
*COMLEX-USA Dimension I — The Clinical Presentation*	2	89
Biomedical/Biomechanical Domain	4	88
*COMLEX-USA Dimension II — The Physician Task*	3	83
Core Competency 6, Practice-Based Learning and Improvement	1	78
*Core Competency 4, Interpersonal and Communication Skills*	1	67
Patient Safety Issues	5	57
*Core Competency 5, Professionalism*	1	55
Core Competency 1, Osteopathic Philosophy and Manipulative Medicine	1	50^†
*Core Competency 7, Systems-Based Practice^‡*	1	NA

Abbreviations: COMLEX-USA, Comprehensive Osteopathic Medical Licensing Examination-USA; NA, not addressed.

*A score of 3 or higher on the 5-point scoring system used for submatrix 1 in the present study signifies that each competency and required element listed for that item was addressed in the standardized patient (SP) encounter. For example, in the case of core competency 2, medical knowledge, all seven elements evaluated received a score of 3 or higher (Table 1).

†Several items in core competency 1, osteopathic philosophy and manipulative medicine, were not assessed because undergraduate students are not evaluated based on the delivery of osteopathic manipulative treatment. Therefore, an expected value (50%) is provided instead.

‡None of the seven items in core competency 7, systems-based practice, was addressed.

View Large

Table 4

Application of Matrix for Quality Enhancement to Medical Microbiology SP Encounters: Summary of Curriculum Assessment *

Evaluation Element	Submatrix	Addressed, %
Core Competency 2, Medical Knowledge	1	100
*Humanistic Domain*	4	*100*
Core Competency 3, Patient Care	1	93
*COMLEX-USA Dimension I — The Clinical Presentation*	2	89
Biomedical/Biomechanical Domain	4	88
*COMLEX-USA Dimension II — The Physician Task*	3	83
Core Competency 6, Practice-Based Learning and Improvement	1	78
*Core Competency 4, Interpersonal and Communication Skills*	1	67
Patient Safety Issues	5	57
*Core Competency 5, Professionalism*	1	55
Core Competency 1, Osteopathic Philosophy and Manipulative Medicine	1	50^†
*Core Competency 7, Systems-Based Practice^‡*	1	NA

Abbreviations: COMLEX-USA, Comprehensive Osteopathic Medical Licensing Examination-USA; NA, not addressed.

‡None of the seven items in core competency 7, systems-based practice, was addressed.

Comment

As previously noted, we applied a novel assessment tool, the Matrix, to an innovative KCOM program in medical microbiology to demonstrate the utility of this instrument for curriculum assessment.

While the Matrix is an efficient tool for osteopathic medical educators in evaluating or developing curricular components, it may be most valuable in assessing student behaviors during clinical encounters (eg, human patient simulators, SPs, real patients). In the present investigation, we applied the Matrix to evaluate a set of SP encounters comprised of 16 viral disease cases used as part of our course in medical microbiology.

Although each student in our exercise saw only two of the 16 cases, all students were encouraged to discuss their cases with their peers during the break between cases because no first-round cases were repeated in the second round. Thus, it is likely that all students received some degree of exposure to all 16 cases. We should reiterate that, prior to undertaking these SP cases, all students were provided access to lectures with the requisite medical knowledge for all cases, and each student was expected to be proficient with each of the randomly assigned cases. The cases allowed students to practice their skills in the context of different case scenarios. While different aspects of medical knowledge were important in each case, many aspects relevant to the core competencies were constant for all students regardless of the cases to which they were assigned.

Conclusion

The Matrix for Quality Enhancement can readily facilitate faculty and administrators in the thorough evaluation of existing curricula. Likewise, it can be applied during the design of new curricular components. The application of this new curriculum-assessment tool will assist osteopathic medical educators in maintaining the integrity of a truly osteopathic curriculum while also ensuring student achievement in national requirements for licensure as well as the highest patient-safety standards.

We believe that the Matrix may be especially valuable when evaluating or designing realistic clinical experiences for educational purposes, which are both, by nature, complex exercises.

A preliminary report on the Matrix for Quality Enhancement was delivered by Dr Lockwood at the 2007 American Association of Colleges of Osteopathic Medicine Annual Meeting on Friday, June 29, 2007, in Baltimore, Md. Dr Lockwood's slide show presentation, “A Matrix for Quality Enhancement Based Upon the Seven Core Competencies of the Osteopathic Profession,” is available on that association's Web site at: http://www.aacom.org/events/annualmtg/past/2007aacom/AACOM07Fri/35-Lockwood-Matrix.ppt.

We thank Julia E. McNabb, DO, MLS, Michelle R. Colen, MD, and Dena Higbee, MS, for their invaluable assistance in the design and operation of the standardized patient encounters involved in this investigation.

American Osteopathic Association. Core Competency Compliance Program (CCCP). OPTI Clearinghouse Web site. https://www.do-online.org/index.cfm?PageID=lcl_opticcp. Accessed August 17, 2009.

Commission on Osteopathic College Accreditation. Accreditation of Colleges of Osteopathic Medicine: COM Accreditation Standards and Procedures. Chicago, Ill: American Osteopathic Association; 2009. http://www.osteopathic.org/pdf/SB03-Standards%20of%20Accreditation%20July%202009.pdf. Accessed August 17, 2009.

American Hospital Association. Quality and patient safety. http://www.aha.org/aha_app/issues/Quality-and-Patient-Safety/index.jsp. Accessed August 17, 2009.

American Hospital Association. Highest Quality Care. http://www.aha.org/aha/content/2009/pdf/09-ip-quality.pdf. Accessed August 17, 2009.

The Leapfrog Group. http://www.leapfroggroup.org/. Accessed August 17, 2009.

Agency for Healthcare Research and Quality. PSNet: AHRQ Patient Safety Network Web site. http://www.psnet.ahrq.gov/index.aspx. Accessed August 17, 2009.

American College of Physicians. Patient safety: the other side of the quality equation [learning modules]. ACP – Internal Medicine Web site. http://www.acponline.org/running_practice/patient_care/safety/modules.htm. Accessed August 17, 2009.

American College of Surgeons. Patient safety principles for office-based surgery utilizing moderate sedation/analgesia, deep sedation/analgesia, or general anesthesia. http://www.facs.org/patientsafety/patientsafety.html. Accessed August 17, 2009.

Rogers FJ, D'Alonzo GE Jr, Glover JC, Korr IM, Osborn GG, Patterson MM, et al. Proposed tenets of osteopathic medicine and principles for patient care. J Am Osteopath Assoc. 2002;102:63-65. http://www.jaoa.org/cgi/reprint/102/2/63. Accessed August 17, 2009.

Gallagher H, Cummings M, Gilman D, McNerney J, Mogil C, Piccinini R, et al, for the AOA Core Competency Task Force. Report of the Core Competency Task Force. Chicago, Ill: American Osteopathic Association; 2003. http://www.com.msu.edu/scs/cc/docs/AOATaskForceReport.pdf. Accessed August 17, 2009.

Kassebaum DG. The measurement of outcomes in the assessment of educational program effectiveness. Acad Med. 1990;65:293-296.

Kassebaum DG, Eaglen RH. Shortcomings in the evaluation of students' clinical skills and behaviors in medical school. Acad Med. 1999;74:842-849.

Kassebaum DG, Eaglen RH, Cutler ER. The objectives of medical education: reflections in the accreditation looking glass. Acad Med. 1997;72:648-656.

Kassebaum DG, Eaglen RH, Cutler ER. The meaning and application of medical accreditation standards [review]. Acad Med. 1997;72:808-818.

Kassebaum DG, Cutler ER, Eaglen RH. The influence of accreditation on educational change in US medical schools. Acad Med. 1997;72:1127-1133.

Cubic BA, Gatewood EE. ACGME core competencies: helpful information for psychologists [published online ahead of print March 14, 2008]. J Clin Psychol Med Settings. 2008;15:28-39.

Heard JK, Allen RM, Clardy J. Assessing the needs of residency program directors to meet the ACGME general competencies. Acad Med. 2002;77:750 .

Frey K, Edwards F, Altman K, Spahr N, Gorman RS. The `Collaborative Care' curriculum: an educational model addressing key ACGME core competencies in primary care residency training. Med Educ. 2003;37:786-789.

The Joint Commission. Official “Do Not Use List.” May 2005. http://www.jointcommission.org/NR/rdonlyres/2329F8F5-6EC5-4E21-B932-54B2B7D53F00/0/dnu_list.pdf. Accessed August 17, 2009.

Centers for Disease Control and Prevention. Universal Precautions for Prevention of Transmission of HIV and Other Bloodborne Infections [fact sheet]. February 5, 1999. http://www.cdc.gov/ncidod/dhqp/bp_universal_precautions.html. Accessed August 17, 2009.

Patient Safety and Quality Improvement Act of 2005, 42 USC §299b-1 (2005). http://frwebgate.access.gpo.gov/cgi-bin/useftp.cgi?IPaddress=162.140.64.21&filename=publ041.109&directory=/diskb/wais/data/109_cong_public_laws. Accessed August 17, 2009.

Ettel DL, Wilson CM. Medication errors: focus on legibility. Patient Safety Quality Healthcare [serial online]. January/February 2006. http://www.psqh.com/janfeb06/mederrors.html. Accessed August 17, 2009.

Consumer-Patient Radiation Health and Safety Act of 1981, 42 USC §300a-6a (1981). http://www.govrecords.org/s-1198-eah-.html. Accessed August 17, 2009.

United States Department of Labor. Regulations (Standards-29 CFR). Bloodborne pathogens. Occupational Safety & Health Administration Web site. http://www.osha.gov/pls/oshaweb/owadisp.show_document?p_table=STANDARDS&p_id=10051. Accessed August 17, 2009.

Centers for Disease Control and Prevention. Hand hygiene guidelines [fact sheet]. http://www.cdc.gov/od/oc/media/pressrel/fs021025.htm. Accessed August 24, 2007.

Willard FH. Autonomic nervous system. In Ward RC, ed. Foundations for Osteopathic Medicine. 2nd ed. Baltimore, Md: Lippincott Williams & Wilkins; 1997:90-119.

Willard FH. Nociception, the neuroendocrine immune system, and osteopathic medicine. In Ward RC, ed. Foundations for Osteopathic Medicine. 2nd ed. Baltimore, Md: Lippincott Williams & Wilkins; 1997: 137-156.

Hruby RH. Thoracic region. In Ward RC, ed. Foundations for Osteopathic Medicine. 2nd ed. Baltimore, Md: Lippincott Williams & Wilkins; 1997:705-717.

Bloom BS. Taxonomy of Educational Objectives: The Classification of Educational Goals. New York, NY: Susan Fauer Company Inc; 1956. Handbook I: Cognitive Domain.

Health Insurance Portability and Accountability Act of 1996, 42 USC §1320d-1320d-8 (1996). http://www.hhs.gov/ocr/privacy/hipaa/administrative/statute/index.html. Accessed August 17, 2009.

Figure 1.

View Original | Slide (.ppt)

Figure 2.

View Original | Slide (.ppt)

Figure 3.

View Original | Slide (.ppt)

Figure 4.

View Original | Slide (.ppt)

Figure 5.

View Original | Slide (.ppt)

Table 1

Application of Submatrix 1 From Matrix for Quality Enhancement to Medical Microbiology Curriculum: SP Encounter With Acute Gastroenteritis Diagnosis

Core Competency Components and Required Elements	Score (1-5)*	Comment
OSTEOPATHIC PHILOSOPHY AND OSTEOPATHIC MANIPULATIVE MEDICINE:
*Demonstrate and apply knowledge of accepted standards in OMT.*	2	*No OMT was applied or expected during the SP encounter though it was an expected element of the SOAP Note “Plan.”*
Train a skilled and competent osteopathic practitioner who remains dedicated to life-long learning.	4	Part of the exercise required students to access an NIH database to use DNA sequence data to identify the agent of infection.
*Train a skilled and competent osteopathic practitioner who remains dedicated to practice habits in osteopathic philosophy and manipulative medicine.*	5	*The physical examination aspect of the exercise required students to use their palpatory skills to search for sympathetic and lymphatic involvement.*
Demonstrate competency in the understanding and application of OMT.	1	No OMT was applied or expected during the SP encounter though it was an expected element of the SOAP Note “Plan.”
Integrate osteopathic concepts and OMT into the medical care provided to patients as appropriate.	2	*No OMT was applied or expected during the SP encounter though it was an expected element of the SOAP Note “Plan.”*
Advance the understanding and integration of osteopathic principles and philosophy into all clinical and patient care activities.	5	The physical examination aspect of the exercise required students to use their palpatory skills to search for sympathetic and lymphatic involvement.
MEDICAL KNOWLEDGE:
Demonstrate and apply knowledge of accepted standards of clinical medicine.	5	Students learned to use The Sanford Guide to Antimicrobial Therapy to determine appropriate treatment. Students wrote a medical record including Assessment and Plan portions of SOAP Note, which was graded with comments and returned.
*Remain current with new developments in medicine.*	4	**Student knowledge was derived from annually updated lectures, class handouts, and the most recent edition of The Sanford Guide to Antimicrobial Therapy.**
Encourage participation in life-long learning activities.	4	Part of the exercise required students to access an NIH database to use DNA sequence data to identify the agent of infection. The course also includes a general, open-book medical informatics exercise, which benefits students in writing their SOAP Notes.
*Encourage development of research.*	3	Part of the exercise required students to access an NIH database to use DNA sequence data to identify the agent of infection. The course also includes a general, open-book medical informatics exercise, which benefits students in writing their SOAP Notes.
Demonstrate competency in the understanding of clinical medicine to patient care.	4	The SP encounters required students to develop an assessment with a minimum of three diagnoses.
Demonstrate competency in the application of clinical medicine to patient care.	4	*The SP encounters required students to develop an assessment with a minimum of three diagnoses and associated plans to verify and treat.*
Aid knowledge and applications of the foundations of clinical and behavioral medicine.	4	Many of the SP encounters involved patients with behavioral issues (eg, alcohol, drugs, smoking, unsafe sexual practice) that students were expected to recognize and address through counseling.
PATIENT CARE:
Demonstrate the ability to effectively treat patients.	5	After writing their SOAP Notes, students were provided additional information from which they were to confirm diagnosis. They were then to submit an evidence-based therapy for the patient via an addendum to the SOAP Note.
*Demonstrate the ability to provide medical care that incorporates the osteopathic philosophy.*	5	The physical examination aspect of the exercise required students to use their palpatory skills to search for sympathetic and lymphatic involvement. Many of the SP encounters involved patients with behavioral issues (eg, alcohol, drugs, smoking, unsafe sexual practice) that students were expected to recognize and address through counseling.
Demonstrate the ability to provide patient empathy.	5	Students were evaluated for a list of items categorized as patient-centered skills, which are based in empathy for the patient.
*Demonstrate the ability to provide awareness of behavioral issues.*	5	*Many of the SP encounters involved patients with behavioral issues (eg, alcohol, drugs, smoking, unsafe sexual practice) that students were expected to recognize and address through counseling.*
Demonstrate the ability to provide the incorporation of preventive medicine.	4	As all of these SP encounters are based on infectious disease, students were evaluated on hand-washing behaviors and wearing gloves at appropriate times during the examination. Patient counseling was to include recognition of contagiousness.
*Demonstrate the ability to provide health promotion.*	5	Many of the SP encounters involved patients with behavioral issues (eg, alcohol, drugs, smoking, unsafe sexual practice) that students were expected to recognize and address through counseling. Patient counseling was to include recognition of contagiousness.
Gather accurate, essential information from all sources.	5	In the open-book phase of writing the SOAP Notes, students were invited to refer to lecture notes, class handouts, and textbooks, as well as online sources of information.
Develop medical interview skills.	5	Students were evaluated three ways; orally immediately postencounter by the SP and peer observer and then in writing by faculty evaluator via SP encounter checklist. Any identified problems required remediation (ie, assessment and discussion) with faculty evaluator.
Develop physical examination skills.	5	Students were evaluated three ways; orally immediately postencounter by the SP and peer observer and then in writing by faculty evaluator via SP encounter checklist. Any identified problems required remediation (ie, assessment and discussion) with faculty evaluator.
Develop medical record-keeping skills.	4	Students were required to produce a SOAP Note with one addendum.
Develop diagnostic/therapeutic plans and treatments.	5	*Students were required to produce a SOAP Note with one addendum. Diagnostic, therapeutic, and treatment plans were required.*
Validate competency in the performance of diagnosis.	4	The SOAP Note is graded on the logic of the diagnoses.
Validate competency in the performance of treatment.	4	*The SOAP Note is graded on the logic of the treatment plan.*
Validate competency in the performance of procedures.	2	No clinical procedures were performed. Some medical microbiology procedures were performed and graded.
Demonstrate ability to provide healthcare services consistent with osteopathic philosophy.	5	Many of the SP encounters involved patients with behavioral issues (eg, alcohol, drugs, smoking, unsafe sexual practice) that students were expected to recognize and address through counseling. Patient counseling was to include recognition of contagiousness.
Demonstrate ability to provide preventive medicine and health promotion services that are based on current scientific evidence.	4	Comments regarding preventive medicine and health promotion were to be based on class handouts and reference materials.
INTERPERSONAL AND COMMUNICATION SKILLS:
Develop and demonstrate interpersonal and communication skills that enable students to establish and maintain professional relationships with patients.	5	Multiple videos have been used to demonstrate or model professional and effective communication skills. Students were assessed and remediated on their communication skills in the SP encounters.
*Develop and demonstrate interpersonal and communication skills that enable students to establish and maintain professional relationships with patients' families.*	1	Not addressed
Develop and demonstrate interpersonal and communication skills that enable them to establish and maintain professional relationships with other members of patient's healthcare team.	1	Not addressed
Demonstrate effectiveness in developing appropriate doctor-patient relationships.	5	*Teaching/assessment/remediation of patient-centered care was central to SP encounters.*
Develop effective listening skills in professional interactions with patients.	5	Teaching/assessment/remediation of listening skills was part of the SP encounters.
Develop effective written communication skills in professional interactions with patients.	4	*Teaching/assessment/remediation of writing skills was central to SP encounters.*
Develop effective oral communication skills in professional interactions with patients.	5	Teaching/assessment/remediation of communication skills was central to SP encounters.
Develop effective oral communication skills with patients' families.	1	*Not addressed*
Develop effective oral communication skills with other health professionals.	4	Students were required to discussed their performance immediately postencounter with the SP and peer observer. Any identified problems required remediation (ie, assessment and discussion) with faculty evaluator.
PROFESSIONALISM:
*Enhance expectations to uphold the osteopathic oath in the conduct of professional activities.*	2	*Although the osteopathic oath is not specifically addressed in the SP encounter model, behaviors addressed in the oath are part of expected behaviors for this exercise.*
Promote advocacy of patient welfare.	1	Not addressed
*Promote adherence to ethical principles.*	4	*Ethics and integrity were discussed and assessed during the SP encounter.*
Promote collaboration with health professionals.	4	The peer observer in the SP encounter may collaborate with the student “physician” in recording data gathered during the interview. The observer is required to provide the “physician” with constructive feedback designed to improve performance in future encounters.
*Promote life-long learning.*	4	*Students were referred to select sources and asked to use computer searches to answer clinical questions.*
Promote sensitivity to a diverse patient population.	4	After each encounter, students received feedback from their SPs, who represented a variety of demographic groups (eg, age, gender, economic diversity).
*Promote avenues of the student's physical and mental health in order to effectively care for patients.*	1	*Not addressed*
Demonstrate respect for patients.	5	Students were assessed directly and indirectly in terms of their respect for the patient.
Demonstrate respect for patients' families.	1	*Not addressed*
Advocate for the primacy of patient welfare and autonomy.	1	Not addressed
Adhere to ethical principles in the practice of medicine.	4	*We discussed and assessed the ethics of accurate SOAP Note records.*
Demonstrate awareness and proper attention to issues of culture, religion, age, gender, sexual orientation, and mental and physical disabilities.	4	Many of the issues listed were embedded in the SP cases, sometimes with direct application to understanding disease etiology.
PRACTICE-BASED LEARNING AND IMPROVEMENT:
Demonstrate the ability to critically evaluate their methods of clinical practice.	1	Not addressed
*Demonstrate the ability to integrate evidence-based medicine into patient care.*	4	*Students were required to seek data from NIH databases for their SOAP Note addenda.*
Demonstrate the ability to show an understanding of research methods.	4	Students were required to seek data from NIH databases for their SOAP Note addenda.
*Demonstrate the ability to improve patient care practices.*	1	*Not addressed*
Treat patients in a manner consistent with the most up-to-date information on diagnostic and therapeutic effectiveness.	5	Students were encouraged to seek data from current source of therapies for infectious disease (eg, The Sanford Guide to Antimicrobial Therapy, online resources).
Perform self-evaluations of clinical practice patterns and practice-based improvement activities using a systematic methodology.	4	*Students were required to write a self-reflection piece that focused on their performance and included any issues identified during peer (observer) assessment.*
Understand research methods as applied to the practice of medicine.	4	Students were given instructions to use an NIH database for genomic information and assessed on their performance through the graded SOAP Note and addendum.
Understand medical informatics as applied to the practice of medicine.	5	*One of the foundation experiences in the course is a medical informatics exercise, which the student is required to master.*
Understand technology as applied to the practice of medicine.	4	Both the medical informatics exercise and the NIH database search were designed around clinical questions.
SYSTEMS-BASED PRACTICE:
Provide an understanding of healthcare delivery systems.	1	Not Adressed
*Provide an understanding of effective and qualitative patient care within the healthcare system.*	1	*Not Adressed*
Provide an understanding of cost-effective medicine.	1	Not Adressed
Promote understanding of national and local healthcare delivery systems and how they impact patient care.	1	*Not Adressed*
Promote understanding of national and local healthcare delivery systems and how they impact professional practice.	1	Not Adressed
Advocate for quality healthcare on behalf of patients.	1	*Not Adressed*
Advance and assist patients in their interactions with the complexities of the medical system.	1	Not Adressed

Abbreviations: NIH, National Institutes of Health (Bethesda, Md); OMT, osteopathic manipulative treatment; SOAP, Subjective, Objective, Assessment, Plan.

View Large

Table 1

Application of Submatrix 1 From Matrix for Quality Enhancement to Medical Microbiology Curriculum: SP Encounter With Acute Gastroenteritis Diagnosis

Core Competency Components and Required Elements	Score (1-5)*	Comment
OSTEOPATHIC PHILOSOPHY AND OSTEOPATHIC MANIPULATIVE MEDICINE:
*Demonstrate and apply knowledge of accepted standards in OMT.*	2	*No OMT was applied or expected during the SP encounter though it was an expected element of the SOAP Note “Plan.”*
Train a skilled and competent osteopathic practitioner who remains dedicated to life-long learning.	4	Part of the exercise required students to access an NIH database to use DNA sequence data to identify the agent of infection.
*Train a skilled and competent osteopathic practitioner who remains dedicated to practice habits in osteopathic philosophy and manipulative medicine.*	5	*The physical examination aspect of the exercise required students to use their palpatory skills to search for sympathetic and lymphatic involvement.*
Demonstrate competency in the understanding and application of OMT.	1	No OMT was applied or expected during the SP encounter though it was an expected element of the SOAP Note “Plan.”
Integrate osteopathic concepts and OMT into the medical care provided to patients as appropriate.	2	*No OMT was applied or expected during the SP encounter though it was an expected element of the SOAP Note “Plan.”*
Advance the understanding and integration of osteopathic principles and philosophy into all clinical and patient care activities.	5	The physical examination aspect of the exercise required students to use their palpatory skills to search for sympathetic and lymphatic involvement.
MEDICAL KNOWLEDGE:
Demonstrate and apply knowledge of accepted standards of clinical medicine.	5	Students learned to use The Sanford Guide to Antimicrobial Therapy to determine appropriate treatment. Students wrote a medical record including Assessment and Plan portions of SOAP Note, which was graded with comments and returned.
*Remain current with new developments in medicine.*	4	**Student knowledge was derived from annually updated lectures, class handouts, and the most recent edition of The Sanford Guide to Antimicrobial Therapy.**
Encourage participation in life-long learning activities.	4	Part of the exercise required students to access an NIH database to use DNA sequence data to identify the agent of infection. The course also includes a general, open-book medical informatics exercise, which benefits students in writing their SOAP Notes.
*Encourage development of research.*	3	Part of the exercise required students to access an NIH database to use DNA sequence data to identify the agent of infection. The course also includes a general, open-book medical informatics exercise, which benefits students in writing their SOAP Notes.
Demonstrate competency in the understanding of clinical medicine to patient care.	4	The SP encounters required students to develop an assessment with a minimum of three diagnoses.
Demonstrate competency in the application of clinical medicine to patient care.	4	*The SP encounters required students to develop an assessment with a minimum of three diagnoses and associated plans to verify and treat.*
Aid knowledge and applications of the foundations of clinical and behavioral medicine.	4	Many of the SP encounters involved patients with behavioral issues (eg, alcohol, drugs, smoking, unsafe sexual practice) that students were expected to recognize and address through counseling.
PATIENT CARE:
Demonstrate the ability to effectively treat patients.	5	After writing their SOAP Notes, students were provided additional information from which they were to confirm diagnosis. They were then to submit an evidence-based therapy for the patient via an addendum to the SOAP Note.
*Demonstrate the ability to provide medical care that incorporates the osteopathic philosophy.*	5	The physical examination aspect of the exercise required students to use their palpatory skills to search for sympathetic and lymphatic involvement. Many of the SP encounters involved patients with behavioral issues (eg, alcohol, drugs, smoking, unsafe sexual practice) that students were expected to recognize and address through counseling.
Demonstrate the ability to provide patient empathy.	5	Students were evaluated for a list of items categorized as patient-centered skills, which are based in empathy for the patient.
*Demonstrate the ability to provide awareness of behavioral issues.*	5	*Many of the SP encounters involved patients with behavioral issues (eg, alcohol, drugs, smoking, unsafe sexual practice) that students were expected to recognize and address through counseling.*
Demonstrate the ability to provide the incorporation of preventive medicine.	4	As all of these SP encounters are based on infectious disease, students were evaluated on hand-washing behaviors and wearing gloves at appropriate times during the examination. Patient counseling was to include recognition of contagiousness.
*Demonstrate the ability to provide health promotion.*	5	Many of the SP encounters involved patients with behavioral issues (eg, alcohol, drugs, smoking, unsafe sexual practice) that students were expected to recognize and address through counseling. Patient counseling was to include recognition of contagiousness.
Gather accurate, essential information from all sources.	5	In the open-book phase of writing the SOAP Notes, students were invited to refer to lecture notes, class handouts, and textbooks, as well as online sources of information.
Develop medical interview skills.	5	Students were evaluated three ways; orally immediately postencounter by the SP and peer observer and then in writing by faculty evaluator via SP encounter checklist. Any identified problems required remediation (ie, assessment and discussion) with faculty evaluator.
Develop physical examination skills.	5	Students were evaluated three ways; orally immediately postencounter by the SP and peer observer and then in writing by faculty evaluator via SP encounter checklist. Any identified problems required remediation (ie, assessment and discussion) with faculty evaluator.
Develop medical record-keeping skills.	4	Students were required to produce a SOAP Note with one addendum.
Develop diagnostic/therapeutic plans and treatments.	5	*Students were required to produce a SOAP Note with one addendum. Diagnostic, therapeutic, and treatment plans were required.*
Validate competency in the performance of diagnosis.	4	The SOAP Note is graded on the logic of the diagnoses.
Validate competency in the performance of treatment.	4	*The SOAP Note is graded on the logic of the treatment plan.*
Validate competency in the performance of procedures.	2	No clinical procedures were performed. Some medical microbiology procedures were performed and graded.
Demonstrate ability to provide healthcare services consistent with osteopathic philosophy.	5	Many of the SP encounters involved patients with behavioral issues (eg, alcohol, drugs, smoking, unsafe sexual practice) that students were expected to recognize and address through counseling. Patient counseling was to include recognition of contagiousness.
Demonstrate ability to provide preventive medicine and health promotion services that are based on current scientific evidence.	4	Comments regarding preventive medicine and health promotion were to be based on class handouts and reference materials.
INTERPERSONAL AND COMMUNICATION SKILLS:
Develop and demonstrate interpersonal and communication skills that enable students to establish and maintain professional relationships with patients.	5	Multiple videos have been used to demonstrate or model professional and effective communication skills. Students were assessed and remediated on their communication skills in the SP encounters.
*Develop and demonstrate interpersonal and communication skills that enable students to establish and maintain professional relationships with patients' families.*	1	Not addressed
Develop and demonstrate interpersonal and communication skills that enable them to establish and maintain professional relationships with other members of patient's healthcare team.	1	Not addressed
Demonstrate effectiveness in developing appropriate doctor-patient relationships.	5	*Teaching/assessment/remediation of patient-centered care was central to SP encounters.*
Develop effective listening skills in professional interactions with patients.	5	Teaching/assessment/remediation of listening skills was part of the SP encounters.
Develop effective written communication skills in professional interactions with patients.	4	*Teaching/assessment/remediation of writing skills was central to SP encounters.*
Develop effective oral communication skills in professional interactions with patients.	5	Teaching/assessment/remediation of communication skills was central to SP encounters.
Develop effective oral communication skills with patients' families.	1	*Not addressed*
Develop effective oral communication skills with other health professionals.	4	Students were required to discussed their performance immediately postencounter with the SP and peer observer. Any identified problems required remediation (ie, assessment and discussion) with faculty evaluator.
PROFESSIONALISM:
*Enhance expectations to uphold the osteopathic oath in the conduct of professional activities.*	2	*Although the osteopathic oath is not specifically addressed in the SP encounter model, behaviors addressed in the oath are part of expected behaviors for this exercise.*
Promote advocacy of patient welfare.	1	Not addressed
*Promote adherence to ethical principles.*	4	*Ethics and integrity were discussed and assessed during the SP encounter.*
Promote collaboration with health professionals.	4	The peer observer in the SP encounter may collaborate with the student “physician” in recording data gathered during the interview. The observer is required to provide the “physician” with constructive feedback designed to improve performance in future encounters.
*Promote life-long learning.*	4	*Students were referred to select sources and asked to use computer searches to answer clinical questions.*
Promote sensitivity to a diverse patient population.	4	After each encounter, students received feedback from their SPs, who represented a variety of demographic groups (eg, age, gender, economic diversity).
*Promote avenues of the student's physical and mental health in order to effectively care for patients.*	1	*Not addressed*
Demonstrate respect for patients.	5	Students were assessed directly and indirectly in terms of their respect for the patient.
Demonstrate respect for patients' families.	1	*Not addressed*
Advocate for the primacy of patient welfare and autonomy.	1	Not addressed
Adhere to ethical principles in the practice of medicine.	4	*We discussed and assessed the ethics of accurate SOAP Note records.*
Demonstrate awareness and proper attention to issues of culture, religion, age, gender, sexual orientation, and mental and physical disabilities.	4	Many of the issues listed were embedded in the SP cases, sometimes with direct application to understanding disease etiology.
PRACTICE-BASED LEARNING AND IMPROVEMENT:
Demonstrate the ability to critically evaluate their methods of clinical practice.	1	Not addressed
*Demonstrate the ability to integrate evidence-based medicine into patient care.*	4	*Students were required to seek data from NIH databases for their SOAP Note addenda.*
Demonstrate the ability to show an understanding of research methods.	4	Students were required to seek data from NIH databases for their SOAP Note addenda.
*Demonstrate the ability to improve patient care practices.*	1	*Not addressed*
Treat patients in a manner consistent with the most up-to-date information on diagnostic and therapeutic effectiveness.	5	Students were encouraged to seek data from current source of therapies for infectious disease (eg, The Sanford Guide to Antimicrobial Therapy, online resources).
Perform self-evaluations of clinical practice patterns and practice-based improvement activities using a systematic methodology.	4	*Students were required to write a self-reflection piece that focused on their performance and included any issues identified during peer (observer) assessment.*
Understand research methods as applied to the practice of medicine.	4	Students were given instructions to use an NIH database for genomic information and assessed on their performance through the graded SOAP Note and addendum.
Understand medical informatics as applied to the practice of medicine.	5	*One of the foundation experiences in the course is a medical informatics exercise, which the student is required to master.*
Understand technology as applied to the practice of medicine.	4	Both the medical informatics exercise and the NIH database search were designed around clinical questions.
SYSTEMS-BASED PRACTICE:
Provide an understanding of healthcare delivery systems.	1	Not Adressed
*Provide an understanding of effective and qualitative patient care within the healthcare system.*	1	*Not Adressed*
Provide an understanding of cost-effective medicine.	1	Not Adressed
Promote understanding of national and local healthcare delivery systems and how they impact patient care.	1	*Not Adressed*
Promote understanding of national and local healthcare delivery systems and how they impact professional practice.	1	Not Adressed
Advocate for quality healthcare on behalf of patients.	1	*Not Adressed*
Advance and assist patients in their interactions with the complexities of the medical system.	1	Not Adressed

Abbreviations: NIH, National Institutes of Health (Bethesda, Md); OMT, osteopathic manipulative treatment; SOAP, Subjective, Objective, Assessment, Plan.

Table 2

Application of Submatrix 1 From Matrix for Quality Enhancement to Medical Microbiology Curriculum: Core Competency 3, Patient Care, as Addressed in SP Encounters *

	Medical Care That Incorporates Osteopathic Philosophy (n=6)	Gather Accurate, Essential Information From All Sources (n=5)	Competent Performance of Diagnosis, Treatment, and Procedures (n=3)	Deliver Healthcare Services Consistent With Osteopathic Philosophy and EBM (n=2)	Elements Addressed (n=16)
SP Encounter	No. of Required Elements Addressed						Expected Score	Actual Score
Acute gastroenteritis	6	5	1	2	14	14
*Acute respiratory disease*	6	5	1	2	14	14
Aseptic meningitis	6	5	1	2	14	14
*Common cold*	6	5	1	2	14	14
Epstein-Barr viral mononucleosis	6	5	1	2	14	14
*Genital warts*	6	5	1	2	14	14
Hepatitis A	6	5	1	2	14	14
*Herpes genitalis*	6	5	1	2	14	14
Herpes simplex pharyngitis	6	5	1	2	14	14
*Human immunodeficiency viremia*	6	5	1	2	14	14
Influenza	6	5	1	2	14	14
*Keratoconjunctivitis*	6	5	1	2	14	14
Laryngitis	6	5	1	2	14	14
*Rubella*	6	5	1	2	14	14
Shingles	6	5	1	2	14	14
*West Nile encephalitis*	6	5	1	2	14	14

Abbreviation: EBM, evidence-based medicine.

View Large

Table 2

Application of Submatrix 1 From Matrix for Quality Enhancement to Medical Microbiology Curriculum: Core Competency 3, Patient Care, as Addressed in SP Encounters *

	Medical Care That Incorporates Osteopathic Philosophy (n=6)	Gather Accurate, Essential Information From All Sources (n=5)	Competent Performance of Diagnosis, Treatment, and Procedures (n=3)	Deliver Healthcare Services Consistent With Osteopathic Philosophy and EBM (n=2)	Elements Addressed (n=16)
SP Encounter	No. of Required Elements Addressed						Expected Score	Actual Score
Acute gastroenteritis	6	5	1	2	14	14
*Acute respiratory disease*	6	5	1	2	14	14
Aseptic meningitis	6	5	1	2	14	14
*Common cold*	6	5	1	2	14	14
Epstein-Barr viral mononucleosis	6	5	1	2	14	14
*Genital warts*	6	5	1	2	14	14
Hepatitis A	6	5	1	2	14	14
*Herpes genitalis*	6	5	1	2	14	14
Herpes simplex pharyngitis	6	5	1	2	14	14
*Human immunodeficiency viremia*	6	5	1	2	14	14
Influenza	6	5	1	2	14	14
*Keratoconjunctivitis*	6	5	1	2	14	14
Laryngitis	6	5	1	2	14	14
*Rubella*	6	5	1	2	14	14
Shingles	6	5	1	2	14	14
*West Nile encephalitis*	6	5	1	2	14	14

Abbreviation: EBM, evidence-based medicine.

Table 3

Application of Matrix for Quality Enhancement to Medical Microbiology SP Encounters: COMLEX-USA Evaluation Elements vs Curriculum

Clinical Presentation Categories	COMLEX-USA Design Parameters,%*	Primary SP Diagnosis, %^†
Respiration and circulation	8-16	25
*Human sexuality and urination*	*3-8*	*12.5*
Digestion and metabolism	4-10	12.5
*Sensory alterations*	*28-38*	*37.5*
Asymptomatic and general symptoms	8-16	12.5
*Thermoregulation*	*2-6*	*NA^†*
Human development	3-8	NA^†
*Motor alterations*	*6-12*	NA
Tissues and trauma	8-16	NA

View Large

Table 3

Application of Matrix for Quality Enhancement to Medical Microbiology SP Encounters: COMLEX-USA Evaluation Elements vs Curriculum

Clinical Presentation Categories	COMLEX-USA Design Parameters,%*	Primary SP Diagnosis, %^†
Respiration and circulation	8-16	25
*Human sexuality and urination*	*3-8*	*12.5*
Digestion and metabolism	4-10	12.5
*Sensory alterations*	*28-38*	*37.5*
Asymptomatic and general symptoms	8-16	12.5
*Thermoregulation*	*2-6*	*NA^†*
Human development	3-8	NA^†
*Motor alterations*	*6-12*	NA
Tissues and trauma	8-16	NA

Table 4

Application of Matrix for Quality Enhancement to Medical Microbiology SP Encounters: Summary of Curriculum Assessment *

Evaluation Element	Submatrix	Addressed, %
Core Competency 2, Medical Knowledge	1	100
*Humanistic Domain*	4	*100*
Core Competency 3, Patient Care	1	93
*COMLEX-USA Dimension I — The Clinical Presentation*	2	89
Biomedical/Biomechanical Domain	4	88
*COMLEX-USA Dimension II — The Physician Task*	3	83
Core Competency 6, Practice-Based Learning and Improvement	1	78
*Core Competency 4, Interpersonal and Communication Skills*	1	67
Patient Safety Issues	5	57
*Core Competency 5, Professionalism*	1	55
Core Competency 1, Osteopathic Philosophy and Manipulative Medicine	1	50^†
*Core Competency 7, Systems-Based Practice^‡*	1	NA

Abbreviations: COMLEX-USA, Comprehensive Osteopathic Medical Licensing Examination-USA; NA, not addressed.

‡None of the seven items in core competency 7, systems-based practice, was addressed.

View Large

Table 4

Application of Matrix for Quality Enhancement to Medical Microbiology SP Encounters: Summary of Curriculum Assessment *

Evaluation Element	Submatrix	Addressed, %
Core Competency 2, Medical Knowledge	1	100
*Humanistic Domain*	4	*100*
Core Competency 3, Patient Care	1	93
*COMLEX-USA Dimension I — The Clinical Presentation*	2	89
Biomedical/Biomechanical Domain	4	88
*COMLEX-USA Dimension II — The Physician Task*	3	83
Core Competency 6, Practice-Based Learning and Improvement	1	78
*Core Competency 4, Interpersonal and Communication Skills*	1	67
Patient Safety Issues	5	57
*Core Competency 5, Professionalism*	1	55
Core Competency 1, Osteopathic Philosophy and Manipulative Medicine	1	50^†
*Core Competency 7, Systems-Based Practice^‡*	1	NA

Abbreviations: COMLEX-USA, Comprehensive Osteopathic Medical Licensing Examination-USA; NA, not addressed.

‡None of the seven items in core competency 7, systems-based practice, was addressed.

Curricular Analysis of Competency-Based Osteopathic Medical Education: Application of a Matrix for Quality Enhancement to a Standardized Patient Encounter Example

Related content

Related Topics