Free
JAOA/AACOM Medical Education  |   September 2019
Integrating the Principles and Practice of Scholarly Activity Into Undergraduate Medical Education: A Narrative Review and Proposed Model for Implementation
Author Notes
  • From the Philadelphia College of Osteopathic Medicine in Pennsylvania. 
  • Financial Disclosures: None reported. 
  • Support: None reported. 
  •  *Address correspondence to Danielle C. Estrada, OMS IV, MS, Philadelphia College of Osteopathic Medicine, 4170 City Ave, Philadelphia, PA 19131-1610. Email: daniellemo@pcom.edu
     
Article Information
Medical Education / Professional Issues / Graduate Medical Education
JAOA/AACOM Medical Education   |   September 2019
Integrating the Principles and Practice of Scholarly Activity Into Undergraduate Medical Education: A Narrative Review and Proposed Model for Implementation
The Journal of the American Osteopathic Association, September 2019, Vol. 119, 598-611. doi:https://doi.org/10.7556/jaoa.2019.103
The Journal of the American Osteopathic Association, September 2019, Vol. 119, 598-611. doi:https://doi.org/10.7556/jaoa.2019.103
Web of Science® Times Cited: 2
Abstract

The osteopathic undergraduate medical education standards have evolved over the past 2 decades to require undergraduate medical student participation in research and scholarly activity. The authors’ objective was to review those evolving standards and develop a model for introducing the principles and practice of research that combines core content with experiential learning. They identified fundamental topics pertinent to the research process and herein provide their recommendations for incorporating these topics into the curriculum as self-study, online modules, and team-based and active learning exercises. The authors’ proposed educational model would provide an avenue for osteopathic medical schools to meet the requirements for scholarly activity. Increasing students' and residents' knowledge of the research process will lay the foundation for their engagement in research and scholarly activity and their practice of evidence-based medicine.

The most widely accepted definition of scholarly activity (SA) by graduate medical education (GME) programs consists of any 1 of 4 components: discovery, integration, application, and teaching.1-4 Many physicians practice basic principles of SA as they advance knowledge, synthesize information, apply knowledge to clinical situations, and educate patients and trainees. All 4 components of SA are present in the practice of evidence-based medicine (EBM) as physicians seek the most up-to-date information, integrate new knowledge into clinical experience, and communicate the basis of clinical decisions.5,6 The importance of EBM in research and SA (RSA) is recognized as essential for clinical practice; therefore, RSA is included within the Accreditation Council for Graduate Medical Education (ACGME) Common Program Requirements.7 The Commission on Osteopathic College Accreditation (COCA) also instituted a requirement for osteopathic medical student participation in RSA, effective in 2017.8,9 This requirement is timely given that all osteopathic and allopathic GME programs will be evaluated by the same standards within the single accreditation system by June 30, 2020.10 
The research process itself provides active learning experiences requiring critical thinking to generate and communicate new knowledge. Broadly speaking, research involves asking a relevant question, then systematically and reproducibly testing a hypothesis before disseminating results for beneficial application in the community.11 According to Residency Review Committee rubrics, publication through a peer-reviewed process is “highly valued,” and 2 of the rubrics consider only research as evidence of SA.3 Furthermore, our analysis of 2016 and 2018 National Resident Matching Program data indicates that research accomplishments, including abstracts, publications, and presentations, had a significant impact on matching success for allopathic medical students.12 Osteopathic medical students had significantly fewer research accomplishments than allopathic medical students, with no significant difference in research accomplishment number between matched and unmatched osteopathic students.12 Given that osteopathic and allopathic medical students will apply for the same residency programs in the 2020 Main Residency Match, it is important to enhance osteopathic medical students’ knowledge of, and experience with, the principles and practice of research. 
In this article, we review and interpret the evolving requirements for RSA in osteopathic undergraduate medical education (UME) cited within the American Osteopathic Association Bureau of Professional Education from 1998 to 2003 and the COCA standards from 2004 to present.8,13,14,15-18 Based on these requirements and the predicted increase in the importance of research accomplishments for residency matching,12 we developed a model for integrating content related to the principles and practice of research into UME that includes experiential learning. The proposed model provides recommendations for combining foundational research-related concepts with supervised active learning experiences for osteopathic medical students to enhance preparedness for engaging in RSA and practicing EBM. 
Literature Review of the Evolution of Osteopathic UME Requirements for RSA
A timeline of inclusion of RSA in the American Osteopathic Association Bureau of Professional Education standards from 1998 to 2003 and COCA standards from 2004 to present is presented in Table 1. Osteopathic UME standards prior to 1998 were not available for review. The first mention of RSA occurred in 1998 under the “Authority and Purpose” section of the Introduction. Until 2004, research was only referred to within the mission/objectives of colleges of osteopathic medicine (COMs) and as a potential area for faculty contribution.13,19 Thereafter, research became a COM standard.8,9,13-29 In 2014, “Standard Seven: Research and Scholarly Activities,” expanded requiring a strategic plan for research that could be met through sponsorship and support for student research fellowships.28 In 2017, “Scholarly Activity” became Standard 8, with “Element 8.1: Research and Scholarly Activity” as a core requirement, stating that a COM must have a strategic plan for SA.8 “Element 8.2: Student Participation” requires a COM to provide support for student-driven RSA.8 In 2019, Element 8.2 became “Research and Scholarly Activity Budget,” requiring a COM to have a budgetary process for research and scholarly activity. “Element 8.3: OMM/OPP Research and Scholarly Activity” was added to state that a COM must demonstrate how osteopathic manipulative medicine and osteopathic principles and practice are incorporated into research and scholarly activity. “Element 8.4: Student Participation in Research and Scholarly Activity” now takes the place of the previous 2017 Element 8.2. The standards effective July 1, 2019, are the current published standards.18 
Table 1.
Osteopathic Undergraduate Medical Education Standards Related to Research and Scholarly Activity From 1998-20198,9,13-29
  Image not available

a Standards are reproduced from the source documents. Minor punctuation adjustments have been made for consistent style.

Abbreviations: COCA, Commission on Osteopathic College Accreditation; COM, college of osteopathic medicine; OMM, osteopathic manipulative medicine; OPP, osteopathic principles and practice.

Table 1.
Osteopathic Undergraduate Medical Education Standards Related to Research and Scholarly Activity From 1998-20198,9,13-29
  Image not available

a Standards are reproduced from the source documents. Minor punctuation adjustments have been made for consistent style.

Abbreviations: COCA, Commission on Osteopathic College Accreditation; COM, college of osteopathic medicine; OMM, osteopathic manipulative medicine; OPP, osteopathic principles and practice.

×
Development of a Model for Integrating the Principles and Practice of RSA Into UME
Guidance for Selection of Core Content and Delivery Methods
The model for integrating the principles and practice of research was created to align with COCA and ACGME standards and enhance students’ preparedness to engage in RSA and the practice of EBM. The following is a narrative review of validated content and education methods related to research principles and practice. Our model for integrating the principles and practice of research incorporates various validated criteria2,30-32 and educational tools and strategies33-40 for the delivery and assimilation of core content. The frameworks for formulating a research question and assessing scholarship were provided by patient/population/problem, intervention/prognostic factor/exposure, comparison, and outcome (PICO)31,32; feasible, interesting, novel, ethical, relevant (FINER),30 and the Glassick criteria.2 
Fundamental topics, content sequencing, and delivery formats were selected in consultation with individual Philadelphia College of Osteopathic Medicine (PCOM) researchers/educators, Library Science faculty, and feedback from the Research and Curriculum Committees and Division of Research. Input also was sought on potential barriers to integrating the model into the curriculum and solutions to overcoming those barriers. Focused interviews were conducted by 2 of the authors (C.N.M. and D.C.E.), both of whom were present for all interviews with PCOM faculty. All interviewees were provided a hard copy of the proposed course model. Discussions began with a brief description of the purpose of the model followed by in-depth outlines of its various components. Suggestions and comments were recorded for future reference. The model also was presented as an oral PowerPoint (Microsoft Corporation) presentation to PCOM's curriculum and research committees. Additional comments and suggestions from PCOM faculty were received via email. A second oral presentation was given to the curriculum committee to solicit additional recommendations and feedback on barriers to integration prior to finalization of the model. 
Content Delivery Through Online Modules
Topics recommended for self-study, online modules are intended to provide students with a foundational understanding of the research process (Table 2). Students would progress at their own pace through the material during the first term of year 1. We recommend early immersion to nurture the development of critical and creative thinking skills that may be used during all phases of medical education. This early immersion may better prepare students to engage in research during the semester gap between years 1 and 2 of medical school and throughout the school year as time permits. Student knowledge would be assessed with multiple-choice questions at the end of each online module accompanied by the rationale for the correct answer. Students would retake the module until a passing grade was achieved. The core-content modules would be available to students after completion to reinforce the material.40 
Table 2.
Core Content for Independent Learning to Integrate the Principles and Practice of Scholarly Activity Into Undergraduate Medical Education
  Image not available

Abbreviations: CEBM, Center for Evidence-Based Medicine; FINER, feasible, interesting, novel, ethical, and relevant; IACUC, Institutional Animal Care and Use Committee; IRB, institutional review board; HIPAA, Health Insurance Portability and Accountability Act; PHI, protected health information; PICO, patient/population/problem, intervention/prognostic factor/exposure, comparison, and outcome; STROBE, strengthening the reporting of observational studies in epidemiology.

Table 2.
Core Content for Independent Learning to Integrate the Principles and Practice of Scholarly Activity Into Undergraduate Medical Education
  Image not available

Abbreviations: CEBM, Center for Evidence-Based Medicine; FINER, feasible, interesting, novel, ethical, and relevant; IACUC, Institutional Animal Care and Use Committee; IRB, institutional review board; HIPAA, Health Insurance Portability and Accountability Act; PHI, protected health information; PICO, patient/population/problem, intervention/prognostic factor/exposure, comparison, and outcome; STROBE, strengthening the reporting of observational studies in epidemiology.

×
Topic content and sequence are intended to be adaptable to an existing curriculum that may already introduce research-related material in the first 2 years of medical school. Topics are aligned with sub-content areas outlined in the National Board of Osteopathic Medical Examiners competency domain practice-based learning and improvement in osteopathic medical practice, including fundamental epidemiologic concepts, clinical decision-making tools, EBM principles and practice, clinical significance of research, and statistics.5 
Core-content areas are provided in Table 2. Beginning early in year 1 with modules about “Types of Research” and “Introduction to Research Compliance,” students would review fundamentals of regulatory committee oversight (institutional review board and Institutional Animal Care and Use Committee). Distinctions between protocols that require or are exempt from informed consent would be identified. Reinforcement of compliance concepts would be achieved with certification training courses sponsored by the Collaborative Institutional Training Initiative or comparable programs. In “Navigating & Evaluating Scientific Literature,” students would gain competency in the process of searching and selecting existing resources and databases, and be able to identify tools that facilitate appraisal of published research studies. These initial core-content areas would create a foundation for students to begin an active learning component of the program. 
The next self-study topic “Formulating a Question, Study Aim & Hypothesis” would teach students how to pose a relevant question using evaluation criteria such as PICO and FINER.30-32 For example, the FINER criteria provide guidelines for developing a question that is feasible to answer and includes considerations of subject number, technical expertise, time, and funds. A research question should be intriguing to the investigator, have clinical relevance, and if answered, impact practices and policies and direct future areas of inquiry.30 In addition, the question must conform to regulatory committee guidelines for ethical conduct of research. 
The next 3 core-content areas, “Study Designs,” “Utilizing Proper Measurement Tools,” and “Research Data Management” would inform students of the differences between retrospective and prospective research, the importance of randomization and blinding, what a power analysis is and how to extract information from public and private databases. A module on “Intellectual Property, Author Rights, Journal Selection, and the Public Domain” would provide introductory information about laws that protect ownership, reproduction, and distribution of original material. Ultimately, through “Communication of Research Findings,” students would learn ways to construct a case report, manuscript, and an oral or poster presentation. 
Team-Based Learning Activities
Team-based learning (TBL) activities would commence at the beginning of the second term of year 2. The activities would be designed to reinforce and assess students’ knowledge gained from the core-content modules and prepare them for small group, experiential learning (Table 3). Students would come to the TBL having reviewed the assigned materials. For the TBL “Research Ethics and Informed Consent,” students would be required to read examples of breaches of ethics and those that fall within the “gray zone,” then demonstrate an understanding of whether or not historical research studies are considered ethical based upon current standards. In the second TBL, “Evaluation of Research Articles,” students would compare and contrast manuscripts describing a well-designed vs ill-conceived study. Students would be expected to identify the various components of the scientific article and evaluate whether the measurement tools were appropriately matched to the study design. Team-based learning activities would follow the standard format, including an individual readiness assurance test, followed by group discussions of the same material and a group readiness assessment test.35,37,39 The TBL would conclude with a discussion that includes an opportunity to challenge answers to the questions. Students would receive a pass/fail grade for the individual readiness assurance and group readiness assurance tests. 
Table 3.
Group Learning Exercises and Activities to Integrate the Principles and Practice of Scholarly Activity Into Undergraduate Medical Education
Topic Process and Assignments Learning Objectives
TBL Format
 1. Research Ethics and Informed Consent ■ Preparation: readings on research ethics violations
■ iRAT: MCQs on research ethics
■ gRAT: MCQs on research ethics
■ Appeal/Discussion: challenge and discuss answers to questions
■ Students will reinforce their comprehension of Module 2 with independent and group evaluations of research ethics violations.
 2. Evaluation of Research Articles ■ Preparation: study core content on navigating the scientific literature
■ gRAT: MCQs on strengths/weaknesses to study design and methodology, components of scientific article, measurement tools, and literature appraisal
■ Appeal/Discussion: challenge and discuss answers to questions
■ Students will apply their knowledge gained from Module 3 with independent and group assessments of published studies.
ALG Format
 1. Hypothesis ■ Discuss potential study areas and questions
■ Select one question and formulate a hypothesis
■ Students will work as a team to formulate a hypothesis.
 2. Background/Introduction ■ Write, review and evaluate background/ introduction for research study
■ Define and assign task distribution
■ Students will identify topics relevant to their proposed study and assign areas for literature review.
 3. Study Design ■ Define an appropriate category of study design as it relates to hypothesis ■ The team will apply their collective knowledge to identify an appropriate study design.
 4. Measurement and Methodology ■ Develop measurement and methodology for selected study
■ Define and assign task distribution
■ The team will apply their collective knowledge to select appropriate tools for measurement and other methods.
 5. Study Proposal ■ Discuss and refine study proposal ■ The team will draft and edit the study proposal.
 6. Limitations and Alternative Approaches ■ Discuss potential limitations, alternative approaches and impact of research study ■ The team will scrutinize their proposal by identifying limitations, proposing alternative approaches.
■ The team will provide a narrative for the potential importance and significance of their research.
 7. Research proposal ■ Finalize research proposal
■ Present research study design
■ Each student will gain experience developing and delivering an oral presentation.

Abbreviations: ALG, active learning group; gRAT, group readiness assurance test; iRAT, individual readiness assurance test; MCQs, multiple-choice questions; TBL, team-based learning.

Table 3.
Group Learning Exercises and Activities to Integrate the Principles and Practice of Scholarly Activity Into Undergraduate Medical Education
Topic Process and Assignments Learning Objectives
TBL Format
 1. Research Ethics and Informed Consent ■ Preparation: readings on research ethics violations
■ iRAT: MCQs on research ethics
■ gRAT: MCQs on research ethics
■ Appeal/Discussion: challenge and discuss answers to questions
■ Students will reinforce their comprehension of Module 2 with independent and group evaluations of research ethics violations.
 2. Evaluation of Research Articles ■ Preparation: study core content on navigating the scientific literature
■ gRAT: MCQs on strengths/weaknesses to study design and methodology, components of scientific article, measurement tools, and literature appraisal
■ Appeal/Discussion: challenge and discuss answers to questions
■ Students will apply their knowledge gained from Module 3 with independent and group assessments of published studies.
ALG Format
 1. Hypothesis ■ Discuss potential study areas and questions
■ Select one question and formulate a hypothesis
■ Students will work as a team to formulate a hypothesis.
 2. Background/Introduction ■ Write, review and evaluate background/ introduction for research study
■ Define and assign task distribution
■ Students will identify topics relevant to their proposed study and assign areas for literature review.
 3. Study Design ■ Define an appropriate category of study design as it relates to hypothesis ■ The team will apply their collective knowledge to identify an appropriate study design.
 4. Measurement and Methodology ■ Develop measurement and methodology for selected study
■ Define and assign task distribution
■ The team will apply their collective knowledge to select appropriate tools for measurement and other methods.
 5. Study Proposal ■ Discuss and refine study proposal ■ The team will draft and edit the study proposal.
 6. Limitations and Alternative Approaches ■ Discuss potential limitations, alternative approaches and impact of research study ■ The team will scrutinize their proposal by identifying limitations, proposing alternative approaches.
■ The team will provide a narrative for the potential importance and significance of their research.
 7. Research proposal ■ Finalize research proposal
■ Present research study design
■ Each student will gain experience developing and delivering an oral presentation.

Abbreviations: ALG, active learning group; gRAT, group readiness assurance test; iRAT, individual readiness assurance test; MCQs, multiple-choice questions; TBL, team-based learning.

×
Active Learning Group Activities
The active learning group (ALG) component of the model would begin in year 1, term 2, following completion of the TBL activities and conclude with submission of the research proposal and oral presentation at the end of year 1, term 2. The goal of the ALG component is to provide students with the opportunity to apply their knowledge and develop a research study under guided mentorship with periodic milestones to mark progress.34 The groundwork for this group project would be provided by the core-content modules, training courses, and TBL exercises. Ideally, each ALG would contain 8 students. The group would be expected to work independently to develop a research proposal. The mentor assigned to the group would be available to answer questions, provide feedback, and review and approve each stage of the process.33,34 The proposal developed within the ALG would constitute a capstone project. 
Active learning group activities are outlined in Table 3. The ALG would be tasked with selecting a question of common interest relevant to clinical practice. By creating their own question, students would be expected to have a greater sense of ownership and investment in the ALG process.38 Background learning from the core-content modules on formulating a question and a preliminary literature review would be used to assess the validity and relevance of the question. Students would then formulate a hypothesis. 
The next task for the ALG would be to delve more extensively into the literature and write a background and significance narrative for the project. Students would collaborate to design a study incorporating biostatistical elements and outcomes measurements. The narrative would also include a discussion of the study's limitations, alternative approaches, and summary of the potential impact of the research. The format for the narrative would follow that of an institutional review board or National Institutes of Health grant application. The pace of completion of each component would be set by the ALG with oversight from the mentor. Each component of the proposal/protocol would be approved by the mentor before progressing to the next stage. 
The written research proposal/protocol would reflect the depth of student comprehension of the principles and practice of research according to the Glassick Criteria that are often applied in GME programs when assessing SA.2,4 The criteria for evaluation include (1) clarity of the hypothesis, (2) relevant literature review, (3) appropriate study design, (4) logical expected outcomes, (5) effective presentation, and (6) meaningful assessment of the study's potential impact. 
An important component of the ALG experience would be to present the capstone project to classmates, mentors, and faculty at the end of the first year. This assignment would provide students with experience assembling and organizing an oral presentation. Each student would present a component of the study to the audience. Pass/fail grades for the written proposal and oral presentation would be assigned to the group by the mentor. 
Students would not be required to implement their project. However, it is likely that some students would work with mentors to submit applications to regulatory committees and seek a source of funding to support their research. It is also possible that research projects may be implemented during residency. Students would be educated about the process of receiving approval from the ALG mentor to conduct the proposed project independently. It is expected that other members of the ALG would receive credit for their input into developing the study design if the results are published. 
Mentorship
Mentoring is central to the process of training students as researchers and physicians. Skilled mentoring increases and improves RSA for GME programs, especially when those providing mentorship have expertise in the area of study and/or research training.42,43 For these reasons, mentors comprising physicians, PhDs, clinicians, alumni, residents, and librarians would be paired with students based on the ALG's proposed topic of study. Mentors would receive both written guidelines and group instruction outlining the goals of the assignment, the process, and standards for evaluation.43 
The availability of mentors is a commonly cited limitation for medical schools.44 While TBL exercises involve large groups of students with only a few facilitators, implementation of the ALG component of the model would depend on both the individual class size and availability of mentors with knowledge of the research process. Engaging faculty, staff, alumni, residents, affiliated clinicians, and librarians would provide students with additional role models and enrich their learning experience. Furthermore, inclusion of faculty mentors from disciplines outside of osteopathic medicine, such as allied health professions, will add the important element of interprofessional education that introduces different perspectives and instills an appreciation for working in interdisciplinary teams. 
Evaluation of the Program
Outcomes assessments are essential components of change. The long-term impact of a research-based curriculum would be measured by tracking typical productivity metrics, such as publications, presentations, and grant awards, as well as national board scores and residency matching.45-47 We recommend collecting outcome data for at least 2 cohorts of students that have participated in the program and 3 cohorts of students that have not been exposed to this content. Students would be informed of an institutional review board–approved study to assess long-term outcomes of the research curriculum before beginning the independent study modules. National board scores and match data would be obtained from the provost's office. Postgraduate information would be collected via SurveyMonkey. 
Acknowledging student feedback improves enthusiasm for and motivation to engage in RSA. Therefore, evaluation surveys are critical for the success and refinement of the program. Surveys would be designed within the conceptual framework of the Kirkpatrick Pyramid48,49 and the MERSQI tool.51-53 The Kirkpatrick Pyramid involves 4 levels of outcome assessments, including Reaction, Learning, Behavior, and Results.48 The MERSQI tool is a 10-item scoring system that examines 6 domains to measure the quality of medical educational research.51,52 Surveys would be distributed after completion of the core-content modules, TBL, and ALG activities. 
Discussion
The expectations surrounding RSA have increased at all levels of a COM (leadership, administration, facilities, curriculum, faculty, students, and policies). Greater osteopathic training in research principles and practice must accompany the elevated expectations for RSA.53,54 Based on our informal review of course catalogs for 34 COMs, varying levels of instruction in research and EBM are provided within the first-year curriculum. Many COMs actively encourage and provide opportunities for students to participate in research either through summer programs or a research elective. However, research programs that involve active mentoring are often available on a competitive basis for a limited number of students. 
Our model is designed to provide all students with the foundations of the principles of research and experiential learning for the practice of research through study design. The model combines independent study, group activities, and experiential learning with mentoring. From a practical standpoint, the model addresses COCA standards of active student participation in SA and prepares them for the practice-based learning and improvement competency domain. The content also fulfills several of the Association of American Medical Colleges’ Entrustable Professional Activities for entering residency, including forming clinical questions and retrieving evidence to advance patient care, collaborating as a member of an interprofessional team, and giving an oral presentation.55 The overarching educational goals of this program are to (1) encourage osteopathic medical students to be curious, innovative, thoughtful, and systematic; (2) lay a sound foundation for the practice of SA and EBM; and (3) cultivate investigative skills as they design their own studies. Nurturing student curiosity and creativity is particularly important in the face of a mounting emphasis on medical board preparation that requires assimilation of existing information. 
In this model, the core online content is intended to be reinforced with TBL exercises and active learning. Active learning group experiences in designing hypothesis-driven research and writing a proposal are important for applying knowledge. In addition to achieving a basic level of competency with study design, the ALG is expected to facilitate a sense of ownership with regard to their education that is critical for lifelong learning and promoting a culture of scholarship. 
Conclusion
Integrating content related to the principles and practice of research into UME is intended to increase students’ familiarity and comfort with the research process and encourage them to participate in research studies. Research accomplishments enhance students’ competitiveness for residency; thus, we anticipate that documented RSA will be a factor in the Main Residency Match under the single accreditation system.12,56-60 Furthermore, research productivity during medical school translates to an increased likelihood of publishing during residency.61,62 Students’ perceptions of the course content and delivery methods, as well as longitudinal outcomes assessments, are critical for measuring whether the program has achieved its goals of instilling a working knowledge and appreciation of the 4 core components of RSA and contributing to the development of lifelong learners who practice EBM. 
Acknowledgments
We thank the following PCOM faculty, staff, and committees: Jane Z. Dumsha, PhD, Chief Research Operations Officer; Pamela Ruoff, MS, Executive Director, Alumni Engagement; Arthur Sesso, DO, Chairman of Surgery, Program Director of General Surgery, Director of Graduate Medical Education for Surgical Services, Medical Director of Clinical Learning Assessment Center, and Chairman of the Curriculum Committee; Matthew Weinberg, MB, Assistant Professor of Medical Ethics and Course Director of Preventive & Community-Based Medicine; Library Staff; Alumni Association; and the Curriculum and Research Committees. We also thank COCA for providing us with previous accreditation standards. 
References
Boyer E. Scholarship Reconsidered: Priorities of the Professoriate. San Francisco, CA: Jossey-Bass; 1990.
Glassick CE, Huber MT, Maeroff GI. Scholarship Assessed: Evaluation of the Professoriate. San Francisco, CA: Jossey-Bass; 1997;10-11.
Grady EC, Roise A, Barr D, et al. Defining scholarly activity in graduate medical education. J Grad Med Edu. 2012;4(4):558-561. doi: 10.4300/JGME-D-12-00266.1 [CrossRef]
Simpson D, Yarris LM, Carek PJ. Defining the scholarly and scholarship common program requirements. J Grad Med Educ. 2013;5(4):539-540. doi: 10.4300/JGME-D-13-00326 [CrossRef] [PubMed]
Practice-based learning and improvement in osteopathic medical practice. National Board of Osteopathic Medical Examiners website. https://www.nbome.org/exams-assessments/comlex-usa/master-blueprint/competency-domains/improvement-in-osteopathic-medical-practice/. Accessed April 5, 2019.
Masic I, Miokovic M, Muhamedagic B. Evidence based medicine - new approaches and challenges. Acta Inform Med. 2008;16(4):219-225. doi: 10.5455/aim.2008.16.219-225 [CrossRef] [PubMed]
Common Program Requirements. Chicago, IL: Accreditation Council for Graduate Medical Education; 2016. https://www.acgme.org/Portals/0/PFAssets/ProgramRequirements/CPRs_2017-07-01.pdf. Accessed July 28, 2019.
Commission on Osteopathic College Accreditation. Accreditation of Colleges of Osteopathic Medicine: COM Continuing Accreditation Standards. Chicago, IL: American Osteopathic Association; 2016. https://health.okstate.edu/site-files/docs/coca-continuing-accreditation-standards-july-1-2017.pdf. Accessed July 28, 2019.
Commission on Osteopathic College Accreditation. Accreditation of Colleges of Osteopathic Medicine: COM Accreditation Standards and Procedures. Chicago, IL: American Osteopathic Association; 2015. https://osteopathic.org/wp-content/uploads/2018/02/com-accreditation-standards-8-29-2016.pdf. Accessed November 5, 2018.
Allopathic and osteopathic medical communities commit to a single graduate medical education accreditation system [letter]. February 26, 2014. American Association of Colleges of Osteopathic Medicine website. https://www.aacom.org/docs/default-source/single-gme-accreditation/joint-press-release.pdf?sfvrsn=c64a7e97_0. Accessed November 5, 2018.
Brannan GD, Dumsha JZ, Yens DP. A research primer: basic guidelines for the novice researcher. J Am Osteopath Assoc. 2013;113(7):556-563. doi: 10.7556/jaoa.2013.007. [CrossRef] [PubMed]
Matthews CN, Estrada DC, Claeson KM, George-Weinstein M. Evaluating the influence of research on match success for osteopathic and allopathic seniors. J Am Osteopath Assoc. 2019;119(9):598-611. [CrossRef] [PubMed]
Bureau of Professional Education. Accreditation of Colleges of Osteopathic Medicine. Chicago, IL: American Osteopathic Association; 1997.
Bureau of Professional Education. Accreditation of Colleges of Osteopathic Medicine. Chicago, IL: American Osteopathic Association; 1998.
Bureau of Professional Education. Accreditation of Colleges of Osteopathic Medicine. Chicago, IL: American Osteopathic Association; 1999.
Bureau of Professional Education. Accreditation of Colleges of Osteopathic Medicine: COM Accreditation Standards and Procedures. Chicago, IL: American Osteopathic Association; 2000.
Bureau of Professional Education. Accreditation of Colleges of Osteopathic Medicine: COM Accreditation Standards and Procedures. Chicago, IL: American Osteopathic Association; 2002.
Commission on Osteopathic College Accreditation. Accreditation of Colleges of Osteopathic Medicine: COM Accreditation Standards and Procedures. Chicago, IL: American Osteopathic Association; 2018. https://osteopathic.org/wp-content/uploads/2018/02/com-continuing-accreditation-standards.pdf. Accessed August 3, 2019.
Commission on Osteopathic College Accreditation. Accreditation of Colleges of Osteopathic Medicine: COM Accreditation Standards and Procedures. Chicago, IL: American Osteopathic Association; 2004.
Commission on Osteopathic College Accreditation. Accreditation of Colleges of Osteopathic Medicine: COM Accreditation Standards and Procedures. Chicago, IL: American Osteopathic Association; 2005.
Commission on Osteopathic College Accreditation. Accreditation of Colleges of Osteopathic Medicine: COM Accreditation Standards and Procedures. Chicago, IL: American Osteopathic Association; 2006.
Commission on Osteopathic College Accreditation. Accreditation of Colleges of Osteopathic Medicine: COM Accreditation Standards and Procedures. Chicago, IL: American Osteopathic Association; 2007.
Commission on Osteopathic College Accreditation. Accreditation of Colleges of Osteopathic Medicine: COM Accreditation Standards and Procedures. Chicago, IL: American Osteopathic Association; 2008.
Commission on Osteopathic College Accreditation. Accreditation of Colleges of Osteopathic Medicine: COM Accreditation Standards and Procedures. Chicago, IL: American Osteopathic Association; 2009.
Commission on Osteopathic College Accreditation. Accreditation of Colleges of Osteopathic Medicine: COM Accreditation Standards and Procedures. Chicago, IL: American Osteopathic Association; 2010.
Commission on Osteopathic College Accreditation. Accreditation of Colleges of Osteopathic Medicine: COM Accreditation Standards and Procedures. Chicago, IL: American Osteopathic Association; 2011.
Commission on Osteopathic College Accreditation. Accreditation of Colleges of Osteopathic Medicine: COM Accreditation Standards and Procedures. Chicago, IL: American Osteopathic Association; 2012.
Commission on Osteopathic College Accreditation. Accreditation of Colleges of Osteopathic Medicine: COM Accreditation Standards and Procedures. Chicago, IL: American Osteopathic Association; 2013.
Commission on Osteopathic College Accreditation. Accreditation of Colleges of Osteopathic Medicine: COM Accreditation Standards and Procedures. Chicago, IL: American Osteopathic Association; 2014.
Cummings SR, Browner WS, Hulley SB. Conceiving the research question and developing the study plan. In: Hulley SB, Cummings SR, Browner WS, Grady DG, Newman TB, eds. Designing Clinical Research. 4th ed. Philadelphia, PA: Lippincott Williams & Wilkins; 2013:14-22.
Oxman AD, Sackett DL, Guyatt GH, et al. Users’ guides to the medical literature I. how to get started. JAMA. 1993;270(17):2093-2095. [CrossRef] [PubMed]
Richardson WS, Wilson MS, Nishikawa J, Hayward RS. The well-built clinical question: a key to evidence based decisions. ACP J Club. 1995;123(3): A12-A13. [PubMed]
Ericsson KA. Acquisition and maintenance of medical expertise: a perspective from the expert-performance approach with deliberate practice. Acad Med. 2015;90(11):1471-1486. [CrossRef] [PubMed]
Gifford KA, Fall LH. Doctor coach: a deliberate practice approach to teaching and learning clinical skills. Acad Med. 2014;89(2):272-276. [CrossRef] [PubMed]
Haidet P, Kubitz K, McCormack WT. Analysis of the team-based learning literature: TBL comes of age. J Excell Coll Teach. 2014;25(3-4):303-333. [PubMed]
Hassan S, Ibrahim MS, Hassan NG. The structural framework, implementation strategies and students’ perception of team-based learning in undergraduate medical education of a medical school in Malaysia. Educ Med J. 2018;10(1):55-68. doi: 10.21315/eimj2018.10.1.7
Jabbar HA, Jarrahi AH, Vamegh MH, Moh'd Alhabahbeh DA, Mahmoud NA, Eladl MA. Original Article: Effectiveness of the team-based learning (TBL) strategy on medical students’ performance. J Taibah Univ Med Sci. 2018;13:70-76. [PubMed]
Lewis CE, Chen DC, Relan A. Implementation of a flipped classroom approach to promote active learning in the third-year surgery clerkship. Am J Surg. 2018;215(2):298-303. [CrossRef] [PubMed]
Orr KK, Feret BM, Lemay VA, et al. Assessment of a hybrid team-based learning (TBL) format in a required self-care course. Curr Pharm Teach Learn. 2015;7(4):470-475. [CrossRef]
Shortt SE, Guillemette JM, Duncan AM, Kirby F. Defining quality criteria for online continuing medical education modules using modified nominal group technique. J Contin Educ Health Prof. 2010;30(4):246-250. doi: 10.1002/chp.20089 [CrossRef] [PubMed]
Kichler K, Kozol R, Buicko J, Lesnikoski B, Tamariz L, Palacio A. A structured step-by-step program to increase scholarly activity. J Surg Educ. 2014;71(6):e19-e21. doi: 10.1016/j.jsurg.2014.08.008 [CrossRef] [PubMed]
Penrose LL, Yeomans ER, Praderio C, Prien SD. An incremental approach to improving scholarly activity. J Grad Med Educ. 2012;4(4):496-499. doi: 10.4300/JGME-D-11-00185.1 [CrossRef] [PubMed]
Ramani S, Gruppen L, Kachur EK. Twelve tips for developing effective mentors. Med Teach. 2006;28(5):404-408. [CrossRef] [PubMed]
Parsonnet J, Gruppuso PA, Kanter SL, Boninger M. Required vs. elective research and in-depth scholarship programs in the medical student curriculum. Acad Med. 2010;85(3):405-408. [CrossRef] [PubMed]
Boninger M, Troen P, Green E, et al. Implementation of a longitudina mentored scholarly project: an approach at two medical schools. Acad Med. 2010;85(3):429-437. doi: 10.1097/ACM.0b013e3181ccc96f [CrossRef] [PubMed]
Jacobs CD, Cross PC. The value of medical student research: the experience at Stanford University School of Medicine. Med Educ. 1995;29:342-346. [CrossRef] [PubMed]
Laskowitz DT, Drucker RP, Parsonnet J, Cross PC, Gesundheit N. Engaging students in dedicated research and scholarship during medical school: the long-term experiences at Duke and Stanford. Acad Med. 2010;85(3):419-428. doi: 10.1097/ACM.0b013e3181ccc77a [CrossRef] [PubMed]
Kirkpatrick D. Great ideas revisited. techniques for evaluating training programs. revisiting Kirkpatrick's four-level model. Technol Devel. 1996;50(1):54-59.
Yardley S, Dornan T. Kirkpatrick's levels and education “evidence.” Med Educ. 2012;46(1):97-106. doi: 10.1111/j.1365-2923.2011.04076.x [CrossRef] [PubMed]
Reed DA, Beckman TJ, Wright SM, Levine RB, Kern DE, Cook DA. Predictive validity evidence for medical education research study quality instrument scores: quality of submissions to JGIM's Medical Education Special Issue. J Gen Int Med. 2008;23(7):903-907. [CrossRef]
Reed DA, Cook DA, Beckham TJ, Levine RB, Kerin DE, Wright SM. Association between funding and quality of published medical education research. JAMA. 2007;(9):1002. doi: 10.1001/jama.298.9.1002
Sullivan GM. Deconstructing quality in education research. J Grad Med Educ. 2011;3(2):121-124. doi: 10.4300/JGME-D-11-00083.1 [CrossRef] [PubMed]
Brannan GD. Growing research among osteopathic residents and medical students: a consortium-based research education continuum model. J Am Osteopath Assoc. 2016;116(5):310-315. [CrossRef] [PubMed]
Smith-Barbaro P, Fulda KG, Coleridge ST. A divisional approach to enhancing research among osteopathic family practice residents. J Am Osteopath Assoc. 2004;104(4):177-179. [PubMed]
Bull J. Literature, Tools and Curricula Supporting the Core Entrustable Professional Activities Pilot Study: Working Document. Washington, DC: Association of American Medical Colleges; 2017.
Hauser LJ, Gebhard GM, Blumhagen R, Carlson NE, Cabrera-Muffly C. Applicant characteristics associated with successful matching into otolaryngology. Laryngoscope. 2017;127(5):1052-1057. doi: 10.1002/lary.26236 [CrossRef] [PubMed]
Jang S, Rosenberg SA, Hullett C, Bradley KA, Kimple RJ. Beyond “charting outcomes” in the radiation oncology match: analysis of self-reported applicant data. Med Educ Online. 2018;23(1):1489691. [CrossRef] [PubMed]
National Resident Matching Program, Data Release and Research Committee: Results of the 2018 NRMP Program Director Survey. Washington, DC: National Resident Matching Program; 2018.
Rinard JR, Garol BD, Shenoy AB, Mahabir RC. Successfully matching into surgical specialties: an analysis of national resident matching program data. J Grad Med Edu. 2010;2(3):316-321. [CrossRef]
Weissbart SJ, Stock JA, Wein AJ. Program directors' criteria for selection into urology residency. Urology. 2015;85(4):731-736. [CrossRef] [PubMed]
Kreitz T, Verma S, Adan A, Verma K. Factors predictive of orthopaedic in-training examination performance and research productivity among orthopaedic residents. J Am Acad Orthop Surg. 2019;27(6):e286-e292. doi: 10.5435/JAAOS-D-17-00257 [CrossRef] [PubMed]
Rezeck I, McDonald RJ, Kallmes DF. Pre-residency publication rate strongly predicts future academic radiology potential. Acad Radiol. 2012;19(5):632-634. doi: 10.1016/j.acra.2011.11.017 [CrossRef] [PubMed]
Table 1.
Osteopathic Undergraduate Medical Education Standards Related to Research and Scholarly Activity From 1998-20198,9,13-29
  Image not available

a Standards are reproduced from the source documents. Minor punctuation adjustments have been made for consistent style.

Abbreviations: COCA, Commission on Osteopathic College Accreditation; COM, college of osteopathic medicine; OMM, osteopathic manipulative medicine; OPP, osteopathic principles and practice.

Table 1.
Osteopathic Undergraduate Medical Education Standards Related to Research and Scholarly Activity From 1998-20198,9,13-29
  Image not available

a Standards are reproduced from the source documents. Minor punctuation adjustments have been made for consistent style.

Abbreviations: COCA, Commission on Osteopathic College Accreditation; COM, college of osteopathic medicine; OMM, osteopathic manipulative medicine; OPP, osteopathic principles and practice.

×
Table 2.
Core Content for Independent Learning to Integrate the Principles and Practice of Scholarly Activity Into Undergraduate Medical Education
  Image not available

Abbreviations: CEBM, Center for Evidence-Based Medicine; FINER, feasible, interesting, novel, ethical, and relevant; IACUC, Institutional Animal Care and Use Committee; IRB, institutional review board; HIPAA, Health Insurance Portability and Accountability Act; PHI, protected health information; PICO, patient/population/problem, intervention/prognostic factor/exposure, comparison, and outcome; STROBE, strengthening the reporting of observational studies in epidemiology.

Table 2.
Core Content for Independent Learning to Integrate the Principles and Practice of Scholarly Activity Into Undergraduate Medical Education
  Image not available

Abbreviations: CEBM, Center for Evidence-Based Medicine; FINER, feasible, interesting, novel, ethical, and relevant; IACUC, Institutional Animal Care and Use Committee; IRB, institutional review board; HIPAA, Health Insurance Portability and Accountability Act; PHI, protected health information; PICO, patient/population/problem, intervention/prognostic factor/exposure, comparison, and outcome; STROBE, strengthening the reporting of observational studies in epidemiology.

×
Table 3.
Group Learning Exercises and Activities to Integrate the Principles and Practice of Scholarly Activity Into Undergraduate Medical Education
Topic Process and Assignments Learning Objectives
TBL Format
 1. Research Ethics and Informed Consent ■ Preparation: readings on research ethics violations
■ iRAT: MCQs on research ethics
■ gRAT: MCQs on research ethics
■ Appeal/Discussion: challenge and discuss answers to questions
■ Students will reinforce their comprehension of Module 2 with independent and group evaluations of research ethics violations.
 2. Evaluation of Research Articles ■ Preparation: study core content on navigating the scientific literature
■ gRAT: MCQs on strengths/weaknesses to study design and methodology, components of scientific article, measurement tools, and literature appraisal
■ Appeal/Discussion: challenge and discuss answers to questions
■ Students will apply their knowledge gained from Module 3 with independent and group assessments of published studies.
ALG Format
 1. Hypothesis ■ Discuss potential study areas and questions
■ Select one question and formulate a hypothesis
■ Students will work as a team to formulate a hypothesis.
 2. Background/Introduction ■ Write, review and evaluate background/ introduction for research study
■ Define and assign task distribution
■ Students will identify topics relevant to their proposed study and assign areas for literature review.
 3. Study Design ■ Define an appropriate category of study design as it relates to hypothesis ■ The team will apply their collective knowledge to identify an appropriate study design.
 4. Measurement and Methodology ■ Develop measurement and methodology for selected study
■ Define and assign task distribution
■ The team will apply their collective knowledge to select appropriate tools for measurement and other methods.
 5. Study Proposal ■ Discuss and refine study proposal ■ The team will draft and edit the study proposal.
 6. Limitations and Alternative Approaches ■ Discuss potential limitations, alternative approaches and impact of research study ■ The team will scrutinize their proposal by identifying limitations, proposing alternative approaches.
■ The team will provide a narrative for the potential importance and significance of their research.
 7. Research proposal ■ Finalize research proposal
■ Present research study design
■ Each student will gain experience developing and delivering an oral presentation.

Abbreviations: ALG, active learning group; gRAT, group readiness assurance test; iRAT, individual readiness assurance test; MCQs, multiple-choice questions; TBL, team-based learning.

Table 3.
Group Learning Exercises and Activities to Integrate the Principles and Practice of Scholarly Activity Into Undergraduate Medical Education
Topic Process and Assignments Learning Objectives
TBL Format
 1. Research Ethics and Informed Consent ■ Preparation: readings on research ethics violations
■ iRAT: MCQs on research ethics
■ gRAT: MCQs on research ethics
■ Appeal/Discussion: challenge and discuss answers to questions
■ Students will reinforce their comprehension of Module 2 with independent and group evaluations of research ethics violations.
 2. Evaluation of Research Articles ■ Preparation: study core content on navigating the scientific literature
■ gRAT: MCQs on strengths/weaknesses to study design and methodology, components of scientific article, measurement tools, and literature appraisal
■ Appeal/Discussion: challenge and discuss answers to questions
■ Students will apply their knowledge gained from Module 3 with independent and group assessments of published studies.
ALG Format
 1. Hypothesis ■ Discuss potential study areas and questions
■ Select one question and formulate a hypothesis
■ Students will work as a team to formulate a hypothesis.
 2. Background/Introduction ■ Write, review and evaluate background/ introduction for research study
■ Define and assign task distribution
■ Students will identify topics relevant to their proposed study and assign areas for literature review.
 3. Study Design ■ Define an appropriate category of study design as it relates to hypothesis ■ The team will apply their collective knowledge to identify an appropriate study design.
 4. Measurement and Methodology ■ Develop measurement and methodology for selected study
■ Define and assign task distribution
■ The team will apply their collective knowledge to select appropriate tools for measurement and other methods.
 5. Study Proposal ■ Discuss and refine study proposal ■ The team will draft and edit the study proposal.
 6. Limitations and Alternative Approaches ■ Discuss potential limitations, alternative approaches and impact of research study ■ The team will scrutinize their proposal by identifying limitations, proposing alternative approaches.
■ The team will provide a narrative for the potential importance and significance of their research.
 7. Research proposal ■ Finalize research proposal
■ Present research study design
■ Each student will gain experience developing and delivering an oral presentation.

Abbreviations: ALG, active learning group; gRAT, group readiness assurance test; iRAT, individual readiness assurance test; MCQs, multiple-choice questions; TBL, team-based learning.

×