Abstract
The importance of increasing scholarly activity has been highlighted among residency programs currently accredited by the American Osteopathic Association (AOA) to ensure a smooth transition to the single accreditation system. The Scholar 7 program, a series of seven 2-hour sessions, was created to aid faculty and residents in the pursuit of scholarly work and to facilitate change in an entire community hospital system's environment by creating a self-replicating scholarly culture in a timely and cost-efficient manner. Skills were taught by means of preparation and submission of a research protocol to the institutional review board (IRB) along with grant proposals. The authors tracked scholarly work, IRB submissions, and grants awarded to participants during the 2015-2016 academic year. The results were compared in a post-hoc fashion with previous classes since 2007-2008 within the same hospitals system. The Scholar 7 program successfully aided faculty in achieving their required pursuit of scholarly work in 8 months. This program has the potential to help AOA-focused residency programs meet the scholarly requirements of the Accreditation Council for Graduate Medical Education.
In February of 2014, the American Osteopathic Association (AOA) along with the American Association of Colleges of Osteopathic Medicine (AACOM) and the Accreditation Council for Graduate Medical Education (ACGME) agreed to form a single accreditation system in the United States.
1 By 2020, AOA-accredited residency programs that wish to continue in the training of residents must achieve pre-accreditation by the ACGME and will be required to demonstrate scholarly activity.
2 However, the difficulty of intensifying scholarly activity among osteopathic-focused community residency programs has, in our experience, been ill-defined and exaggerated.
In the 1990s, Boyer et al redefined scholarly work into 4 components (discovery, integration, application, and teaching) to better align with contemporary ideas.
3,4 These components were soon accepted by the Carnegie-Melon Foundation, which challenged academia. This challenge was accepted by the ACGME, which implemented the 4 components into its accredited residency programs. The AOA was less aggressive in implementation of the components within its residency programs. The fact that this particular challenge did not translate sufficiently into the AOA's postgraduate environment has played a crucial role in the amount of scholarly activity produced yearly when compared with the ACGME postgraduate environment.
Reports have identified the need to increase the quantity and quality of scholarly activity in the osteopathic medical community.
5,6 Clark and Blazyk
5 proposed a series of actions needed to accomplish this goal, which included the need to train faculty. However, an identifiable education plan on how to train faculty to accomplish increased scholarly activity was not offered.
5
Program Development
To address the need to create a culture of scholarly activity in the osteopathic community, the Scholar 7 program
7 was developed to be implemented among community hospitals using a mentor to mentee approach. The process of program development began in 2007 as a series of lectures to incoming residents and culminated in what, as of 2015, is now called Scholar 7. The program comprises 7 interactive workshop sessions that take place over a period of 6 to 8 months. The goal of this initiative was to enable the postgraduate training programs at community hospitals to increase their scholarly activities, to apply for grant funding, to reach scholarly requirements in a timely and cost-effective manner, and to train faculty in each aspect of scholarly activity.
Scholar 7: 2015-2016 Program
The process began in fall 2015 with an e-mail sent by the director of graduate medical education (GME) notifying University Hospitals Regional Hospitals (UHRH) departments of medical education and their respective community-based residency programs to participate in a faculty development seminar. A central venue was chosen and contained all the necessary equipment to make this process flow smoothly, including adequate workspace, wireless internet, a projector, and a white screen. Scholarly skills were taught by means of preparation and submission of a research protocol submitted to the institutional review board (IRB) along with grant proposals. All participants were required to complete the Collaborative Institutional Training Initiative before IRB submission.
Each of the seven 2-hour sessions was performed after work hours to accommodate multiple specialties. Participating specialties included allergy and immunology, emergency medicine, family medicine, internal medicine, physical medicine and rehabilitation, obstetrics and gynecology, orthopedic surgery, podiatry, sports medicine, and traditional internship programs, with each specialty representing an individual group. The participants included the directors of medical education, program directors, faculty members, and residents.
The first 6 sessions were held in a multi-specialty group setting, and the last session was held privately with individual groups. The Scholar 7 core mentors, who designed the program, led the workshops. Perceived barriers to engaging in research were accessed in an open forum at the beginning of session 1. Sessions 2 through 6 were held about 4 weeks apart and conducted in a similar format, in which each group presented their progress to the rest of the participants for feedback. Session 7 was held 4 to 8 weeks after session 6. Scholarly work, IRB submissions, IRB approval/waivers, and grants awarded to participants during the postgraduate year 2015-2016 were compared with classes since 2007-2008 within the same hospitals system.
Session 1: Demystifying Scholarly Work
The first session was designed to give the participants a preview of sessions 2 through 7 on a smaller scale, to make the process more fun and less stressful. The goal of session 1 was to address the misconceptions and perceived barriers associated with scholarly activity expressed among participants, including the idea that the process of creating a research protocol is too tedious, time consuming, and overwhelming; that scholarly activity can only be done in large academic centers; and that research has to be done on high-impact topics. The session began with a brief discussion outlining topic headings and formatting for clinical research proposals and grants. This traditional didactic medical forum quickly transcended into a 7-step activity meant to mirror and introduce each of the following 6 sessions. On completion of each step, the groups were required to present their work to the rest of the participants for feedback.
▪ Step 1: Development of a Hypothesis. A batch of small toys, including a dart gun, whoopee cushion, and balsa wood airplane, were placed on a table in the middle of the room. Each faculty member was asked to select an item and return to his or her respective tables. They were instructed to develop a hypothesis that could be tested immediately in the room. For example, if the faculty member had chosen the whoopee cushion, their hypothesis could be, “Every person who sits on the whoopee cushion will inspire a laugh by the audience.”
▪ Step 2: Development of an Introduction. Next, participants were asked to write an introduction of 3 lines regarding their chosen item. The third line of the introduction had to conclude with a statement including their hypothesis. This statement would prove or disprove the established hypothesis.
▪ Step 3: Development of Specific Aims. The faculty members were asked to develop 3 specific aims in accordance with their hypothesis. Specific aims had to address the following parameters: state how the hypothesis will be investigated, define the population, and describe data collection and analysis. Each specific aim required a small 2-line paragraph describing the general aim.
▪ Step 4: Development of Preliminary Data. A paragraph for each sentence in the 3-sentence introduction was created. The data for each paragraph in the first session was obtained from the internet, the item's container, or a collection of observations made using the item. This information established a background for their hypothesis and provided a basis for conducting their research.
▪ Step 5: Development of Materials and Methods. During this step, the participants worked on developing each specific aim into a paragraph describing in detail the process of identification of the chosen population, collection of the data from the population, and evaluation of collected data.
▪ Step 6: Development of Conclusion. The conclusions developed by participants were 3 lines long, similar to the introduction but emphasizing the outcomes of the methods. The paragraph ended in summation of the hypothesis.
▪ Step 7: IRB Submission. In this step, the entire research protocol was compiled, completed, and streamlined for submission to the IRB.
On completion of session 1, participants were assessed on their comfort level moving forward as mentors to fellows, residents, and interns. Their thoughts were solicited on the barriers and misconceptions of research that were mentioned at the beginning of the session. The first session concluded with an assignment that was to be completed before session 2. The participants were instructed to choose a disease, procedure, or diagnosis that was common within or specific to their medical specialty to be the subject of their final project. They were instructed to choose a team of 1 to 3 fellows, residents, or interns to complete this project. The teams, once formed, were required to have a research meeting on their own between each session to complete assigned tasks and monitor their progress. Faculty members were also advised to set a timeline for the final project by determining a meeting or conference to submit their research to.
Session 2: Development of a Hypothesis
During this session, each faculty member and their team refined a hypothesis pertaining to their specific clinical research project using the skills that they learned in step 1 of session 1. As in session 1, the teams presented their work to the rest of the participants and received feedback. They were also asked to perform an extensive literature review, address any conflicts found from the literature search (eg, the project has already been done, critical flaws in the hypothesis), and choose 3 key pieces of literature pertaining to their hypothesis with their team before session 3. Suggested literature search engines included PubMed, OVID, and OSTMED.
Session 3: Development of an Introduction
The third session was shaped around the information taught in step 2 of session 1, which involved creating an introduction. The format of the 3-sentence introduction began with a general statement regarding the hypothesis, a leading linking statement, and a concluding statement, using the references that the teams gathered before the session. The assigned task for the teams to complete before session 4 was to prepare 3 specific aims regarding the hypothesis using the information they learned in step 3 of session 1.
Session 4: Development of Specific Aims
As they were taught in step 3 of session 1, the aims had to address how they were going to investigate the hypothesis, define the population, and describe their methods of data collection and analysis. Each aim required a short paragraph to describe it. The teams presented, received feedback on, and refined their aims during this session. The participants were assigned to begin their preliminary data section by developing 3 paragraphs, 1 for each of the sentences in the introduction, and supporting them with references.
Session 5: Development of Preliminary Data
In session 5, the teams were given feedback from Scholar 7 mentors on the 3 paragraphs they were assigned to work on after the previous session, and the teams then refined and presented the work to the rest of the teams to receive additional feedback. The assigned task to be completed for session 6 was to create a methods section by expanding the 3 specific aims into 3 that outlined how the aims would be tested and measured.
Session 6: Development of Materials, Methods, and Conclusion
Session 6 was conducted similar to step 5 and 6 of session 1. In this session the teams worked on refining previously drafted methods and describing the process of identification of the chosen population, collection of the data from the population, and evaluation of collected data in detail. This was the last group session that included all teams in which constructive feedback was provided for each research project. The task assigned to be completed before scheduling the final session was to edit any spelling or grammar errors, ensure the general flow of their final research protocol, and then use it as a template to be shortened to make a consent form, if needed.
Session 7: IRB Submission
The final version of each team's protocol was uploaded onto the IRB website for submission. Because each study was different and required tailored guidance through the IRB software, this session was held separately for each group. Teams had the option of scheduling additional meetings with the Scholar 7 core mentors to address concerns indicated by academic departmental or IRB reviewers; grant writing; data collection, processing, and interpretation; and manuscript preparation. E-mail and telephone support was also available for specific questions.
Program Successes
Since the creation of the initial Scholar program in 2007 (now known as the Scholar 7 program) at UHRH, there has been an increase in the number of scholarly projects presented at UHRH's Annual Research Symposium (
Figure 1). The first UHRH Annual Research Symposium held in 2008 had 13 scholarly projects presented. Eight years later, 73 scholarly projects were presented. Additionally,
Figure 2 demonstrates an increase in the number of peer-reviewed publications, national poster presentations, and oral presentations in the Allergy and Immunology fellowship program, which had been using a form of the Scholar 7 program since 2009.
In 2015, the faculty members that took part in session 1 noted that they felt more comfortable teaching their team how to develop a hypothesis and guide them through the process of creating a research proposal. By the end of session 6, the teams became familiar with the process of developing study protocols for IRB submission regardless of the type of study.
The 2015 Scholar 7 sessions included 45 participants from 10 specialty groups. Each group consisted of 1 to 2 advisors and 1 to 3 residents. Five of the 10 groups had submitted research protocols to the IRB, received subsequent approvals or waivers, and began data collection. One of these 5 groups stopped attending sessions and attempted their first IRB submission on their own. This group later contacted the program for example manuscripts and help with navigating the IRB's software for their final submission. The remaining 5 groups that have not yet submitted a protocol to an IRB made a similar choice to finish on their own. To date, their projects have not been started or submitted to the IRB. One of these groups has reported that they are close to submission. All faculty involved earned their pursuit of scholarly activity required by ACGME-accredited programs in only 8 months, regardless of IRB submission. Among the 5 groups that completed the Scholar 7 program, 8 research projects were created, including questionnaires, a case report, and retrospective and prospective studies. The overall grant funding for research projects at UHRH doubled from the postgraduate years 2015-2016 to 2016-2017. Results from 3 of the 8 projects have already been submitted as abstracts to national conferences this coming year.
Discussion
Scholarly work continues to be an important component of postgraduate education. The 4 components—discovery, integration, application, and teaching—have since been the mainstay for ACGME training programs throughout the United States. Training programs that have academic affiliations with larger universities with more abundant resources have found it easier to develop venues for scholarly work.
5 The osteopathic medical profession has a greater percentage of residency programs at smaller institutions based in community settings.
8 The Scholar 7 program is designed to address these specific cultural needs in osteopathic postgraduate programs starting with faculty development.
According to Ray et al,
9 barriers regarding scholarly activity include lack of knowledge about institutional research programs, confidence in medical writing skills, time, and understanding of the value of research. Although training programs in community-based settings have scholarly activities, we believe it has been more arduous for such programs because of the high-volume nature of its clinical practice, limited research-orientated faculty members, and various on-site research resources. However, what is seen as an obstacle for community-based scholarly work is also one of its largest assets: access to high patient volume.
Electronic medical records can help produce an abundance of scholarly work in a retrospective and prospective manner. In addition, with advances in telecommunications, collaborative projects and resources are more readily obtainable. An important reason for suboptimal scholarly work in smaller community-based hospitals may simply be inadequate nurturing of simple research skill sets and lack of familiarity with available research-related resources. In developing the current Scholar 7 program, the difference between the level of success of the Allergy and Immunology Fellowship program with respect to peer-reviewed scholarly work (
Figure 2) and the other postgraduate programs (
Figure 1) was the level of faculty involvement in guiding the research projects. This involvement led to the telescoping mentorship project within the Scholar 7 program and successfully produced 4 clinical research projects among 4 separate disciplines that have not regularly engaged in this level of research.
The potential effectiveness of establishing a culture of scholarly work can be seen in
Figure 1 when comparing the increase in presentations from 2015 to 2016 (49 to 73, respectively). This increase marks the largest growth between years since 2007. Importantly, the additional time commitment the Scholar 7 program asked of the resident participants did not negatively affect the number of presentations at the annual symposium. Furthermore, none of the presentations were on projects designed through the Scholar 7 program. This growth in the face of more work may be initial signs of increased proficiency in the skill set and successfully establishing expectations of a scholarly work culture within the hospital system. Even the medical librarian has noted in graduate medical committee meetings that for the first time, incoming residents (2016-2017) were requesting to meet before their already-scheduled and mandatory medical library and literary search orientation.
Other processes have been suggested in the literature to encourage scholarly work in the postgraduate environment. One such method by Nocera et al
10 showed a “Think, Do, Write” process. This model assumes that all participants have the skill set to proceed to completion of scholarly work and would not be as effective in most osteopathic residency programs. Other models proposed a departmental process for scholarly work that involved a core group of medical education research groups.
11 This group had facilitators that assisted in the development of scholarly work. Although the process was successful, it lacked the structure needed to change a culture within an entire hospital system and, in some cases, required funding from the begining.
12,13
Mayo et al
14 showed that a scholarly environment requires key foundational requirements: communication of expectations and development of a robust program structure, dedicated protected time, research curriculum, programmatic support, mentorship, oversight, accountability, and tracking of accomplishments. The Scholar 7 process satisfies the above fundamentals, including accountable protected time by scheduling a weekly research meeting that would be held during a traditional didactic session.
Brandon et al
15 suggested that a lack of time, confidence, and optimal writing practices can inhibit the growth of scholarly work of clinical faculty within a department. They demonstrated that organized writing circles can encourage clinical faculty to proceed beyond the first draft. All 7 sessions of the Scholar 7 incorporated a “writing circle” by providing large group and mentored feedback.
The biggest limitation to the Scholar 7 program is that the approach is based on volunteerism from both the core Scholar 7 program mentors and the participants. This point is well illustrated by the fact that none of the groups that stopped participating in the Scholar 7 process completed an IRB submission without returning to the program compared with 100% IRB submission for those that finished the process. The 5 groups that did not complete the program or go on to finish cited work and family obligations as the main reasons for not finishing. It should be noted that several of the groups that stopped participating have faculty that have successfully submitted IRBs and have had peer-reviewed publications within the past 5 years. For groups attempting their first IRB submissions, it is clear, in our experience, that without the aid of mentors who have experienced the process, there is a significant failure rate and delay in submissions. The volunteer nature of participation is important because producing quality scholarly work of this kind ends up being the result of genuine interest and personal drive. Other forms of valuable scholarly activities are available that do not require IRB review. The Scholar 7 program's focus on IRB-proposal manuscripts and grant writing is because these skill sets can easily be used in all forms of scholarly activity.
Conclusion
The Scholar 7 program is a step-by-step process designed to stimulate the culture of scholarly activities in AOA-focused residency programs that currently may lack a research curriculum and have inadequate grant funding. The challenge that remains for this program is to determine the level and frequency of continued intervention by the core Scholar 7 mentors to keep the initial embers of scholarly work burning. This continued support is vital to ensure that a self-replicating scholarly culture has completely taken root and is utilizing the skill sets effectively and addressing unique barriers that may develop and thus require additional sessions.
Author Contributions
All authors provided substantial contributions to conception and design, acquisition of data, or analysis and interpretation of data; all authors drafted the article or revised it critically for important intellectual content; authors gave final approval of the version of the article to be published; and all authors agree to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.
Editor's Note: The Scholar 7 program can be accessed on the following webpages:
References
The Accreditation Council for Graduate Medical Education. Executive summary of the agreement among ACGME, AOA, and AACOM. Chicago, IL: Accreditation Council for Graduate Medical Education; 2014.
Boyer EL. Scholarship Reconsidered: Priorities of the Professoriate. Princeton, NJ: Carnegie Institute for the Advancement of Teaching; 1990.
Toledo-Pereyra LH. In the pursuit of scholarly activities.
J Invest Surg. 2010;
23(6):335-341. doi: 10.3109/08941939.2010.534002
[CrossRef] [PubMed]Clark BC, Blazyk J. Research in the osteopathic medical profession: roadmap to recovery.
J Am Osteop Assoc. 2014;
114(8):608-614. doi: 10.7556/jaoa.2014.124
[CrossRef] Walkowski SA, Eland D, Hain S, Byron S. Research in the osteopathic medical profession: roadmap to recovery-conundrums in osteopathic research require consensus and collaboration.
J Am Osteopath Assoc. 2015;
115(2):70-71. doi: 10.7556/jaoa.2015.016
[PubMed]Gevitz N. The D.O.‘s: Osteopathic Medicine in America. Baltimore, MA: The Johns Hopkins University Press; 1991.
Basu R I, Henry TL, Davis W, Alam J, Amedee RG, Pinsky WW. Consolidated academic and research exposition: a pilot study of an innovative education method to increase residents’ research involvement.
Ochsner J. 2012;
12(4):367-372.
[PubMed]Nocera R, Ramoska EA, Hamilton RJ. Building a resident research program in emergency medicine.
Intern Emerg Med. 2016;
11(2):245-248. doi: 10.1007/s11739-015-1354-x
[CrossRef] [PubMed]Perry M, Hopson L, House JB, et al Model for developing educational research productivity: the medical education research group.
West J Emerg Med. 2015;
16(6):947-951. doi: 10.5811/westjem.2015.9.27306
[CrossRef] [PubMed]Brannan GD. Growing research among osteopathic residents and medical students: a consortium-based research education continuum model.
J Am Osteopath Assoc. 2016;
116(5):310. doi: 10.7556/jaoa.2016.06112
[CrossRef] [PubMed]Nottingham KL, Rush LJ, Beverly EA. Building primary care research capacity in a college of osteopathic medicine.
J Am Osteopath Assoc. 2016;
116(5):278. doi: 10.7556/jaoa.2016.05513
[CrossRef] [PubMed]Mayo MJ, Rockey DC. Development of a successful scholarly activity and research program for subspecialty trainees.
Am J Med Sci. 2015;
350(3):222-227. doi: 10.1097/maj.000000000000048914
[CrossRef] [PubMed]Brandon C, Jamadar D, Girish G, Dong Q, Morag y, Mullan P. Peer support of a faculty “writers’ circle” increases confidence and productivity in generating scholarship.
Acad Radiol. 2015;
22(4):534-538. doi: 10.1016/j.acra.2014.12.006
[CrossRef] [PubMed]