Abstract
All postdoctoral training programs approved by the American Osteopathic Association are required to be part of an Osteopathic Postdoctoral Training Institution (OPTI) consortium. The author reviews recent activities related to OPTI operations, including the transfer the OPTI Annual Report to an electronic database, revisions to the OPTI Accreditation Handbook, training at the 2010 OPTI Workshop, and new requirements of the American Osteopathic Association Commission on Osteopathic College Accreditation. The author also reviews the OPTI accreditation process, cites common commendations and deficiencies for reviews completed from 2008 to 2010, and provides an overview of plans for future improvements.
The Osteopathic Postdoctoral Training Institution (OPTI) system, established by the American Osteopathic Association (AOA) in 1995, has grown considerably as a method for providing structure to osteopathic graduate medical education (OGME). Each OPTI is a community-based training consortium composed of at least 1 college of osteopathic medicine (COM) and 1 hospital accredited by the AOA Bureau of Healthcare Facilities Accreditation, the Joint Commission, or another recognized healthcare facility accrediting entity.
1 Additional hospitals and ambulatory care facilities may also join this core partnership. Since July 1999, OPTI partnerships are required for institutions conducting AOA-approved osteopathic postdoctoral training programs.
2 Nineteen OPTIs are accredited by the AOA Bureau of Osteopathic Education (BOE) (
Figure 1).
1
Recent Activities
During the past 2 years, the AOA Council on Osteopathic Postdoctoral Training Institutions (COPTI), under the leadership of its chairperson, D. Keith Watson, DO, initiated the electronic OPTI Annual Report, completed revisions to the OPTI Accreditation Handbook,
3 held the 2010 OPTI Workshop, and worked with OPTIs to assist member COMs meet new requirements of the American Osteopathic Association Commission on Osteopathic College Accreditation (AOA COCA).
OPTI Annual Report
The OPTI Annual Report is a survey that queries OPTIs on a series of operational issues. The report is based on the OPTI reviewers' inspection worksheet (ie, Standards Crosswalk). The format of the OPTI Annual Report was recently revised to make the data easily transferrable to electronic databases, improving the AOA's electronic data reporting capabilities.
4
OPTI Accreditation Handbook
Recent additions to the handbook include a process for filing corrective action plans for deficiencies and information for the self-study, a systematic self-review that OPTIs are required to complete prior to accreditation site review. As a result of discussions at the inaugural Osteopathic Heritage Foundation Medical Education Summit,
5 the AOA and COPTI use new metrics to conduct comparative analyses of OPTI operations. Data are gathered from the Standards Crosswalk and the OPTI Annual Report, which is transferable to electronic databases. The transferable data provide a more detailed picture of OPTI operations and their value.
2010 OPTI Workshop
Reviewers for OPTIs must review a training document every year and attend a training session every 3 years. During the 2010 OPTI Workshop in San Antonio, Texas, the AOA provided training for reviewers. The training included a mock site review with a sample self-study, which incorporated actual responses from past OPTI self-studies. Participants were organized into small groups that examined the self-study to determine if the standard was met. The workshop also included a presentation on OPTI internal communication and relationships with COMs.
New AOA COCA Requirements
As of 2010-2011, requirements of AOA COCA call for COMs to “utilize the clinical education capabilities of its OPTI partners to offer predoctoral clinical education clerkships to its students.”
6 All OPTIs are working with member COMs in meeting this standard by disseminating information and making the students within OPTIs aware of education clerkships.
OPTI Accreditation Process
The Council reviews OPTIs in intervals based on type of accreditation status, ranging from 1 to 5 years. For example, an OPTI that was awarded a 3-year accreditation in 2009 will undergo a visit in 2012. The Council conducts 2 types of reviews: full site reviews, which are structured on the accreditation cycle, and focused site reviews, which are focused on one or more specific concerns within the OPTI.
The AOA and COPTI provide a fair and balanced approach to the review process. Institutions are notified at least 1 year before a full site review. In addition, they are informed that a self-study will be due 60 days before the scheduled review. Starting in fall 2010, the AOA began sending a second letter to OPTIs to confirm the review dates. Assignment of reviewers begins once review dates have been set. The AOA provides resources to OPTIs that are preparing for the self-study and onsite review, including the
Standard Crosswalk; the AOA's OPTI Accreditation Handbook,
3 which includes a list of required documentation for the self-study; and staff assistance on the construction of the self-study.
Full site reviews last 1.5 days and include interviews with OPTI staff and the review of OPTI documents. The site team consists of 4 members (3 inspectors and 1 AOA staff member). At least 1 osteopathic physician reviews the clinical OPTI standards. Individuals who serve as OPTI reviewers are involved with OGME and are required to attend training before being approved. Review team members sign a conflict of interest form before each visit.
The assigned review team reviews the self-study, the last site review report, and past annual reports of the OPTI being reviewed. The team then meets by means of a teleconference before the OPTI visit to discuss the submitted documentation and to ensure logistical support is covered. The team also reviews a draft copy of the review schedule and recommends any necessary changes. The leader of the review team and the chief academic officer of the OPTI being reviewed sign an attestation form that requires all OPTI documents to be released for review at the end of the visit. The OPTI should ensure that documents are clearly and concisely organized according to OPTI standards to ensure that the review team has quick access to materials.
During the onsite visit, the review team awards the OPTI a certain number of points (ie, number of accreditation elements met) for each of the 8 standard sections (
Figure 2), for a maximum possible score of 64 points. Depending on the total score received, the OPTI can receive an accreditation status ranging from 1 year (45-47 points) to 5 years (61-64 points). As shown in
Figure 3, OPTIs with a total score of less than 44 points are placed on public probation. If, after 1 year, an onsite inspection reveals another total score of less than 44 points, the OPTI's accreditation is withdrawn.
At the end of each review, the review team and the OPTI have an exit conference. The goal of the exit conference is to provide a general review of the OPTI site visit and to outline the process of AOA accreditation. Teams can provide initial findings to the OPTI, but must stress that the findings are subject to change.
After the OPTI review, AOA staff works with the team to finalize a first draft of the site report. Once approved by the team, the report is forwarded to the OPTI for an optional written response. In instances in which the OPTI disagrees with the findings of the report, the OPTI has 15 business days to submit a written response to the AOA Division of Postdoctoral Training. The OPTI response must be based on errors in fact and provide all supporting documentation.
Once the OPTI's response to the initial site report has been received by the AOA, it is forwarded to the site team for review. The team meets by teleconference, reviews the initial findings and the OPTI response, and makes any necessary changes to the report. The report is then forwarded to the COPTI for review. A copy of the updated report is sent to the OPTI for its records.
The COPTI considers team reports and OPTI responses during COPTI meetings, which occur 3 times a year and each comprise 3 sessions. The review of this documentation occurs in the “Accreditation Session,” where COPTI hears testimony from OPTI representatives and the review team leader. The COPTI makes a final recommendation in the “Executive Session,” without the OPTI representatives or team leader present.
The COPTI forwards its final recommendation to the BOE for final action. The BOE's decisions on actions regarding OPTI accreditation actions may be appealed to the BOE appeals committee. After the BOE meeting, a final decision letter is mailed to the OPTI that outlines the accreditation award, commendations, and deficiencies.
Once the review is complete and the final accreditation letter has been sent to the OPTI, the AOA sends an electronic survey to the OPTI to evaluate both the OPTI review process and the accreditation process. The survey, which was initiated in the summer of 2009, has questions that determine the quality of the review process. For example, the survey queries the OPTI about the review team's professionalism and the quality of the communication between the AOA and the OPTI. The findings of these evaluations help the AOA and COPTI make informed decisions regarding future improvements to the OPTI review process.
If an OPTI is cited for a deficiency, it is required to submit a corrective action plan for correction of the deficiency to COPTI. The Commission reviews the plan at its next meeting and provides feedback to the OPTI. If COPTI accepts the plan, the OPTI reports on the implementation of the plan in its annual report. If the COPTI does not accept the plan, the OPTI is required to submit a new plan or a focused site visit may be scheduled.
OPTI Commendations
Reviewers of OPTI sites score results for AOA basic standards according to 3 categories: “not met,” “met,” or “met with excellence.” Institutions do not receive extra points for standards that are “met with excellence.” However, they do receive commendations for those standards. Multiple commendations can be awarded to a single OPTI. In the 14 full OPTI reviews conducted from 2008 to 2010, AOA basic standards were scored as “met with excellence” 22 times (
Table). One OPTI received the distinction of being awarded a “blue ribbon” status—a perfect score on the review. In the 14 reviews conducted from 2008 to 2010, COPTI awarded 1 commendation in prerequisites for accreditation (Standard Section A); 2 in organization, governance, and finance (Standard Section B); and 1 in program evaluation (Standard Section C). Six commendations were also awarded in research (Standard Section D), 5 were awarded in faculty and instruction (Standard Section E) and 0 were awarded in intern and resident status and services (Standard Section F). In addition, 4 commendations were awarded in curriculum (Standard Section G), and 3 commendations were awarded to the OPTIs for facilities (Standard Section H).
Table.
Commendations and Deficiencies Given to Osteopathic Postdoctoral Training Institutions for Compliance With AOA Standards, 2008 to 2010 (N=14)
|
Standard Section*
| | | | | | | |
|---|
Commendations or Deficiencies, No.
| A
| B
| C
| D
| E
| F
| G
| H
|
|---|
| ▪ Commendations | 1 | 2 | 1 | 6 | 5 | 0 | 4 | 3 |
▪ Deficiencies
| 3 | 26 | 23 | 17 | 8 | 8 | 6 | 0 |
Table.
Commendations and Deficiencies Given to Osteopathic Postdoctoral Training Institutions for Compliance With AOA Standards, 2008 to 2010 (N=14)
|
Standard Section*
| | | | | | | |
|---|
Commendations or Deficiencies, No.
| A
| B
| C
| D
| E
| F
| G
| H
|
|---|
| ▪ Commendations | 1 | 2 | 1 | 6 | 5 | 0 | 4 | 3 |
▪ Deficiencies
| 3 | 26 | 23 | 17 | 8 | 8 | 6 | 0 |
×
The AOA basic standards that were most often awarded as “met with excellence” were Standards D 9.2 and E 9.5:
D 9.2: “Each OPTI shall facilitate and provide research education, assistance and resources directly to interns, residents and institutions to encourage research and to meet the specialty college requirements.”
1 E 9.5: “Each OPTI and its member institutions and designated faculty, shall integrate osteopathic principles and practice (OPP) into all teaching services as appropriate and shall have designated faculty to provide OPP teaching.”
1
OPTI Deficiencies
In the 14 full OPTI reviews completed from 2008 to 2010, COPTI cited 91 deficiencies, or AOA basic standards found as “not met” (
Table), with multiple deficiencies cited to a single OPTI. Three deficiencies were found for prerequisites of accreditation (Standard Section A); 26 for organization, governance, and finance (Standard Section B); and 23 in program evaluation (Standard Section C). Seventeen deficiencies were also cited in research (Standard Section D), and 8 deficiencies were cited in faculty and instruction (Standard Section E) and in intern and resident status and services (Section F). In addition, 6 standards were cited in curriculum (Standard Section G) and no deficiencies were cited for facilities (Standard Section H).
Among these deficiencies, the following AOA basic standards were most often cited as “not met”:
B 9.9: “Each OPTI shall document in its self-studies and annual reports that ongoing or periodic strategic planning is occurring to meet its stated mission, objectives, and outcome measurements.”
1 B 9.16: “Each OPTI shall define a financial plan and budget that is linked to its strategic plan, annual educational plan and outcomes, and that reflects profit/loss detailed allocations to all members.”
1 C 9.4: “The responsibility of the OGME committee shall be clearly stated. The OGME shall document its effectiveness through outcome measures consistent with the OPTI strategic plan goals.”
1 D 9.5: “The OPTI shall seek funding, either externally generated or internally budgeted funding (not only from the COM), to provide for OPTI-wide or program specific research efforts of its member faculty and residents or students.”
1
Future of the OPTI Accreditation Process
For 2011, COPTI is planning an initiative to revise the OPTI standards and provide several major recommendations on OPTI functioning. These recommendations will be based on the results of an OPTI study conducted by the outside consulting firm Michael Hamm and Associates. The OPTI study is jointly sponsored by the AOA and the American Association of Colleges of Osteopathic Medicine. The recommendations will also consider the number of deficiencies and commendations according to the collected OPTI review data.
In addition to initiating a more transparent review process that is founded upon effective communication and thorough procedures, the AOA will begin to collect information on individual reviewers and specific review teams in regards to the type of deficiencies and commendations cited during reviews. With this information, the AOA will be able to examine if a particular reviewer has a tendency to cite one standard over another as a deficiency, which will assist in creating more individualized reviewer training modules. The AOA continues to compile statistics on the most cited standard deficiencies and commendations. These data are being used by the AOA and COPTI to assist in the development of curricula that will improve OPTI operations.
In 2011, the OPTI Workshop will concentrate on assisting OPTIs in various operations. It will also focus on providing a clear mission and vision statement that captures the purpose and responsibilities of OPTIs. The AOA also will develop examples of various OPTI policies and procedures to assist OPTIs in meeting accreditation standards.
Conclusion
The COPTI's review of and subsequent conversations on the role of OPTIs and their effectiveness are expected to bring a robust discussion in 2011. With these discussions, the COPTI can begin to define the evolving role of OPTIs in OGME.
Financial Disclosures: None reported.
American Osteopathic Association.
Accreditation Document for Osteopathic Postdoctoral Training Institutions and the Basic Document for Postdoctoral Training Programs. Chicago, IL: American Osteopathic Association; 2010.
https://www.do-online.org/pdf/sir_postdoctrainproced.pdf. Accessed March 3, 2011.
Meyer CT, Portanova R, Riley C, et al.
The anatomy of an OPTI: Part 2. The CORE system. Ohio Osteopathic Hospital Association. Ohio Association of Osteopathic Medical Directors. Ohio Osteopathic Association [abstract].
J Am Osteopath Assoc. 1997;97(11):686.
http://www.jaoa.org/cgi/content/abstract/97/11/686. Accessed March 3, 2011.
Duffy T, Martinez B. Examining OPTI operations with a new light. J Am Osteopath Assoc. 2010;110(3):160-167.
Watson DK, Nichols KJ. Medical education summits: building a solid foundation for the future of the osteopathic medical profession.
J Am Osteopath Assoc. 2008;108(3):110-115.
http://www.jaoa.org/cgi/content/full/108/3/110. Accessed March 3, 2011.