From the early beginnings of the AOA's certification program, the AOA has been concerned with the quality and standardization of its certifications. In 1992, the AOA Board of Trustees established a policy on uniform standards for all AOA certifying boards for test construction, evaluation, and test use and, in 1995, resolved that the BOS provide for rigorous statistical validation of the examinations of all AOA certifying boards. In 1997, the Guidelines for AOA Certification Examination Standards was formally approved, which describes the psychometric standards that all AOA examinations must meet. From this document, a plan for a formal evaluation process to assess the validity and reliability of each AOA certification examination was devised and, in June 2000, the first official reviews of three AOA certifying boards were conducted. This is an area that certainly brings value to osteopathic certification and those holding it. The review process is in essence a continuous improvement mechanism that in the end benefits not only the certifying board but the current and future diplomates as well.
The format of the certifying board review is a self-study with completion of a report. These reports are then reviewed by the Standards Review Committee, which includes six elected members plus the BOS public member, two alternate members, and the AOA psychometrician (nonvoting). After the first cycle review, each board is reviewed again in 3.5 years. After the second review, the time interval between reviews is 5 years. At each BOS meeting, three certifying boards are reviewed and conjoint examinations are reviewed at the last meeting of the evaluation cycle.
After review of the board's self-study report, the Standards Review Committee may make recommendation of compliance to the BOS, or they may defer making any recommendation at that time. Within 30 days of the Standards Review Committee's evaluation, the certifying board will receive a written evaluation, and examination activities found not in compliance with the standards will be clearly described to the Board. In the first evaluation cycle if the board was not approved by the BOS as compliant, the certifying board has 120 days from the date of the written evaluation to respond in writing with their action plan, specifying how the activities not in compliance will be addressed. For second and later review cycles, the submission of an action plan is not required.
For the first review cycle, at the next committee [D2]meeting, the board's action plan is studied and any comments are forwarded to the certifying board. Usually, the board's action plan is formally accepted at this time or further information from the board may be requested. Within 1 year of the committee's appraisal of the action plan, the board must submit an updated report to the committee with evidence that shows the board is in compliance with the standards.
For second and later review cycles, if any examination activities are found not to be in compliance with the standards, the board must submit an updated report within 1 year of the review date, along with acceptable evidence that shows that all the examination activities are in compliance with the standards. The board must submit its updated report at least 45 days in advance of the meeting when the committee will study it.
The committee will then review the updated report and make a recommendation to the BOS for action. If the BOS action finds the board not in compliance, the BOS may impose a 1-year probation period and the practice affiliate will be notified of the board's probationary status. Certifying board failure to comply with the standards results in a BOS recommendation to the Board of Trustees that the certifying board's directors and/or members may be replaced and certification activities may be suspended until the board demonstrates compliance with the standards. At the end of the probation period, the board must then demonstrate compliance with the standards.
The Committee began meeting in conjunction with the BOS meetings (January and June) starting in June 2000; the second review cycle began in January 2004. Subsequent review cycles start in January 2009 and occur every 5 years after that. At each meeting, three certifying boards will be reviewed. The schedule for the first and second evaluation is shown in
Figure 2.
In addition to ensuring the validity of the AOA certification process, the BOS is also beginning to explore how to enhance the certification process through incorporation of the seven core competencies into the process. The seven core competencies, already espoused by AOA-approved residency programs, are, and will be items of discussion at near future BOS leadership meetings.