From the beginnings of the AOA's certification program, the AOA has been concerned with the quality and standardization of its certifications. In 1992, the AOA Board of Trustees established a policy on uniform standards for all AOA certifying boards for test construction, evaluation, and use. In 1995, the Board of Trustees resolved that the BOS provide for rigorous statistical validation of the examinations of all AOA certifying boards. In 1997, formal approval was granted to the Guidelines for AOA Certification Examination Standards, which describe the psychometric standards that all AOA examinations must meet. From this document, a plan for a formal evaluation process to assess the validity and reliability of each AOA certification examination was devised and, in June 2000, the first official reviews of three AOA certifying boards were conducted. This review process brings value to osteopathic certification and to those holding it. The process is in essence a continuous improvement mechanism that benefits not only the certifying board but the current and future diplomates as well.
The format of the certifying board review is a self-study with completion of a report. These reports are then reviewed by the Standards Review Committee, which includes six elected members plus the BOS public member, two alternate members, and the AOA psychometrician (nonvoting). After the first cycle review, each board is reviewed again in 3 ½ years. After the second review, the time interval between reviews is 5 years. At each BOS meeting, three certifying boards are reviewed. Conjoint examinations are reviewed at the last meeting of the evaluation cycle.
After review of the certifying board's self-study report, the Standards Review Committee may make recommendation of compliance to the BOS, or the committee may defer making any recommendation at that time. Within 30 days of the Standards Review Committee's evaluation, the certifying board will receive a written evaluation, and examination activities found not in compliance with the standards will be clearly described to the board. In the first evaluation cycle, if the certifying board was not approved by the BOS as compliant, the board has 120 days from the date of the written evaluation to respond in writing with its action plan, specifying how the activities not in compliance will be addressed. For the second and later review cycles, the submission of an action plan is not required.
For the first review cycle, at the next meeting of the Standards Review Committee, the certifying board's action plan is studied, and any comments are forwarded to the board. In most cases, the board's action plan is formally accepted at this time, but in some cases, further information from the board may be requested. Within 1 year of the Standards Review Committee's appraisal of the action plan, the certifying board must submit an updated report to the committee with evidence that the board is in compliance with the standards.
For the second and later review cycles, if any examination activities are found not in compliance with the standards, the certifying board must submit an updated report within 1 year of the review date, along with acceptable evidence showing that all the examination activities are in compliance with the standards. The board must submit its updated report at least 45 days in advance of the meeting in which the Standards Review Committee will study it.
The Standards Review Committee will review the updated report and make a recommendation to the BOS for action. If the BOS finds the certifying board not in compliance, the BOS may impose a 1-year probation period and the practice affiliate will be notified of the board's probationary status. The failure of the certifying board to comply with the standards results in a BOS recommendation to the Board of Trustees that the certifying board's directors and/or members may be replaced and that certification activities may be suspended until the board demonstrates compliance with the standards. At the end of the probation period, the board must demonstrate compliance with the standards.
The Standards Review Committee began meeting in conjunction with the BOS meetings (held in January and June) in June 2000. The second review cycle began in January 2004. Subsequent review cycles start in January 2009 and will occur every 5 years after that. The schedule for the first and second review cycles is shown in
Figure 3.