Since 2008, The Journal's annual osteopathic medical education theme issue has included a series of articles that addresses major areas of the American Osteopathic Association Commission on Osteopathic College Accreditation (AOA COCA). The series, prepared by the AOA's Department of Accreditation, has focused on several topics that may be of interest to the profession and the many stakeholders in the accreditation process. In the March 2008 article¹ we focused on key changes in the accreditation process for colleges of osteopathic medicine (COMs) in the United States² and on the recent growth of new COMs. In March 2009, we addressed AOA COCA procedures regarding “substantive changes” to institutional missions and programs—the chief example of such change currently being class size increases.³ In March 2010, we discussed AOA COCA's process of continual evaluation of COMs.⁴ In April 2011, we discussed the AOA COCA's process of identifying and training the evaluators who conduct the site visits to COMs.⁵ 

The AOA COCA is the only accrediting agency for predoctoral osteopathic medical education and, as such, is recognized by the US Department of Education (USDE). Like all accrediting agencies who seek to maintain their recognition with the USDE, the AOA COCA is also required to adhere to all federal laws^6,7 and regulations⁸ associated with recognition of accrediting agencies. Included in the regulations are requirements for the evaluation of standards and procedures.⁹ The Commission has established protocols for the review and revision of its standards and procedures.¹⁰ As part of its stated goals and objectives, AOA COCA serves the public by establishing, maintaining, applying, and reviewing its standards and procedures published in Accreditation of Colleges of Osteopathic Medicine: COM Accreditation Standards and Procedures,² which can be found on AOA COCA Web site, http://www.aoacoca.org. In the present article, we discuss AOA COCA's established processes for systematic and comprehensive review of its standards and procedures of accreditation. 

The Commission acknowledges the need for constant and systematic review of its standards, processes, and procedures. In practice, this method of review takes on a dual nature, with 2 processes running in parallel. The first process is the comprehensive, systematic review of all standards and is initiated with survey analysis. We describe this process in greater detail in a separate section of the present article. 

The second process is the ongoing, more selective review of specific standards and procedures. This process arises from changing factors in the medical education environment and from specific requests for revisions to a standard. Changes to the Accreditation of Colleges of Osteopathic Medicine: COM Accreditation Standards and Procedures may be proposed by stakeholders, including COM presidents, deans, faculty, and students; site visit team evaluators; members of osteopathic specialty colleges and osteopathic licensure boards; hospital administrators; the AOA Board of Trustees; and AOA COCA members. Proposed changes are initially considered by AOA COCA staff and then forwarded to AOA COCA for discussion. 

When AOA COCA receives proposed changes to its standards, it begins a standards review by forwarding proposed changes to the Standards Review Committee (SRC) for review and recommendation. The SRC is a standing committee of AOA COCA that is responsible for reviewing all proposed changes to the standards and procedures on a continuous basis. The SRC includes a chairperson, committee members who are commissioners on AOA COCA and appointed by the AOA COCA chair, the AOA COCA chair as an ex officio member, and AOA COCA staff as nonvoting members. The SRC meets by teleconference at the direction of the AOA COCA when new standards and procedures are proposed or when clarification of current standards and procedures is needed. 

The Commission receives the SRC's proposed revisions, reviews the revisions at its December meeting, and determines whether to release them for public commentary. If the proposed changes are released, stakeholders and the public are notified of the proposed changes, which are posted for public commentary on the AOA COCA Web site for 45 days. Members of AOA COCA review the public commentary prior to deciding whether to approve the proposed changes, a decision that will take place at the April/May meeting of AOA COCA. Changes to the standards or procedures usually become effective on July 1 of that same calendar year. 

Some of these concerns may be implemented into standards quickly. For example, in January 2011, AOA COCA student services evaluators met at the AOA COCA Committee on College Accreditation Training Workshop to review Standard 5: Students. As a result of this meeting, the evaluators proposed changes to this standard, which were reviewed in April 2011 at a Standards Forum (discussed in the following section) and then by the SRC. The proposed changes were accepted by AOA COCA at its May 2011 meeting and became effective on July 1, 2011. Such proposed changes may also be studied further by the SRC or addressed during the periodic comprehensive survey review. 

A key part of the public commentary process has been the Standards Forum. In recent years, AOA COCA has hosted a Standards Forum for stakeholders to openly debate proposed changes to the standards after written public comments have been received and catalogued. The stakeholders exchange ideas, recommend and fine-tune language, and try to reach consensus in their recommendations to AOA COCA on the proposed standards changes. The results of the Standards Forum, as well as any public commentary, are considered by the SRC before it prepares its final recommendations to AOA COCA for a decision on any standard changes. 

This process began in December 2007, when AOA COCA reviewed proposed revisions from multiple sources to accreditation standards for clinical education. Members of AOA COCA directed the SRC to invite COMs and other stakeholders to participate in a Standards Forum to discuss the proposed standards changes. The first Standards Forum took place in September 2008. Participants of the Forum proposed revisions to the standards changes, which were forwarded to AOA COCA for consideration. Members of AOA COCA determined further discussion was needed on the proposed standards changes in clinical education and called for the SRC to host another Standards Forum for all stakeholders. Stakeholders were informed of the Standards Forum by e-mail and received the updated proposed standards changes to be discussed. Thirty-two stakeholders participated in the Standards Forum, which was held by teleconference. The SRC presented its recommendations based on the outcome of the teleconference to AOA COCA at its May 2009 meeting, and AOA COCA voted to approve the proposed revisions in the standards in clinical education, which became effective on July 1, 2009. 

In 2010 and 2011, similar Standards Forums were conducted prior to the spring AOA COCA meetings. This method of discussion and review is currently being used in changes proposed to standards and procedures regarding the continuum between predoctoral and graduate medical education. 

It is a policy of AOA COCA to conduct a comprehensive survey review of its standards and procedures on an ongoing basis to assess the relevancy, clarity, and reliability of those standards and procedures. The most recent survey review began in the fall of 2008. An outside consultant assisted AOA COCA staff in developing a survey tool that directed survey participants to evaluate all standards in chapter 1 of Accreditation of Colleges of Osteopathic Medicine: COM Accreditation Standards and Procedures. The survey focused on the standards' relevancy, clarity, and usefulness and gave participants an opportunity to provide open-ended comments regarding all standards. The responses were scored on a Likert scale from 1 to 5, with “1” indicating “not at all relevant,” “ not at all clear,” “definitely not,” or “not at all well,” and “5” indicating “very relevant,” very clear,” “yes definitely,” or “very well.” The questionnaire contained a total of 65 questions. 

The consultant also developed correspondence to explain to participants and stakeholders the process and rationale for the standard survey and to encourage them to participate in the survey. Participants were invited to take part in the survey by means of e-mail. After the initial letter, periodic reminder e-mails were sent. Using the online survey tool, participants and stakeholders received a link directly to the survey tool, which could also be accessed from the AOA COCA Web site. The survey was active for 6 weeks. Of the approximately 200 participants that received the initial e-mail request, 82 completed the survey for a 41% response rate (assuming unique responses). The response rate for the 2008 survey was 20 percentage points higher than the 2005 survey. 

The results of this survey (Table 1) indicated that the standards were rated the highest with regards to relevancy (average, 4.4), followed by clarity to internal users (ie, COM personnel; average, 4.0). Ratings for the categories regarding clarity to external users and usefulness to AOA COCA evaluators were nearly equal or above the target score of 3.75. When averages across all standards were compared, Standard 7: Research scored lower than all other standards. In addition to standards, procedures pertaining to the self-study, substantive change, and branch campuses were also surveyed for clarity and usefulness to AOA COCA evaluators. 

After compiling the results of the 2008 survey, AOA COCA staff developed a work plan and presented all of the survey data and analyses to the SRC for review at the spring 2009 AOA COCA meeting. The results of the analysis demonstrated high overall scoring in all parameters (Table 1). On the basis of these results, the SRC adopted the proposed work plan for the standard review and revision process that would begin with those domains that scored less than 3.75 (1, lowest; 5, highest). 

According to this work plan, Standard 7: Research, Standard 4: Faculty, and Standard 6: Curriculum would be the first 3 standards to be reviewed and revised because they had the lowest overall scores according the survey results. The SRC determined the next priority to be the handbook sections on procedures, including chapters 2, “Applicant and Accreditation Procedures”; 3, “The Self-Study Process”; 4, “On-site Visit Procedures”; 5, “Accreditation Procedures”; 6, “USDE Requirements”; and 7, “In the Public Interest,” because they had not been revised since January 2004. The final priority would be to review and revise the 4 standards that received the highest overall averages: Standard 1: Mission, Goals, and Objectives; Standard 2: Governance, Administration, and Finance; Standard 3: Facilities; Standard 5: Students; and Standard 8: Prerequisites for Accreditation. The work plan proposed that the SRC begin this process in June 2009. 

At the December 2009 AOA COCA meeting, the SRC presented its proposed revisions to Standards 7, 4, and 6 and chapters 2, 3, 4, and 7 and requested that AOA COCA authorize the posting of the proposed revisions for public commentary and that all stakeholders be informed of this action. In January 2010, all stakeholders were notified by e-mail of this action and were directed to the AOA COCA Web site, where the proposed revisions were posted. All comments were directed to the AOA COCA assistant secretary, and a Standards Forum was scheduled for March 2010. 

The Standards Forum took place on March 6, 2010. In a day-long meeting, participants discussed each of the proposed changes to the standards and procedures. The group proposed additional changes as a result of this discussion and, by the end of the meeting, reached consensus regarding the proposed changes. The proposed revised standards and procedures were presented at the AOA COCA meeting in May 2010. The AOA COCA voted to approve the proposed changes to Standards 4, 6, and 7 and chapters 2, 3, 4, and 7. These changes became effective on July 1, 2010. 

In addition to the revisions to the standards and procedures that were implemented as a result of the standards survey of 2008, revisions were initiated and completed in the spring of 2010 to be in compliance with the USDE's final rule on the federal regulation “Institutional Eligibility Under the Higher Education Act of 1965, as Amended, and the Secretary's Recognition of Accrediting Agencies,” which was issued October 27, 2009.¹¹ A first draft of proposed revisions to address the final rule were prepared and shared with AOA COCA at its meeting in December 2009. Staff presented 4 questions and a work plan for accomplishing the revisions and making them effective by the mandated July 1, 2010, deadline. Changes were required primarily in chapter 5, “Accreditation Procedures,“ and chapter 6, “USDE Requirements.” New terms were also added to the glossary and 2 changes were made to chapter 1, “Standards.” These changes went through a legal review by the AOA general counsel and the SRC. Stakeholders were notified by e-mail that revisions were posted on the AOA COCA Web site for commentary. At the May 2010 AOA COCA meeting, the revisions to the standards from the USDE final rule were approved, and the changes became effective on July 1, 2010. 

In the fall of 2012, AOA COCA will conduct a new survey of all standards and procedures in Accreditation of Colleges of Osteopathic Medicine: COM Accreditation Standards and Procedures. A survey tool similar to that used in 2008 will be used to gather information from stakeholders regarding the relevancy, clarity, and reliability of all standards and procedures used in the accreditation of COMs. Although all standards will be assessed, some attention will be focused on the standards that were not changed during the 2008 review to ensure that they remain at a high level of relevancy, clarity, and reliability. 

For the past 10 academic years, AOA COCA–approved class sizes have steadily increased. During the 2002-2003 academic year, 19 COMs operating on 19 campuses were approved for 2968 total matriculants. During the current 2011-2012 academic year, 26 COMs, 4 branch campuses, and 4 additional locations—34 sites total—were approved for 5442 matriculants (Table 2). 

The aggregate number of authorized matriculants in the approved class sizes increased by 2474 over academic years 2002-2003 through 2011-2012, inclusive, for a 83.4% increase since the 2002-2003 academic year. The larger fraction of this increase—1554 authorized matriculants (62.8%) of the 2474 increase—was the result of the establishment of 14 new campuses and the expansion of 6 existing campuses authorized by AOA COCA. The remaining 920 authorized matriculants (37.2%) are accounted for by the expansion of class sizes at existing COM campuses. 

The review of the standards and procedures used by AOA COCA is a systematic, continuous process, just as the evaluation of COMs is a continuous process. The standards and procedures result from collective input from not only AOA COCA and its internal publics, but also from external public entities including but not limited to other accrediting agencies, other entities that recognize accrediting agencies, the US Congress, the US secretary of education, and the public at large.