The present study has some limitations. First, although all panelists had experience with training and assessing medical students and residents, they predominately demonstrated expertise in applying the clinical competencies in a graduate medical education setting. This selection bias may have confounded study results, and future study could include representatives from osteopathic undergraduate, graduate, and continuing medical education. Second, for the 8368 judgments made by the 8 panelists, interrater reliability was quite high, but reliability could be strengthened further by repeating the exercise with a second group of medical educators. Third, by integrating OPP and OMT into the original competencies, only 6 competencies were used for classification instead of the 7 competencies defined by the AOA. Fourth, previously written examination items, designed specifically to meet current COMLEX-USA blueprint specifications, were reviewed for the purposes of this study. Imposing competency-based classifications onto items that were created using an entirely different blueprint may not only be impractical, but may also explain the difficulty in identifying one discrete competency for each item. Future study could involve classification of items that are constructed specifically for a particular competency.
Although the present study supports the claim that knowledge of a variety of competencies can be assessed using multiple-choice questions, it is important to remember that COMLEX-USA Level 2-Performance Evaluation (PE) augments the COMLEX-USA Level 2-CE by providing an assessment of fundamental clinical skills of osteopathic medical students who are preparing to enter graduate medical education.
13,14 The clinical skills examination measures patientphysician communication, interpersonal skills, professionalism, medical history-taking and physical examination skills, integration of osteopathic principles into practice and use of OMT, written communication skills and ability to synthesize information, and the ability to develop a differential diagnosis and formulate a diagnostic and treatment plan. Results of the present study suggest that COMLEX-USA cognitive examinations can be used to some extent to assess knowledge of each of the competencies; COMLEX-USA Level 2-PE can be used to assess performance of each of the competencies.