Free
Medical Education  |   April 2015
Comprehensive Osteopathic Medical Licensing Examination-USA Level 1 and Level 2-Cognitive Evaluation Preparation and Outcomes
Danielle E. Maholtz, OMS IV; Michael J. Erickson, OMS IV; Tyler Cymet, DO
Author Notes
  • From the Philadelphia College of Osteopathic Medicine in Pennsylvania (Student Doctor Maholtz); the Touro College of Osteopathic Medicine in New York, New York (Student Doctor Erickson); and the American Association of Colleges of Osteopathic Medicine in Chevy Chase, Maryland (Dr Cymet). Student Doctor Maholtz is the current and Student Doctor Erickson is the immediate past national medical education representative for the Council of Osteopathic Student Government Presidents. The portion of this article on the topic of the Council of Osteopathic Student Government Presidents' survey on students' preparation for the Comprehensive Osteopathic Medical Licensing Examination-USA was previously presented by Student Doctor Erickson at the American Association of Colleges of Osteopathic Medicine Annual Conference held April 2-5, 2014, in Washington, DC. The presentation was titled "Level Up!: COMLEX Preparation." E-mail: daniellema@pcom.edu 
Article Information
Medical Education / COMLEX-USA
Medical Education   |   April 2015
Comprehensive Osteopathic Medical Licensing Examination-USA Level 1 and Level 2-Cognitive Evaluation Preparation and Outcomes
The Journal of the American Osteopathic Association, April 2015, Vol. 115, 232-235. doi:https://doi.org/10.7556/jaoa.2015.046
The Journal of the American Osteopathic Association, April 2015, Vol. 115, 232-235. doi:https://doi.org/10.7556/jaoa.2015.046
Abstract

The Comprehensive Osteopathic Medical Licensing Examination-USA (COMLEX-USA) assesses the competence of osteopathic physicians in training. It is designed to protect the public by setting minimum competence standards. All osteopathic medical students must pass COMLEX-USA Level 1, Level 2-Cognitive Evaluation, and Level 2-Performance Evaluation before being allowed to graduate from an osteopathic medical school. Residency training programs use COMLEX-USA scores as a major factor in deciding whom they will interview and admit into their programs. In addition, colleges of osteopathic medicine use student COMLEX-USA scores as an external assessment of their success in educating students. Because COMLEX-USA is a high-stakes examination series, it is important to understand predictive factors for performance. The authors review the literature on the relationship between COMLEX-USA scores and correlated student variables. Results from the Council on Osteopathic Student Government Presidents' survey on students' preparation methods and performance are also provided.

Licensure to practice medicine occurs at the state level. Initially, each state or licensing district created and administered their own examination. Because of the need for higher quality and more standardized examinations, organizations such as the National Board of Osteopathic Medical Examiners (NBOME) were established.1 The NBOME and its allopathic counterpart, the National Board of Medical Examiners, are charged with creating the standardized examinations that ultimately determine licensure for today's physicians. The Comprehensive Osteopathic Medical Licensing Examination-USA (COMLEX-USA) is the standardized test series designed to assess an osteopathic physician (ie, DO) trainee for a minimum level of competence. The examination evaluates osteopathic medical students in 2 ways based on the Angoff and Hofstee statistical criteria.2 The Angoff method sets a standard score for items one would need to know to practice osteopathic medicine safely. It uses a cut score that relies on experts who examine test content to predict how a minimally qualified candidate would be expected to perform on an item. The Hofstee method measures students' performance relative to that of other students taking the examination. It is used to set a minimum and maximum failure rate expectation. These 2 criteria are used to validate one another when they produce similar passing scores. 
Passage of all 3 levels of COMLEX-USA or the United States Medical Licensing Examination (USMLE) is a major step that osteopathic medical students must complete before receiving a license to practice osteopathic medicine. Osteopathic medical students usually take the COMLEX-USA Levels 1 and 2 during their second and third years in school, respectively. The basic scientific concepts and principles taught in the first 2 years of undergraduate osteopathic medical education are emphasized on COMLEX-USA Level 1. Level 2 comprises 2 separate examinations: a written examination called the Cognitive Evaluation (CE) and a patient encounter examination called the Performance Evaluation. Level 2-CE focuses on biomedical concepts, clinical diagnostic reasoning, and the ability to treat patients. Level 3 of COMLEX-USA is typically taken while in graduate medical education (GME) training. 
Although designed to test the competence of physician trainees, licensure examination scores are increasingly being used by GME training programs as a way to screen applicants, and scores are an important factor used to decide whom to accept into those programs. Of 1793 surveyed Accreditation Council for Graduate Medical Education program directors, 94% indicated using USMLE Step 1 or COMLEX-USA Level 1 scores in selecting applicants to interview, with an importance rating of 4.1 out of 5 points.3 In this same survey, 36% of respondents indicated that they look for COMLEX-USA Level 1 pass scores only, whereas 41% indicated that they look for target scores.3 
In our experience, students today wonder whether their COMLEX-USA scores are more important than the osteopathic medical school they attend or any other factors. The use of COMLEX-USA scores in this way has caused much stress and anxiety for osteopathic medical students. To our knowledge, students use their COMLEX-USA results as a major determinant in potential placement into a GME program, both for specialties to pursue and programs to which they should apply. Adding to this stress is the lack of COMLEX-USA–specific preparation materials. Frequently, students from a specific COM use similar resources, which we believe occurs because either the COM provides them with preparation materials or the students base their study methods on peer-guided recommendations. 
In the present article, we review the data currently available for students to use in their preparation methods. These data include individual variables that place students at a greater likelihood of passing or failing, as well as successful preparation methods used to attain certain COMLEX-USA scores on both Level 1 and Level 2-CE. We also summarize findings of a survey on COMLEX-USA preparation and performance. 
Review
COMLEX-USA Scores vs Premedical School Variables
To provide both COMs and osteopathic medical students with methods to predict how students will perform on licensing examinations, many studies have analyzed predictive values for student outcomes on their examinations taken before entering a COM, such as the Medical College Admission Test (MCAT). Correlations between MCAT scores and undergraduate grade point averages (GPAs) can predict a student's likely success in a COM overall, as well as his or her COMLEX-USA outcomes. Dixon4 found that physical sciences and biological sciences MCAT subscores were correlated with COMLEX-USA Level 1 performance, whereas verbal reasoning, physical, and biological MCAT subscores were correlated with Level 2 performance. Cope et al5 used preadmission variables in predictive models and reported that biological MCAT scores and cumulative undergraduate GPA were predictive of COMLEX-USA Level 1 scores. In contrast, Baker et al6 found no correlations between COMLEX-USA Level 1 scores, preadmission variables, and medical school performance. 
COMLEX-USA Scores vs Medical School and Residency Variables
Some studies have looked at the relationship between Comprehensive Osteopathic Medical Achievement Test (COMAT) outcomes and COMLEX-USA scores. Li et al7 found that internal medicine COMAT scores were the strongest predictor of Level 2-CE performance. Lewis et al8 examined specific course grades in relation to COMLEX-USA outcomes and found a correlation between performance on the second-year osteopathic manipulative medicine examination and COMLEX-USA Level 1 outcomes. In yet another study, Li et al9 looked at the predictive power of COMLEX-USA performance on resident outcomes and performance on the American Osteopathic Board of Emergency Medicine Part 1 certification examination. They found the greatest correlation with COMLEX-USA Level 3, followed by Level 2-CE, with the least correlation with COMLEX-USA Level 1 scores.9 
COMLEX-USA Scores vs Student Preparation Methods
Aditya et al10 looked at predictors for students scoring at least 600 on the COMLEX-USA Level 1 in relation to both correlated student variables and their preparation strategies. A score of 600 or higher was associated with a higher medical school GPA, a higher MCAT score, earlier preparation initiation, and students not ranking the Comprehensive Osteopathic Medical Self-Assessment Examination as the most helpful practice examination.10 A second regression removing GPA and MCAT scores showed that a score of 600 was associated with earlier initiation of examination preparation and not ranking COMBANK (a question bank for osteopathic examinations) as the most helpful question bank.10 These findings emphasize the importance of establishing a consistent program of study and maintaining a high GPA throughout medical school, as these factors will most likely lead to successful performance on the COMLEX-USA. 
Survey on COMLEX-USA Preparation
Methods
Using the study by Aditya et al10 as a guide, in 2013, the Council of Osteopathic Student Government Presidents medical education representative (M.J.E.) developed surveys to analyze the most important factors in the first 2 years of medical school for student success, specific licensing examination preparation methods, and outcomes on COMLEX-USA Levels 1 and 2-CE. The survey was reviewed by leadership at the American Association of Colleges of Osteopathic Medicine. The study was reviewed by the institutional review board at the Touro College of Osteopathic Medicine in New York, New York, which determined it to be exempt from approval and informed consent. 
The surveys (one for Level 1 and one for Level 2-CE and created using Google Forms) invited all osteopathic medical students who had taken COMLEX-USA in the past year to complete the questionnaires. The surveys were e-mailed to osteopathic medical students by their student government leaders via student e-mail listservs. Participation was not required, and all responses were anonymous. The surveys were conducted in 2013 and in 2014. 
Results
Students from 13 schools responded to the surveys, for a total of 399 responses on the COMLEX-USA Level 1 survey and 306 responses on the COMLEX-USA Level 2-CE survey. The mean score of Level 1 respondents was 547 (range, 385-840). The mean score of Level 2-CE respondents was 578 (range, 400-850). 
When surveyed regarding COMLEX-USA Level 1, the majority of students reported spending either 3 to 4 weeks or 4 to 8 weeks preparing solely for their licensing examinations. Students who were in the top 20% of their class were found to have a mean COMLEX-USA Level 1 score of 610, whereas all other students were found to have a mean score of 522, indicating that the top 20% of the class was more likely to achieve higher scores on the examination. The top 4 student preparation resources were First Aid for the USMLE, USMLEWorld, COMBANK, and Doctors in Training. Of our respondents, 48% also took the USMLE Step 1, and they had a mean score of 224 (out of a possible 300). 
Regarding COMLEX-USA Level 2-CE, the majority of students stated that their COM helped prepare them by using COMATs on clinical rotations. In contrast to Level 1 respondents, Level 2-CE respondents were much more likely to consider their education, or clinical rotations, as their licensing examination preparation and stated that they spent more than 6 weeks preparing for Level 2-CE. 
For both COMLEX-USA Levels 1 and 2-CE, many students stressed that the harder one worked in the first 2 years of medical school, the easier it was to prepare for licensing examinations. Others stressed the continued importance of practice tests and practice questions. 
Discussion
Outcomes on Levels 1 and 2 of the COMLEX-USA series are important assessment tools used by COMs and GME programs. Students are therefore placing increasing importance on preparation for these examinations. Previous research has demonstrated the correlation of undergraduate GPA, MCAT scores, medical school GPA, performance on individual medical school examinations, and performance on COMAT with student COMLEX-USA outcomes. More recent research has demonstrated the possible relevance of students' perspectives on the COMLEX-USA and their licensing examination–focused preparation methods to their performance on COMLEX-USA. Our survey data showed that students value practice questions and time for test preparation above other variables. 
The outcomes of the Council of Osteopathic Student Government Presidents survey are limited. Our findings provide a snapshot of COMLEX-USA and student perspectives at this time and not a comprehensive review. The study described herein also used students' survey responses only and did not include those of other stakeholders. 
Conclusion
Standardized licensing examinations such as the COMLEX-USA series are an integral part of developing competent physicians. Research will likely continue to investigate the most effective preparation tools and predictors for performance outcomes. 
References
Mohr JC. Licensed to Practice: The Supreme Court Defines the American Medical Profession. Baltimore, MD: Johns Hopkins University Press; 2013.
The approach to standard setting. National Board of Osteopathic Medical Examiners website. http://www.nbome.org/standardsetting_app.asp. Accessed February 25, 2015.
The National Resident Matching Program. Data Release and Research Committee: Results of the 2014 NRMP Program Director Survey. Washington, DC: The National Resident Matching Program; June 2014. http://www.nrmp.org/wp-content/uploads/2014/09/PD-Survey-Report-2014.pdf. Accessed February 25, 2015.
Dixon D. Relation between variables of preadmission, medical school performance, and COMLEX-USA Levels 1 and 2 Performance. J Am Osteopath Assoc. 2004;104(8):332-336. http://www.jaoa.osteopathic.org/content/104/8/332.full. Accessed February 25, 2015. [PubMed]
Cope MK, Baker HH, Fisk R, Gorby JN, Foster RW. Prediction of student performance on the Comprehensive Osteopathic Medical Licensing Examination Level 1 based on admission data and course performance. J Am Osteopath Assoc. 2001;101(2):84-90. http://www.jaoa.osteopathic.org/content/101/2/84.full.pdf+html. Accessed February 25, 2015. [PubMed]
Baker HH, Cope MK, Fisk R, Gorby JN, Foster RW. Relationship of pre-admission variables and first- and second-year course performance to performance on the National Board of Osteopathic Medical Examiners' COMLEX-USA Level 1 examination. J Am Osteopath Assoc. 2000;100(3):153-161. http://www.jaoa.osteopathic.org/content/100/3/153.full.pdf+html. Accessed February 25, 2015. [PubMed]
Li F, Kalinowski KE, Song, H, Bales BP. Relationships between the Comprehensive Osteopathic Medical Achievement Test (COMAT) subject examinations and the COMLEX-USA Level 2-Cognitive Evaluation. J Am Osteopath Assoc. 2014;114(9):714-721. doi:10.7556/jaoa.2014.140. [CrossRef] [PubMed]
Lewis DD, Johnson MT, Finnerty EP. Predictive relationship of osteopathic manual medicine grades and COMLEX-USA Level 1 total scores and osteopathic principles and practice subscores. J Am Osteopath Assoc. 2014;114(6):480-485. doi:10.7556/jaoa.2014.097. [CrossRef] [PubMed]
Li F, Gimpel JR, Arenson, E, Song H, Bates BP, Ludwin F. Relationship between COMLEX-USA scores and performance on the American Osteopathic Board of Emergency Medicine Part I certifying examination. J Am Osteopath Assoc. 2014;114(4):260-266. doi:10.7556/jaoa.2014.051. [CrossRef] [PubMed]
Aditya V, Maltezos N, Alfonzo L, Hernandez N, Calix E, Fernandez MI. Predictors of scoring at least 600 on COMLEX-USA Level 1: successful preparation strategies. J Am Osteopath Assoc. 2013;113(2):164-173. http://www.jaoa.osteopathic.org/content/113/2/164.full. Accessed February 25, 2015. [PubMed]