Predictors of Scoring at Least 600 on COMLEX-USA Level 1: Successful Preparation Strategies

Aditya Vora; Nathan Maltezos; Lauren Alfonzo; Nilda Hernandez; Erica Calix; M. Isabel Fernandez

doi:10.7556/jaoa.2013.113.2.164

Abstract

Context: Comprehensive Osteopathic Medical Licensing Examination-USA (COMLEX-USA) Level 1 scores are an important criterion used by residency directors to make residency placement decisions.

Objective: To explore the association between scoring at least 600 on COMLEX-USA Level 1 and grade point average (GPA), scores on the Medical College Admission Test (MCAT), and different test preparation strategies.

Methods: Third-year osteopathic medical students at Nova Southeastern University were invited to complete a self-administered survey regarding their COMLEX-USA preparation strategies and to provide consent for the researchers to access their preclinical GPA and their MCAT and COMLEX-USA scores. Descriptive analyses were conducted to understand examination preparation procedures and resources used, and bivariate analyses were conducted to identify the statisically significant predictors of scoring 600 or higher. Two separate logistic regressions were also run. The first included all of the statisically significant factors that emerged from the bivariate analyses, and the second examined which candidate predictors remained statistically significant once the effects of GPA and MCAT scores were removed.

Results: One hundred twenty-two students completed the survey, and 113 (93%) provided informed consent to access their preclinical GPA and their MCAT and COMLEX-USA scores. In the first regression, scoring 600 or higher was associated with a higher GPA (P<.02), a higher MCAT score (P<.05), earlier preparation initiation (P<.05), and not ranking the Comprehensive Osteopathic Medical Self-Assessment Examination (COMSAE) as the most helpful practice examination (P<.04). In the second regression, scoring 600 or higher was associated with earlier initiation of examination preparation (P<.01) and not ranking COMBANK (question bank for COMLEX-USA) as the most helpful question bank (P<.03). Among the different examination preparation methods, the specific resources ranked as most helpful were First Aid for the USMLE (United States Medical Licensing Examination) (review book), the COMSAE (practice examination); COMBANK (question bank); and Kaplan USMLE (lecture videos).

Conclusion: Preclinical GPA and MCAT scores continue to be important predictors of scoring at least 600 on COMLEX-USA Level 1. However, the findings underscore the importance of maintaining a high GPA during the first 2 years of medical school and initiating COMLEX-USA preparation early.

During the past few years, osteopathic medicine in the United States has experienced tremendous growth in both the number of physicians and the awareness of the profession, as evidenced by the increasing number of osteopathic medical graduates.¹ This trend is continuing. In 2012, the number of newly graduated doctors of osteopathic medicine reached the new peak of 4773,¹ and osteopathic physicians are increasingly more visible across the range of medical specialties. To ensure that osteopathic physicians are proficient medical providers, the osteopathic medical profession maintains strict criteria for osteopathic licensure, which includes passing all 3 levels of the Comprehensive Osteopathic Medical Licensing Examination-USA (COMLEX-USA). This examination is comparable to the allopathic licensing examination, the United States Medical Licensing Examination (USMLE). Level 1 of COMLEX-USA, the first examination in the 3-part series, is typically taken at the end of the second year of osteopathic medical school and consists of two 4-hour test sessions completed in 1 day.² It is “a problem-based and symptom-based assessment integrating the basic sciences of anatomy, behavioral science, biochemistry, microbiology, osteopathic principles, pathology, pharmacology, physiology, and other areas of medical knowledge as they are relevant to solving medical problems.”²

Despite evidence questioning the validity of licensing test scores as measures of clinical competence,³ many residency directors continue to use COMLEX-USA Level 1 scores to make residency placement decisions. Studies have shown that COMLEX-USA scores are an important criterion used by program directors in selecting residents.^4-6 A similar practice has been reported in allopathic medical training. Studies of residency directors in obstetrics and gynecology, family medicine, emergency medicine, anesthesiology, general surgery, and orthopedic surgery have shown that a candidate's USMLE Step 1 score is one of the most highly weighted variables in deciding whether to grant a student an interview and select him or her as a resident.^7-11 Although licensing examination scores are not correlated with reliable measures of clinical skill acquision,³ it is clear that Level 1 and Level 2 board scores continue to be used as important factors in both the osteopathic and the allopathic residency selection process.

Emerging literature has examined factors associated with test performance on COMLEX-USA Level 1.^12-14 There is strong evidence that performance on COMLEX-USA Level 1 is correlated with scores on the Medical College Admission Test (MCAT)^15-17 and grade point averages (GPAs) during the first 2 years of medical school (ie, preclinical GPA).^12,13,15,18 The role of other factors—specifically examination preparation approaches, time spent studying, and types of examination preparation resources used—on COMLEX-USA Level 1 performance has, to our knowledge, not been rigorously examined. Understanding the relative contribution of such factors is important because it can provide additional guidance to medical students on how best to prepare for this examination. Despite the importance of these test scores on the future careers of medical students, we could not find a single study directly focused on COMLEX-USA Level 1 to guide osteopathic medical students to prepare for this important examination. The present study directly addresses this need.

The goal of this exploratory study was to identify the factors associated with scoring 600 or higher on COMLEX-USA Level 1. More specifically, we examined the relationship between scores on COMLEX-USA Level 1 and examination preparation resources (eg, question banks, review books, practice examinations, lecture videos), length of time preparing, study format (individual or group), preclinical GPA, and MCAT scores.

Methods

Participants

Participants were third-year osteopathic medical students at Nova Southeastern University College of Osteopathic Medicine (NSU-COM) who attended a residency fair on November 10, 2011, on campus. All study participants had taken COMLEX-USA Level 1 by September 30, 2011, as required by NSU-COM policy.

Design

This study used a cross-sectional design. The protocol was approved in September 2011 by the institutional review board at Nova Southeastern University.

Procedures

While third-year osteopathic medical students were gathered in a school auditorium, we briefly presented the study and informed consent procedures for obtaining access to participants' preclinical GPAs and their MCAT and COMLEX-USA Level 1 scores. Students could voluntarily agree to (1) complete the survey and provide consent to access their scores, (2) complete only the survey, or (3) not participate in either study component. The researchers stressed that participation was voluntary and that there would be no consequences for electing not to participate. A question and answer period followed the presentation. The researchers then distributed envelopes containing an introductory letter, informed consent forms, a survey instrument, and a $1 bill as a token of appreciation. Students who agreed to participate completed the survey; those who granted permission to access their scores signed the informed consent. Participants placed the completed documents into the envelopes, sealed them, and placed them in a collection box. To protect the confidentiality of participants, each survey was assigned a unique study identification number, which was preprinted on the survey forms; the informed consent contained the only personally identifying information.

Faculty mentors on the research team at NSU-COM opened the sealed envelopes and prepared a spreadsheet linking each participant's name to their study identification number. The survey data were entered into an SPSS data file (IBM Corporation, Chicago, Illinois) and merged with a data file containing preclinical GPA and MCAT and COMLEX-USA Level 1 scores.

Survey Instrument

The research team developed the items by using examples from the published literature and methodological experts at NSU-COM. It consisted of 35 items that tapped 6 domains: demographics, examination methods, perceived performance as an osteopathic medical student, study format, student tips, and GPA and MCAT and COMLEX-USA Level 1 scores.

Demographics

Participants reported their age, sex, race, ethnicity, highest-earned academic degree, year of degree completion, undergraduate major, marital status, number of children, and annual family income.

Examination Preparation Methods

Participants reported when they began preparing for the COMLEX-USA Level 1 (first year of osteopathic medical school, fall of second year [September to December], winter of second year [January to March], spring of second year [April to May]), and the number of hours they studied per week for each of the 8 weeks leading up to the examination. For each of 5 broad categories of examination preparation resources (review books, practice examinations, question banks, lecture videos, and class notes), participants reported how often they used it and, if used, how helpful it was. Both were rated using a 5-point scale, from “never” (1) to “always” (5) for the first question and from “unhelpful” (1) to “very helpful” (5) for the follow-up question. To gain more specific information regarding the resources used, participants rank-ordered the 3 most helpful resources within each of the 5 broad categories.

Performance as a Medical Student

Participants self-ranked their own perceived performance as osteopathic medical students on a 5-point scale from “poor” (1) to “outstanding” (5).

Study Format

Participants reported the percentage of time they studied alone and the percentage of time they studied in groups.

Student Tips

Using an open-ended format, participants provided specific tips or advice they would give to other students to help them prepare for the examination.

Preclinical GPA and MCAT and COMLEX-USA Scores

Administrative staff at NSU-COM provided the preclinical GPA and MCAT and official COMLEX-USA Level 1 scores. These data were then linked to each participant's survey responses prior to analysis.

Dependent Variable

The dependent variable was COMLEX-USA Level 1 scores. Students were classified as having a high score if they scored 600 or higher on the examination. A score of 600 to 800 is within the top 25% of the scores on COMLEX-USA Level 1. Therefore, we used this cutoff point to differentiate students with high scores from all others.

Analyses

Data were entered into a data file (SPSS version 19) and cross-checked for reliability by using standard quality assurance procedures. We first conducted descriptive analyses to describe the sample and to understand medical students' examination preparation procedures and resources used. Next we classified participants according to their COMLEX-USA Level 1 scores. Participants who scored 600 or higher were classified as 1 and those who scored lower than 600 were classified as 0. We conducted bivariate analyses on theoretically relevant factors to examine the relationship between each factor and the dependent variable. Specifically, we investigated demographic variables (age, sex, race, annual family income, marital status), ratings of performance in medical school, examination preparation resources, initiation of examination preparation, study format, preclinical GPA, and MCAT score. Next, we conducted 2 multivariable logistic regressions to examine the relationship between theoretically relevant variables and COMLEX-USA Level 1 scores. Following the recommendations of Hosmer and Lemeshow,¹⁹ we included factors with P values of .20 or lower in the bivariate analysis as candidate predictors in the first logistic regression because use of more traditional significance values might fail to identify important relationships. We ran a second regression excluding preclinical GPA and MCAT scores as independent variables to explore whether the remaining candidate predictors were associated with higher scores on COMLEX-USA Level 1 once the effect of these 2 factors was removed.

Results

Population Sample

Of the 152 students in attendance at the fair, 122 (80%) completed a self-administered survey, 113 of whom also provided informed consent to access their preclinical GPA and MCAT and COMLEX-USA Level 1 scores. Participants who did not provide consent (n=9) were not included in the analyses. All surveys that were turned in were included in the analyses even though a few participants did not complete all of the items.

The characteristics of the population sample are available in Table 1. A majority of the respondents were male (66 [58.4%]), white (81 [72.3%]), and single (80 [71.4%]). Nearly a quarter of respondents (29 [23.6%]) had an annual family income greater than $200,000. Sixty-two participants (55.4%) rated their performance in medical school to be outstanding or excellent. The mean (standard deviation [SD]) preclinical GPA was 88.4 (4.2); MCAT score, 26.7 (3); and COMLEX-USA Level 1 score, 524.9 (86.1). Twenty-one participants (18.6%) scored 600 or higher on COMLEX-USA Level 1.

Table 1.

Predictors of COMLEX-USA Level 1 Performance: Characteristics of Survey Respondents (N=113)

Characteristic	No. (%)^a
Age, mean (SD), y (n=110)	26.3 (2.2)
Sex (n=113)
Male	66 (58.4)
Female	47 (41.6)
Race (n=112)
Asian	18 (16.1)
Black/African American	1 (0.9)
White	81 (72.3)
Other	12 (10.7)
Ethnicity (n=105)
Hispanic/Latino	16 (15.2)
Non-Hispanic/Latino	89 (84.8)
Highest-Earned Academic Degree (n=107)
Bachelor's	85 (79.4)
Master's	19 (17.8)
Doctorate	3 (2.8)
Marital Status (n=112)
Single	80 (71.4)
Married	29 (25.9)
Divorced	2 (1.8)
Domestic Partnership	1 (0.9)
Number of Children (n=109)
0	100 (91.7)
1	6 (5.5)
≥2	3 (2.8)
Annual Family Income (n=110)
<$20,000	25 (20.9)
$20,000-$60,000	20 (17.3)
$60,001-$100,000	25 (20.9)
$100,001-$200,000	19 (17.3)
$200,001+	29 (23.6)
Perceived Performance as Medical Student (n=112)^a
Outstanding	14 (12.5)
Excellent	48 (42.9)
Good	38 (33.9)
Fair	12 (10.7)
Poor	0
MCAT Score, mean (SD) (n=111)^b	26.7 (3.0)
COMLEX-USA Level 1 Score, mean (SD) (n=113)^c	524.9 (86.1)
Grade Point Average, mean (SD) (n=113)^d	88.4 (4.2)

^a Data are presented as No. (%) except where otherwise noted.

^b The maximum Medical College Admission Test (MCAT) score is 45.

^c The maximum score on the Comprehensive Osteopathic Medical Licensing Examination-USA (COMLEX-USA) Level 1 is 800.

^d The maximum grade point average is 100.

Abbreviation: SD, standard deviation.

View Large

Table 1.

Predictors of COMLEX-USA Level 1 Performance: Characteristics of Survey Respondents (N=113)

Characteristic	No. (%)^a
Age, mean (SD), y (n=110)	26.3 (2.2)
Sex (n=113)
Male	66 (58.4)
Female	47 (41.6)
Race (n=112)
Asian	18 (16.1)
Black/African American	1 (0.9)
White	81 (72.3)
Other	12 (10.7)
Ethnicity (n=105)
Hispanic/Latino	16 (15.2)
Non-Hispanic/Latino	89 (84.8)
Highest-Earned Academic Degree (n=107)
Bachelor's	85 (79.4)
Master's	19 (17.8)
Doctorate	3 (2.8)
Marital Status (n=112)
Single	80 (71.4)
Married	29 (25.9)
Divorced	2 (1.8)
Domestic Partnership	1 (0.9)
Number of Children (n=109)
0	100 (91.7)
1	6 (5.5)
≥2	3 (2.8)
Annual Family Income (n=110)
<$20,000	25 (20.9)
$20,000-$60,000	20 (17.3)
$60,001-$100,000	25 (20.9)
$100,001-$200,000	19 (17.3)
$200,001+	29 (23.6)
Perceived Performance as Medical Student (n=112)^a
Outstanding	14 (12.5)
Excellent	48 (42.9)
Good	38 (33.9)
Fair	12 (10.7)
Poor	0
MCAT Score, mean (SD) (n=111)^b	26.7 (3.0)
COMLEX-USA Level 1 Score, mean (SD) (n=113)^c	524.9 (86.1)
Grade Point Average, mean (SD) (n=113)^d	88.4 (4.2)

^a Data are presented as No. (%) except where otherwise noted.

^b The maximum Medical College Admission Test (MCAT) score is 45.

^c The maximum score on the Comprehensive Osteopathic Medical Licensing Examination-USA (COMLEX-USA) Level 1 is 800.

^d The maximum grade point average is 100.

Abbreviation: SD, standard deviation.

Summaries of the different preparation strategies used by participants are found in Table 2. The majority of participants began preparing for COMLEX-USA Level 1 from January to March (39 [34.8%]) or from April to May (50 [44.6%]) of their second year of medical school. The most frequently used and most helpful examination preparation resources were review books and question banks. One hundred two participants (93.6%) reported review books as helpful or very helpful, while 101 (92.7%) reported question banks as helpful or very helpful. Among the different examination preparation methods, the specific resources ranked as most helpful were First Aid for the USMLE (review book), the Comprehensive Osteopathic Medical Self-Assessment Examination (COMSAE) (practice examination), COMBANK (question bank [for COMLEX-USA]), and Kaplan USMLE (lecture videos). Eighty-one percent of students reported that they studied alone.

Table 2.

Predictors of COMLEX-USA Level 1 Performance: Summary of Examination Preparation Strategies Among Survey Respondents (N=113)

Survey Question	n	No. (%)^a
Start of Examination Preparation
During first year	112	6 (5.4)
September to December of second year	112	17 (15.2)
January to March of second year	112	39 (34.8)
April to May of second year	112	50 (44.6)
Time Studying Alone, mean (SD), %	108	81.3 (25.3)
Study Aids: Almost Always/Always Used
Review books	111	106 (95.5)
Practice examinations	112	56 (50.0)
Question banks	113	103 (91.2)
Lecture videos	112	49 (43.8)
Class notes	112	3 (2.7)
Study Aids: Usually/Very Helpful
Review books	109	102 (93.6)
Practice examinations	102	71 (69.6)
Question banks	109	101 (92.7)
Lecture videos	95	63 (66.3)
Class notes	46	12 (26.1)
Review Book Ranked Most Helpful
First Aid for the USMLE	112	94 (83.9)
Practice Examination Ranked Most Helpful
COMSAE	97	60 (61.9)
Question Bank Ranked Most Helpful
COMBANK	109	56 (51.4)
Lecture Video Ranked Most Helpful
Kaplan USMLE	96	69 (72.6)

^a Data are presented as No. (%) except where otherwise noted.

Abbreviations: COMBANK, question bank for the Comprehensive Osteopathic Medical Licensing Examination-USA (COMLEX-USA); COMSAE, Comprehensive Osteopathic Medical Self Assessment Examination; SD, standard deviation; USMLE, United States Medical Licensing Examination.

View Large

Table 2.

Predictors of COMLEX-USA Level 1 Performance: Summary of Examination Preparation Strategies Among Survey Respondents (N=113)

Survey Question	n	No. (%)^a
Start of Examination Preparation
During first year	112	6 (5.4)
September to December of second year	112	17 (15.2)
January to March of second year	112	39 (34.8)
April to May of second year	112	50 (44.6)
Time Studying Alone, mean (SD), %	108	81.3 (25.3)
Study Aids: Almost Always/Always Used
Review books	111	106 (95.5)
Practice examinations	112	56 (50.0)
Question banks	113	103 (91.2)
Lecture videos	112	49 (43.8)
Class notes	112	3 (2.7)
Study Aids: Usually/Very Helpful
Review books	109	102 (93.6)
Practice examinations	102	71 (69.6)
Question banks	109	101 (92.7)
Lecture videos	95	63 (66.3)
Class notes	46	12 (26.1)
Review Book Ranked Most Helpful
First Aid for the USMLE	112	94 (83.9)
Practice Examination Ranked Most Helpful
COMSAE	97	60 (61.9)
Question Bank Ranked Most Helpful
COMBANK	109	56 (51.4)
Lecture Video Ranked Most Helpful
Kaplan USMLE	96	69 (72.6)

^a Data are presented as No. (%) except where otherwise noted.

Abbreviations: COMBANK, question bank for the Comprehensive Osteopathic Medical Licensing Examination-USA (COMLEX-USA); COMSAE, Comprehensive Osteopathic Medical Self Assessment Examination; SD, standard deviation; USMLE, United States Medical Licensing Examination.

In terms of open-ended suggestions or “student tips” for examination preparation, of the 100 respondents to this question, 48 suggested completing a lot of practice questions, 26 suggested using the review book First Aid for the USMLE while studying for courses, 26 recommended starting the study process early, and 10 suggested making and following a study schedule (Table 3).

Table 3.

Predictors of COMLEX-USA Level 1 Performance: Tips From Survey Respondents (n=100)

Tip	No. (%)
Do a lot of practice questions	48 (48)
Use First Aid for the USMLE with courses	26 (26)
Start early	26 (26)
Make a schedule	10 (10)

Abbreviations: COMLEX-USA, Comprehensive Osteopathic Medical Licensing Examination-USA; USMLE, United States Medical Licensing Examination.

View Large

Table 3.

Predictors of COMLEX-USA Level 1 Performance: Tips From Survey Respondents (n=100)

Tip	No. (%)
Do a lot of practice questions	48 (48)
Use First Aid for the USMLE with courses	26 (26)
Start early	26 (26)
Make a schedule	10 (10)

Abbreviations: COMLEX-USA, Comprehensive Osteopathic Medical Licensing Examination-USA; USMLE, United States Medical Licensing Examination.

Bivariate Analysis

We found no statistically significant association between a score of 600 or higher on COMLEX-USA Level 1 and any of the demographic variables or ratings of self-perceived performance in osteopathic medical school. There was a significant association with examination preparation methods, including earlier initiation of examination preparation (

χ_{4}^{2} = 10.07

, P=.04), ranking COMSAE as the most helpful practice examination (

χ_{1}^{2} = 7.81

, P=.005) and ranking COMBANK as the most helpful question bank (

χ_{1}^{2} = 11.24

, P=.001). A score of 600 or higher on COMLEX-USA Level 1 was associated with a significantly higher preclinical GPA during the first 2 years of osteopathic medical school (P<.0001) and a higher MCAT score (P=.02).

Multivariate Analysis

The results of the first logistic regression is found in Table 4. In the final model, preclinical GPA, MCAT scores, practice examination, and initiation of examination preparation were statistically significant in predicting a COMLEX-USA Level 1 score of 600 or higher. Participants with COMLEX-USA scores of 600 or higher were 1.4 (95% confidence interval [CI], 1.06-1.71; P<.02) times more likely to have a higher preclinical GPA, 1.3 (95% CI, 1.00-1.69; P<.05) times more likely to have higher MCAT scores, and 2.5 (95% CI, 1.00-6.33; P<.05) times more likely to have initiated their examination preparation earlier than those who had COMLEX-USA Level 1 scores less than 600.

Table 4.

Multivariate Logistic Regression: Predictors of Scoring ≥600 on COMLEX-USA Level 1 Among Survey Respondents (N=113)

Predictor	Odds Ratio (95% Confidence Interval)	P Value
Sex	1.46 (0.34-6.25)	.608
Race	1.47 (0.68-3.20)	.327
COMBANK as most helpful question bank	0.24 (0.05-1.31)	.099
Time started studying	2.53 (1.004-6.33)	<.05
COMSAE as most helpful practice examination	0.19 (0.04-0.86)	<.04
Medical College Admission Test score	1.30 (1.001-1.69)	<.05
Preclinical grade point average	1.35 (1.06-1.71)	<.02

Abbreviations: COMBANK, question bank for the Comprehensive Osteopathic Medical Licensing Examination-USA; COMSAE, Comprehensive Osteopathic Medical Self-Assessment Examination.

View Large

Table 4.

Multivariate Logistic Regression: Predictors of Scoring ≥600 on COMLEX-USA Level 1 Among Survey Respondents (N=113)

Predictor	Odds Ratio (95% Confidence Interval)	P Value
Sex	1.46 (0.34-6.25)	.608
Race	1.47 (0.68-3.20)	.327
COMBANK as most helpful question bank	0.24 (0.05-1.31)	.099
Time started studying	2.53 (1.004-6.33)	<.05
COMSAE as most helpful practice examination	0.19 (0.04-0.86)	<.04
Medical College Admission Test score	1.30 (1.001-1.69)	<.05
Preclinical grade point average	1.35 (1.06-1.71)	<.02

Abbreviations: COMBANK, question bank for the Comprehensive Osteopathic Medical Licensing Examination-USA; COMSAE, Comprehensive Osteopathic Medical Self-Assessment Examination.

Participants who did not rank COMSAE as the most helpful practice examination were 5.2 (95% CI, 1.16-22.73; P<.04) times more likely to have COMLEX-USA scores of 600 or higher than those who did rank it as the most help practice examination. The final model correctly classified 87% of participants: 94.6% of those who scored lower than 600 and 55.6% of participants who scored 600 or higher were correctly classified, and this finding was statistically significant (

χ_{7}^{2} = 38.22

, P=.0001).

In the second logistic regression, in which MCAT scores and preclinical GPA were not included as independent variables, initiation of examination preparation and question banks were the 2 statistically significant predictors (Table 5). Participants with COMLEX-USA Level 1 scores of 600 or higher were 2.9 (95% CI, 1.38-6.29; P<.01) times more likely to have initiated examination preparation earlier than those with lower scores. Participants who did not rank COMBANK as the most helpful question bank were 5.5 (95% CI, 1.25-23.81; P<.03) times more likely to have COMLEX-USA scores of 600 or higher than those who did rank it as the most helpful question bank. The final model correctly classified 84.9% of participants: 94.7% of those who scored lower than 600 and 44.4% of participants who scored 600 or higher were classified correctly. This finding was statistically significant (

χ_{5}^{2} = 25.67

, P=.0001).

Table 5.

Multivariate Logistic Regression: Predictors of Scoring ≥600 on COMLEX-USA Level 1 Among Survey Respondents, Excluding Grade Point Average and Medical College Admission Test Score (N=113)

Predictor	Odds Ratio (95% Confidence Interval)	P Value
Sex	2.34 (0.67-8.20)	.186
Race	1.50 (0.71-3.16)	.288
COMBANK as most helpful question bank	0.18 (0.04-0.80)	<.03
Time started studying	2.90 (1.38-6.29)	<.01
COMSAE as most helpful practice examination	0.30 (0.08-1.05)	.06

Abbreviations: COMBANK, question bank for the Comprehensive Osteopathic Medical Licensing Examination-USA (COMLEX-USA); COMSAE, Comprehensive Osteopathic Medical Self-Assessment Examination.

View Large

Table 5.

Multivariate Logistic Regression: Predictors of Scoring ≥600 on COMLEX-USA Level 1 Among Survey Respondents, Excluding Grade Point Average and Medical College Admission Test Score (N=113)

Predictor	Odds Ratio (95% Confidence Interval)	P Value
Sex	2.34 (0.67-8.20)	.186
Race	1.50 (0.71-3.16)	.288
COMBANK as most helpful question bank	0.18 (0.04-0.80)	<.03
Time started studying	2.90 (1.38-6.29)	<.01
COMSAE as most helpful practice examination	0.30 (0.08-1.05)	.06

Abbreviations: COMBANK, question bank for the Comprehensive Osteopathic Medical Licensing Examination-USA (COMLEX-USA); COMSAE, Comprehensive Osteopathic Medical Self-Assessment Examination.

Comment

Our findings are consistent with those of other studies^12-17 in demonstrating that preclinical GPA and MCAT scores are statistically significant predictors of higher scores on COMLEX-USA. These findings underscore the importance of maintaining a high GPA during the first 2 years of medical school. Students need to forgo the misperception that studying intensively during the few months preceding the examination alone will be sufficient to guarantee optimal performance. Preparing for this examination is more like running a marathon than a 500-m dash. A consistent program of study and a high GPA during the first 2 years of medical school is one of the best test preparation strategies.

The present study is, to our knowledge, among the first to show the importance of starting earlier in preparing for COMLEX-USA Level 1. The earlier participants began to prepare, the more likely they scored 600 or higher. Future studies need to further elucidate this variable and determine the optimal time to initiate preparing for the test. Information regarding the variability in the number of hours spent in examination preparation and the resources used at different time points would also be helpful.

Surprisingly, there was an association with not ranking the COMSAE as the most helpful practice examination and scoring 600 or higher. This finding is difficult to interpret and must be viewed within the limitations of our data collection tool. It is possible that this finding is an artifact of how the item was worded because we asked participants to rank order the 3 most helpful practice examinations. Students who used more than 1 practice examination might have had difficulty answering the question, and the rank order they provided might have been a function of which examination first came to their mind as they answered the question. Additionally, 20% of participants included question banks in their responses to this item, which suggests some confusion about the phrasing of the item. It could also be that some students are not clear on the differences between practice examinations such as COMSAE, which are shorter, and question banks, which are much longer. Students who spent many hours answering question banks may have found COMSAE to be less useful because of its limited number of items. These students may not have realized that the purpose of COMSAE is to help assess readiness for taking COMLEX-USA because it provides a simulated structure and format that mirrors the actual COMLEX-USA. Alternatively, some students may have used question banks as a substitute for practice examinations. Given the format that we used to collect these data, this finding should not be interpreted as a recommendation not to incorporate COMSAE as part of a comprehensive examination preparation strategy. Future studies need to improve the format of this item to ensure that respondents differentiate between practice examinations and question banks.

In our second regression model, we did not include the preclinical GPA and MCAT scores to determine which variables were associated with scoring 600 or higher once the effect of these 2 variables were removed. Similar to the first regression, initiation of examination preparation earlier was associated with higher scores on COMLEX-USA Level 1. This finding suggests that studying earlier is important for scoring high on COMLEX-USA Level 1. However, not ranking COMSAE as the most helpful practice examination did not emerge as a statistically significant predictor, but not ranking COMBANK as the most helpful question bank was a statistically significant predictor.

We conducted post hoc analyses to help explain these associations. The frequency distribution of the responses to this question were interesting in that 56 participants (51.4%) ranked COMBANK as the most helpful question bank and 45 (41.3%) ranked USMLE WORLD (UWORLD) as the most helpful question bank. Given the similarity of the distributions, we conducted additional bivariate analyses comparing those who ranked UWORLD as the most helpful question bank with those who did not and found a statistically significant association between ranking UWORLD as the most helpful question bank and scoring 600 or higher on COMLEX-USA Level 1. We then included UWORLD rather than COMBANK as an independent variable and ran a logistic regression, and UWORLD did not emerge as a statistically significant predictor of scoring 600 or higher in the final model.

Given that our sample size was moderate and the distribution of participants ranking each question bank as most helpful were comparable, it could be that the effects were attenuated because of insufficient power. It is noteworthy that a high proportion of students stressed the importance of question banks as an examination preparation resource. Unfortunately, given the limitations of our study we are not able to address this issue more fully. Future studies with larger sample sizes are needed to explore more fully the contributions of question banks as COMLEX-USA preparation methods. These studies should gather quantitative data on the number of practice questions each student completes, the specific question banks used, and when students started to use these question banks to provide more specific guidance regarding examination preparation strategies.

The present study was not without limitations. First, our sample was restricted to osteopathic medical students attending NSU-COM; thus, the generalizability of our findings is limited. Future studies should recruit participants from across the spectrum of osteopathic medical schools in the country. Further attention needs to be given to the data collection tool we used. Some of the items could have been worded to elicit direct responses rather than Likert-scale responses. Also, it may have been helpful if we provided definitions of the examination preparation resources to prevent some of the confusion that emerged between a question bank and a practice examination. Rank-ordering of the specific examination preparation tools used may also have been difficult for respondents.

Conclusion

Our study investigated and described how students prepared for the licensing examination and which examination preparation resources were the most useful and most helpful. Our study was among the first to demonstrate the association between earlier initiation of examination preparation activities and test performance. We also confirmed the previously established association between strong academic performance during the first 2 years of osteopathic medical school and obtaining high scores on the licensing examination. Many medical students are concerned about their performance on the licensing examinations and seek guidance in developing effective examination preparation strategies. At the start of medical school, professors, administrators, and academic advisors should strongly encourage students to establish a consistent program of study and maintain a high GPA to maximize their performance on the licensing examinations. Students should be encouraged to consider starting their examination preparation as early as possible. Future studies are needed to examine in greater detail which are the best examination preparation resources, the most optimal time to initiate examination preparation activities, and how to develop an effective strategy for preparing for examinations.

Acknowledgments

We thank NSU-COM for funding this study, and Dean Anthony J. Silvagni, DO, PharmD, MSc, for his continued support of the research fellowship program. We also thank the osteopathic medical students who took the time to complete our survey.

Financial Disclosures: None reported.

Support: This work was supported by the Research Fellowship Program of Nova Southeastern University College of Osteopathic Medicine.

References

1

Osteopathic Medical Profession Report. Chicago, IL: American Osteopathic Association; 2012. http://www.osteopathic.org/inside-aoa/about/aoa-annual-statistics/Documents/2012-OMP-report.pdf. Accessed January 14, 2012.

2

COMLEX-USA Bulletin of Information. Chicago, IL: National Board of Osteopathic Medical Examiners; 2012:8. http://www.nbome.org/docs/comlexBOI.pdf. Accessed September 18, 2012.

3

Osteopathic GME Match Report 2009. Chevy Chase, MD: American Association of Colleges of Osteopathic Medicine; 2011. http://data.aacom.org/media/DO_GME_match_2009.pdf. Accessed October 5, 2011.

4

Teitelbaum HS. Osteopathic Medical Education in the United States: Improving the Future of Medicine. Chevy Chase, MD: American Association of Colleges of Osteopathic Medicine; 2005:147. http://www.do-online.org/pdf/acc_mededstudy05.pdf. Accessed January 14, 2013.

5

Bates BP. Selection criteria for applicants in primary care osteopathic graduate medical education. J Am Osteopath Assoc. 2002;102(11):621-626. [PubMed]

6

Taylor CA, Weinstein L, Mayhew HE. The process of resident selection: a view from the residency director's desk. Obstet Gynecol. 1995;85(2):299-303. [CrossRef] [PubMed]

7

Crane JT, Ferraro CM. Selection criteria for emergency residency applicants. Acad Emerg Med. 2000;7(1):54-60. http://onlinelibrary.wiley.com/doi/10.1111/j.1553-2712.2000.tb01892.x/pdf. Accessed January 14, 2013

8

Guffey RC, Rusin K, Chidiac EJ, Marsh HM. The utility of pre-residency standardized tests for anesthesiology resident selection: the place of United States Medical Licensing Examination scores. Anesth Analg. 2011;112(1):201-206. http://www.anesthesia-analgesia.org/content/112/1/201.full. Accessed January 14, 2013.

9

Rinard JR, Mahabir RC. Successfully matching into surgical specialties: an analysis of national resident matching program data. J Grad Med Educ. 2010;2(3):316-321. http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2951766/?tool=pubmed. Accessed January 14, 2013.

10

Bernstein AD, Jazrawi LM, Elbeshbeshy B, Della Valle CJ, Zuckerman JD. An analysis of orthopaedic residency selection criteria. Bull Hosp Jt Dis. 2002-2003;61(1,2):49-57. http://www.nyuhjdbulletin.org/Permalink.aspx?permalinkId=d240fb53-2d4e-4947-8725-319d69956ba0. Accessed January 14, 2013.

11

McGaghie WC, Cohen ER, Wayne DB. Are United States Medical Licensing Exam Step 1 and 2 scores valid measures for postgraduate medical residency selection decisions? Acad Med. 2011;86(1):48-52. [CrossRef] [PubMed]

12

Hartman SE, Bates BP, Sprafka SA. Correlation of scores for the Comprehensive Osteopathic Medical Licensing Examination with osteopathic medical school grades. J Am Osteopath Assoc. 2001;101(6):347-349. http://www.jaoa.org/content/101/6/347.full.pdf+html. Accessed January 14, 2013.

13

Sefcik DJ, Prozialeck WC, O'Hare TH. Characteristics of the courses that best predict COMLEX-USA Level 1 performance. J Am Osteopath Assoc. 2003;103(10):491-494. http://www.jaoa.org/content/103/10/491.full.pdf. Accessed January 14, 2013.

14

Wong SK, Ramirez JR, Helf SC. Student performance on Levels 1 and 2-CE of COMLEX-USA: do elective upper-level undergraduate science courses matter? J Am Osteopath Assoc. 2009;109(11):592-598. http://www.jaoa.org/content/109/11/592.full. Accessed January 14, 2013.

15

Dixon D. Relation between variables of preadmission, medical school performance, and COMLEX-USA Levels 1 and 2 performance. J Am Osteopath Assoc. 2004;104(8):332-336. http://www.jaoa.org/content/104/8/332.full. Accessed January 14, 2013.

16

Evans P, Wen FK. Does the Medical College Admission Test predict global academic performance in osteopathic medical school? J Am Osteopath Assoc. 2007;107(4):157-162. http://www.jaoa.org/content/107/4/157.full. Accessed January 14, 2013.

17

Meoli FG, Wallace WS, Kaiser-Smith J, Shen L. Relationship of osteopathic medical licensure examinations with undergraduate admission measures and predictive value of identifying future performance in osteopathic principles and practice/osteopathic manipulative medicine courses and rotations. J Am Osteopath Assoc. 2002;102(11):615-620. http://www.jaoa.org/content/102/11/615.long. Accessed January 14, 2013.

18

Baker HH, Foster RW, Bates BPet al. Relationship between academic achievement and COMLEX-USA Level 1 performance: a multisite study. J Am Osteopath Assoc. 2000;100(4):238-242. http://www.jaoa.org/content/100/4/238.full.pdf+html. Accessed January 14, 2013.

19

Hosmer DW, Lemeshow S. Applied Logistic Regression. 2nd ed. John Wiley and Sons, Inc: New York, NY; 2000. http://books.google.com/books?hl=en&lr=&id=Po0RLQ7USIMC&oi=fnd&pg=PA1&dq=applied+logistic+regression+hosmer&ots=Dn70qh_jCU&sig=VnqPoPl6_94KZt5o03UJb61d3qc#v=onepage&q=applied%20logistic%20regression%20hosmer&f=false. Accessed February 5, 2012.

Table 1.

Predictors of COMLEX-USA Level 1 Performance: Characteristics of Survey Respondents (N=113)

Characteristic	No. (%)^a
Age, mean (SD), y (n=110)	26.3 (2.2)
Sex (n=113)
Male	66 (58.4)
Female	47 (41.6)
Race (n=112)
Asian	18 (16.1)
Black/African American	1 (0.9)
White	81 (72.3)
Other	12 (10.7)
Ethnicity (n=105)
Hispanic/Latino	16 (15.2)
Non-Hispanic/Latino	89 (84.8)
Highest-Earned Academic Degree (n=107)
Bachelor's	85 (79.4)
Master's	19 (17.8)
Doctorate	3 (2.8)
Marital Status (n=112)
Single	80 (71.4)
Married	29 (25.9)
Divorced	2 (1.8)
Domestic Partnership	1 (0.9)
Number of Children (n=109)
0	100 (91.7)
1	6 (5.5)
≥2	3 (2.8)
Annual Family Income (n=110)
<$20,000	25 (20.9)
$20,000-$60,000	20 (17.3)
$60,001-$100,000	25 (20.9)
$100,001-$200,000	19 (17.3)
$200,001+	29 (23.6)
Perceived Performance as Medical Student (n=112)^a
Outstanding	14 (12.5)
Excellent	48 (42.9)
Good	38 (33.9)
Fair	12 (10.7)
Poor	0
MCAT Score, mean (SD) (n=111)^b	26.7 (3.0)
COMLEX-USA Level 1 Score, mean (SD) (n=113)^c	524.9 (86.1)
Grade Point Average, mean (SD) (n=113)^d	88.4 (4.2)

^a Data are presented as No. (%) except where otherwise noted.

^b The maximum Medical College Admission Test (MCAT) score is 45.

^c The maximum score on the Comprehensive Osteopathic Medical Licensing Examination-USA (COMLEX-USA) Level 1 is 800.

^d The maximum grade point average is 100.

Abbreviation: SD, standard deviation.

View Large

Table 1.

Predictors of COMLEX-USA Level 1 Performance: Characteristics of Survey Respondents (N=113)

Characteristic	No. (%)^a
Age, mean (SD), y (n=110)	26.3 (2.2)
Sex (n=113)
Male	66 (58.4)
Female	47 (41.6)
Race (n=112)
Asian	18 (16.1)
Black/African American	1 (0.9)
White	81 (72.3)
Other	12 (10.7)
Ethnicity (n=105)
Hispanic/Latino	16 (15.2)
Non-Hispanic/Latino	89 (84.8)
Highest-Earned Academic Degree (n=107)
Bachelor's	85 (79.4)
Master's	19 (17.8)
Doctorate	3 (2.8)
Marital Status (n=112)
Single	80 (71.4)
Married	29 (25.9)
Divorced	2 (1.8)
Domestic Partnership	1 (0.9)
Number of Children (n=109)
0	100 (91.7)
1	6 (5.5)
≥2	3 (2.8)
Annual Family Income (n=110)
<$20,000	25 (20.9)
$20,000-$60,000	20 (17.3)
$60,001-$100,000	25 (20.9)
$100,001-$200,000	19 (17.3)
$200,001+	29 (23.6)
Perceived Performance as Medical Student (n=112)^a
Outstanding	14 (12.5)
Excellent	48 (42.9)
Good	38 (33.9)
Fair	12 (10.7)
Poor	0
MCAT Score, mean (SD) (n=111)^b	26.7 (3.0)
COMLEX-USA Level 1 Score, mean (SD) (n=113)^c	524.9 (86.1)
Grade Point Average, mean (SD) (n=113)^d	88.4 (4.2)

^a Data are presented as No. (%) except where otherwise noted.

^b The maximum Medical College Admission Test (MCAT) score is 45.

^c The maximum score on the Comprehensive Osteopathic Medical Licensing Examination-USA (COMLEX-USA) Level 1 is 800.

^d The maximum grade point average is 100.

Abbreviation: SD, standard deviation.

Table 2.

Predictors of COMLEX-USA Level 1 Performance: Summary of Examination Preparation Strategies Among Survey Respondents (N=113)

Survey Question	n	No. (%)^a
Start of Examination Preparation
During first year	112	6 (5.4)
September to December of second year	112	17 (15.2)
January to March of second year	112	39 (34.8)
April to May of second year	112	50 (44.6)
Time Studying Alone, mean (SD), %	108	81.3 (25.3)
Study Aids: Almost Always/Always Used
Review books	111	106 (95.5)
Practice examinations	112	56 (50.0)
Question banks	113	103 (91.2)
Lecture videos	112	49 (43.8)
Class notes	112	3 (2.7)
Study Aids: Usually/Very Helpful
Review books	109	102 (93.6)
Practice examinations	102	71 (69.6)
Question banks	109	101 (92.7)
Lecture videos	95	63 (66.3)
Class notes	46	12 (26.1)
Review Book Ranked Most Helpful
First Aid for the USMLE	112	94 (83.9)
Practice Examination Ranked Most Helpful
COMSAE	97	60 (61.9)
Question Bank Ranked Most Helpful
COMBANK	109	56 (51.4)
Lecture Video Ranked Most Helpful
Kaplan USMLE	96	69 (72.6)

^a Data are presented as No. (%) except where otherwise noted.

Abbreviations: COMBANK, question bank for the Comprehensive Osteopathic Medical Licensing Examination-USA (COMLEX-USA); COMSAE, Comprehensive Osteopathic Medical Self Assessment Examination; SD, standard deviation; USMLE, United States Medical Licensing Examination.

View Large

Table 2.

Predictors of COMLEX-USA Level 1 Performance: Summary of Examination Preparation Strategies Among Survey Respondents (N=113)

Survey Question	n	No. (%)^a
Start of Examination Preparation
During first year	112	6 (5.4)
September to December of second year	112	17 (15.2)
January to March of second year	112	39 (34.8)
April to May of second year	112	50 (44.6)
Time Studying Alone, mean (SD), %	108	81.3 (25.3)
Study Aids: Almost Always/Always Used
Review books	111	106 (95.5)
Practice examinations	112	56 (50.0)
Question banks	113	103 (91.2)
Lecture videos	112	49 (43.8)
Class notes	112	3 (2.7)
Study Aids: Usually/Very Helpful
Review books	109	102 (93.6)
Practice examinations	102	71 (69.6)
Question banks	109	101 (92.7)
Lecture videos	95	63 (66.3)
Class notes	46	12 (26.1)
Review Book Ranked Most Helpful
First Aid for the USMLE	112	94 (83.9)
Practice Examination Ranked Most Helpful
COMSAE	97	60 (61.9)
Question Bank Ranked Most Helpful
COMBANK	109	56 (51.4)
Lecture Video Ranked Most Helpful
Kaplan USMLE	96	69 (72.6)

^a Data are presented as No. (%) except where otherwise noted.

Abbreviations: COMBANK, question bank for the Comprehensive Osteopathic Medical Licensing Examination-USA (COMLEX-USA); COMSAE, Comprehensive Osteopathic Medical Self Assessment Examination; SD, standard deviation; USMLE, United States Medical Licensing Examination.

Table 3.

Predictors of COMLEX-USA Level 1 Performance: Tips From Survey Respondents (n=100)

Tip	No. (%)
Do a lot of practice questions	48 (48)
Use First Aid for the USMLE with courses	26 (26)
Start early	26 (26)
Make a schedule	10 (10)

Abbreviations: COMLEX-USA, Comprehensive Osteopathic Medical Licensing Examination-USA; USMLE, United States Medical Licensing Examination.

View Large

Table 3.

Predictors of COMLEX-USA Level 1 Performance: Tips From Survey Respondents (n=100)

Tip	No. (%)
Do a lot of practice questions	48 (48)
Use First Aid for the USMLE with courses	26 (26)
Start early	26 (26)
Make a schedule	10 (10)

Abbreviations: COMLEX-USA, Comprehensive Osteopathic Medical Licensing Examination-USA; USMLE, United States Medical Licensing Examination.

Table 4.

Multivariate Logistic Regression: Predictors of Scoring ≥600 on COMLEX-USA Level 1 Among Survey Respondents (N=113)

Predictor	Odds Ratio (95% Confidence Interval)	P Value
Sex	1.46 (0.34-6.25)	.608
Race	1.47 (0.68-3.20)	.327
COMBANK as most helpful question bank	0.24 (0.05-1.31)	.099
Time started studying	2.53 (1.004-6.33)	<.05
COMSAE as most helpful practice examination	0.19 (0.04-0.86)	<.04
Medical College Admission Test score	1.30 (1.001-1.69)	<.05
Preclinical grade point average	1.35 (1.06-1.71)	<.02

Abbreviations: COMBANK, question bank for the Comprehensive Osteopathic Medical Licensing Examination-USA; COMSAE, Comprehensive Osteopathic Medical Self-Assessment Examination.

View Large

Table 4.

Multivariate Logistic Regression: Predictors of Scoring ≥600 on COMLEX-USA Level 1 Among Survey Respondents (N=113)

Predictor	Odds Ratio (95% Confidence Interval)	P Value
Sex	1.46 (0.34-6.25)	.608
Race	1.47 (0.68-3.20)	.327
COMBANK as most helpful question bank	0.24 (0.05-1.31)	.099
Time started studying	2.53 (1.004-6.33)	<.05
COMSAE as most helpful practice examination	0.19 (0.04-0.86)	<.04
Medical College Admission Test score	1.30 (1.001-1.69)	<.05
Preclinical grade point average	1.35 (1.06-1.71)	<.02

Abbreviations: COMBANK, question bank for the Comprehensive Osteopathic Medical Licensing Examination-USA; COMSAE, Comprehensive Osteopathic Medical Self-Assessment Examination.

Table 5.

Multivariate Logistic Regression: Predictors of Scoring ≥600 on COMLEX-USA Level 1 Among Survey Respondents, Excluding Grade Point Average and Medical College Admission Test Score (N=113)

Predictor	Odds Ratio (95% Confidence Interval)	P Value
Sex	2.34 (0.67-8.20)	.186
Race	1.50 (0.71-3.16)	.288
COMBANK as most helpful question bank	0.18 (0.04-0.80)	<.03
Time started studying	2.90 (1.38-6.29)	<.01
COMSAE as most helpful practice examination	0.30 (0.08-1.05)	.06

Abbreviations: COMBANK, question bank for the Comprehensive Osteopathic Medical Licensing Examination-USA (COMLEX-USA); COMSAE, Comprehensive Osteopathic Medical Self-Assessment Examination.

View Large

Table 5.

Multivariate Logistic Regression: Predictors of Scoring ≥600 on COMLEX-USA Level 1 Among Survey Respondents, Excluding Grade Point Average and Medical College Admission Test Score (N=113)

Predictor	Odds Ratio (95% Confidence Interval)	P Value
Sex	2.34 (0.67-8.20)	.186
Race	1.50 (0.71-3.16)	.288
COMBANK as most helpful question bank	0.18 (0.04-0.80)	<.03
Time started studying	2.90 (1.38-6.29)	<.01
COMSAE as most helpful practice examination	0.30 (0.08-1.05)	.06

Abbreviations: COMBANK, question bank for the Comprehensive Osteopathic Medical Licensing Examination-USA (COMLEX-USA); COMSAE, Comprehensive Osteopathic Medical Self-Assessment Examination.

Predictors of Scoring at Least 600 on COMLEX-USA Level 1: Successful Preparation Strategies

Related content

Related Topics