Abstract
Context: Comprehensive Osteopathic Medical Licensing Examination-USA (COMLEX-USA) Level 1 scores are an important criterion used by residency directors to make residency placement decisions.
Objective: To explore the association between scoring at least 600 on COMLEX-USA Level 1 and grade point average (GPA), scores on the Medical College Admission Test (MCAT), and different test preparation strategies.
Methods: Third-year osteopathic medical students at Nova Southeastern University were invited to complete a self-administered survey regarding their COMLEX-USA preparation strategies and to provide consent for the researchers to access their preclinical GPA and their MCAT and COMLEX-USA scores. Descriptive analyses were conducted to understand examination preparation procedures and resources used, and bivariate analyses were conducted to identify the statisically significant predictors of scoring 600 or higher. Two separate logistic regressions were also run. The first included all of the statisically significant factors that emerged from the bivariate analyses, and the second examined which candidate predictors remained statistically significant once the effects of GPA and MCAT scores were removed.
Results: One hundred twenty-two students completed the survey, and 113 (93%) provided informed consent to access their preclinical GPA and their MCAT and COMLEX-USA scores. In the first regression, scoring 600 or higher was associated with a higher GPA (P<.02), a higher MCAT score (P<.05), earlier preparation initiation (P<.05), and not ranking the Comprehensive Osteopathic Medical Self-Assessment Examination (COMSAE) as the most helpful practice examination (P<.04). In the second regression, scoring 600 or higher was associated with earlier initiation of examination preparation (P<.01) and not ranking COMBANK (question bank for COMLEX-USA) as the most helpful question bank (P<.03). Among the different examination preparation methods, the specific resources ranked as most helpful were First Aid for the USMLE (United States Medical Licensing Examination) (review book), the COMSAE (practice examination); COMBANK (question bank); and Kaplan USMLE (lecture videos).
Conclusion: Preclinical GPA and MCAT scores continue to be important predictors of scoring at least 600 on COMLEX-USA Level 1. However, the findings underscore the importance of maintaining a high GPA during the first 2 years of medical school and initiating COMLEX-USA preparation early.
During the past few years, osteopathic medicine in the United States has experienced tremendous growth in both the number of physicians and the awareness of the profession, as evidenced by the increasing number of osteopathic medical graduates.
1 This trend is continuing. In 2012, the number of newly graduated doctors of osteopathic medicine reached the new peak of 4773,
1 and osteopathic physicians are increasingly more visible across the range of medical specialties. To ensure that osteopathic physicians are proficient medical providers, the osteopathic medical profession maintains strict criteria for osteopathic licensure, which includes passing all 3 levels of the Comprehensive Osteopathic Medical Licensing Examination-USA (COMLEX-USA). This examination is comparable to the allopathic licensing examination, the United States Medical Licensing Examination (USMLE). Level 1 of COMLEX-USA, the first examination in the 3-part series, is typically taken at the end of the second year of osteopathic medical school and consists of two 4-hour test sessions completed in 1 day.
2 It is “a problem-based and symptom-based assessment integrating the basic sciences of anatomy, behavioral science, biochemistry, microbiology, osteopathic principles, pathology, pharmacology, physiology, and other areas of medical knowledge as they are relevant to solving medical problems.”
2
Despite evidence questioning the validity of licensing test scores as measures of clinical competence,
3 many residency directors continue to use COMLEX-USA Level 1 scores to make residency placement decisions. Studies have shown that COMLEX-USA scores are an important criterion used by program directors in selecting residents.
4-6 A similar practice has been reported in allopathic medical training. Studies of residency directors in obstetrics and gynecology, family medicine, emergency medicine, anesthesiology, general surgery, and orthopedic surgery have shown that a candidate's USMLE Step 1 score is one of the most highly weighted variables in deciding whether to grant a student an interview and select him or her as a resident.
7-11 Although licensing examination scores are not correlated with reliable measures of clinical skill acquision,
3 it is clear that Level 1 and Level 2 board scores continue to be used as important factors in both the osteopathic and the allopathic residency selection process.
Emerging literature has examined factors associated with test performance on COMLEX-USA Level 1.
12-14 There is strong evidence that performance on COMLEX-USA Level 1 is correlated with scores on the Medical College Admission Test (MCAT)
15-17 and grade point averages (GPAs) during the first 2 years of medical school (ie, preclinical GPA).
12,13,15,18 The role of other factors—specifically examination preparation approaches, time spent studying, and types of examination preparation resources used—on COMLEX-USA Level 1 performance has, to our knowledge, not been rigorously examined. Understanding the relative contribution of such factors is important because it can provide additional guidance to medical students on how best to prepare for this examination. Despite the importance of these test scores on the future careers of medical students, we could not find a single study directly focused on COMLEX-USA Level 1 to guide osteopathic medical students to prepare for this important examination. The present study directly addresses this need.
The goal of this exploratory study was to identify the factors associated with scoring 600 or higher on COMLEX-USA Level 1. More specifically, we examined the relationship between scores on COMLEX-USA Level 1 and examination preparation resources (eg, question banks, review books, practice examinations, lecture videos), length of time preparing, study format (individual or group), preclinical GPA, and MCAT scores.
Methods
Participants
Participants were third-year osteopathic medical students at Nova Southeastern University College of Osteopathic Medicine (NSU-COM) who attended a residency fair on November 10, 2011, on campus. All study participants had taken COMLEX-USA Level 1 by September 30, 2011, as required by NSU-COM policy.
Design
This study used a cross-sectional design. The protocol was approved in September 2011 by the institutional review board at Nova Southeastern University.
Procedures
While third-year osteopathic medical students were gathered in a school auditorium, we briefly presented the study and informed consent procedures for obtaining access to participants' preclinical GPAs and their MCAT and COMLEX-USA Level 1 scores. Students could voluntarily agree to (1) complete the survey and provide consent to access their scores, (2) complete only the survey, or (3) not participate in either study component. The researchers stressed that participation was voluntary and that there would be no consequences for electing not to participate. A question and answer period followed the presentation. The researchers then distributed envelopes containing an introductory letter, informed consent forms, a survey instrument, and a $1 bill as a token of appreciation. Students who agreed to participate completed the survey; those who granted permission to access their scores signed the informed consent. Participants placed the completed documents into the envelopes, sealed them, and placed them in a collection box. To protect the confidentiality of participants, each survey was assigned a unique study identification number, which was preprinted on the survey forms; the informed consent contained the only personally identifying information.
Faculty mentors on the research team at NSU-COM opened the sealed envelopes and prepared a spreadsheet linking each participant's name to their study identification number. The survey data were entered into an SPSS data file (IBM Corporation, Chicago, Illinois) and merged with a data file containing preclinical GPA and MCAT and COMLEX-USA Level 1 scores.
Survey Instrument
The research team developed the items by using examples from the published literature and methodological experts at NSU-COM. It consisted of 35 items that tapped 6 domains: demographics, examination methods, perceived performance as an osteopathic medical student, study format, student tips, and GPA and MCAT and COMLEX-USA Level 1 scores.
Demographics
Participants reported their age, sex, race, ethnicity, highest-earned academic degree, year of degree completion, undergraduate major, marital status, number of children, and annual family income.
Examination Preparation Methods
Participants reported when they began preparing for the COMLEX-USA Level 1 (first year of osteopathic medical school, fall of second year [September to December], winter of second year [January to March], spring of second year [April to May]), and the number of hours they studied per week for each of the 8 weeks leading up to the examination. For each of 5 broad categories of examination preparation resources (review books, practice examinations, question banks, lecture videos, and class notes), participants reported how often they used it and, if used, how helpful it was. Both were rated using a 5-point scale, from “never” (1) to “always” (5) for the first question and from “unhelpful” (1) to “very helpful” (5) for the follow-up question. To gain more specific information regarding the resources used, participants rank-ordered the 3 most helpful resources within each of the 5 broad categories.
Performance as a Medical Student
Participants self-ranked their own perceived performance as osteopathic medical students on a 5-point scale from “poor” (1) to “outstanding” (5).
Study Format
Participants reported the percentage of time they studied alone and the percentage of time they studied in groups.
Student Tips
Using an open-ended format, participants provided specific tips or advice they would give to other students to help them prepare for the examination.
Preclinical GPA and MCAT and COMLEX-USA Scores
Administrative staff at NSU-COM provided the preclinical GPA and MCAT and official COMLEX-USA Level 1 scores. These data were then linked to each participant's survey responses prior to analysis.
Dependent Variable
The dependent variable was COMLEX-USA Level 1 scores. Students were classified as having a high score if they scored 600 or higher on the examination. A score of 600 to 800 is within the top 25% of the scores on COMLEX-USA Level 1. Therefore, we used this cutoff point to differentiate students with high scores from all others.
Analyses
Data were entered into a data file (SPSS version 19) and cross-checked for reliability by using standard quality assurance procedures. We first conducted descriptive analyses to describe the sample and to understand medical students' examination preparation procedures and resources used. Next we classified participants according to their COMLEX-USA Level 1 scores. Participants who scored 600 or higher were classified as 1 and those who scored lower than 600 were classified as 0. We conducted bivariate analyses on theoretically relevant factors to examine the relationship between each factor and the dependent variable. Specifically, we investigated demographic variables (age, sex, race, annual family income, marital status), ratings of performance in medical school, examination preparation resources, initiation of examination preparation, study format, preclinical GPA, and MCAT score. Next, we conducted 2 multivariable logistic regressions to examine the relationship between theoretically relevant variables and COMLEX-USA Level 1 scores. Following the recommendations of Hosmer and Lemeshow,
19 we included factors with
P values of .20 or lower in the bivariate analysis as candidate predictors in the first logistic regression because use of more traditional significance values might fail to identify important relationships. We ran a second regression excluding preclinical GPA and MCAT scores as independent variables to explore whether the remaining candidate predictors were associated with higher scores on COMLEX-USA Level 1 once the effect of these 2 factors was removed.
Results
Population Sample
Of the 152 students in attendance at the fair, 122 (80%) completed a self-administered survey, 113 of whom also provided informed consent to access their preclinical GPA and MCAT and COMLEX-USA Level 1 scores. Participants who did not provide consent (n=9) were not included in the analyses. All surveys that were turned in were included in the analyses even though a few participants did not complete all of the items.
The characteristics of the population sample are available in
Table 1. A majority of the respondents were male (66 [58.4%]), white (81 [72.3%]), and single (80 [71.4%]). Nearly a quarter of respondents (29 [23.6%]) had an annual family income greater than $200,000. Sixty-two participants (55.4%) rated their performance in medical school to be outstanding or excellent. The mean (standard deviation [SD]) preclinical GPA was 88.4 (4.2); MCAT score, 26.7 (3); and COMLEX-USA Level 1 score, 524.9 (86.1). Twenty-one participants (18.6%) scored 600 or higher on COMLEX-USA Level 1.
Table 1.
Predictors of COMLEX-USA Level 1 Performance: Characteristics of Survey Respondents (N=113)
| Characteristic | No. (%)a |
| Age, mean (SD), y (n=110) | 26.3 (2.2) |
| Sex (n=113) | |
| Male | 66 (58.4) |
| Female | 47 (41.6) |
| Race (n=112) | |
| Asian | 18 (16.1) |
| Black/African American | 1 (0.9) |
| White | 81 (72.3) |
| Other | 12 (10.7) |
| Ethnicity (n=105) | |
| Hispanic/Latino | 16 (15.2) |
| Non-Hispanic/Latino | 89 (84.8) |
| Highest-Earned Academic Degree (n=107) |
| Bachelor's | 85 (79.4) |
| Master's | 19 (17.8) |
| Doctorate | 3 (2.8) |
| Marital Status (n=112) | |
| Single | 80 (71.4) |
| Married | 29 (25.9) |
| Divorced | 2 (1.8) |
| Domestic Partnership | 1 (0.9) |
| Number of Children (n=109) | |
| 0 | 100 (91.7) |
| 1 | 6 (5.5) |
| ≥2 | 3 (2.8) |
| Annual Family Income (n=110) | |
| <$20,000 | 25 (20.9) |
| $20,000-$60,000 | 20 (17.3) |
| $60,001-$100,000 | 25 (20.9) |
| $100,001-$200,000 | 19 (17.3) |
| $200,001+ | 29 (23.6) |
| Perceived Performance as Medical Student (n=112)a | |
| Outstanding | 14 (12.5) |
| Excellent | 48 (42.9) |
| Good | 38 (33.9) |
| Fair | 12 (10.7) |
| Poor | 0 |
| MCAT Score, mean (SD) (n=111)b | 26.7 (3.0) |
| COMLEX-USA Level 1 Score, mean (SD) (n=113)c | 524.9 (86.1) |
| Grade Point Average, mean (SD) (n=113)d | 88.4 (4.2) |
Table 1.
Predictors of COMLEX-USA Level 1 Performance: Characteristics of Survey Respondents (N=113)
| Characteristic | No. (%)a |
| Age, mean (SD), y (n=110) | 26.3 (2.2) |
| Sex (n=113) | |
| Male | 66 (58.4) |
| Female | 47 (41.6) |
| Race (n=112) | |
| Asian | 18 (16.1) |
| Black/African American | 1 (0.9) |
| White | 81 (72.3) |
| Other | 12 (10.7) |
| Ethnicity (n=105) | |
| Hispanic/Latino | 16 (15.2) |
| Non-Hispanic/Latino | 89 (84.8) |
| Highest-Earned Academic Degree (n=107) |
| Bachelor's | 85 (79.4) |
| Master's | 19 (17.8) |
| Doctorate | 3 (2.8) |
| Marital Status (n=112) | |
| Single | 80 (71.4) |
| Married | 29 (25.9) |
| Divorced | 2 (1.8) |
| Domestic Partnership | 1 (0.9) |
| Number of Children (n=109) | |
| 0 | 100 (91.7) |
| 1 | 6 (5.5) |
| ≥2 | 3 (2.8) |
| Annual Family Income (n=110) | |
| <$20,000 | 25 (20.9) |
| $20,000-$60,000 | 20 (17.3) |
| $60,001-$100,000 | 25 (20.9) |
| $100,001-$200,000 | 19 (17.3) |
| $200,001+ | 29 (23.6) |
| Perceived Performance as Medical Student (n=112)a | |
| Outstanding | 14 (12.5) |
| Excellent | 48 (42.9) |
| Good | 38 (33.9) |
| Fair | 12 (10.7) |
| Poor | 0 |
| MCAT Score, mean (SD) (n=111)b | 26.7 (3.0) |
| COMLEX-USA Level 1 Score, mean (SD) (n=113)c | 524.9 (86.1) |
| Grade Point Average, mean (SD) (n=113)d | 88.4 (4.2) |
×
Summaries of the different preparation strategies used by participants are found in
Table 2. The majority of participants began preparing for COMLEX-USA Level 1 from January to March (39 [34.8%]) or from April to May (50 [44.6%]) of their second year of medical school. The most frequently used and most helpful examination preparation resources were review books and question banks. One hundred two participants (93.6%) reported review books as helpful or very helpful, while 101 (92.7%) reported question banks as helpful or very helpful. Among the different examination preparation methods, the specific resources ranked as most helpful were
First Aid for the USMLE (review book), the Comprehensive Osteopathic Medical Self-Assessment Examination (COMSAE) (practice examination), COMBANK (question bank [for COMLEX-USA]), and Kaplan USMLE (lecture videos). Eighty-one percent of students reported that they studied alone.
Table 2.
Predictors of COMLEX-USA Level 1 Performance: Summary of Examination Preparation Strategies Among Survey Respondents (N=113)
| Survey Question | n | No. (%)a |
| Start of Examination Preparation | | |
| During first year | 112 | 6 (5.4) |
| September to December of second year | 112 | 17 (15.2) |
| January to March of second year | 112 | 39 (34.8) |
| April to May of second year | 112 | 50 (44.6) |
| Time Studying Alone, mean (SD), % | 108 | 81.3 (25.3) |
| Study Aids: Almost Always/Always Used | | |
| Review books | 111 | 106 (95.5) |
| Practice examinations | 112 | 56 (50.0) |
| Question banks | 113 | 103 (91.2) |
| Lecture videos | 112 | 49 (43.8) |
| Class notes | 112 | 3 (2.7) |
| Study Aids: Usually/Very Helpful | | |
| Review books | 109 | 102 (93.6) |
| Practice examinations | 102 | 71 (69.6) |
| Question banks | 109 | 101 (92.7) |
| Lecture videos | 95 | 63 (66.3) |
| Class notes | 46 | 12 (26.1) |
| Review Book Ranked Most Helpful | | |
| First Aid for the USMLE | 112 | 94 (83.9) |
| Practice Examination Ranked Most Helpful | | |
| COMSAE | 97 | 60 (61.9) |
| Question Bank Ranked Most Helpful | | |
| COMBANK | 109 | 56 (51.4) |
| Lecture Video Ranked Most Helpful | | |
| Kaplan USMLE | 96 | 69 (72.6) |
Table 2.
Predictors of COMLEX-USA Level 1 Performance: Summary of Examination Preparation Strategies Among Survey Respondents (N=113)
| Survey Question | n | No. (%)a |
| Start of Examination Preparation | | |
| During first year | 112 | 6 (5.4) |
| September to December of second year | 112 | 17 (15.2) |
| January to March of second year | 112 | 39 (34.8) |
| April to May of second year | 112 | 50 (44.6) |
| Time Studying Alone, mean (SD), % | 108 | 81.3 (25.3) |
| Study Aids: Almost Always/Always Used | | |
| Review books | 111 | 106 (95.5) |
| Practice examinations | 112 | 56 (50.0) |
| Question banks | 113 | 103 (91.2) |
| Lecture videos | 112 | 49 (43.8) |
| Class notes | 112 | 3 (2.7) |
| Study Aids: Usually/Very Helpful | | |
| Review books | 109 | 102 (93.6) |
| Practice examinations | 102 | 71 (69.6) |
| Question banks | 109 | 101 (92.7) |
| Lecture videos | 95 | 63 (66.3) |
| Class notes | 46 | 12 (26.1) |
| Review Book Ranked Most Helpful | | |
| First Aid for the USMLE | 112 | 94 (83.9) |
| Practice Examination Ranked Most Helpful | | |
| COMSAE | 97 | 60 (61.9) |
| Question Bank Ranked Most Helpful | | |
| COMBANK | 109 | 56 (51.4) |
| Lecture Video Ranked Most Helpful | | |
| Kaplan USMLE | 96 | 69 (72.6) |
×
In terms of open-ended suggestions or “student tips” for examination preparation, of the 100 respondents to this question, 48 suggested completing a lot of practice questions, 26 suggested using the review book
First Aid for the USMLE while studying for courses, 26 recommended starting the study process early, and 10 suggested making and following a study schedule (
Table 3).
Table 3.
Predictors of COMLEX-USA Level 1 Performance: Tips From Survey Respondents (n=100)
| Tip | No. (%) |
| Do a lot of practice questions | 48 (48) |
| Use First Aid for the USMLE with courses | 26 (26) |
| Start early | 26 (26) |
| Make a schedule | 10 (10) |
Table 3.
Predictors of COMLEX-USA Level 1 Performance: Tips From Survey Respondents (n=100)
| Tip | No. (%) |
| Do a lot of practice questions | 48 (48) |
| Use First Aid for the USMLE with courses | 26 (26) |
| Start early | 26 (26) |
| Make a schedule | 10 (10) |
×
Bivariate Analysis
We found no statistically significant association between a score of 600 or higher on COMLEX-USA Level 1 and any of the demographic variables or ratings of self-perceived performance in osteopathic medical school. There was a significant association with examination preparation methods, including earlier initiation of examination preparation ( , P=.04), ranking COMSAE as the most helpful practice examination ( , P=.005) and ranking COMBANK as the most helpful question bank ( , P=.001). A score of 600 or higher on COMLEX-USA Level 1 was associated with a significantly higher preclinical GPA during the first 2 years of osteopathic medical school (P<.0001) and a higher MCAT score (P=.02).
Multivariate Analysis
The results of the first logistic regression is found in
Table 4. In the final model, preclinical GPA, MCAT scores, practice examination, and initiation of examination preparation were statistically significant in predicting a COMLEX-USA Level 1 score of 600 or higher. Participants with COMLEX-USA scores of 600 or higher were 1.4 (95% confidence interval [CI], 1.06-1.71;
P<.02) times more likely to have a higher preclinical GPA, 1.3 (95% CI, 1.00-1.69;
P<.05) times more likely to have higher MCAT scores, and 2.5 (95% CI, 1.00-6.33;
P<.05) times more likely to have initiated their examination preparation earlier than those who had COMLEX-USA Level 1 scores less than 600.
Table 4.
Multivariate Logistic Regression: Predictors of Scoring ≥600 on COMLEX-USA Level 1 Among Survey Respondents (N=113)
| Predictor | Odds Ratio (95% Confidence Interval) | P Value |
| Sex | 1.46 (0.34-6.25) | .608 |
| Race | 1.47 (0.68-3.20) | .327 |
| COMBANK as most helpful question bank | 0.24 (0.05-1.31) | .099 |
| Time started studying | 2.53 (1.004-6.33) | <.05 |
| COMSAE as most helpful practice examination | 0.19 (0.04-0.86) | <.04 |
| Medical College Admission Test score | 1.30 (1.001-1.69) | <.05 |
| Preclinical grade point average | 1.35 (1.06-1.71) | <.02 |
Table 4.
Multivariate Logistic Regression: Predictors of Scoring ≥600 on COMLEX-USA Level 1 Among Survey Respondents (N=113)
| Predictor | Odds Ratio (95% Confidence Interval) | P Value |
| Sex | 1.46 (0.34-6.25) | .608 |
| Race | 1.47 (0.68-3.20) | .327 |
| COMBANK as most helpful question bank | 0.24 (0.05-1.31) | .099 |
| Time started studying | 2.53 (1.004-6.33) | <.05 |
| COMSAE as most helpful practice examination | 0.19 (0.04-0.86) | <.04 |
| Medical College Admission Test score | 1.30 (1.001-1.69) | <.05 |
| Preclinical grade point average | 1.35 (1.06-1.71) | <.02 |
×
Participants who did not rank COMSAE as the most helpful practice examination were 5.2 (95% CI, 1.16-22.73; P<.04) times more likely to have COMLEX-USA scores of 600 or higher than those who did rank it as the most help practice examination. The final model correctly classified 87% of participants: 94.6% of those who scored lower than 600 and 55.6% of participants who scored 600 or higher were correctly classified, and this finding was statistically significant ( , P=.0001).
In the second logistic regression, in which MCAT scores and preclinical GPA were not included as independent variables, initiation of examination preparation and question banks were the 2 statistically significant predictors (
Table 5). Participants with COMLEX-USA Level 1 scores of 600 or higher were 2.9 (95% CI, 1.38-6.29;
P<.01) times more likely to have initiated examination preparation earlier than those with lower scores. Participants who did not rank COMBANK as the most helpful question bank were 5.5 (95% CI, 1.25-23.81;
P<.03) times more likely to have COMLEX-USA scores of 600 or higher than those who did rank it as the most helpful question bank. The final model correctly classified 84.9% of participants: 94.7% of those who scored lower than 600 and 44.4% of participants who scored 600 or higher were classified correctly. This finding was statistically significant (
,
P=.0001).
Table 5.
Multivariate Logistic Regression: Predictors of Scoring ≥600 on COMLEX-USA Level 1 Among Survey Respondents, Excluding Grade Point Average and Medical College Admission Test Score (N=113)
| Predictor | Odds Ratio (95% Confidence Interval) | P Value |
| Sex | 2.34 (0.67-8.20) | .186 |
| Race | 1.50 (0.71-3.16) | .288 |
| COMBANK as most helpful question bank | 0.18 (0.04-0.80) | <.03 |
| Time started studying | 2.90 (1.38-6.29) | <.01 |
| COMSAE as most helpful practice examination | 0.30 (0.08-1.05) | .06 |
Table 5.
Multivariate Logistic Regression: Predictors of Scoring ≥600 on COMLEX-USA Level 1 Among Survey Respondents, Excluding Grade Point Average and Medical College Admission Test Score (N=113)
| Predictor | Odds Ratio (95% Confidence Interval) | P Value |
| Sex | 2.34 (0.67-8.20) | .186 |
| Race | 1.50 (0.71-3.16) | .288 |
| COMBANK as most helpful question bank | 0.18 (0.04-0.80) | <.03 |
| Time started studying | 2.90 (1.38-6.29) | <.01 |
| COMSAE as most helpful practice examination | 0.30 (0.08-1.05) | .06 |
×
Comment
Our findings are consistent with those of other studies
12-17 in demonstrating that preclinical GPA and MCAT scores are statistically significant predictors of higher scores on COMLEX-USA. These findings underscore the importance of maintaining a high GPA during the first 2 years of medical school. Students need to forgo the misperception that studying intensively during the few months preceding the examination alone will be sufficient to guarantee optimal performance. Preparing for this examination is more like running a marathon than a 500-m dash. A consistent program of study and a high GPA during the first 2 years of medical school is one of the best test preparation strategies.
The present study is, to our knowledge, among the first to show the importance of starting earlier in preparing for COMLEX-USA Level 1. The earlier participants began to prepare, the more likely they scored 600 or higher. Future studies need to further elucidate this variable and determine the optimal time to initiate preparing for the test. Information regarding the variability in the number of hours spent in examination preparation and the resources used at different time points would also be helpful.
Surprisingly, there was an association with not ranking the COMSAE as the most helpful practice examination and scoring 600 or higher. This finding is difficult to interpret and must be viewed within the limitations of our data collection tool. It is possible that this finding is an artifact of how the item was worded because we asked participants to rank order the 3 most helpful practice examinations. Students who used more than 1 practice examination might have had difficulty answering the question, and the rank order they provided might have been a function of which examination first came to their mind as they answered the question. Additionally, 20% of participants included question banks in their responses to this item, which suggests some confusion about the phrasing of the item. It could also be that some students are not clear on the differences between practice examinations such as COMSAE, which are shorter, and question banks, which are much longer. Students who spent many hours answering question banks may have found COMSAE to be less useful because of its limited number of items. These students may not have realized that the purpose of COMSAE is to help assess readiness for taking COMLEX-USA because it provides a simulated structure and format that mirrors the actual COMLEX-USA. Alternatively, some students may have used question banks as a substitute for practice examinations. Given the format that we used to collect these data, this finding should not be interpreted as a recommendation not to incorporate COMSAE as part of a comprehensive examination preparation strategy. Future studies need to improve the format of this item to ensure that respondents differentiate between practice examinations and question banks.
In our second regression model, we did not include the preclinical GPA and MCAT scores to determine which variables were associated with scoring 600 or higher once the effect of these 2 variables were removed. Similar to the first regression, initiation of examination preparation earlier was associated with higher scores on COMLEX-USA Level 1. This finding suggests that studying earlier is important for scoring high on COMLEX-USA Level 1. However, not ranking COMSAE as the most helpful practice examination did not emerge as a statistically significant predictor, but not ranking COMBANK as the most helpful question bank was a statistically significant predictor.
We conducted post hoc analyses to help explain these associations. The frequency distribution of the responses to this question were interesting in that 56 participants (51.4%) ranked COMBANK as the most helpful question bank and 45 (41.3%) ranked USMLE WORLD (UWORLD) as the most helpful question bank. Given the similarity of the distributions, we conducted additional bivariate analyses comparing those who ranked UWORLD as the most helpful question bank with those who did not and found a statistically significant association between ranking UWORLD as the most helpful question bank and scoring 600 or higher on COMLEX-USA Level 1. We then included UWORLD rather than COMBANK as an independent variable and ran a logistic regression, and UWORLD did not emerge as a statistically significant predictor of scoring 600 or higher in the final model.
Given that our sample size was moderate and the distribution of participants ranking each question bank as most helpful were comparable, it could be that the effects were attenuated because of insufficient power. It is noteworthy that a high proportion of students stressed the importance of question banks as an examination preparation resource. Unfortunately, given the limitations of our study we are not able to address this issue more fully. Future studies with larger sample sizes are needed to explore more fully the contributions of question banks as COMLEX-USA preparation methods. These studies should gather quantitative data on the number of practice questions each student completes, the specific question banks used, and when students started to use these question banks to provide more specific guidance regarding examination preparation strategies.
The present study was not without limitations. First, our sample was restricted to osteopathic medical students attending NSU-COM; thus, the generalizability of our findings is limited. Future studies should recruit participants from across the spectrum of osteopathic medical schools in the country. Further attention needs to be given to the data collection tool we used. Some of the items could have been worded to elicit direct responses rather than Likert-scale responses. Also, it may have been helpful if we provided definitions of the examination preparation resources to prevent some of the confusion that emerged between a question bank and a practice examination. Rank-ordering of the specific examination preparation tools used may also have been difficult for respondents.
Conclusion
Our study investigated and described how students prepared for the licensing examination and which examination preparation resources were the most useful and most helpful. Our study was among the first to demonstrate the association between earlier initiation of examination preparation activities and test performance. We also confirmed the previously established association between strong academic performance during the first 2 years of osteopathic medical school and obtaining high scores on the licensing examination. Many medical students are concerned about their performance on the licensing examinations and seek guidance in developing effective examination preparation strategies. At the start of medical school, professors, administrators, and academic advisors should strongly encourage students to establish a consistent program of study and maintain a high GPA to maximize their performance on the licensing examinations. Students should be encouraged to consider starting their examination preparation as early as possible. Future studies are needed to examine in greater detail which are the best examination preparation resources, the most optimal time to initiate examination preparation activities, and how to develop an effective strategy for preparing for examinations.
Acknowledgments
We thank NSU-COM for funding this study, and Dean Anthony J. Silvagni, DO, PharmD, MSc, for his continued support of the research fellowship program. We also thank the osteopathic medical students who took the time to complete our survey.
Financial Disclosures: None reported.
Support: This work was supported by the Research Fellowship Program of Nova Southeastern University College of Osteopathic Medicine.
References
Teitelbaum HS.
Osteopathic Medical Education in the United States: Improving the Future of Medicine. Chevy Chase, MD: American Association of Colleges of Osteopathic Medicine; 2005:147.
http://www.do-online.org/pdf/acc_mededstudy05.pdf. Accessed January 14, 2013.
Bates BP. Selection criteria for applicants in primary care osteopathic graduate medical education.
J Am Osteopath Assoc. 2002;
102(11):621-626.
[PubMed]
Taylor CA, Weinstein L, Mayhew HE. The process of resident selection: a view from the residency director's desk.
Obstet Gynecol. 1995;
85(2):299-303.
[CrossRef] [PubMed]
Guffey RC, Rusin K, Chidiac EJ, Marsh HM. The utility of pre-residency standardized tests for anesthesiology resident selection: the place of United States Medical Licensing Examination scores.
Anesth Analg. 2011;
112(1):201-206.
http://www.anesthesia-analgesia.org/content/112/1/201.full. Accessed January 14, 2013.
McGaghie WC, Cohen ER, Wayne DB. Are United States Medical Licensing Exam Step 1 and 2 scores valid measures for postgraduate medical residency selection decisions?
Acad Med. 2011;
86(1):48-52.
[CrossRef] [PubMed]
Hartman SE, Bates BP, Sprafka SA. Correlation of scores for the Comprehensive Osteopathic Medical Licensing Examination with osteopathic medical school grades.
J Am Osteopath Assoc. 2001;
101(6):347-349.
http://www.jaoa.org/content/101/6/347.full.pdf+html. Accessed January 14, 2013.
Sefcik DJ, Prozialeck WC, O'Hare TH. Characteristics of the courses that best predict COMLEX-USA Level 1 performance.
J Am Osteopath Assoc. 2003;
103(10):491-494.
http://www.jaoa.org/content/103/10/491.full.pdf. Accessed January 14, 2013.
Wong SK, Ramirez JR, Helf SC. Student performance on Levels 1 and 2-CE of COMLEX-USA: do elective upper-level undergraduate science courses matter?
J Am Osteopath Assoc. 2009;
109(11):592-598.
http://www.jaoa.org/content/109/11/592.full. Accessed January 14, 2013.
Dixon D. Relation between variables of preadmission, medical school performance, and COMLEX-USA Levels 1 and 2 performance.
J Am Osteopath Assoc. 2004;
104(8):332-336.
http://www.jaoa.org/content/104/8/332.full. Accessed January 14, 2013.
Evans P, Wen FK. Does the Medical College Admission Test predict global academic performance in osteopathic medical school?
J Am Osteopath Assoc. 2007;
107(4):157-162.
http://www.jaoa.org/content/107/4/157.full. Accessed January 14, 2013.
Meoli FG, Wallace WS, Kaiser-Smith J, Shen L. Relationship of osteopathic medical licensure examinations with undergraduate admission measures and predictive value of identifying future performance in osteopathic principles and practice/osteopathic manipulative medicine courses and rotations.
J Am Osteopath Assoc. 2002;
102(11):615-620.
http://www.jaoa.org/content/102/11/615.long. Accessed January 14, 2013.
Baker HH, Foster RW, Bates BPet al. Relationship between academic achievement and COMLEX-USA Level 1 performance: a multisite study.
J Am Osteopath Assoc. 2000;
100(4):238-242.
http://www.jaoa.org/content/100/4/238.full.pdf+html. Accessed January 14, 2013.