Abstract
The purpose of this study was to investigate the relation between preadmission academic variables, osteopathic medical school performance in the first 2 years, and performance on the Comprehensive Osteopathic Medical Licensing Examination (COMLEX–USA) Levels 1 and 2. The study group comprised 174 students in the class of 2001 of the New York College of Osteopathic Medicine of the New York Institute of Technology, Old Westbury. Preadmission academic variables were the Medical College Admission Test (MCAT) subscores and undergraduate grade point averages (UGPAs).
Physical sciences (physical MCAT) and biological sciences MCAT (biological MCAT) subscores were significantly correlated with COMLEX–USA Level 1 performance, and verbal reasoning, physical, and biological MCAT subscores were correlated with Level 2 performance. COMLEX–USA Level 1 performance was correlated with the year 1 grade point average (GPA) (0.78) and the year 2 GPA (0.83). COMLEX–USA Level 2 performance was correlated with the year 1 GPA (0.64) and the year 2 GPA (0.68). Strong correlations existed between all year 1 and most year 2 course grades and COMLEX–USA Level 1 scores. School-specific regression models that were highly predictive of school performance and COMLEX-USA Level 1 performance were developed. COMLEX–USA Level 1 predictive models used preadmission variables combined with the year 1 and year 2 course grades. The year 2 courses' model had a higher predictive value for COMLEX–USA Level 1 performance (R2 = 0.81) than the year 1 courses' model (R2 = 0.77). Significant predictors of COMLEX–USA Level 1 performance in the combined year 1 and 2 courses' model were the pharmacology II, neuropathology, and pulmonary pathology grades, and the verbal and physical MCAT subscores (R2 = 0.820).
Medical schools have traditionally used Medical College Admission Test (MCAT) scores and undergraduate grade point averages (UGPAs) in the selection of candidates for admission. The validity of using preadmission academic variables to predict student performance has been studied. Most studies have focused on the relation of these variables to medical licensing examination scores as performance end-points. Only a few studies have examined the relationship between preadmission variables, osteopathic medical school performance, and Comprehensive Osteopathic Medical Licensing Examination–USA (COMLEX–USA) performance, and the results were not consistent. Medical College Admission Test scores were reported to be correlated with COMLEX–USA Level 1 and Level 2 scores,
1,2 whereas another study found no significant correlations between preadmission data and COMLEX–USA Level 1 scores.
3 The relation of preadmission variables to osteopathic medical school performance and COMLEX–USA scores has not been sufficiently investigated at other schools. There has also been a lack of predictive validity studies of the relation of pre-medical and medical school performance to COMLEX–USA Levels 1 and 2.
Researchers in the current study previously reported significant correlations between MCAT subscores and year 1 and year 2 GPAs and COMLEX–USA Level 1 performance at the New York College of Osteopathic Medicine of the New York Institute of Technology (NYCOM), Old Westbury.
4 The purpose of the present study was to further examine the relation between preadmission variables and student academic performance, including year 1 and 2 GPAs, individual course grades, and COMLEX–USA Level 1 performance at NYCOM.
Preadmission academic variables were correlated with performance on the United States Medical Licensing Examination (USMLE) Step 1.
5-7 Elam and Johnson
5 found that the biological sciences MCAT (biological MCAT) subscore was the only significant preadmission predictor of USMLE Step 1 performance. A study using data from 14 allopathic medical schools reported that total MCAT scores and undergraduate GPAs were predictors of USMLE Step 1 performance.
6 Silver and Hodgson
7 found that total MCAT scores and science UGPAs were related to performance on the National Board of Medical Examiners Part I (NBME I), a forerunner of the present USMLE Step 1.
Baker et al
3 examined the relation between COMLEX–USA Level 1 scores, preadmission variables, and medical school performance in detail, and they found no correlations between preadmission variables and COMLEX–USA Level 1 performance. Cope et al
8 used preadmission variables in predictive models and reported that the biological MCAT and cumulative UGPA were predictive of COMLEX–USA Level 1 scores. Moderate correlations were found between COMLEX–USA Levels 1 and 2 and total MCAT scores.
1,2
COMLEX–USA Level 1 performance was reported to be strongly correlated with the year 1 and year 2 GPAs, as was Level 2 performance.
1,3,8,9 A multisite study using data from 16 osteopathic medical schools found strong correlations between GPAs and COMLEX Level 1 scores within schools.
10
The class of 2001 study group comprised 174 students (93 men and 81 women). These students completed the first 2 years of the standard NYCOM curriculum in 1999 and took the COMLEX–USA Level 1 examination in 1999 and the Level 2 examination in 2000. The database for analysis included preadmission academic variables, medical school course grades in years 1 and 2, and COMLEX–USA Level 1 and 2 scores. Individual course grades were obtained from institutional databases, and total COMLEX–USA scores were those reported by the National Board of Osteopathic Medical Examiners to the institution. Individual course grades used were those for written examinations. Grade point averages were calculated for the first year (year 1 GPA), the second year (year 2 GPA), and years 1 and 2 (2-year cumulative GPA).
Preadmission data were obtained from the American Association of Colleges of Osteopathic Medicine Application Service. The preadmission academic variables used in the analysis were MCAT subscores and UGPAs. Medical College Admission Test subscores used were verbal reasoning (verbal MCAT), physical sciences (physical MCAT), and biological MCAT. Pearson's correlation coefficients were calculated with Bonferroni adjustments. Researchers performed multiple linear regression analysis by entering all independent variables in a single step and constructed regression models to identify an optimal combination of predictors of COMLEX–USA Level 1 performance. All statistical analyses were calculated with Statistical Program for the Social Sciences (SPSS) statistical software for Windows, version 10.0.
Preadmission data for the study group were mean MCAT subscores (verbal, 7.8; physical, 8.2; biological, 8.5) and mean UGPAs (science, 3.33; nonscience, 3.45; cumulative, 3.38). The study group had a mean COMLEX–USA Level 1 score of 514.87 (SD 73.90) and a mean COMLEX–USA Level 2 score of 525.87 (SD 74.12).
The correlations of preadmission academic variables with year 1 and 2 GPAs are shown in
Table 1. The physical MCAT subscore, biological MCAT subscore, and the science and nonscience UGPA were significantly correlated with the year 1 GPA. The biological MCAT subscore, science UGPA, and nonscience UGPA were correlated with the year 2 GPA. The five preadmission variables were used in a regression model to predict the year GPAs. The physical MCAT subscore, biological MCAT subscore, and science UGPA were significant predictor variables for the year 1 GPA (
R2 = 0.284), and the biological MCAT subscore was a predictor for the year 2 GPA (
R2 = 0.157).
Physical and biological MCAT subscores were correlated with COMLEX Level 1 and the verbal, physical, and biological MCAT subscores were correlated with COMLEX Level 2 (
Table 1). The science and nonscience UGPAs were not correlated. Scientists used the five preadmission variables in a regression model (
Table 2) and found that the physical and biological MCAT subscores were modest predictors of COMLEX Level 1 performance (
R2 = 0.232). The verbal MCAT subscore was a weak predictor of COMLEX Level 2 performance (
R2 = 0.168).
The correlations between medical school GPAs and COMLEX–USA scores are shown in
Table 3. COMLEX–USA Level 1 scores were strongly correlated with both the year 1 and 2 GPAs. The correlations between year 1 and 2 GPAs and COMLEX–USA Level 2 scores were lower than for Level 1 scores. COMLEX–USA Level 1 performance and COMLEX–USA Level 2 performance were correlated (0.723). The year 1 and year 2 GPAs were predictors of COMLEX–USA Level 1 scores in a regression model (
R2 = 0.702). The addition of preadmission variables to the model with the year 1 and year 2 GPAs model (
Table 2) resulted in a slightly higher
R2 value (0.774).
Relationship Between Preadmission Variables, Year 1 Courses, and COMLEX–USA Level 1 Performance
First-year basic science courses included anatomy, biochemistry, general pathology, genetics, histology, microbiology, neuroscience, osteopathic principles I, pharmacology, and physiology. Most of the first-year courses were moderately correlated with only the physical and biological MCAT subscores (
P<.05). Strong correlations existed between all year 1 course grades and COMLEX–USA Level 1 scores (
Table 4).
Scientists used all year 1 course grades as variables in a regression analysis to predict COMLEX–USA Level 1 performance (
Table 2;
R2 = 0.652). Physiology and pharmacology grades were predictors of COMLEX–USA Level 1 performance. The addition of preadmission variables to the year 1 courses' model resulted in a higher
R2 value (0.663).
Relationship Between Preadmission Variables, Year 2 Courses, and COMLEX–USA Level 1 Performance
Year 2 courses included the following pathology courses: cardiovascular, gastrointestinal, musculoskeletal, obstetric and gynecologic, pulmonary, renal, and neuropathology. The systems courses were cardiovascular, gastrointestinal, nervous, obstetrics-gynecology, renal, respiratory, and rheumatologic systems. Other specialty courses in year 2 included dermatology, endocrinology, family practice, hematology, immunology, pharmacology II, osteopathic principles II, pediatrics, psychiatry, surgery, and toxicology. When correlations between preadmission variables and individual year 2 grades were studied, the only significant correlations were between the biological MCAT subscore and cardiovascular pathology and between the verbal and physical MCAT subscores and pediatrics (P<.05).
Table 5 shows the correlations between individual year 2 course grades and COMLEX–USA Level 1 scores. All courses, except musculoskeletal pathology, family practice, and psychiatry, were significantly correlated with Level 1 (
P<.05). All year 2 course grades were used in a regression model to predict COMLEX–USA Level 1 performance. Pharmacology II and pediatrics grades were found to be predictors of COMLEX–USA Level 1 performance (
R2 = 0.730). When preadmission variables were added to the model, the predictive variables were pharmacology II, neuropathology, pulmonary pathology, and the physical and biological MCAT subscores. The
R2 for this model was 0.807.
When all year 1 and 2 course grades were used in a predictive model for COMLEX–USA Level 1, physiology, pharmacology II, and neuropathology were significant variables (
R2 = 0.771). When preadmission variables were added to the model, the significant predictors were pharmacology II, neuropathology, pulmonary pathology, and the verbal and physical MCAT subscores (
R2 = 0.820) (
Table 2).
This study has demonstrated significant correlations between academic preadmission variables, osteopathic medical school performance in the first 2 years, and COMLEX–USA Level 1 and 2 scores. The physical and biological MCAT subscores were correlated with both year 1 and 2 osteopathic medical school GPAs. Regression models identified the physical and biological MCAT subscores as predictor variables for COMLEX–USA Level 1 and the verbal MCAT subscore as a predictor for COMLEX–USA Level 2. Most previous studies have used composite MCAT scores (sum of the individual subscores) as a variable. Researchers reporting on the predictive validity of MCAT scores noted that the use of MCAT composite scores limits its use as a predictor and recommended that future studies investigate the differential validities of subscores.
11 The use of MCAT subscores in the present study was more informative. While the preadmission variables alone were not highly predictive of either COMLEX–USA Level 1 or 2 performance, together with other performance measures preadmission variables improved the total predictive validity.
Only the physical and biological MCAT subscores were moderately correlated with first-year grades. Individual course grades in the first 2 years were strongly correlated to COMLEX–USA Level 1 scores with the exception of a few year 2 courses. Although many of the correlations between course grades and COMLEX–USA Level 1 are similar to some previously reported,
3 differences among school courses make data comparisons between individual schools difficult to interpret.
Individual year 1 and 2 course grades with preadmission variables were used in predictive models of COMLEX–USA Level 1 scores. In all the models tested, the addition of preadmission variables raised the predictive value. The year 2 courses' model had a higher predictive validity than the year 1 model. The combined model, using all year 1 and 2 course grades and preadmission variables, had a higher predictive validity than the cumulative year 1 and 2 GPAs model. The predictive value of these models is higher than previously reported Level 1 models.
8 The higher values found for COMLEX–USA Level 1 predictive models at NYCOM may be due to greater class heterogeneity and larger class size. More detailed studies of data from other NYCOM classes are needed to further confirm the predictive value of the COMLEX–USA Level 1 predictive models. A factor that might improve the predictive value of the preadmission variables would be the use of an undergraduate school selectivity index.
This study has identified many academic performance variables that predict both year 1 and 2 GPAs and COMLEX–USA Level 1 performance. These findings may be useful in predicting future performance of NYCOM students on Level 1 examinations. Further research is needed to determine the relationships between these variables, performance in the third and fourth years, and performance on COMLEX–USA Levels 2 and 3.
The author thanks Arnold L. Nagler, PhD, (NYCOM) for his support and Larry R. Stepp, PhD (also of NYCOM), for his helpful comments on this manuscript.