In this study, total UGPA appeared to be the strongest predictor of global academic performance in osteopathic medical school. Overall, MCAT score appears to have limited value in predicting academic performance. We found a small difference between total UGPA and science UGPA in predicting overall academic performance, with total UGPA having only slightly stronger predictive value.
There are mixed findings regarding the strengths of using UGPA and MCAT score to predict medical school performance. Kulatunga-Moruzi and Norman
18 found that UGPA had the most utility in predicting academic and clinical performance, a finding that is further supported by our results. However, other studies suggest that MCAT score is a better predictor of academic performance than UGPA. Wiley and Koenig
4 found that MCAT scores had a slightly higher correlation (r=0.615-0.67) with academic performance when compared with UGPA (r=0.54-0.58). Veloski and colleagues
5 reported that science MCAT was a better predictor of performance on the National Board of Medical Examiners Part I (a predecessor of the present USMLE Step 1) than UGPA. Swanson and colleagues
12 found that MCAT score was a better predictor of USMLE Step 1 performance than UGPA. Basco and colleagues
7 showed that MCAT score was more strongly related to USMLE Step 1 performance than science UGPA. In addition, Julian
8 recently reported that MCAT score is substantially better than UGPA in predicting performance on USMLE Steps 1, 2, and 3, meaning there is virtually no need to use UGPA to predict these scores in the future.
While biology MCAT and physical MCAT were significant predictors of basic GPA as well as COMLEX-USA Level 1 and Level 2, they were not significant predictors of clinical GPA. Of note, none of the MCAT subtests were found to be significantly correlated with clerkship performance alone (
Table 1), nor were any found to be a significant predictor in combination with UGPA (
Table 2). The high average GPA in clerkships at OSU-COM (mean, 3.79) may limit the ability to discriminate high performers from low performers. Another limitation could be the result of the nonstandardized clerkship examinations given in each rotation. It is important to note that OSU-COM does not use COMLEX-USA shelf examinations in clinical clerkship testing, differing from many allopathic medical schools that use standardized USMLE shelf examinations, a practice that may influence clerkship grades.
While written MCAT was found to correlate significantly, albeit modestly, with total GPA (
Table 1), it was not found to be a significant predictor of any of the five performance variables when combined with the other MCAT subtests and total UGPA in the regression models (
Table 2). This finding suggests limited predictive value for the written MCAT.
Conversely, verbal MCAT was found to be both a significant correlate (
Table 1) and a significant predictor (
Table 2) within the regression model of COMLEX-USA Level 2 scores. Though MCAT score was previously found to be correlated with COMLEX-USA Level 2 performance, this result was from a study
16 that looked at only one graduating class (also at OSU-COM) and used average values of MCAT scores rather than differentiating between MCAT subtests as specified in the present study design. Therefore, it is difficult to compare directly the results of these studies. In contrast with the results of the regression analyses used in our study, Hojat and colleagues
14 reported that written MCAT results were more closely associated with clinical competence and class rank than with basic science performance as measured by basic GPA. Kulatunga-Moruzi and Norman
18 noted that verbal MCAT was useful in predicting communication skills on the MCCQE Part II. Roth and colleagues
19 similarly reported that the most highly predictive factor for USMLE Step 2 performance was verbal MCAT (r=0.33). Daugherty and colleagues
13 also suggested that verbal MCAT may have predictive value for identifying poor preclinical performers who would do better in clerkships.
The findings of the present study are limited in several ways. First, the dataset is comprised of academic records from a single osteopathic medical institution across a 5-year time span, thus limiting our ability to generalize the results across the osteopathic medical profession. Second, this dataset (as with all data taken from medical students) is subject to self-selection bias (ie, comprised only of individuals selected using the predictor variables), and includes a small number of students (2.8%) who showed poor performance on the general academic measures. In such cases, where there is a restricted and homogeneous sample, statistical theory suggests that a restriction in the range of possible values of normally distributed variables (and many nonnormally distributed variables) may occur. This range restriction, in turn, would reduce correlations that may otherwise be seen in unrestricted populations (eg, all MCAT examinees). The issue of restriction in range was analyzed by calculating corrected correlations between the dependent and independent variables using a standard formula.
20 However, corrected correlations were found to show a nearly 10% increase in absolute value above those presented in
Table 1. This increase suggests that restricted range is not a substantial issue in this dataset. Lastly, the variance seen in MCAT scores may be the result of students' test-taking abilities.
Biology MCAT and physical MCAT were significantly correlated with most general academic performance variables, with the notable exception of clinical GPA. However, when compared in the regression analyses, biology MCAT emerged as the stronger predictor of most outcomes. These findings are likely the result of the degree of overlap between the two subtests (r=0.46). The extent of the overlap did not invalidate the regression analyses (ie, through multicollinearity), and the nature of the commonalities (eg, content, test-taking approach) is beyond the scope of this paper.
Results from this study raise several important questions:
Our results indicate that MCAT scores have limitations when used to predict academic success. Though Swanson and colleagues
12 found a much larger variance (at r=0.615, the variance was 38%), our findings (at r=0.22, the variance was 4.8%) are substantially lower.
It is unclear whether the results of our study were unique to osteopathic medical education. More studies are needed to answer these questions, particularly through integrative approaches, such as examining multiple variables predictive of academic success with larger samples drawn from multiple schools and diverse geographic regions over time. Using a larger dataset that includes a higher number of unsuccessful students from other osteopathic medical schools may shed more light on this topic.