For the current study we selected participants from 2 COMAT cycles in 2011 and 2012. The 2011 cycle, the initial cycle for COMAT, extended from July 1, 2011, to June 30, 2012, and the 2012 cycle from July 1, 2012, to June 30, 2013. According to surveys conducted by the NBOME in 2011 and 2012, most osteopathic medical schools used COMAT scores either as part of students' final grade or for a pass-fail decision in their clerkships and clinical rotations. However, a few schools used COMAT only for formative purposes and did not ascribe COMAT scores to actual performance; this applied to 39.6% of the COMAT administrations in the 2011 cycle. For the 2012 cycle, a larger proportion of schools used COMAT scores to assess performance, with only 8.3% of the examinations used formatively. For the current study, we classified the scores from schools who used COMAT for clerkship or clinical rotation grades or pass-fail determination as being used for “high-stakes” purposes, and we classified the rest as being used for “low-stakes” purposes. We used only existing data that were properly obtained; participants could not be identified, directly or statistically. Therefore, the research was exempt from institutional review board approval.
After first-attempt COMAT score records were matched with first-attempt COMLEX-USA Level 2-CE scores, we conducted correlation analyses between the scores from the 7 COMAT subject examinations and between COMAT and COMLEX-USA Level 2-CE scores for the following 4 groups: (1) 2011 low-stakes administrations; (2) 2011 high-stakes administrations; (3) 2012 low-stakes administrations; and (4) 2012 high-stakes administrations. We also performed a simultaneous multiple linear regression analysis of the groups to determine how much of the variance in COMLEX-USA Level 2-CE scores was explained by COMAT scores. All analyses were conducted using SAS/STAT software (SAS for Windows, version 9.3; SAS Institute). The P value cutoff (α) was set a priori to .05 for all tests of statistical significance.