Abstract
Context:
Bloom's Taxonomy is a widely accepted tool for analyzing learning objectives, creating assessment materials, and ensuring that students move progressively through various levels of knowledge and cognition. Competency-based osteopathic medical education has, to the authors’ knowledge, yet to be subjected to systematic bloomian analysis.
Objective:
To advance a bloomian analysis of competencies published by the American Association of Colleges of Osteopathic Medicine (AACOM) and draw consequences of that analysis for undergraduate osteopathic medical education.
Methods:
Content analysis of the verbs and contexts used in the AACOM competencies was conducted, followed by categorization by the various cognitive and knowledge dimensions as specified by the Revised Bloom's Taxonomy.
Results:
The majority of AACOM competencies gravitate toward the taxonomy's apply and procedural cognitive and knowledge dimensions, respectively. In addition, 58% of the top 10 most-used verbs in the document were identified to be in the apply cognitive dimension. An analysis of the competencies revealed that 69% can be categorized under the procedural knowledge dimension. Of 657 code co-occurances, 45% were from the apply cognitive and procedural knowledge dimensions. These figures suggest a skewing toward application-based cognitive and procedural-based knowledge over other types of cognitive and knowledge dimensions.
Conclusion:
Bloom's Taxonomy offers a useful framework for understanding how competency-based osteopathic medical education is organized. Such analysis underscores the importance of attending to the language and construction of competencies to better understand how the language of competencies shapes pedagogic practices and begin a broader conversation about the appropriate use and distribution of lower- and higher-order cognitive processes within competency-based osteopathic medical education.
Since the 1950s, Bloom's Taxonomy has served as an important tool used by educational professionals.
1-3 The taxonomy is a system for classifying instructional objectives by level of cognitive complexity, primarily through analyzing verbs used in written objectives. In organizing cognitive processes from lower-order to higher-order,
4 the taxonomy offers “a method of improving the exchange of ideas and materials among test workers, as well as other persons concerned with educational research and curriculum development.”
5 This method enables curriculum developers to understand the relationship between learning experiences and the cognitive changes that occur through instruction.
In 2001, Anderson et al
6 updated the taxonomy to reflect new developments in the science of cognitive development. To allow for more robust categorization, the revised taxonomy groups instructional objectives into 6 cognitive (remember, understand, apply, analyze, evaluate, and create) and 4 knowledge (factual, conceptual, procedural, and metacognitive) dimensions, while retaining the original taxonomy's method.
Over the years, Bloom's Taxonomy has also gained traction in medical education,
7-9 as an instrument for evaluating instructional outcomes and developing learning materials.
2,10,11 Yet, the taxonomy has not been used to assess the growing trend of competency-based osteopathic medical education.
12-14 In response to a growing concern that some medical students lack the competence needed in residency,
15 key organizations
14 have published competencies categorized in content domains and composed of specific knowledge, skills, and attitudes. For example, in 2012, the American Association of Colleges of Osteopathic Medicine (AACOM) convened a Core Competency Liaison Group to devise a set of curricular outcomes for common use among all osteopathic schools. The product of the committee's work,
Osteopathic Core Competencies for Medical Students,16 sought to “[guide] curriculum committees in educating osteopathic medical students, [provide] a template for sharing ideas on learning activities and evaluation tools, and [discuss] ways in which osteopathic medical schools might work together to meet these needs.”
The purpose of the current study is to demonstrate the value of Bloom's Taxonomy for assessing the cognitive complexity of the AACOM competencies. We believe that such a demonstration provides an illuminating and newly critical angle for rethinking the competencies and the language used to construct them. Accordingly, one of the goals of this study is to invite future osteopathic medical education researchers to apply the taxonomy's approach to their own analyses.
A content analysis of verbs and noun phrases used in the AACOM competencies was undertaken. Content analysis allows researchers to draw inferences from texts to capture not only the words themselves but also their contexts.
17 Analysis of this type allows for the identification of thematic elements and other patterns that may exist within a contained document.
18 We categorized the AACOM competencies into relevant cognitive and knowledge dimensions as specified by the taxonomy (
Table 1).
We deployed a 2-step process. First, we classified verbs into the taxonomy's cognitive dimensions: remember, understand, apply, analyze, evaluate, and create. This process involved taking each verb from the competencies and identifying the corresponding category in the taxonomy. When faced with competencies containing multiple verbs, the editors of the taxonomy suggested inferring “which process the teacher intended in order to classify that objective.”
6 Accordingly, we distinguished between nonaction and action verbs. For example, “Identify the patient's chief complaints and appropriately perform a logical physical examination in order to properly diagnose the condition” contains 3 verbs,
identify,
perform, and
diagnose. However, not all 3 verbs function as action verbs for pedagogic purposes.
Identify and
perform are action verbs because they describe actions required to determine student competence.
Diagnose, however, does not drive the cognitive activity with which students are being tasked and so was not coded in our study. A similar approach was taken by and is advocated for by Ven and Chuang.
19
Second, the competencies were categorized into the 4 knowledge dimensions specified by the taxonomy: factual, conceptual, procedural, and metacognitive. Again, in accordance with the taxonomy's guidelines, we analyzed each verb's context as it appears in the competencies.
6 For example, “Identify the association between organ systems, function, and structural findings” has the verb
identify and the context, or noun phrase, of “the association between organ systems, function, and structural findings.” This phrase was most closely associated with Bloom's conceptual knowledge dimension, described as “the interrelationships among the basic elements within a larger structure that enable them to function together.”
6
In cases in which the verb or noun phrase used in the competency was not immediately clear, the taxonomy's deeper descriptions of domains provided further guidance for settling interpretative differences. In instances in which a competency did not fit directly into any dimensions, we coded them as “unclassified” for the cognitive dimension and “unclassified knowledge” for the knowledge dimensions. For example, the competency, “Seek qualified care from a health professional outside the family of the physician” contains the verb seek but cannot be subsumed into the taxonomy because it is a sentence fragment. Although these very few (18 of 978) instances may have been oversights on the part of the competency's authors, for purposes of methodologic consistency, they were coded as unclassified. Similarly, we used the code “unclassified knowledge” in only 1 instance.
Coding was performed collaboratively over multiple working sessions by K.R. and J.M. using the qualitative online research software Dedoose. While the classification of most of the competency's verbs garnered easy consensus among the researchers, in some cases a closer look at context was required. In these cases, following research from Graneheim and Lundman
20 on the role of interpretation in qualitative content analysis, we (K.R. and J.M.) discussed the verbs with a recognition that qualitative content analysis “focuses on the subject and context, and emphasizes differences between and similarities within codes and categories,” while also recognizing that both manifest and latent content is embedded in the language of the competencies. This view is consistent with Finfgeld-Connett's
21 recognition that there is great diversity in approaches to qualitative content, ranging from “impressionistic interpretation to systematic analysis of text-based data,” the latter of which was the aim of the current study.
After initial coding was completed, we undertook an analysis of the data collected by using Dedoose code co-occurrence and application frequency tools. This process, in turn, facilitated further discussion to identify results and notable findings from the data.
We identified 385 competency and subcompetency statements to analyze. Of these statements, 545 verbs were identified and coded.
Figure 1 presents the coding results of the top 10 most frequently used verbs and their distribution among the taxonomy's cognitive dimensions. These data indicate that a majority of verbs fall within the apply domain (148 [58%]), followed by the understand (63 [25%]) and remember (23 [9%]) domains.
Figure 2 shows the frequency of competencies classified within the taxonomy's 4 knowledge dimensions. There were 426 codes applied. The procedural knowledge domain was the most common (296 [69%]), followed by the conceptual domain (96 [23%]). This increase exhibits the extensive focus the competencies have on application-based cognitive and procedural-based knowledge dimensions of the taxonomy.
Table 2 includes an additional category of “unclassified.” As described above, this code was applied when a verb could not be categorized using the descriptions of the taxonomy's cognitive and knowledge dimensions.
Osteopathic medical educators may deem our findings useful in at least 2 key ways. First, we show that Bloom's Taxonomy provides an important opportunity for understanding and reflecting on what the AACOM competencies tell us about the goals of osteopathic medical education. Specifically, it illuminates the assumptions of cognitive complexity, if not in distribution or sequence then in frequency, that the authors of the AACOM competencies brought to bear in their work. Second, our analysis suggests some logistical considerations as osteopathic curriculum developers adjust priorities to meet the aims of the AACOM competencies, address workforce needs, and develop instructional and assessment methods.
The overwhelming number of competencies require application-based cognition (n=148) and procedural-based knowledge (n=296). This finding is consistent with the taxonomy, asserting that the apply cognitive and procedural knowledge domains are closely linked.
6 As
Figure 1 and
Figure 2 show, these competencies tend to be clustered in the higher level of the lower-order cognitive dimensions and not in a linear or evenly distributed fashion. This finding suggests that the competencies bring students to a point at which their education becomes more complex, reflexive, and cognitively intensive, but it declines into the higher-order cognitive processes. It may be that this is a function of practical limitations in the goals of preresidency education, for which the AACOM competencies were written. In other words, higher-order cognitive processes that the AACOM competencies do not reach may be expected to be undertaken during residency. For example, self-evaluation, which is associated with higher-order cognitive and knowledge processes, is commonplace in many residency programs.
22-24
At the same time, most competencies not only neglect the highest levels of cognitive activity but also divest from the lower-order cognitive dimensions. This distribution may have been driven by the laudable goal of bringing students to the taxonomy's application-based dimensions as a response to a sense that medical education too often skews toward rote memorization and retention.
25-27 This observation suggests, paradoxically, that the competencies may have swung too far toward the highest level of the lower-order cognitive dimensions, possibly at the expense of higher-order cognitive processes. This swing also diminishes the specificity of lower-order processes that are needed before students can reach the level of application that may be helpful for instructors and curriculum developers. Although this is not in itself a problem, it does help us understand the instructional aims of the competencies. However, before we can affirm whether this distribution is optimal, future research will need to compare the distribution of the AACOM competencies with AACOM's ultimate understanding of what 21st century physicians need to know and be able to do.
The move to an increased emphasis on application-focused learning has consequences for medical education. It requires commitments to and investments in teaching modes that engage students in active learning, such as team-based learning and hands-on experiences in which skills are developed in actual or simulated clinical spaces.
28-30 This shift also provides considerable evidence for the value of moving students out of closed pedagogic spaces and into clinics, where they can engage in hands-on learning that requires the use of application-based cognitive skills.
31
A level of knowledge is needed before certain procedures can be carried out by medical students; thus, lower-order thinking processes of the taxonomy are needed. Unsurprisingly, in a way that supports the AACOM competencies’ emphasis on application-based cognitive modes of learning, medical schools are already prioritizing the early development of students’ clinical skills. For example, schools such as Case Western Reserve University School of Medicine and Duke University School of Medicine are moving their students into clinics earlier in their education compared with other traditional models.
32,33 This move toward early field experience and active learning strategies supports the move away from traditional lecture-style classes that seem to remain the norm of many medical institutions.
Beyond early placement in clinics, schools are prioritizing active learning strategies so that students may actively practice and learn skills that they will be using in clinical spaces.
34,35 For example, the University of Stanford's reimagining undergraduate medical education initiative aims to promote active learning and innovative approaches to student instruction on a large scale.
36 The use of mediums such as simulation- and team-based learning fosters the active development and use of knowledge and skills as opposed to passive acquisition through the rote memorization method perpetuated by lectures. In addition, this move toward improving students’ skills shores up the case for earlier clinical exposure, which has been shown to reinforce clinical skills while also making learning more relevant to students.
37-39 These pedagogic modes can also increase the need for resources, especially faculty facilitators. As competency-based education becomes more cognitively complex, additional resources may be necessary to meet curricular needs. As such, medical colleges with limited resources may struggle to find ways to carry out such curricula, or they will need to develop innovative models when limited resources pose barriers.
Finally, the growing interest in competency-based medical education has led to the widespread use of a related metric known as Entrustable Professional Activities (EPAs), understood as “units of professional practice” associated with tasks or responsibilities that medical students and residents are expected to be able to perform at certain stages of their development.
13 The Association of American Medical Colleges (AAMC) has put forward a list of “integrated activities to be expected of all M.D. graduates making the transition from medical school to residency” into 13 core EPAs.
13
The
Osteopathic Considerations for Core Entrustable Professional Activities for Entering Residency40 directly ties AACOM's core competencies with the 13 AAMC EPAs, suggesting that the results of our study extend beyond competencies and students’ application and procedural-based knowledge to EPAs. Although the EPAs were not subjected to a full bloomian analysis in the current study, a review of the language used by the AAMC in the EPAs seems to reflect this relationship, particularly with regard to the taxonomy's apply and procedural domains. For example, EPA 12, “Perform general procedures of a physician” uses the verb
perform, which finds a home in the taxonomy's apply cognitive and procedural knowledge domains. Additional verbs contained in the EPAs, such as
gather,
give,
document,
provide, and
retrieve also correspond to these domains. Rather than suggest that EPAs do not require higher-order cognitive and knowledge processes, it reinforces our findings that the AACOM competencies emphasize application- and procedural-based skills before entering residency programs.
Our findings are limited to an osteopathic medical educational context. Several other sets of competencies have been published by organizations such as the AAMC and the Royal College of Physicians and Surgeons of Canada. Although our analysis is not generalizable to these contexts, we hope that our analysis does open the door for applying a similar method of evaluation. Our findings suggest that Bloom's Taxonomy may be a valuable tool for better understanding the complexities of competency-based medical education generally. Our analysis does not take a position on what would be an appropriate distribution through the taxonomy and its cognitive and knowledge dimensions of how to best train physicians. These positions include changes in the way medicine itself is being conceived in broad terms, as well as specific questions about the future of osteopathic medicine.
41
Bloom's Taxonomy and the published revisions offer a useful framework for understanding how AACOM's vision of competency-based osteopathic medical education is organized. Although the AACOM competencies are meant to serve as a foundation of minimal competence, they are similar to other curriculum drivers, such as graduate outcomes, assessments, and learning activities. Through a bloomian analysis, educators can attain greater specificity with regard to such drivers. For example, this approach suggests the need for more intensive learning processes, which could include earlier clinical experiences serving as patient navigators,
42 and active learning. The full implementation of such cognitively intensive processes may require significant faculty resources, which may pose challenges for some institutions. Such an evaluation can also help osteopathic medical educators understand the extent to which the competencies move students through various levels of cognition. Critical reflection on the distribution of the competencies can help us to be clear about what we expect students to know and be able to do and at what stage in their education. Our finding that the AACOM competencies emphasize lower-order cognitive processes supports the goals of AACOM as well as those of other organizations, all of which address growing concerns that students are too often unprepared for residency.
43,44
Our study suggests that few competencies ask students to engage in higher levels of cognition. Therefore, we advise that curriculum developers at osteopathic medical schools be mindful of this finding as students progress through curricula. It is important to assess higher-order cognitive processes and more diverse knowledge types, such as metacognition. Although the aims of this study may have been narrow, our larger argument gestures toward research beyond the AACOM competencies. Future studies may include a bloomian analysis of other competencies, both national and international. Ultimately, this line of research would ensure that institutions using competencies are clear about the cognitive skills that they believe all osteopathic physicians should possess. Medical education is always a work in progress, and the present research provides an additional analytic lens if and when such revisions to the competencies are undertaken.