Developing Technology-Enhanced Active Learning for Medical Education: Challenges, Solutions, and Future Directions

Lise McCoy; Robin K. Pettit; Joy H. Lewis; Thomas Bennett; Noel Carrasco; Stanley Brysacz; Inder Raj S. Makin; Ryan Hutman; Frederic N. Schwartz

doi:10.7556/jaoa.2015.042

Abstract

Growing up in an era of video games and Web-based applications has primed current medical students to expect rapid, interactive feedback. To address this need, the A.T. Still University–School of Osteopathic Medicine in Arizona (Mesa) has developed and integrated a variety of approaches using technology-enhanced active learning for medical education (TEAL-MEd) into its curriculum. Over the course of 3 years (2010-2013), the authors facilitated more than 80 implementations of games and virtual patient simulations into the education of 550 osteopathic medical students. The authors report on 4 key aspects of the TEAL-MEd initiative, including purpose, portfolio of tools, progress to date regarding challenges and solutions, and future directions. Lessons learned may be of benefit to medical educators at academic and clinical training sites who wish to implement TEAL-MEd activities.

Technology-enhanced active learning (TEAL) is a growing trend in higher education.^1-4 This trend is in response to the learning needs of the current generation of medical students, who grew up in a video-game era. They are accustomed to "plug and play" video games and rapid feedback.^3,5,6 Education technology reports published by the Educause Center for Analysis and Research⁴ and Horizon⁷ indicate that the current generation of students requires interactive, technology-enhanced learning approaches that support a variety of learning styles and modalities. Studies point to the potential benefits of TEAL for health or health care education. For example, virtual simulations provide safe environments for students to practice clinical reasoning before live patient encounters,^{8, 9} and video games enhance aspects of visual processing.¹⁰

Games and simulations promote self-directed learning. de Bilde et al¹¹ asserted that self-directed learning results in intrinsically motivated participation and better learning outcomes. Teachers act as facilitators, rather than sources, of content (ie, guides on the side). Learners take responsibility for their own learning and self-assessment of learning outcomes.¹² Interactive, technology-based activities allow students to problem solve and engage in collaborative discussion in a fun environment.^{1, 6} A 2004 review¹³ of the literature concluded that actively engaging students allows them to retain information longer, as it promotes deeper learning. According to education researchers Offir et al,¹⁴ deep learning is "a process that takes place when students translate new information into engraved concepts and relate it to their life experience."

In 2010, in an effort to provide engaging, contextual learning, the A.T. Still University– School of Osteopathic Medicine in Arizona (Mesa) (ATSU-SOMA) embarked on an initiative to infuse educational games, simulations, and other TEAL for medical education (TEAL-MEd) activities into the school's curricula. The project's 5-year objectives included producing 24 interactive electronic games related to clinical presentations and subsequently measuring student satisfaction with this mode of instruction.

At the start of the project, ATSU-SOMA founded an ad hoc steering committee to investigate the use of electronic games or simulation exercises for student practice of clinical decision making and patient care and to assist other educators in developing and implementing these activities. Over time, the group expanded its repertoire of projects and currently operates as the "TEAL-MEd committee." Using consensus-building strategies,¹⁵ the TEAL-MEd committee has gradually evolved into a productive community of teaching practice.¹⁶ The core team meets weekly and includes 11 members: 7 physicians (osteopathic and allopathic), 1 basic scientist, 1 physician assistant, 1 education specialist, and 1 technology specialist. Associated work groups are developing video-podcast guidelines, virtual anatomy, and games for community clinic health care. These projects involve 10 additional faculty and staff: 5 physicians, 3 basic scientists, 1 librarian, and 1 curriculum coordinator.

The ATSU-SOMA TEAL-MEd initiative's pedagogical framework infuses clinical presentation,^{17, 18} situational learning,²¹ active learning,^2,4,20 deliberate practice in a clinical context,^{21, 22} team-based learning,²³ medical cognition,^{24, 25} and learner-centered approaches.¹ The TEAL-MEd initiative aligns with national medical education priorities such as the American Association of Colleges of Osteopathic Medicine (AACOM) Core Competencies,²⁶ patient safety,⁸ interprofessional education,^{27, 28} primary care,²⁹ 21st century skills,³⁰ and the National Association of Community Health Centers' mission of training physicians to work in community health centers.³¹ Principles of action research³² and design-based research^{2, 33} guided the process of designing and testing these interventions. Other industries describe these iterative implementations as "Plan-Do-Study-Assess" cycles.³⁴ Specific research questions (eg, Which virtual patient case mechanisms allow students to effectively make clinical decisions?) guided each design cycle.

Portfolio of Tools

The TEAL-MEd committee conducted landscape analyses through literature searches on electronic games and virtual simulations for health care education. In the process, the committee reviewed games and virtual patient simulations (VPSs) available commercially, interviewed game developers, and participated in game design courses, webinars, and national conferences.

On the basis of their findings, the TEAL-MEd committee developed ATSU-SOMA's current portfolio of TEAL-MEd classroom tools, which consist of custom activities created in 4 platforms: TurningPoint (Turning Technologies) for pause activities and games (ie, mini-activities interspersed throughout lessons), Bravo (C3 Softworks) quiz games for basic science practice, Prognosis-ATSU (Medical Joyworks) for just-in-time learning (ie, seeking knowledge anytime, anywhere as needed to solve real-world problems¹) on mobile devices, and DecisionSim (Decision Simulation) for nonlinear, VPS training scenarios.

TurningPoint

The advent of lecture capture and video podcasting allows residential students to learn by means of distance training. In some medical schools, this learning approach has affected classroom attendance. For example, researchers at Harvard medical school surveyed students regarding their learning preferences and found that 29.4% of the students preferred to watch lectures by video podcast.³⁵ These changes in the learning environment challenge faculty who wish to engage with students to increase the interactive nature of lectures.

Pause activities are one way faculty can make in-person lectures more engaging. These mini-activities interspersed throughout lessons provoke discussion and separate learning into 15-minute segments. TurningPoint pause activities invite physical and cognitive participation from the students. In addition, as Prince^13(pp3-4) stated, "Many proponents of active learning suggest that the effectiveness of this approach has to do with student attention span during lecture … 15 minutes."

Bravo

During 2011 to 2013, ATSU-SOMA faculty experimented with Bravo audience-response quiz games with large groups of first-year osteopathic medical students. Bravo offers several types of competitive education game formats, allowing faculty to prepare quiz-like games for review of medical knowledge using the game-builder interface (Figure 1). Students participate in games either in person using audience response "clickers," or remotely using the Blackboard learning management system (Blackboard, Inc). Responses can be anonymous or identified, and the performance results for each quiz item appear in aggregate after polling has closed, providing immediate feedback. This instructional medium allows faculty to pause and review the reasons for correct and incorrect answers. Student performance results may be downloaded for analysis after the classroom session.

Figure 1.

Screen capture of the game-builder interface in the Bravo (C3 Softworks) games. As part of a technology-enhanced active learning initiative, faculty at the A.T. Still University– School of Osteopathic Medicine in Arizona (Mesa) used the software to prepare and incorporate quiz-like games into their lectures for review of medical knowledge.

View Original | Slide (.ppt)

Prognosis-ATSU

Smart phones and tablets are common in medical education and health care interactions. Exercises using mobile devices offer opportunities for just-in-time classroom and clinical learning.³⁶ After a review of available mobile clinical simulation exercises, the TEAL-MEd committee decided to adapt Prognosis Your Diagnosis, a mobile application that challenges players to make 8 clinical decisions during 5-minute virtual patient cases. The medical school entered into a joint venture with the developers of Medical Joyworks to publish 2 new series of games based on this platform. Figure 2 shows a screen capture from the Prognosis-ATSU "Family Medicine" game series. After managing a virtual case on a mobile device, students receive immediate scores and feedback regarding the evaluation and management choices made during game play.

Figure 2.

Screen capture of a Prognosis-ATSU (Prognosis Your Diagnosis [Medical Joyworks]) mobile clinical simulation game, incorporated into the curriculum at the A.T. Still University–School of Osteopathic Medicine in Arizona (Mesa) as part of a technology-enhanced active learning initiative. After managing a virtual case on a mobile device, students receive immediate scores and feedback regarding the evaluation and management choices made during game play.

View Original | Slide (.ppt)

DecisionSim

The medical education literature⁸ suggests that VPSs provide students with opportunities to make mistakes and self-assess in a safe learning environment. The TEAL-MEd committee hypothesized that VPSs could also be leveraged for clinical decision making and learner construction of illness scripts, or mental maps for solving medical issues.²⁵ All of the VPSs include osteopathic considerations and scheme-inductive reasoning exercises designed to reinforce problem-solving illness scripts.²⁵ During the pilot implementation phase, we were also able to integrate 21st-century soft skills²⁶ such as professionalism; teamwork; and whole-person, patient-centered care²⁷ into the VPSs. For example, we incorporated elements of professionalism and social determinants into virtual scenarios and required students to collaborate in teams of 3 to solve the cases. The TEAL-MEd team presented these simulations at the Osteopathic Medical Education Leadership conference in 2014.

In 2011, ATSU-SOMA purchased Decision Simulation authoring licenses for faculty, as well as student accounts. The TEAL-MEd faculty constructed virtual case modules (Figure 3) using the case-builder wizard. The ATSU-SOMA VPS modules align with AACOM competencies²⁶ for clinical reasoning, communication, and teamwork. The case player allows for multimedia such as embedded video, as well as hyperlinks to Web-based resources such as treatment guidelines published by the National Institutes of Health.³⁷

Figure 3.

Screen capture of a virtual patient simulation (DecisionSim [Decision Simulation]), incorporated into the curriculum at the A.T. Still University–School of Osteopathic Medicine in Arizona (Mesa) as part of a technology-enhanced active learning initiative. Students accessed the virtual patient simulation online using their laptops and then collaborated in small groups to achieve consensus regarding decisions for the case.

View Original | Slide (.ppt)

During small group case practice, student teams access the cases online using their laptops and navigate through patient encounters for approximately 20 minutes. Students collaborate to achieve consensus regarding decisions in the case, receiving immediate written feedback and a score for each decision. Performance results output to students at the end of each case. Electronic performance reports are accessible on the Decision Simulation website.

Results

The TEAL-MEd committee assesses the effectiveness of ATSU-SOMA's initiative by tracking the number of TEAL-MEd activities implemented, the number of faculty that are trained, student and faculty perceptions of the activities, and evidence of improvement in the domains of medical knowledge, clinical reasoning, professionalism, and collaboration in the context of community-oriented primary care.

From 2010 to 2013, the TEAL-MEd committee facilitated more than 80 implementations of games and VPSs into the education of 550 osteopathic medical students, as well as multiple faculty training sessions. Table 1 summarizes the 81 TEAL-MEd activities implemented during the study period. Twenty-four simulations aligned with AACOM core competencies and scheme-inductive reasoning.

Table 1.

TEAL-MEd Activities Implemented During Academic Years 2010-2013 at ATSU-SOMA

Platform	Activity Description	No.	Learners	Sample Courses	Sample Topics
TurningPoint	Pause activities and games^a	18	OMS I	Gastrointestinal Neuromusculoskeletal Cardiopulmonary Renal-endocrine	Helicobacter pylori Soft tissue infection Endocarditis, tuberculosis, fungal pneumonia Urinary tract infection
Bravo	Basic science quiz-show game	26	OMS I	Renal-endocrine Gastrointestinal Neuromusculoskeletal Foundations of health Human development	Infections in diabetic patients Intestinal protozoa, oral infections Bone and joint infections Bacteriology, mycology, parasitology Patient safety
Prognosis-ATSU	Mobile clinical simulation exercises	13	OMS I-IV Faculty	Neuromusculoskeletal Cardiopulmonary Renal-endocrine Faculty development	Headache, rheumatoid arthritis Dyspnea Urinary frequency Mastoiditis
DecisionSim	Virtual patient simulations	24	OMS I	Foundations of health Cardiopulmonary Gastrointestinal Neuromusculoskeletal	Fever Cough, wheezing Constipation Headache, seizure, vertigo

^a Mini activities that professors intersperse throughout lessons to provoke discussion and chunk learning into 15-minute segments.Abbreviations: ATSU-SOMA, A.T. Still University–School of Osteopathic Medicine in Arizona; OMS, osteopathic medical student; TEAL-MEd, technology-enhanced active learning for medical education.

View Large

Table 1.

TEAL-MEd Activities Implemented During Academic Years 2010-2013 at ATSU-SOMA

Platform	Activity Description	No.	Learners	Sample Courses	Sample Topics
TurningPoint	Pause activities and games^a	18	OMS I	Gastrointestinal Neuromusculoskeletal Cardiopulmonary Renal-endocrine	Helicobacter pylori Soft tissue infection Endocarditis, tuberculosis, fungal pneumonia Urinary tract infection
Bravo	Basic science quiz-show game	26	OMS I	Renal-endocrine Gastrointestinal Neuromusculoskeletal Foundations of health Human development	Infections in diabetic patients Intestinal protozoa, oral infections Bone and joint infections Bacteriology, mycology, parasitology Patient safety
Prognosis-ATSU	Mobile clinical simulation exercises	13	OMS I-IV Faculty	Neuromusculoskeletal Cardiopulmonary Renal-endocrine Faculty development	Headache, rheumatoid arthritis Dyspnea Urinary frequency Mastoiditis
DecisionSim	Virtual patient simulations	24	OMS I	Foundations of health Cardiopulmonary Gastrointestinal Neuromusculoskeletal	Fever Cough, wheezing Constipation Headache, seizure, vertigo

^a Mini activities that professors intersperse throughout lessons to provoke discussion and chunk learning into 15-minute segments.Abbreviations: ATSU-SOMA, A.T. Still University–School of Osteopathic Medicine in Arizona; OMS, osteopathic medical student; TEAL-MEd, technology-enhanced active learning for medical education.

According to faculty development training records, 32 of the 80 full- and part-time ATSU-SOMA faculty (40%) were involved in designing or implementing TEAL-MEd activities, including 12 basic and clinical science faculty members who explored active learning with TurningPoint for individual vs group play, 5 faculty members who created games in Bravo, 10 faculty members who developed case-based games in Prognosis-ATSU, and 5 who designed DecisionSim VPSs.

TurningPoint

Student response to the TurningPoint pause activities was overwhelmingly positive. Common, unsolicited student comments provided in teaching evaluations stated that they appreciated the opportunity to immediately apply information and to gain practice with examination-style questions during large-group sessions.

Bravo

In 2013, TEAL-MEd committee researchers surveyed 107 first-year osteopathic medical students who had played the games in 3 different courses, with a response rate of 68%. Results indicated that a majority of students strongly agreed or agreed that Bravo games offered engaging formats, provided positive learning environments, clarified concepts, and encouraged clinical thinking.^38(p5)

Prognosis-ATSU

The TEAL-MEd committee hypothesized that Prognosis-ATSU games with primary care scenarios would enable student practice of clinical decision-making skills. In 2011-2012, the team produced 5 primary care games designed for play on mobile devices and conducted an institutional-level trial. Four cohorts of students downloaded the games and played them, and student clinical reasoning scores were successfully tracked. The results of this trial provided proof of concept for use of these games for deliberate practice in medical curricula. The TEAL-MEd committee presented these games at the Osteopathic Medical Education Leadership conference in 2013. Based on this pilot, a new series of games is currently being developed and implemented at ATSU-SOMA.

DecisionSim

Over the course of 2 years, TEAL-MEd faculty developed a series of 24 VPSs for primary care clinical presentations using a scheme-inductive, progressive-disclosure approach.¹⁸ These VPSs provide deliberate practice in clinical decision making by requiring students to apply the conceptual knowledge presented earlier in the curriculum. In 2012 and 2013 field tests, the committee integrated VPSs for weekly small group case practice during first-year osteopathic medical school foundations of health, neuromusculoskeletal, gastrointestinal, and cardiopulmonary courses (Table 1). Triangulated evidence from student exit survey results, faculty feedback, and classroom photographs revealed that the response from students was positive and suggested that students were cognitively engaged during exercises.³⁹

Discussion

Our finding that more than one-third of ATSU-SOMA faculty is involved with TEAL-MEd activities reflects a shift in the instructional culture at our institution, which did not previously use game-based or VPS instruction. We have completed initial pilot testing of this initiative and are now in the stage of evaluating data related to the potential learning gains from these activities.

The TEAL-MEd initiative has increased the TEAL-MEd options for faculty and students. Our experience indicates that interspersing electronic game and simulation activities throughout a curriculum provides students more variety, more opportunities for engagement and collaboration, and more deliberate practice with medical knowledge and clinical decision making.

While piloting TEAL-MEd activities, faculty explored the fundamentals of game software, AACOM competencies, scheme-inductive reasoning, peer collaboration, rubrics, feedback, skill tracking, individual vs group play, and the role of faculty as guides on the side during small-group tutoring. In our view, the instructional skills of faculty improved in these areas as they experimented with TEAL-MEd tools and methods. For example, 8 small-group facilitators practiced managing virtual simulations as coaches.

Challenges and Solutions

The TEAL-MEd committee and ATSU-SOMA faculty encountered and solved many challenges with regard to implementation, pedagogy, and research of TEAL-MEd activities (Table 2).

Table 2.

Challenges and Solutions to TEAL-MEd Activities Implemented During Academic Years 2010-2013 at ATSU-SOMA

TEAL-MEd Component	Challenge	Solution
Implementation
Acceptance	Implementing virtual activities requires faculty and student acceptance.	Provide experiential training. Measure satisfaction, and address areas of concern.
Technological integration	Integrating the new technology tools so that they complement the existing technology framework.	Consider classroom equipment and student mobile technology. Train students and faculty to access Web-based accounts or download applications for game and simulation activities.
Technology glitches	Technology issues may occur during initial sessions.	Pilot the activities to work out the technology bugs. Rehearse before classroom use.
Rights-free images	Virtual cases require rights-free images.	Publish precise faculty guidelines regarding use of permissioned images.
Mobile applications	Mobile applications must work on many platforms.	Ensure that mobile activities publish to iPhone, Android, iPad, and other tablets.
Pedagogical
Faculty role	Faculty can feel displaced during technology-enhanced learning.	Prepare faculty for their important role as facilitators during TEAL-MEd activities. Provide guidelines and instructions.
Feedback	It is important to provide rich feedback to students.	Ensure that games or activities provide students with immediate feedback and a terminal score.
Variety	Students easily burn out when 1 type of learning activity is overprescribed.	Provide a variety of TEAL-MEd activities.
Deep learning	Case simulation activities 5 to 10 minutes in length are sometimes too brief for deep learning.	Change brief clinical decision games to longer case discussions or add other extension exercises such as study questions, debriefs, and replay opportunities.
Motivation	How do we motivate students to apply effort to practice through virtual cases in nongraded situations?	Schedule short and focused activities, and implement them during in-class activities.
Research
Control groups	How do we set up experimental control groups in the classroom?	Use natural control groups and crossover designs so that curriculum delivered is equitable.
Video recording of classroom activities	Obtaining digital media of game play requires careful planning and consent.	Develop a consent protocol acceptable to the institutional review board. Obtain written consent from students and other stakeholders.
Avoiding survey fatigue	Alternative tools are needed to obtain feedback from students.	Add specific questions to course evaluations. Observe students during interactive sessions using observation protocols.
Measuring learning gain	How do we measure improvement after a single class session?	Design studies to obtain at least 4 sessions of student use with each media.
Proof of efficacy	It takes a long time to complete multiple iterative cycles of research to prove learning gains. In each cycle, the team must work out flaws in the game or research design.	1. Use PDSA cycles to collect 3 types of data: (a) feasibility of new TEAL-MEd intervention (b) learning gain, or affective change (c) change in clinical behavior 2. Use mixed-methods research to triangulate findings.

Abbreviations: ATSU-SOMA, A.T. Still University–School of Osteopathic Medicine in Arizona; PDSA, Plan-Do-Study-Assess; TEAL-MEd, technology-enhanced active learning for medical education.

View Large

Table 2.

Challenges and Solutions to TEAL-MEd Activities Implemented During Academic Years 2010-2013 at ATSU-SOMA

TEAL-MEd Component	Challenge	Solution
Implementation
Acceptance	Implementing virtual activities requires faculty and student acceptance.	Provide experiential training. Measure satisfaction, and address areas of concern.
Technological integration	Integrating the new technology tools so that they complement the existing technology framework.	Consider classroom equipment and student mobile technology. Train students and faculty to access Web-based accounts or download applications for game and simulation activities.
Technology glitches	Technology issues may occur during initial sessions.	Pilot the activities to work out the technology bugs. Rehearse before classroom use.
Rights-free images	Virtual cases require rights-free images.	Publish precise faculty guidelines regarding use of permissioned images.
Mobile applications	Mobile applications must work on many platforms.	Ensure that mobile activities publish to iPhone, Android, iPad, and other tablets.
Pedagogical
Faculty role	Faculty can feel displaced during technology-enhanced learning.	Prepare faculty for their important role as facilitators during TEAL-MEd activities. Provide guidelines and instructions.
Feedback	It is important to provide rich feedback to students.	Ensure that games or activities provide students with immediate feedback and a terminal score.
Variety	Students easily burn out when 1 type of learning activity is overprescribed.	Provide a variety of TEAL-MEd activities.
Deep learning	Case simulation activities 5 to 10 minutes in length are sometimes too brief for deep learning.	Change brief clinical decision games to longer case discussions or add other extension exercises such as study questions, debriefs, and replay opportunities.
Motivation	How do we motivate students to apply effort to practice through virtual cases in nongraded situations?	Schedule short and focused activities, and implement them during in-class activities.
Research
Control groups	How do we set up experimental control groups in the classroom?	Use natural control groups and crossover designs so that curriculum delivered is equitable.
Video recording of classroom activities	Obtaining digital media of game play requires careful planning and consent.	Develop a consent protocol acceptable to the institutional review board. Obtain written consent from students and other stakeholders.
Avoiding survey fatigue	Alternative tools are needed to obtain feedback from students.	Add specific questions to course evaluations. Observe students during interactive sessions using observation protocols.
Measuring learning gain	How do we measure improvement after a single class session?	Design studies to obtain at least 4 sessions of student use with each media.
Proof of efficacy	It takes a long time to complete multiple iterative cycles of research to prove learning gains. In each cycle, the team must work out flaws in the game or research design.	1. Use PDSA cycles to collect 3 types of data: (a) feasibility of new TEAL-MEd intervention (b) learning gain, or affective change (c) change in clinical behavior 2. Use mixed-methods research to triangulate findings.

Abbreviations: ATSU-SOMA, A.T. Still University–School of Osteopathic Medicine in Arizona; PDSA, Plan-Do-Study-Assess; TEAL-MEd, technology-enhanced active learning for medical education.

Implementation challenges included achieving support and consent from ATSU-SOMA faculty and leadership to integrate TEAL-MEd activities into courses. To accomplish this task, it was critical to train faculty in multiple sessions and then revise activities after receiving their input. The tipping point occurred after the second year of TEAL-MEd implementation, after faculty had had a chance to witness the learning activities in class and receive positive student comments in faculty evaluations.

The growing technological portfolio of the school required the team to integrate new game technology into the existing technological framework. This integration involved considering classroom equipment and student mobile technology. For this reason, it was important to include a technology expert on the TEAL-MEd committee. Over the course of 3 years, the school's technology team gradually increased technical support as the school integrated new technologies. The team engaged in continuous quality improvement cycles, technology discussions, and frequent faculty development seminars.

Faculty contact time during lessons did not change substantially, as TEAL-MEd activities were implemented during normal teaching episodes. Pedagogical challenges included redefining the faculty role during TEAL-MEd activities; ensuring that students received rich, timely feedback; formatting brief games to elicit deep learning; and embedding TEAL-MEd activities during structured, in-class time.

Research challenges such as assigning students to control groups, video recording in-class activities, avoiding survey fatigue, and measuring learning gains were overcome through iterative research cycles and consensus solutions generated by the TEAL-MEd committee.

Future Directions

It has taken 3 years for ATSU-SOMA to develop TEAL-MEd activities and pilot various research designs. A typical design sequence includes 1 year to develop a series to beta test and 2 years of pilot testing to resolve technology flaws and finalize research designs. The TEAL-MEd research in progress includes investigations of domains such as learning outcomes, clinical decision making, professional collaboration (ie, participation, communication, and soliciting opinions from team members), and engagement (ie, flow⁴⁰ [concentration, enjoyment], interest, and relevance).

In addition, the TEAL-MEd committee members have recently published a description of Bravo games for formative assessment, including student perceptions of this TEAL-MEd format.³⁸ We are currently preparing manuscripts describing gamification of TurningPoint, student clinical reasoning, collaboration and engagement during DecisionSim activities, and student perceptions of the effectiveness of Prognosis-ATSU games for student learning. During these activities, we have observed students collaborating and faculty evolving in their new roles as guides on the side.

Going forward, we will continue to design and test innovative models for engaging, high-quality health care training. We have found that TEAL-MEd activities may be useful for teaching concepts related to community medicine and changes in practice necessary to meet the requirements of the Patient Protection and Affordable Care Act.⁴¹ For example, TEAL-MEd committee faculty are currently working on thematic VPS modules and virtual environments related to community health center care delivery, including innovations that integrate virtual anatomy teaching materials and electronic health records.

Conclusion

In response to the current educational movement toward experiential, technology-based learning, the TEAL-MEd committee at ATSU-SOMA facilitated the implementation of more than 80 TEAL-MEd activities within the curriculum, with more than one-third of the school's faculty actively participating in game development or implementation. Using Plan-Do-Study-Assess cycles, the committee has improved TEAL-MEd activities and optimized lesson design. Research on learning outcomes of TEAL-MEd activities is in process.

References

1.

Cullen R, Harris M, Hill R. The Learner-Centered Curriculum: Design and Implementation. San Francisco, CA: Jossey-Bass; 2012.

2.

Wang F, Hannafin, MJ. Design-based research and technology-enhanced learning environments. Educ Tech Res Dev. 2005;53:4. [CrossRef]

3.

Roberts G. Technology and learning experiences of the net generation. In: Oblinger D, Oblinger J, eds. Educating the Net Generation. Louisville, CO: Educause; 2005.

4.

Educause Center for Applied Research (ECAR). ECAR Study of Undergraduate Students and Education Technology. Louisville, CO: Educause; 2012.

5.

Kron FW, Gjerde CL, Sen A, Fetters MD. Medical student attitudes toward video games and related new media technologies in medical education. BMC Med Educ. 2010;10:50. doi:10.1186/1472-6920-10-50. [CrossRef] [PubMed]

6.

Gee JP. Pleasure, learning, video games, and life: the projective stance. E-Learn. 2005;2(3):211. [CrossRef]

7.

Johnson L, Adams S, Cummins M. NMC Horizon Report: 2012 Higher Education Edition. Austin, TX: The New Media Consortium; 2012.

8.

Ziv A, Wolpe PR, Small SD, Glick S. Simulation-based medical education: an ethical imperative. Acad Med. 2003;78(8):783-788. [CrossRef] [PubMed]

9.

Bateman J, Allen ME, Kidd J, Parsons N, Davies D. Virtual patients design and its effect on clinical reasoning and student experience: a protocol for a randomised factorial multi-centre study. BMC Med Educ. 2012;12:162. doi:10.1186/1472-6920-12-62. [CrossRef]

10.

Green CS, Bavelier D. Action video game modifies visual selective attention. Nature. 2003;423(6939):534-537. [CrossRef] [PubMed]

11.

de Bilde J, Vansteekiste M, Lens W. Understanding the association between future time perspective and self-regulated learning through the lens of self-determination theory. Learn Instr. 2011;21:332-344. [CrossRef]

12.

Mazmanian P, Feldman M. Theory is needed to improve education assessment and policy in self-directed learning. Med Educ. 2011;45(4):324-326. doi:10.1111/j.1365-2923.2011.03937.x. [CrossRef] [PubMed]

13.

Prince M. Does active learning work? a review of the research. J Eng Educ. 2004;93(3):223-231. doi:10.1002/j.2168-9830.2004.tb00809.x. [CrossRef]

14.

Offir B, Lev Y, Bezalel R. Surface and deep learning processes in distance education: synchronous versus asynchronous systems. Comp Edu. 2008;51:1172-1183. doi:10.1016/j.compedu. 2007.10.009. [CrossRef]

15.

Reeves D. Level 5 networks: making significant change in complex organizations. In: Hargreaves A, Fullan M, eds. Change Wars. Bloomington, IN: Solution Tree; 2010.

16.

Wenger E. Communities of Practice: Learning, Meaning, and Identity. New York, NY: Oxford University Press; 1998.

17.

Schwartz FN, Hover M, Kinney M, McCoy L. Student assessment of an innovative approach to medical education. J Int Assoc Med Sci Educ. 2012;22(3):102-107. [CrossRef]

18.

Mandin H, Harasym P, Eagle C, Watanabe M. Developing a "clinical presentation" curriculum at the University of Calgary. Acad Med. 1995;70(3):186-193. [CrossRef] [PubMed]

19.

Lave J, Wenger E. Situated Learning: Legitimate Peripheral Participation. New York, NY: Cambridge University Press; 1991.

20.

Wood W. Clickers: a teaching gimmick that works. Dev Cell. 2004;7(6):796-179. [CrossRef]

21.

Dhaliwal G. Clinical excellence: make it a habit. Acad Med. 2012;87(11):1473. doi:10.1097/ACM.0b013e31826d68d9. [CrossRef] [PubMed]

22.

Ericsson KA. Deliberate practice and the acquisition and maintenance of expert performance in medicine and related domains. Acad Med. 2004;79(10):70-81. [CrossRef]

23.

Michaelsen LK, Parmelee D, Levine R. Team-Based Learning for Health Professions Education: A Guide to Using Small Groups for Improving Learning. Sterling, VA: Stylus Publishing, LLC; 2008.

24.

Patel VL, Arocha JF, Zhang J. Thinking and reasoning in medicine. In: Cambridge Handbook of Thinking and Reasoning. Cambridge, UK: Cambridge University Press; 2004.

25.

Charlin B, Tardif J, Henny P, Boshuizen A. Scripts and medical diagnostic knowledge: theory and applications for clinical reasoning. Acad Med. 2000;75:182-190. [CrossRef] [PubMed]

26.

American Association of Colleges of Osteopathic Medicine (AACOM). Osteopathic Core Competencies for Medical Students. Chevy Chase, MD: AACOM; August 2012. http://www.aacom.org/docs/default-source/core-competencies/corecompetencyreport2012.pdf?sfvrsn=4. Accessed March 6, 2015.

27.

Frenk J, Chen L, Bhutta Z et al Health professionals for a new century: transforming education to strengthen health systems in an interdependent world. Lancet. 2010;376(9756):1923-1958. doi:10.1016/S0140-6736(10)61854-5. [CrossRef] [PubMed]

28.

Buring SM, Bhushan A, Brazeau G, Conway S, Hansen L, Westberg S. Keys to successful implementation of interprofessional education: learning location, faculty development, and curricular themes. Am J Pharm Educ. 2009;73(4):60. [CrossRef] [PubMed]

29.

Walker T. Interest in primary care up, but shortage still looms. Medical Economics. May 10, 2013. http://medicaleconomics.modernmedicine.com/medical-economics/content/modernmedicine/modern-medicine-feature-articles/interest-primary-care-shor. Accessed February 12, 2015.

30.

Kereluik K, Mishra P, Fahnoe C, Terry L. What knowledge is of most worth: teacher knowledge for 21st century learning. J Digital Learn Teach Educ. 2013;29(4):127-140. [CrossRef]

31.

National Association of Community Health Centers (NACHC) website. http://www.nachc.com. Accessed February 12, 2015.

32.

Mills GE. Action Research: A Guide for the Teacher Researcher. Boston, MA: Pearson Education; 2011.

33.

Barab SA, Squire K. Design-based research: putting a stake in the ground. J Learn Sci. 2004;13(1):1-14. doi:10.1207/s15327809jls1301_1. [CrossRef]

34.

Langley G, Moen R, Nolan K, Nolan T, Norman C, Provost L. The Improvement Guide: A Practical Approach To Enhancing Organizational Performance. San Francisco, CA: Jossey Bass; 2009.

35.

Cardall, S, Krupat, E, Ulrich, M. Live lecture versus video-recorded lecture: are students voting with their feet? Acad Med. 2008;83(12):1174-1178. doi:10.1097/ACM.0b013e31818c6902. [CrossRef] [PubMed]

36.

Tews M, Brennan K, Begaz T, Treat R. Medical student case presentation performance and perception when using mobile learning technology in the emergency department [published online October 14, 2011]. Med Educ Online. 2011;16:1-7. doi:10.3402/meo.v16i0.7327. [CrossRef]

37.

National Institutes of Health (NIH) website. http://health.nih.gov/. Accessed February 12, 2015.

38.

Pettit RK, McCoy L, Kinney MB, Schwartz FN. A multimedia audience response game show for medical education. Med Sci Educ. 2014;24:181-187 doi:10.1007/s40670-014-0038-x. [CrossRef]

39.

McCoy L. Virtual Patient Simulations for Medical Education: Increasing Clinical Reasoning Skills through Deliberate Practice [dissertation]. Tempe: Arizona State University; March 2014.

40.

Schiefele U, Raabe A. Skills-demands compatibility as a determinant of flow experience in an inductive reasoning task. Psychol Rep. 2011;109(2):428-444. [CrossRef] [PubMed]

41.

About the law. US Dept of Health and Human Services website. http://www.hhs.gov/healthcare/rights/index.html. Accessed February 12, 2015.

Figure 1.

Screen capture of the game-builder interface in the Bravo (C3 Softworks) games. As part of a technology-enhanced active learning initiative, faculty at the A.T. Still University– School of Osteopathic Medicine in Arizona (Mesa) used the software to prepare and incorporate quiz-like games into their lectures for review of medical knowledge.

View Original | Slide (.ppt)

Figure 2.

Screen capture of a Prognosis-ATSU (Prognosis Your Diagnosis [Medical Joyworks]) mobile clinical simulation game, incorporated into the curriculum at the A.T. Still University–School of Osteopathic Medicine in Arizona (Mesa) as part of a technology-enhanced active learning initiative. After managing a virtual case on a mobile device, students receive immediate scores and feedback regarding the evaluation and management choices made during game play.

View Original | Slide (.ppt)

Figure 3.

Screen capture of a virtual patient simulation (DecisionSim [Decision Simulation]), incorporated into the curriculum at the A.T. Still University–School of Osteopathic Medicine in Arizona (Mesa) as part of a technology-enhanced active learning initiative. Students accessed the virtual patient simulation online using their laptops and then collaborated in small groups to achieve consensus regarding decisions for the case.

View Original | Slide (.ppt)

Table 1.

TEAL-MEd Activities Implemented During Academic Years 2010-2013 at ATSU-SOMA

Platform	Activity Description	No.	Learners	Sample Courses	Sample Topics
TurningPoint	Pause activities and games^a	18	OMS I	Gastrointestinal Neuromusculoskeletal Cardiopulmonary Renal-endocrine	Helicobacter pylori Soft tissue infection Endocarditis, tuberculosis, fungal pneumonia Urinary tract infection
Bravo	Basic science quiz-show game	26	OMS I	Renal-endocrine Gastrointestinal Neuromusculoskeletal Foundations of health Human development	Infections in diabetic patients Intestinal protozoa, oral infections Bone and joint infections Bacteriology, mycology, parasitology Patient safety
Prognosis-ATSU	Mobile clinical simulation exercises	13	OMS I-IV Faculty	Neuromusculoskeletal Cardiopulmonary Renal-endocrine Faculty development	Headache, rheumatoid arthritis Dyspnea Urinary frequency Mastoiditis
DecisionSim	Virtual patient simulations	24	OMS I	Foundations of health Cardiopulmonary Gastrointestinal Neuromusculoskeletal	Fever Cough, wheezing Constipation Headache, seizure, vertigo

^a Mini activities that professors intersperse throughout lessons to provoke discussion and chunk learning into 15-minute segments.Abbreviations: ATSU-SOMA, A.T. Still University–School of Osteopathic Medicine in Arizona; OMS, osteopathic medical student; TEAL-MEd, technology-enhanced active learning for medical education.

View Large

Table 1.

TEAL-MEd Activities Implemented During Academic Years 2010-2013 at ATSU-SOMA

Platform	Activity Description	No.	Learners	Sample Courses	Sample Topics
TurningPoint	Pause activities and games^a	18	OMS I	Gastrointestinal Neuromusculoskeletal Cardiopulmonary Renal-endocrine	Helicobacter pylori Soft tissue infection Endocarditis, tuberculosis, fungal pneumonia Urinary tract infection
Bravo	Basic science quiz-show game	26	OMS I	Renal-endocrine Gastrointestinal Neuromusculoskeletal Foundations of health Human development	Infections in diabetic patients Intestinal protozoa, oral infections Bone and joint infections Bacteriology, mycology, parasitology Patient safety
Prognosis-ATSU	Mobile clinical simulation exercises	13	OMS I-IV Faculty	Neuromusculoskeletal Cardiopulmonary Renal-endocrine Faculty development	Headache, rheumatoid arthritis Dyspnea Urinary frequency Mastoiditis
DecisionSim	Virtual patient simulations	24	OMS I	Foundations of health Cardiopulmonary Gastrointestinal Neuromusculoskeletal	Fever Cough, wheezing Constipation Headache, seizure, vertigo

^a Mini activities that professors intersperse throughout lessons to provoke discussion and chunk learning into 15-minute segments.Abbreviations: ATSU-SOMA, A.T. Still University–School of Osteopathic Medicine in Arizona; OMS, osteopathic medical student; TEAL-MEd, technology-enhanced active learning for medical education.

Table 2.

Challenges and Solutions to TEAL-MEd Activities Implemented During Academic Years 2010-2013 at ATSU-SOMA

TEAL-MEd Component	Challenge	Solution
Implementation
Acceptance	Implementing virtual activities requires faculty and student acceptance.	Provide experiential training. Measure satisfaction, and address areas of concern.
Technological integration	Integrating the new technology tools so that they complement the existing technology framework.	Consider classroom equipment and student mobile technology. Train students and faculty to access Web-based accounts or download applications for game and simulation activities.
Technology glitches	Technology issues may occur during initial sessions.	Pilot the activities to work out the technology bugs. Rehearse before classroom use.
Rights-free images	Virtual cases require rights-free images.	Publish precise faculty guidelines regarding use of permissioned images.
Mobile applications	Mobile applications must work on many platforms.	Ensure that mobile activities publish to iPhone, Android, iPad, and other tablets.
Pedagogical
Faculty role	Faculty can feel displaced during technology-enhanced learning.	Prepare faculty for their important role as facilitators during TEAL-MEd activities. Provide guidelines and instructions.
Feedback	It is important to provide rich feedback to students.	Ensure that games or activities provide students with immediate feedback and a terminal score.
Variety	Students easily burn out when 1 type of learning activity is overprescribed.	Provide a variety of TEAL-MEd activities.
Deep learning	Case simulation activities 5 to 10 minutes in length are sometimes too brief for deep learning.	Change brief clinical decision games to longer case discussions or add other extension exercises such as study questions, debriefs, and replay opportunities.
Motivation	How do we motivate students to apply effort to practice through virtual cases in nongraded situations?	Schedule short and focused activities, and implement them during in-class activities.
Research
Control groups	How do we set up experimental control groups in the classroom?	Use natural control groups and crossover designs so that curriculum delivered is equitable.
Video recording of classroom activities	Obtaining digital media of game play requires careful planning and consent.	Develop a consent protocol acceptable to the institutional review board. Obtain written consent from students and other stakeholders.
Avoiding survey fatigue	Alternative tools are needed to obtain feedback from students.	Add specific questions to course evaluations. Observe students during interactive sessions using observation protocols.
Measuring learning gain	How do we measure improvement after a single class session?	Design studies to obtain at least 4 sessions of student use with each media.
Proof of efficacy	It takes a long time to complete multiple iterative cycles of research to prove learning gains. In each cycle, the team must work out flaws in the game or research design.	1. Use PDSA cycles to collect 3 types of data: (a) feasibility of new TEAL-MEd intervention (b) learning gain, or affective change (c) change in clinical behavior 2. Use mixed-methods research to triangulate findings.

Abbreviations: ATSU-SOMA, A.T. Still University–School of Osteopathic Medicine in Arizona; PDSA, Plan-Do-Study-Assess; TEAL-MEd, technology-enhanced active learning for medical education.

View Large

Table 2.

Challenges and Solutions to TEAL-MEd Activities Implemented During Academic Years 2010-2013 at ATSU-SOMA

TEAL-MEd Component	Challenge	Solution
Implementation
Acceptance	Implementing virtual activities requires faculty and student acceptance.	Provide experiential training. Measure satisfaction, and address areas of concern.
Technological integration	Integrating the new technology tools so that they complement the existing technology framework.	Consider classroom equipment and student mobile technology. Train students and faculty to access Web-based accounts or download applications for game and simulation activities.
Technology glitches	Technology issues may occur during initial sessions.	Pilot the activities to work out the technology bugs. Rehearse before classroom use.
Rights-free images	Virtual cases require rights-free images.	Publish precise faculty guidelines regarding use of permissioned images.
Mobile applications	Mobile applications must work on many platforms.	Ensure that mobile activities publish to iPhone, Android, iPad, and other tablets.
Pedagogical
Faculty role	Faculty can feel displaced during technology-enhanced learning.	Prepare faculty for their important role as facilitators during TEAL-MEd activities. Provide guidelines and instructions.
Feedback	It is important to provide rich feedback to students.	Ensure that games or activities provide students with immediate feedback and a terminal score.
Variety	Students easily burn out when 1 type of learning activity is overprescribed.	Provide a variety of TEAL-MEd activities.
Deep learning	Case simulation activities 5 to 10 minutes in length are sometimes too brief for deep learning.	Change brief clinical decision games to longer case discussions or add other extension exercises such as study questions, debriefs, and replay opportunities.
Motivation	How do we motivate students to apply effort to practice through virtual cases in nongraded situations?	Schedule short and focused activities, and implement them during in-class activities.
Research
Control groups	How do we set up experimental control groups in the classroom?	Use natural control groups and crossover designs so that curriculum delivered is equitable.
Video recording of classroom activities	Obtaining digital media of game play requires careful planning and consent.	Develop a consent protocol acceptable to the institutional review board. Obtain written consent from students and other stakeholders.
Avoiding survey fatigue	Alternative tools are needed to obtain feedback from students.	Add specific questions to course evaluations. Observe students during interactive sessions using observation protocols.
Measuring learning gain	How do we measure improvement after a single class session?	Design studies to obtain at least 4 sessions of student use with each media.
Proof of efficacy	It takes a long time to complete multiple iterative cycles of research to prove learning gains. In each cycle, the team must work out flaws in the game or research design.	1. Use PDSA cycles to collect 3 types of data: (a) feasibility of new TEAL-MEd intervention (b) learning gain, or affective change (c) change in clinical behavior 2. Use mixed-methods research to triangulate findings.

Abbreviations: ATSU-SOMA, A.T. Still University–School of Osteopathic Medicine in Arizona; PDSA, Plan-Do-Study-Assess; TEAL-MEd, technology-enhanced active learning for medical education.

Developing Technology-Enhanced Active Learning for Medical Education: Challenges, Solutions, and Future Directions

Related content

Related Topics