Abstract
Context: Traditional medical education is shifting to incorporate learning technologies and online educational activities with traditional face-to-face clinical instruction to engage students, especially at remote clinical training sites.
Objective: To describe and evaluate the effectiveness of the blended learning format (combining online and face-to-face instruction) for third-year osteopathic medical students during their pediatric rotation.
Methods: Third-year medical students who completed the 4-week clerkship in pediatrics during the 2014-2015 academic year were divided into a standard learning group and a blended learning group with online activities (discussion boards, blogs, virtual patient encounters, narrated video presentations, and online training modules). Comprehensive Osteopathic Medical Achievement Test scores and final course grades were compared between the standard learning and blended learning groups. Students in the blended learning group completed a postsurvey regarding their experiences.
Results: Of 264 third-year students who completed the 4-week clerkship in pediatrics during the 2014-2015 academic year, 78 (29.5%) participated in the blended learning supplement with online activities. Of 53 students who completed the postsurvey in the blended learning group, 44 (83.0%) agreed or strongly agreed that “The integration of e-learning and face-to-face learning helped me learn pediatrics.” Open-ended comments supported this overall satisfaction with the course format; however, 26 of 100 comments reflected a desire to increase the amount of clinical exposure and face-to-face time with patients. No statistical differences were seen between the standard learning (n=186) and blended learning (n=78) groups with regard to Comprehensive Osteopathic Medical Achievement Test scores (P=.321). Compared with the standard learning group, more students in the blended learning group received a final course grade of honors (P=.015).
Conclusion: Results of this study support the use of blended learning in a clinical training environment. As more medical educators use blended learning, it is important to investigate the best balance between learning with technology and learning in a face-to-face setting. Online activities may enhance but should never fully replace face-to-face learning with real patients.
Keywords: blended learning, clinical training, e-learning, medical education, online learning, pediatric education
The rapid growth of new osteopathic medical schools and branch campuses over the past decade has created concerns about the availability and adequacy of clinical training sites in the osteopathic medical profession. This concern is shared by the medicine, nursing, laboratory technology, and other health care professions.
1-4 As traditional hospital-based clinical training opportunities become more limited, medical educators look to develop new clinical training sites and new educational models to meet demand. Clinical training sites vary in setting, location, and teaching method; despite these variations, medical students can have similar learning experiences, performances on evaluations, and postrotation examination scores.
5-7 For decades, community-based training sites have served a crucial role in osteopathic undergraduate medical education. In a 2013 national survey, 57% of the deans from osteopathic medical schools were concerned about the quality of clinical training sites, and many of these schools have created supplemental didactic or computer-based curriculum to address this concern.
1
E-learning initiatives, such as online and blended learning, are increasingly being integrated into a number of educational programs for medicine, nursing, physical therapy, nutrition, social work, and pharmacy.
4,8-27 These educational initiatives provide a consistent academic experience, engage learners at a time and location that is most convenient for them, encourage learners to build on previous knowledge and experiences, maximize learning opportunities, and promote authentic learning by allowing learners to participate in e-learning activities while working in a clinical environment. The National Council of State Boards of Nursing has embarked on a landmark study substituting traditional face-to-face clinical education with simulation and e-learning programs.
2,3 This strategy is meant to address the need to increase the number and quality of training programs to accommodate the growing student population that will in turn care for the growing US patient population.
2,3
Although many formal definitions exist, blended learning is essentially the integration of online and face-to-face engagement to facilitate learning between students, teachers, and educational resources.
27-29 Blended learning is more than a collection of digital technology, games, and tools; rather, it is an educational strategy to integrate learning technologies with face-to-face instruction.
27-30 Through this type of learning, learners may engage in a variety of e-learning activities, such as online content review, discussion boards, interactive blogs, wiki websites, Web conferencing, self-reflection, and group activities. Blended learning is particularly well suited for clinical education, especially for medical students rotating in geographically distributed training sites. These programs could improve consistency across training sites, maximize learning opportunities, reduce the burden of clinical preceptors at the onsite training facilities, allow students to link experiences to previous knowledge, and increase the number of educational opportunities for students. In addition, blended learning may help to centralize educational activities and reduce the didactic burden for clinical preceptors at remote training sites. Previous studies provide rudimentary support and evidence for blended learning; however, rigorous research is still lacking.
24,25
In this study, we describe a third-year clinical rotation in pediatrics facilitated partially online as a blended learning supplement. The program combined online learning (ie, asynchronous discussion boards and blogs, podcasts, video demonstrations, didactic presentations, scenario-based instruction, menu-driven simulation and virtual patients, and online reference material and resources), and face-to-face clinical instruction with a faculty preceptor. We also evaluated the effectiveness of this blended learning program through course evaluation (ie, postsurvey) and performance outcomes (ie, end-of-rotation examination scores and final course grades).
Methods
The study design was nonexperimental with a convenience sample of third-year osteopathic medical students in clinical rotations from a single osteopathic medical school during the 2014-2015 academic year. Of the 18 clinical rotation sites, 4 sites implemented the 4-week blended learning course (combining online and face-to-face instruction with clinical preceptors), and the remaining 14 sites maintained the standard 4-week curriculum. The Philadelphia College of Osteopathic Medicine’s institutional review board approved this study.
Course Description
The blended learning program was constructed on 4 pedagogic principles: problem-based learning,
31 constructivism,
32 connectivism,
33 and online collaborative learning.
32 In these models, the role of the teacher is to provide support as the learner builds on his or her knowledge and previous experiences, stimulate and facilitate discussion, and motivate and challenge learners. Instructional strategies for teaching in these models are learner centered and include authentic learning, reflection, and collaboration. Students were presented with a number of learning resources, educational content, and assessments using a Web-based learning management system. Course activities included a combination of the elements and assessments in
Table 1.
Table 1.
Third-Year Pediatric Clinical Rotation Course Activities in the Blended Learning Group
| Activities | Description |
| Face-to-Face Components |
| Clinical teaching | Clinical precepting with a pediatrician 3 to 4 days per week. |
| Case log | Students maintain a log of patients seen during the rotation. |
| History and physical examinations | Students submit history and physical examination forms for patients of specified ages. |
| E-learning Components |
| Discussion boards | Students provide original comments and responses to discussion board threads, such as “Identify 1 practice guideline intended to prevent illness or promote health in children. Summarize the evidence-based references and/or recommendations, and cite your reference.” |
| Blogs | Students post an introduction blog and share information about themselves (interests, professional goals, and experience with children), and they note 3 specific goals to achieve during this rotation. |
| Podcasts | Students listen to orientation and summary of learning objectives that are presented as podcasts for each week of the course. |
| Virtual patient encounters | Students complete 12 to 32 pediatric menu-driven simulations (ie, Computer-Assisted Learning in Pediatrics Program cases). |
| Website links/WebQuests | Students explore various Web-based resources: Centers for Disease Control and Prevention, American Academy of Pediatrics Bright Futures, KidsHealth, and GeneTests. |
| Video demonstrations | Students watch videos such as the American Academy of Pediatrics’ A View Through the Otoscope: Distinguishing Acute Otitis Media From Otitis Media With Effusion. |
| Narrated presentations | Students review faculty-created PowerPoint presentations. |
| Articles and resources | Students review a number of articles, clinical guidelines, and references. |
| Community resource summary | Students identify a Philadelphia-based community resource for patients, write a summary, and post it on the course website. |
| Case write-up | Students prepare a formal case write-up and submit it as an assignment. |
| Online training modules | Students complete the California Vaccines for Children EZIZ Vaccine Administration Online Training program. |
Table 1.
Third-Year Pediatric Clinical Rotation Course Activities in the Blended Learning Group
| Activities | Description |
| Face-to-Face Components |
| Clinical teaching | Clinical precepting with a pediatrician 3 to 4 days per week. |
| Case log | Students maintain a log of patients seen during the rotation. |
| History and physical examinations | Students submit history and physical examination forms for patients of specified ages. |
| E-learning Components |
| Discussion boards | Students provide original comments and responses to discussion board threads, such as “Identify 1 practice guideline intended to prevent illness or promote health in children. Summarize the evidence-based references and/or recommendations, and cite your reference.” |
| Blogs | Students post an introduction blog and share information about themselves (interests, professional goals, and experience with children), and they note 3 specific goals to achieve during this rotation. |
| Podcasts | Students listen to orientation and summary of learning objectives that are presented as podcasts for each week of the course. |
| Virtual patient encounters | Students complete 12 to 32 pediatric menu-driven simulations (ie, Computer-Assisted Learning in Pediatrics Program cases). |
| Website links/WebQuests | Students explore various Web-based resources: Centers for Disease Control and Prevention, American Academy of Pediatrics Bright Futures, KidsHealth, and GeneTests. |
| Video demonstrations | Students watch videos such as the American Academy of Pediatrics’ A View Through the Otoscope: Distinguishing Acute Otitis Media From Otitis Media With Effusion. |
| Narrated presentations | Students review faculty-created PowerPoint presentations. |
| Articles and resources | Students review a number of articles, clinical guidelines, and references. |
| Community resource summary | Students identify a Philadelphia-based community resource for patients, write a summary, and post it on the course website. |
| Case write-up | Students prepare a formal case write-up and submit it as an assignment. |
| Online training modules | Students complete the California Vaccines for Children EZIZ Vaccine Administration Online Training program. |
×
With support from staff in the Department of Academic Technologies, 2 clinical faculty with additional training in education and online learning facilitated the online portion of the blended learning course. In addition to the clinical preceptors who facilitated the online instruction, students were supervised and instructed by clinical site directors and preceptors at each of the respective clinical training sites. These face-to-face (standard learning group) preceptors did not participate in online activity.
Course Evaluation by Students
Students in the blended learning and standard learning groups both provided feedback in the usual manner by completing an anonymous online course evaluation. Questions did not pertain directly to blended or online learning, but students provided comments that may or may not have pertained to the blended learning supplement.
Students in the blended learning group were also provided an opportunity to complete a postsurvey regarding their experience with the blended learning format. This survey was constructed with input from the course director, participating faculty, associate dean of clinical education, and academic technologies staff. Students in the blended learning group voluntarily completed the survey online. The survey instrument included items specifically related to the online components of the course and comprised 38 Likert-type items, an opportunity to provide open-ended comments, and a 15-item adjective checklist.
34
Student Performance Outcomes
Both groups completed the 120-item Comprehensive Osteopathic Medical Achievement Test (COMAT) at the end of rotation.
35 All students received final grades for the course: honors, high pass, pass, or fail. These grades were assigned based on the end-of-rotation evaluations by the preceptor and COMAT scores, each contributing to 50% of the final grade.
Preceptor evaluations included 18 Likert-type global assessment scores on a scale of 1 (substandard) to 10 (outstanding) that were divided across the 6 osteopathic core competencies; the preceptor assigned a preceptor evaluation of fail, pass, or honor pass. Scores for the COMAT examination included honors (ie, greater than 1 SD above the school mean), pass (ie, between 1 SD above and 2 SDs below the school mean), and fail (greater than 2 SDs below the school mean).
These preceptor evaluations and COMAT scores were combined for a final course grade: honors, high pass, pass, or fail.
Analysis
Data from postsurveys were extracted from SurveyMonkey using Microsoft Excel (Microsoft Corporation). Rotation evaluations were anonymous and were extracted from CoursEval. Scores for the COMAT examination and final course grades were extracted using Microsoft Excel and analyzed using SPSS statistical software version 22.0 (SPSS Inc). A 2-tailed t test was used to evaluate COMAT score differences between groups, and a test for independence (using χ2 distribution) was used to compare final grades between groups.
Results
Of the 264 third-year students who completed the 4-week clerkship in the 2014-2015 academic year, 78 (29.5%) participated in the blended learning group, and 186 (70.5%) participated in the standard learning group. Of 264 students, 256 (97.0%) completed the anonymous course evaluation. Students shared 446 open-ended responses to their overall experience with the rotation on this course evaluation, and 8 comments specifically addressed their experience with blended learning. Comments included requests to limit online learning without reducing the amount of clinical time for face-to-face education, overall satisfaction with the course and educational value, frustration with critical assignments being due at the end (rather than the beginning) of the rotation, desire to have a more transparent scoring rubric for discussion board participation, and satisfaction with menu-driven simulations.
Of the 78 students in the blended learning group, 53 (67.9%) completed the postsurvey (
Table 2). Of the 53 respondents, 46 (86.8%) agreed or strongly agreed with the statement, “This was a practical learning experience,” and 44 (83.0%) agreed or strongly agreed with “The integration of e-learning and face-to-face learning helped me learn pediatrics.” Overall, 26 students (49.1%) agreed or strongly agreed with the statement, “I prefer this hybrid learning format to traditional face-to-face clinical rotations,” and 38 (71.7%) agreed or strongly agreed with the statement, “The amount of work required for this course was appropriate.” When asked “how much time did you spend per week on the online component of this course?,” 11 (21%) reported 0 to 5 hours, 31 (58%) reported 6 to 10 hours, 9 (17%) reported 11 to 15 hours, and 1 (2%) reported spending 16 to 20 hours per week. Overall, 10 (18.9%) reported spending 11 hours or more per week, and 44 (83.0%) reported “I was satisfied with the overall learning experience.”
Table 2.
Survey Responses of Third-Year Osteopathic Medical Students in a Blended Learning Group in a Pediatric Clinical Rotation (n=53)a
| Statement | Strongly Disagree | Disagree | Agree | Strongly Agree |
| Course Format |
| The integration of e-learning and face-to-face learning was convenient for me. | 2 | 6 | 24 | 20 |
| This was a practical learning experience. | 3 | 3 | 31 | 15 |
| The integration of e-learning and face-to-face learning helped me learn pediatrics topics. | 2 | 6 | 25 | 19 |
| Overall Technology |
| I have taken online or hybrid courses in the past. | 11 | 16 | 17 | 8 |
| I have used Blackboard in the past. | 0 | 0 | 20 | 32 |
| I found Blackboard easy to use in this course. | 2 | 2 | 25 | 23 |
| I found the online preceptor to be helpful when addressing technology-related issues or problems. | 0 | 0 | 14 | 37 |
| I did not experience any technical difficulties during the course. | 0 | 2 | 27 | 22 |
| Discussion Boards |
| Discussion boards improved my understanding of pediatric topics. | 4 | 12 | 29 | 7 |
| Discussion boards were valuable. | 5 | 9 | 30 | 8 |
| Discussions were relevant. | 3 | 2 | 32 | 15 |
| Discussions were facilitated well by the online preceptor. | 2 | 3 | 26 | 21 |
| Blogs |
| The introduction blog was valuable. | 7 | 13 | 25 | 7 |
| The reflection blog helped me review what I learned during the course. | 5 | 10 | 27 | 10 |
| CLIPP Cases |
| CLIPP cases were valuable. | 1 | 3 | 21 | 27 |
| CLIPP cases covered topics and clinical presentations that were not seen during my face-to-face clinical experience with patients at my training site. | 1 | 2 | 17 | 32 |
| Case Write-Upb |
| The case write-up was a valuable exercise. | 3 | 6 | 33 | 8 |
| Completing the case write-up helped me to learn a pediatric topic in detail. | 2 | 4 | 31 | 13 |
| Preparing the case write-up increased my comfort with medical writing. | 2 | 9 | 30 | 9 |
| Community Resource Summaryb |
| Preparing the community resource summary was a valuable learning experience. | 1 | 10 | 24 | 15 |
| Preparing the community resource summary helped me learn about a community resource available to families in Philadelphia. | 1 | 5 | 26 | 18 |
| Learning Objectives |
| I developed an understanding for communication with children and their families. | 1 | 0 | 23 | 28 |
| I developed basic skills in conducting newborn, pediatric, and adolescent physical examinations. | 1 | 0 | 23 | 28 |
| I can describe significant developmental milestones in childhood. | 0 | 4 | 27 | 21 |
| I can identify nutrition requirements for infants and children. | 0 | 4 | 29 | 19 |
| I recognize “normal values” for vital signs and anthropomorphic data. | 1 | 11 | 24 | 16 |
| I appreciate the importance of vaccination in promoting health and wellness. | 0 | 0 | 16 | 36 |
| I understand the influence of family, community, and society on the child in health and disease. | 0 | 1 | 14 | 37 |
| I understand important preventive and anticipatory guidance strategies for children. | 0 | 0 | 23 | 29 |
| I have started to understand common diseases in children (eg, asthma, anemia). | 0 | 3 | 20 | 29 |
| I can identify important clinical resources to help me care for children. | 1 | 2 | 31 | 18 |
| Overall Experience |
| I prefer this blended learning format (e-learning combined with face-to-face clinical education) to traditional face-to-face clinical rotations. | 8 | 18 | 15 | 11 |
| The amount of work required for this course was appropriate. | 2 | 12 | 30 | 8 |
| As a result of this course, I feel more confident in communicating with children. | 1 | 3 | 30 | 18 |
| I would recommend this course to my fellow students. | 5 | 7 | 30 | 10 |
| I would sign up for another blended course like this in the future. | 4 | 11 | 27 | 10 |
| I was satisfied with the overall learning experience. | 3 | 5 | 31 | 13 |
Table 2.
Survey Responses of Third-Year Osteopathic Medical Students in a Blended Learning Group in a Pediatric Clinical Rotation (n=53)a
| Statement | Strongly Disagree | Disagree | Agree | Strongly Agree |
| Course Format |
| The integration of e-learning and face-to-face learning was convenient for me. | 2 | 6 | 24 | 20 |
| This was a practical learning experience. | 3 | 3 | 31 | 15 |
| The integration of e-learning and face-to-face learning helped me learn pediatrics topics. | 2 | 6 | 25 | 19 |
| Overall Technology |
| I have taken online or hybrid courses in the past. | 11 | 16 | 17 | 8 |
| I have used Blackboard in the past. | 0 | 0 | 20 | 32 |
| I found Blackboard easy to use in this course. | 2 | 2 | 25 | 23 |
| I found the online preceptor to be helpful when addressing technology-related issues or problems. | 0 | 0 | 14 | 37 |
| I did not experience any technical difficulties during the course. | 0 | 2 | 27 | 22 |
| Discussion Boards |
| Discussion boards improved my understanding of pediatric topics. | 4 | 12 | 29 | 7 |
| Discussion boards were valuable. | 5 | 9 | 30 | 8 |
| Discussions were relevant. | 3 | 2 | 32 | 15 |
| Discussions were facilitated well by the online preceptor. | 2 | 3 | 26 | 21 |
| Blogs |
| The introduction blog was valuable. | 7 | 13 | 25 | 7 |
| The reflection blog helped me review what I learned during the course. | 5 | 10 | 27 | 10 |
| CLIPP Cases |
| CLIPP cases were valuable. | 1 | 3 | 21 | 27 |
| CLIPP cases covered topics and clinical presentations that were not seen during my face-to-face clinical experience with patients at my training site. | 1 | 2 | 17 | 32 |
| Case Write-Upb |
| The case write-up was a valuable exercise. | 3 | 6 | 33 | 8 |
| Completing the case write-up helped me to learn a pediatric topic in detail. | 2 | 4 | 31 | 13 |
| Preparing the case write-up increased my comfort with medical writing. | 2 | 9 | 30 | 9 |
| Community Resource Summaryb |
| Preparing the community resource summary was a valuable learning experience. | 1 | 10 | 24 | 15 |
| Preparing the community resource summary helped me learn about a community resource available to families in Philadelphia. | 1 | 5 | 26 | 18 |
| Learning Objectives |
| I developed an understanding for communication with children and their families. | 1 | 0 | 23 | 28 |
| I developed basic skills in conducting newborn, pediatric, and adolescent physical examinations. | 1 | 0 | 23 | 28 |
| I can describe significant developmental milestones in childhood. | 0 | 4 | 27 | 21 |
| I can identify nutrition requirements for infants and children. | 0 | 4 | 29 | 19 |
| I recognize “normal values” for vital signs and anthropomorphic data. | 1 | 11 | 24 | 16 |
| I appreciate the importance of vaccination in promoting health and wellness. | 0 | 0 | 16 | 36 |
| I understand the influence of family, community, and society on the child in health and disease. | 0 | 1 | 14 | 37 |
| I understand important preventive and anticipatory guidance strategies for children. | 0 | 0 | 23 | 29 |
| I have started to understand common diseases in children (eg, asthma, anemia). | 0 | 3 | 20 | 29 |
| I can identify important clinical resources to help me care for children. | 1 | 2 | 31 | 18 |
| Overall Experience |
| I prefer this blended learning format (e-learning combined with face-to-face clinical education) to traditional face-to-face clinical rotations. | 8 | 18 | 15 | 11 |
| The amount of work required for this course was appropriate. | 2 | 12 | 30 | 8 |
| As a result of this course, I feel more confident in communicating with children. | 1 | 3 | 30 | 18 |
| I would recommend this course to my fellow students. | 5 | 7 | 30 | 10 |
| I would sign up for another blended course like this in the future. | 4 | 11 | 27 | 10 |
| I was satisfied with the overall learning experience. | 3 | 5 | 31 | 13 |
×
A total of 100 open-ended comments were provided by students from the blended learning group in the postsurvey. These responses were coded and divided into overlapping themes: supportive, constructive, and other. Students’ responsed often addressed a combination of themes. Sixty supportive statements addressed overall satisfaction with e-learning, course facilitation, clinical integration, reflection, menu-driven simulations, discussion boards, blogs, and active learning. In addition, 55 constructive statements addressed general dissatisfaction with e-learning, “busy work,” increased educational demands and accountability, overall preference for face-to-face learning, and challenges with video playback, discussion boards, and blog navigation. A total of 34 other statements largely addressed a desire to increase the amount of clinical exposure and face-to-face time with patients, and 26 comments were requests to replace discussion boards with blogs, add weekly case presentations, and provide more detailed student orientation for blended learning.
When asked to identify descriptors to represent their attitudes toward the course in a 15-adjective checklist (
Figure 1), the most frequent responses were positive (eg, useful, informative, interesting, engaging), and the least frequent responses were negative (eg, worthless, difficult to follow, confusing, unrealistic).
Student Performance Outcomes
Mean COMAT scores, reported as percent score, were 66.2 for all students, 66.4 for the standard learning group, and 65.6 for the blended learning group. Using the Levene test of equality of variance, the assumption of homogeneity of variance between groups was met (F=0.269, P=.604), permitting the use of a 2-tailed t test. Using a 2-tailed t test for analysis, no statistical differences were seen between standard learning and blended learning groups with regard to COMAT scores (P=.321).
All students passed the course for the 2014-2015 academic year (
Figure 2). A test for independence (χ
2 distribution) found that final grades between groups were substantially different. A greater proportion of students in the blended learning group (39 of 78 [50.0%]) received honors compared with students in the standard learning group (63 of 186 [33.9%]) (
P=.015).
Discussion
The results of the current study support findings from previous studies indicating that students are generally satisfied with online learning.
34,36,37 In the present study, a majority of the students indicated that the blended learning approach helped them learn the material. This approach is particularly important for clinical rotations, whereby students can combine authentic learning using real patients with online activities to augment face-to-face learning.
Consistent with another study investigating performance outcomes,
37 end-of-rotation examination scores were not different between students in the blended learning group compared with the standard learning group. However, students in the blended learning group had much higher final grades compared with the standard learning group. Higher grades among these students may be explained by (1) clinical preceptors unintentionally assigned higher grades to these students (presumably because these students learned the material better and performed better than other students), (2) clinical preceptors intentionally assigned higher grades to these students (presumably because these students “worked harder,” and extra effort for completing the online activities should be reflected in their final grade), or (3) clinical preceptors demonstrated a rater bias and were improperly using the final grade evaluation tool.
Rowe et al
25 suggested that blended learning may not necessarily result in higher grades but may address clinical competencies that are not routinely assessed by examinations. Perhaps we have exposed this gap between knowledge and performance, and blended learning indeed leads to better authentic learning and higher end-of-rotation grades. Although the results of the current study were statistically significant, additional research is warranted to investigate the underlying cause for this increase in higher end-of-rotation grades among those who participated in the blended learning group.
An unanticipated theme emerged from the current study: students valued educational online activities but clearly reported a continued desire to keep, or even increase, the amount of patient contact during clinical rotations. Many of the students in the blended learning group saw patients 3 or 4 days per week rather than the standard 5 days per week. As articulated by one student, “The blended learning is a nice idea, but it is not a replacement for patient encounters.” The clinical training sites provide medical students educational opportunities with patients in a real-world setting that cannot be replicated in the classroom. To our knowledge, this is the first study to report this finding with regard to blended learning. With so many technology advances and available digital tools, educators may be tempted to replace live clinical experiences with digital or simulated ones. However online learning should not replace face-to-face clinical teaching with patients, but rather augment those face-to-face educational opportunities. Studies such as the National Council of State Board of Nursing’s investigation of replacing clinical training hours with simulation2,3 are important to explore and identify the best balance of blending technology (whether it be in a simulated or online environment) with real patient experiences for medical education.
Most students reported spending 6 to 10 hours per week with online activities and reported that the amount of work for the course was appropriate. The course was designed to engage students for 8 to 10 hours per week with online course material, and these self-reported findings supported that goal. Although most students reported satisfaction with the amount of work, some students reported that their workload was too much and unrealistic. This dissatisfaction may have been due to varying expectations between different clinical training sites, a sense of unfairness because other students (ie, those in the standard learning group and students from previous years) were not required to complete online activities, or the expectations were indeed too high. Anecdotally, students appeared to access online content “off-hours” (ie, evenings, weekends, and school closure days due to snow emergencies). Future study is needed to provide additional insight into time spent completing online educational activities.
Course evaluation and improvement strategies were incorporated into the design of this educational program using action research.38 As a result of student feedback and findings from the current study, a number of course modifications and enhancements were made for subsequent academic years: the requirement to complete the 3- to 5-page case report was removed, the Web-based learning management system was reorganized and redesigned to improve ease of use, the amount of clinical time was increased to ensure a minimum of 4 days of clinical experience at all training sites, and the program was expanded to include all third-year students. These improvements will likely address some of the student concerns regarding time, fairness, consistency, course site navigation, and amount of face-to-face training with real patients in the clinical setting.
The current study has a number of limitations. First, findings may not be generalizable; the study is limited to 1 institution, 4 clinical training sites, and 1 discipline. Second, clinical training varied between sites with regard to site location, patient demographics, exposure to clinical pathology, number of patients seen, and amount of time spent with patients (4 sites required 3-4 days of clinical experience; 1 site required 5 days of clinical experience). This variation, especially with regard to the amount of time required for seeing patients, may account for some of the comments articulating frustrations with expectations for completing the required online activities. Third, students in the blended learning group were not aware of the online requirements before accepting the clinical assignment. Not all students may benefit from a blended learning environment, and these students were not provided the opportunity to select the best learning format for them. Fourth, unlike previous studies,26 the use of specific educational tools was not evaluated; rather, the current study addressed the overall pedagogic approach of blended learning, combining a number of tools to engage students. Fifth, learning analytics were not evaluated in this study; therefore, time spent with online activities could not be verified and could not be linked with outcome measures. Sixth, feedback was not collected from participating clinical preceptors; this feedback would be important for future research regarding blended and online learning initiatives.
Conclusion
The findings of the current study support the use of blended learning in a clinical training environment. Students valued the blended learning approach, and although their end-of-rotation examination scores were not improved, they may have benefited by receiving higher course grades. As more medical educators use blended learning, the best balance between learning with technology and learning in a face-to-face setting must be investigated. Online activities may enhance but should never fully replace face-to-face learning with real patients.
Acknowledgments
We thank each of the student participants; the faculty for their dedication and enthusiasm to explore new educational strategies and technologies; Marcus Bell, PhD, for his assistance with statistical analysis; and Christine Black-Langenau, DO, and David Teter for their critical reviews of the manuscript.
References
American Association of Colleges of Nursing, the American Association of Colleges of Osteopathic Medicine, the Physician Assistant Education Association, and the Association of American Medical Colleges.
Recruiting and Maintaining U.S. Clinical Training Sites: Joint Report of the 2013 Multi-discipline Clerkship/Clinical Training Site Survey. 2014.
https://members.aamc.org/eweb/upload/13-225%20WC%20Report%202%20update.pdf. Accessed February 17, 2017.
Hayden J, Keegan M, Kardong-Edgren S, Smiley RA. Reliability and validity testing of the Creighton Competency Evaluation Instrument for use in the NCSBN National Simulation Study.
Nurs Educ Perspect. 2014;
35(4):244-252.
[CrossRef] [PubMed]Hayden JK, Smiley RA, Alexander M, Kardong-Edgren S, Jeffries PR. The NCSBN National Simulation Study: a longitudinal, randomized, controlled study replacing clinical hours with simulation in prelicensure nursing education. J Nurse Reg. 2014;5(2 suppl):S1-S64.
Conway-Klaassen JM, Brennecke PJ, Wiesner SM, Spannaus-Martin DJ. Development of online conferencing and web-based in-service modules for preceptor training. Clin Lab Sci. 2012;25(4 suppl): 4-26-33.
Kurlandsky LE, Potts M, Kumar A. Pediatric clerkship performance in diverse community clinical settings.
Pediatrics. 1994;
93(4):608-610.
[PubMed]McCurdy FA, Beck GL, Kollath JP, Harper JL. Pediatric clerkship experience and performance in the Nebraska Education Consortium: a community vs university comparison.
Arch Pediatr Adolesc Med. 1999;
153(9):989-994.
[CrossRef] [PubMed]Masters PA, Nester C. A study of primary care teaching comparing academic and community-based settings.
J Gen Intern Med. 2001;
16(1):9-13.
[CrossRef] [PubMed]Gordon JA, Pawlowski J. Education on-demand: the development of a simulator-based medical education service.
Acad Med. 2002;
77(7):751-752.
[CrossRef] [PubMed]Choules AP. The use of elearning in medical education: a review of the current situation.
Postgrad Med J. 2007;
83(978):212-216.
[CrossRef] [PubMed]Peska DN, Lewis KO. Uniform instruction using web-based, asynchronous technology in a geographically distributed clinical clerkship: analysis of osteopathic medical student participation and satisfaction.
J Am Osteopath Assoc. 2010;
110(3):135-142.
[PubMed]Langenau E, Kachur E, Horber D. Web-based objective structured clinical examination with remote standardized patients and Skype: resident experience.
Patient Educ Couns. 2014;
96(1):55-62. doi:10.1016/j.pec.2014.04.016
[CrossRef] [PubMed]Markova T, Roth LM. E-conferencing for delivery of residency didactics.
Acad Med. 2002;
77(7):748-749.
[CrossRef] [PubMed]Lang VJ, Kogan J, Berman N, Torre D. The evolving role of online virtual patients in internal medicine clerkship education nationally.
Acad Med. 2013;
88(11):1713-1718. doi:10.1097/ACM.0b013e3182a7f28f
[CrossRef] [PubMed]Guzic BL, McIlhenny CV, Knee DR, et al Distance learning and clinical simulation in senior baccalaureate nursing education.
Clin Simulation Nurs. 2012;
8(9):e459-e467.
[CrossRef] Prober CG, Khan S. Medical education reimagined: a call to action.
Acad Med. 2013;
88(10):1407-1410. doi:10.1097/ACM.0b013e3182a368bd
[CrossRef] [PubMed]Dyrbye L, Cumyn A, Day H, Heflin M. A qualitative study of physicians’ experiences with online learning in a masters degree program: benefits, challenges, and proposed solutions. Med Teach. 2009;31(2):e40-e46. doi:10.1080/01421590802366129
Sandars J, Langlois M, Waterman H. Online collaborative learning for healthcare continuing professional development: a cross-case analysis of three case studies.
Med Teach. 2007;
29(1):e9-e17.
[CrossRef] [PubMed]Linjawi AI, Walmsley AD, Hill KB. Online discussion boards in dental education: potential and challenges.
Eur J Dent Educ. 2012;
16(1):e3-e9. doi:10.1111/j.1600-0579.2010.00662.x
[CrossRef] [PubMed]Brahler CJ, Quitadamo IJ, Johnson EC. Student critical thinking is enhanced by developing exercise prescriptions using online learning modules.
Adv Physiol Educ. 2002;
26(1-4):210-221.
[CrossRef] [PubMed]Christianson L, Tiene D, Luft P. Web-based teaching in undergraduate nursing programs.
Nurse Educ. 2002:
27(6):276-282.
[CrossRef] [PubMed]Hoegerl C, John S. E-learning in medical education.
J Am Osteopath Assoc. 2010;
110(3):193-194.
[PubMed]Lapidus M, McCord SK, McCloskey WW, Kostka-Rokosz MD. Combined use of online tutorials and hands-on group exercises in bibliographic instruction for pharmacy students.
Med Ref Serv Q. 2012;
31(4):383-899. doi:10.1080/02763869.2012.724277
[CrossRef] [PubMed]Margolis LM, Grediagin A, Koenig C, Sanders LF. Effectiveness and acceptance of web-based learning compared to traditional face-to-face learning for performance nutrition education.
Mil Med. 2009;
174(10):1095-1099.
[CrossRef] [PubMed]McCutcheon K, Lohan M, Traynor M, Martin D. A systematic review evaluating the impact of online or blended learning vs. face-to-face learning of clinical skills in undergraduate nurse education.
J Adv Nurs. 2015;
71(2):255-270. doi:10.1111/jan.12509
[CrossRef] [PubMed]Rowe M, Frantz J, Bozalek V. The role of blended learning in the clinical education of healthcare students: a systematic review.
Med Teach. 2012;
34(4):e216-e221. doi:10.3109/0142159X.2012.642831
[CrossRef] [PubMed]Cook DA, Levinson AJ, Garside S, Dupras DM, Erwin PJ, Montori VM. Instructional design variations in internet-based learning for health professions education: a systematic review and meta-analysis.
Acad Med. 2010;
85(5):909-922. doi:10.1097/ACM.0b013e3181d6c319
[CrossRef] [PubMed]Garrison DR, Kanuka H. Blended learning: uncovering its transformative potential in higher education.
Internet Higher Educ. 2004;
7(2):95-105.
[CrossRef] Barrows HS. Problem-based learning in medicine and beyond: a brief overview.
N Dir Teaching Learning. 1996;
68:3-12.
[CrossRef] Harasim L. Learning Theory and Online Technologies. New York, NY: Routledge; 2012.
Rudestam KE, Schoenholtz-Read J. Handbook of Online Learning. 2nd ed. Thousand Oaks, CA: SAGE Publications; 2010.
Thomas SJ. Designing Surveys That Work! A Step-by-Step Guide. Thousand Oaks, CA: Corwin; 1999.
Gormley GJ, Collins K, Boohan M, Bickle IC, Stevenson M. Is there a place for e-learning in clinical skills? a survey of undergraduate medical students’ experiences and attitudes.
Med Teach. 2009;
31(1):e6-e12. doi:10.1080/01421590802334317
[CrossRef] [PubMed]Khasawneh R, Simonsen K, Snowden J, Higgins J, Beck G. The effectiveness of e-learning in pediatric medical student education. Med Educ Online. 2016;21:29516. doi:10.3402/meo.v21.29516
Mills GE. Action Research: A Guide for the Teacher Researcher. 4th ed. Boston, MA: Pearson; 2010.