Abstract
Many medical students have difficulty learning basic science, either because they find the material challenging to comprehend or because they believe it has limited clinical application. Computer-assisted instruction (CAI)—ie, computer animation—can clarify instruction by allowing students to visualize complex, dynamic processes in an interesting presentation. At West Virginia School of Osteopathic Medicine (WVSOM) in Lewisburg, a series of computer animations have been developed to present concepts in molecular and cellular biology. The author conducted an investigation to compare the efficacy of one representative computer animation with that of traditional textbook material. The subjects were 22 students who had been admitted to WVSOM but who had not yet begun classes. The experimental design of the study consisted of a prelesson test, a lesson, and a postlesson test. The lesson explained the process of deoxyribonucleic acid (DNA) replication using either a computer animation (n=12) or a chapter from a textbook (n=10). Lesson comprehension as measured by the tests was significantly higher for subjects who used the computer animation than for subjects who used the textbook (P<.01). Furthermore, reviewing the text after studying with the computer animation did not raise test scores, suggesting that the animation was sufficient for learning and the text was unnecessary. After the study, a majority of subjects indicated a preference for the animation over the text. These results demonstrate that CAI can be an effective tool for relating basic science to medical students by improving comprehension and eliciting interest in the lessons.
Medical students often have difficulty learning basic science. This is partially because some of the material is conceptually challenging, but also because many students are so eager to master clinical applications that they lack motivation to study fundamental science material. The advent of computer-assisted instruction (CAI) offers educators a means for addressing this problem. Computers can be used to generate detailed animations depicting molecules and cells interacting in three-dimensional space. These animations have the potential to make it easier for students to understand difficult science concepts. They allow students to watch dynamic biomolecular processes unfold step by step, rather than forcing them to rely solely on their imaginations. Computer animations can also help to hold students' attention by providing visually stimulating presentations.
Numerous studies have analyzed the efficacy of CAI by comparing the performance of students using these programs with that of students using traditional educational methods, such as lectures or textbooks. Although results vary, most of these studies
1–7 report test performances for CAI that are equal or superior to those obtained with traditional teaching methods. Students exposed to CAI usually consider it to be a useful learning tool.
6–8 Students often prefer access to both CAI and traditional methods,
3,9 and study time is usually shorter with CAI.
2,10,11 Such reports suggest that the use of CAI can be an effective instructional tool.
At West Virginia School of Osteopathic Medicine (WVSOM) in Lewisburg, a series of CAI animations has been developed to teach biology at the molecular and cellular levels (
Figure 1). These animations consist of moving images displaying subcellular processes, such as transcription factors binding recognition sequences, nucleosomes coiling into solenoids, transport protein action, fluid mosaic diffusion, and primitive streak elongation. Student evaluations regarding these animations have been mostly favorable, demonstrating that the students appreciate the computerized presentations. However, there has been no empirical evidence that the WVSOM animations actually enhance learning. Therefore, to verify that the computer animations are effective tools for relating basic science to medical students, an investigation was conducted comparing the efficacy of one representative animation—presenting the topic of deoxyribonucleic acid (DNA) replication—with the efficacy of textbook material presenting the same topic.
Methods
The institutional review board at WVSOM approved all procedures for the present study. The subjects were 22 osteopathic medical students from the classes of 2007 and 2008 at WVSOM who had not yet begun classes. When a candidate paid his or her retainer fee for admission to the school, the candidate was mailed an invitation to volunteer for the study. Personnel in the admissions office mediated all mailings, and the only identifier seen by the author was an arbitrary number assigned to each subject. There were two main purposes for this blinded experimental design: (1) to guarantee that there was no inadvertent coercion of students to participate, and (2) to discourage subjects from returning responses that they thought the principle investigator wanted to hear.
The invitation mailing included a cover letter with a basic explanation of the study, an informed consent form, a questionnaire, and a 20-question prelesson test. After a subject returned these materials, he or she was assigned to either the computer group (n=12) or the textbook group (n=10). Three criteria were evaluated to maintain equivalent expertise levels between the groups:
the subjects' prelesson test scores;
the subjects' estimation of their computer skills (ranked on a scale of 1 to 5); and
the subjects' estimation of their understanding of DNA replication (ranked on a scale of 1 to 5).
After the subjects were assigned to their study groups, a packet was mailed to each volunteer containing instructions, a log to record their study time, and a series of six envelopes marked “step 1” through “step 6.” Subjects were instructed to open each envelope sequentially.
For the computer group, the step 1 envelope contained a CD-ROM featuring a computer animation explaining the topic of DNA replication. This animation was composed with the Corel Presentations slide-show program (Corel Presentations 10; Corel Corporation, Ottawa, Ontario). The animation began by presenting a list of seven characteristics of DNA replication, then it graphically showed the process of replication step by step. The presentation ended with a quiz reviewing key terms.
For the textbook group, the step 1 envelope contained the printed text from chapter 12 of
Basic Medical Biochemistry: A Clinical Approach.
12 This textbook is used at WVSOM to teach lessons in DNA replication. The last seven pages of the chapter cover two topics not presented by the computer animation—DNA repair and recombination—so only the first seven pages were included in the study packet.
The step 2 envelope contained a 20-question postlesson test, which was identical for both the computer group and the textbook group. The step 3 envelope contained material for a reciprocal review: the computer group received the material from the textbook, while the textbook group received the CD-ROM. The step 4 envelope contained a 20-question final test, which was identical for both groups. The 20 guestions on the prelesson test, postlesson test, and final test were different for each test.
The step 5 envelope contained a questionnaire asking subjects to indicate which media they preferred (text, computer, both, or no preference) and to rank how well the animation and text aided their understanding of the subject matter. The questionnaire also asked for written responses to three subjective questions:
What did you like most about the animation?
What did you dislike about the animation?
How did the animation compare with the text as a learning tool?
The step 6 envelope contained a self-addressed envelope for returning all the completed materials.
For statistical analyses of the test and questionnaire responses, means were compared with 2-tailed t tests using a spreadsheet program (Microsoft Excel 2000; Microsoft Corporation, Redmond, Wash). To determine reliability, a test management program (LXR 6.0; Applied Measurement Professionals Inc, Alta Loma, Calif) was used to compute Cronbach's α coefficients. In an effort to quantitatively analyze the responses to the three subjective questions on the questionnaire, descriptive phrases from each response were counted. For example, the following response was credited with six phrases:
I found that I had to read the text material a couple of times for everything to sink in. I watched the computer animation once and felt that I had a clear understanding of the material. Basically, I feel that the computer animation saves time, and it is a fun, easy way to learn!
The phrases credited to this response were couple of times, clear, understanding, saves time, fun, and easy.
Results
The results of this study verify that students in both the computer group and the textbook group began with equivalent levels of expertise. As shown in
Figure 2, there was no significant difference between group means for the prelesson test scores (computer group, 51.7%; textbook group, 48.5%;
P>.6). Nor were there significant differences between group means in the self-estimation ratings of computer skills (computer group, 3.9; textbook group, 3.6;
P>.05) or the understanding of the topic (computer group, 2.8; textbook group, 2.9;
P>.8).
The computer group's postlesson test results indicate that the tests were reliable (Cronbach's α coefficient=0.83). However, the variability of the prelesson test results for both the computer and textbook groups was too great to assess reliability. The variability of the textbook group's postlesson test results and both groups' final test results was too low to assess reliability.
As shown in
Figure 2, the mean test score of students in the computer group was 22.3% higher on the postlesson test than the mean test score of students in the textbook group (computer group, 96.3%; textbook group, 74%;
P<.01). This difference demonstrates that comprehension was markedly higher for students who used the computer animation than for students who used the material from the textbook.
After completing the postlesson test, students in each group conducted a reciprocal review using the opposite medium. The computer group read the text, while the textbook group studied with the computer animation. The reciprocal review was conducted to give students the opportunity to compare the two media, and to see if reviewing both media was any better than using either media alone. The reciprocal review was followed by a final test.
The results of the final test must be interpreted with caution because there was no control group to determine if any improvement in scores was produced by the repeated lessons or by repeated testing. However, as shown in
Figure 2, it is relevant that the computer group's mean score actually decreased slightly, but significantly, from the postlesson test to the final test (postlesson test, 96.3%; final test, 90.8%;
P<.05), and that there was no significant difference between the computer group's postlesson test mean score and the textbook group's final test mean score (computer group, 96.3%; textbook group, 97%;
P>.6). If reviewing both media does not elevate comprehension above the level achieved by studying the computer animation alone, then it is reasonable to conclude that the animation is sufficient and the text is unnecessary.
Figure 3 indicates that for both the computer and textbook group, most students ranked the computer animation higher than the textbook chapter on a preference scale of 1 (not prefer) to 5 (greatly prefer). For the computer group, the mean preference rating of the animation over the text was significant (computer animation, 4.9; textbook chapter, 3.5;
P<.001). The textbook group also ranked the computer animation higher, but the difference in preference rating was not significant (computer animation, 4.3; textbook chapter, 3.8;
P>.3). These results demonstrate that the sequence of use of the computer animation and textbook chapter influenced preference for the animation. Students reacted more favorably toward the computer animation if they used it before they used the textbook chapter.
To further analyze preference, we asked students, both before and after conducting the study, to indicate which media they preferred. Experience with the computer animation altered their opinions on this question. Before the study, no students gave a preference for using the computer animation alone, but after the study, a small majority of subjects (12 [54.6%]) said they preferred using the animation alone (
Figure 4). Interestingly, the sequence of use affected opinions. At the end of the study, most of the computer group subjects (8 [66.7%]) preferred computers alone, while most of the textbook group subjects (6 [60%]) preferred access to both media. This observation suggests that students were less accepting of the text if they read it after using the animation.
Yet another method for evaluating preference was to count the critical or complimentary phrases in the written responses to the three subjective questions. For the question that asked subjects what they liked most about the computer animation, 100 complimentary phrases for the animation and nine criticisms of the text were counted. For the question that asked students what they disliked about the animation, only 14 (63.6%) students responded. Each of these students pointed to specific properties of the animation, such as certain graphics that moved too quickly. For the question that asked students to compare the animation with the textbook chapter as a learning tool, all responses favored the animation. Fifty-five phrases that complimented the animation were counted in the responses to this question. There were only three phrases critical of the animation, while there were 12 criticisms of the text. Six responses noted certain advantages of the text, but even these conceded the superiority of the animation. This analysis demonstrates that most of the commentary favored the animation.
Many of the phrases used by students to praise the computer animation consisted of emphatically positive terms. These included alert, come to life, eager to learn, enabled, fun, more please!, superior, and vivid. In comparison, most of the terms used to describe the textbook chapter were emphatically negative, including boring, confusing, dry, inferior, intimidating, relieved when finished, stagnant, and yawn reflex. The preponderance and fervency of the praise for the animation suggests that many students were enthusiastic about it, while the paucity of praise and the fervency of criticism for the text suggests that students were antagonistic toward it.
As shown in
Figure 5, when the average amount of time needed to complete a lesson was analyzed, no significant differences were observed in either group in study time for the computer animation versus study time for the textbook chapter (
P>.8). The computer group completed the animation lesson in an average of 50 minutes and the textbook lesson in an average of 44.6 minutes; the textbook group completed the animation lesson in an average of 39.7 minutes and the textbook lesson in an average of 43.1 minutes. In addition, there were no significant differences in the number of breaks taken or in the length of each break for the animation versus the textbook lesson (data on file). These results demonstrate that, contrary to what might be expected, the animation did not shorten study time when compared with the textbook chapter.
Comment
The present investigation reveals that the WVSOM computer animation on DNA replication is a more effective learning tool for medical students than the corresponding material in Basic Medical Biochemistry. Students learned the topic more thoroughly using the animation. In addition, this experience increased their preference for computer-assisted instruction. These results support the use of CAI as an effective tool for teaching basic science concepts to medical students.
The evidence from this study indicates not only that the computer animation was more effective at building students' comprehension, but also that the textbook was unnecessary. After completing their reciprocal reviews, neither the computer group nor the textbook group scored significantly higher than the computer group scored after it had used the animation alone. Therefore, if reviewing the textbook chapter does not enhance comprehension, it is reasonable to conclude that the animation is sufficient for learning, and reading the textbook chapter constitutes an ineffectual use of study time.
Consistent with previous reports,
13,14 the present investigation found that experience with computer animation altered preferences in favor of computers. However, almost half the students in the present study continued to desire traditional textbook material in conjunction with computer animation. This finding is consistent with previous reports in which subjects were reluctant to abandon traditional teaching methods.
4,5,15 These observations illustrate that interest in CAI does not preclude student appreciation of traditional methods of instruction.
A correlation between sequence of media use and preference of media was evident in the present study. Students who studied the computer animation before reading the textbook chapter reacted more favorably toward the animation and more negatively toward the text. This finding is consistent with previously reported results.
3 A possible explanation for this correlation is that the animation explained the topic so well that reviewing the textbook chapter seemed unnecessary or even counterproductive. For students who began studying with the textbook chapter, the animation may have seemed complementary because it clarified certain points of confusion. If this explanation is correct, then reading text before viewing a computer animation may promote appreciation for the text.
The written responses in the questionnaires demonstrate that most students found the computer animation to be entertaining and interesting. Thirteen responses included the phrases enjoyable, entertaining, fun, held attention, or interesting. The entertainment value probably contributes to the effectiveness of the animation, because it serves to hold students' attention. The more entertaining a presentation is, the easier it is to concentrate on the lesson and the more information can be assimilated for longer periods of time.
The computer animation and textbook chapter required roughly equivalent study times. This finding was unexpected because most previous reports have noted decreased study time with CAI.
2,10,11 However, computer study time that is equal to or longer than text study time has previously been observed with specific CAI applications.
1,9 A possible explanation for the present study's observed time equivalence is that, although information could probably be assimilated more rapidly with the animation than with the text, the animation was formatted to include steps that the text did not have. The animation began with a preview of seven characteristics of DNA replication and then presented the replication process in a step-by-step manner. The text, on the other hand, introduced the characteristics and described the replication process simultaneously. In addition, the animation ended with a practice quiz reviewing key terms—a step that the text lacked. This difference in formatting may have contributed to the rough equivalence in study times between the computer animation and textbook chapter.
Numerous responses indicate that students appreciated the preview and quiz portions of the computer animation, and that each of these elements probably contributed to the effectiveness of the animation. To verify that the efficacy of the animation resulted from properties specific to the computerized presentation and not merely from superior formatting, a future investigation should study textbook material that has formatting similar to that of the computer animation.
Conclusion
This preliminary investigation addressed a narrowly defined question: How does the WVSOM computer animation on DNA replication compare with a specific textbook chapter on the same subject, when short-term memory is tested after homework study by first-year medical students? The results verify that this animation is an effective educational tool.
Despite the findings of this study, further investigation is required to evaluate the general applicability of the WVSOM computer animation. Remaining questions to address include the following:
How would the animation compare with other textbooks?
How would the animation compare with lectures or with other CAI programs?
What is the best strategy for incorporating animations into coursework?
Should animations be provided as supplementary material, or should they be used to replace textbooks or lectures?
The ultimate objective of this line of research will be to determine how best to prepare computer animations on science and medical topics for course use in medical schools.
The author acknowledges the invaluable consultation provided by Helen H. Baker, PhD, both in the planning stages of this project and in her critiques of the manuscript. Robert M. Fisk, PhD; Hanno Kirk, PhD; Kelly M. Jackson, PhD; and James P. Wells, PhD, also critiqued the manuscript. As chairman of the West Virginia School of Osteopathic Medicine's (WVSOM) institutional review board, David E. Brown, PhD, not only helped to obtain approval for this project, he also provided valuable advice in the planning stage. William T. Blue, PhD, approved financial support for the project from the Structural Biology Division at WVSOM. Carol C. Berger, under the supervision of Donna S. Varney in the admissions office at WVSOM, mediated all mailings between the subjects and the author. Michael Kasey, DO, a senior student at WVSOM at the time, previewed all the materials to verify that the tests did not inadvertently favor the animation over the text. Elyssa Rutledge, a first-year student at WVSOM at the time, helped to prepare mailings.
Chew FS, Stiles RS. Joseph E. Whitley, MD, Award. Computer-assisted instruction with interactive videodisc versus textbook for teaching radiology. Acad Radiol. 1994;1:326-331.
Shomaker TS, Ricks DJ, Hale DC. A prospective, randomized controlled study of computer-assisted learning in parasitology. Acad Med. 2002;77:446-449.
Froman RD, Hence C, Neafsey PJ. A comparative assessment interactive videodisc instruction. Comput Nurs. 1993;11:236-241.
Seabra D, Srougi M, Baptista R, Nesrallah LJ, Ortiz V, Sigulem D. Computer aided learning versus standard lecture for undergraduate education in urology. J Urol. 2004;171:1220-1222.
Bissell V, McKerlie RA, Kinane DF, McHugh S. Teaching periodontal pocket charting to dental students: a comparison of computer assisted learning and traditional tutorials. Br Dent J. 2003;195:333-336.
Maag M. The effectiveness of an interactive multimedia learning tool on nursing students' math knowledge and self-efficacy. Comput Inform Nurs. 2004;22:26-33.
King CR. Innovations in human genetics education. Alternative methods of instruction in medical genetics. Am J Hum Genet. 1989;45:182-184.
Chew FS, Smirniotopoulos JG. Teaching skeletal radiology with use of computer-assisted instruction with interactive videodisc. J Bone Joint Surg Am. 1995;77:1080-1086.
Day R, Payne L. Computer-managed instruction: an alternative teaching strategy. J Nurs Educ. 1987;26:30-36.
Effken JA, Kim NG, Shaw RE. Making the relationships visible: testing alternative display design strategies for teaching principles of hemodynamic monitoring and treatment. Proc Annu Symp Comput Appl Med Care. 1994:949-953.
Mars M, McLean M. Students' perceptions of a multimedia computer-aided instruction resource in histology. S Afr Med J. 1996;86:1098-1102.
Marks DB, Marks AD, Smith CM. Basic Medical Biochemistry: A Clinical Approach. Baltimore, Md: Lippincott Williams & Wilkins; 1996.
Langer I, Schewe S, Haedecke C, Puppe F, Rheinhardt T. Learning at the computer: evaluation of an intelligent tutoring system. Eur J Med Res. 1998;3:119-126.
Porter RS. Efficacy of computer-assisted instruction in the continuing education of paramedics. Ann Emerg Med. 1991;20:380-384.
Herriot AM, Bishop JA, Kelly M, Murphy M, Turby H. Evaluation of computer assisted instruction resource in nursing education. Nurse Educ Today. 2003;23:537-545.