Editorial  |   May 2014
Wikipedia: Proceed With Caution
Author Notes
  • Address correspondence to Lori Fitterling, MLS, Digital Services/Reference Librarian, D'Angelo Library, Kansas City University of Medicine and Biosciences, 1750 Independence Ave, Kansas City, MO 64106-1453. E-mail:  
Article Information
Evidence-Based Medicine / Medical Education / Neuromusculoskeletal Disorders / Professional Issues / Graduate Medical Education
Editorial   |   May 2014
Wikipedia: Proceed With Caution
The Journal of the American Osteopathic Association, May 2014, Vol. 114, 334-335. doi:
The Journal of the American Osteopathic Association, May 2014, Vol. 114, 334-335. doi:
Previous studies1-8 have evaluated the usage rates and contents of the Internet resource Wikipedia. From a medical librarian's perspective, this subject demands not only judicious research, but also continual critical review. In the current issue of The Journal of the American Osteopathic Association, Hasty et al9 present a randomized, blinded study that singles out the 10 most costly medical conditions and compares articles from Wikipedia vs articles in other peer-reviewed sources. The findings of their study demonstrate that assertions (facts) presented in Wikipedia for these medical conditions were mostly in discordance with peer-reviewed literature. These results cast serious doubt on Wikipedia's authority as a medical reference repository. Overall, the study adds credence to the message librarians have heralded: medical professionals should be educated about and engaged in the critical analysis of online information. In other words, information literacy should provide a basis for evidence-based practice. 
The basis of the inquiry of Hasty et al9 challenges the quality of content found in Wikipedia and builds on previous studies3,4 that have shown widespread use by physicians and medical students of Web 2.0 tools, including Wikipedia, to find information for patient care. Hasty et al9 do a very good job of reviewing the medical literature on Wikipedia use and underscoring concerns about its reliability in light of its open-editing and revision policies. Two strong elements of their study are (1) the focus on the costliest medical conditions and (2) the hypothesis that discrepancies between Wikipedia articles and peer-reviewed resources on these conditions call Wikipedia's credibility as a medical reference into question. Further, the randomized study design used 2 independent reviewers for each topic and then enlisted 2 additional reviewers to conduct a meta-analysis of the results. By providing term definitions and scoring using the McNemar test for concordance, the researchers could maintain objectivity. This way of measuring the accuracy of Wikipedia articles yielded compelling results: of the 10 articles selected, 9 showed statistically significant discordance. 
Despite the study's strengths, a minor obstacle in the methodology should be noted. Hasty et al9 point out that a reporting bias by the reviewers may have caused a discrepancy in dissimilar assertions that failed to show discordance. What 1 reviewer counted as an assertion, another might not have, and thus this was purely left to the subjectivity of the reviewer. Different results might have been found—and a greater or lesser number of assertions or levels of concordance or discordance might have occurred—if expert clinicians were used to review medical topics associated with their specialty. As it was, all reviewers were internal medicine residents or rotating interns; although this demographic allowed for redundancy, their specialization and limited experience may have also influenced the results. Thus the validity of the reviewer assertions might be questioned. 
Also, the legitimacy the individual, peer-reviewed articles used in the study is in question. Although the databases used to search for peer-reviewed articles were identified (UpToDate, PubMed, and Google Scholar), the articles retrieved from these databases were not verified by the 2 independent reviewers, who were tasked only with checking assertions in the Wikipedia articles. Both PubMed and Google Scholar contain non–peer-reviewed articles, and there is a chance that an article that was used to discredit an assertion was not peer-reviewed. Hasty et al9 acknowledge that the use of “any” peer-reviewed source was a limitation and that checking the peer-reviewed articles with the assertions would be beneficial for a future study. 
Some reviews of Wikipedia articles have criticized its open-editing policies, as well as the writing and presentation of the topics themselves.1,6,10 Because some of the articles are poorly written and hard to understand, the identification of assertions might be difficult. As previously noted, the validity of reviewer assertions might be questioned. 
Overall, Hasty et al9 rightly acknowledge content problems with Wikipedia, but they skirt other issues such as why medical professionals choose Wikipedia over peer-reviewed resources. Whether it is because of subscription costs, convenience, popularity, or ease of use, the researchers do not say. Further, medical students and physicians may lack information literacy skills and may not know how to find information, research medical topics, understand evidence-based clinical reviews, or evaluate information resources. If physicians are lacking these skills, then these subjects should all be part of the curricula in medical informatics classes and continuing medical education. Librarians are uniquely positioned to bring this knowledge forward. Wikipedia has a place in literature searching, but it is best used as a starting point rather than an ending point. Wikipedia is attempting to upgrade its health content by offering a voluntary peer-review process.10,11 If physicians can make a commitment to edit Wikipedia content, then Wikipedia might have a chance at becoming a more reliable, peer-edited clinical decision tool. 
As Hasty et al9 write, “Although the effect of Wikipedia's information on medical decision making is unclear, it almost certainly has an influence.” The problem is that no study can directly tie the use of Wikipedia with a physician's patient care. Whichever resource is consulted, it is the physician's knowledge and application of the information that would determine patient outcomes. 
Nonetheless, the study by Hasty et al9 is a wonderful eye-opener for the medical community because it presents compelling evidence that assertions from collaborative, community-edited media lack the quality of content that a peer-reviewed resource provides. It is a call to improve the awareness of quality medical decision-making resources. 
Would I want my physician to consult Wikipedia about my condition? No. Physicians and medical students, spend your time consulting a credible, peer-reviewed, evidence-based resource. And if you do not know how to do this, let your reference librarian teach you. 
   Financial Disclosures: None reported.
   Support: None reported.
Bould M, Hladkowicz E, Pigford AAet al. References that anyone can edit: review of Wikipedia citations in peer reviewed health science literature. BMJ. 2014;348:g1585. doi:10.1136/bmj.g1585. [CrossRef] [PubMed]
Grajales FJIII, Sheps S, Ho K, Novak-Lauscher H, Eysenbach G. Social media: a review and tutorial of applications in medicine and health care. J Med Internet Res. 2014;16(2):e13. doi:10.2196/jmir.2912. [CrossRef] [PubMed]
Hughes B, Joshi I, Lemonde H, Wareham J. Junior physician's use of Web 2.0 for information seeking and medical education: a qualitative study [published online June 5, 2009]. Int J Med Inform. 2009;78(10):645-655. doi:10.1016/j.ijmedinf.2009.04.008. [CrossRef] [PubMed]
Judd T, Kennedy G. Expediency-based practice? medical students' reliance on Google and Wikipedia for biomedical inquiries. Br J Educ Technol. 2011;42(2):351-360. doi:10.1111/j.1467-8535.2009.01019.x. [CrossRef]
Kupferberg N, Protus BM. Accuracy and completeness of drug information in Wikipedia: an assessment. J Med Libr Assoc. 2011;99(4):310-313. doi:10.3163/1536-5050.99.4.010. [CrossRef] [PubMed]
Thomas GR, Eng L, de Wolff JF, Grover SC. An evaluation of Wikipedia as a resource for patient education in nephrology [published online February 22, 2013]. Semin Dial. 2013;26(2):159-163. doi:10.1111/sdi.12059. [CrossRef] [PubMed]
Yasseri T, Sumi R, Rung A, Kornai A, Kertész J. Dynamics of conflicts in Wikipedia [published online June 20, 2012]. PLoS One. 2012;7(6):1-12. doi:10.1371/journal. pone.0038869. [CrossRef]
Younger P. Beyond Wikipedia: how good a reference source are medical wikis? Ref Rev. 2010;24(1):7-9. doi:10.1108/09504121011019899. [CrossRef]
Hasty RT, Garbalosa RC, Barbato VAet al. Wikipedia vs peer-reviewed medical literature for information about the 10 most costly medical conditions. J Am Osteopath Assoc. 2014;114(5):368-373. [CrossRef] [PubMed]
Wikipedia seeks to improve its content quality. PT In Motion. 2013;5(2):10.
Heilman JM, Kemmann E, Bonert Met al. Wikipedia: a key tool for global public health promotion. J Med Internet Res. 2011;13(1):e14. doi:10.2196/jmir.1589. [CrossRef] [PubMed]