Abstract
Context: Since its launch in 2001, Wikipedia has become the most popular general reference site on the Internet and a popular source of health care information. To evaluate the accuracy of this resource, the authors compared Wikipedia articles on the most costly medical conditions with standard, evidence-based, peer-reviewed sources.
Methods: The top 10 most costly conditions in terms of public and private expenditure in the United States were identified, and a Wikipedia article corresponding to each topic was chosen. In a blinded process, 2 randomly assigned investigators independently reviewed each article and identified all assertions (ie, implication or statement of fact) made in it. The reviewer then conducted a literature search to determine whether each assertion was supported by evidence. The assertions found by each reviewer were compared and analyzed to determine whether assertions made by Wikipedia for these conditions were supported by peer-reviewed sources.
Results: For commonly identified assertions, there was statistically significant discordance between 9 of the 10 selected Wikipedia articles (coronary artery disease, lung cancer, major depressive disorder, osteoarthritis, chronic obstructive pulmonary disease, hypertension, diabetes mellitus, back pain, and hyperlipidemia) and their corresponding peer-reviewed sources (P<.05) and for all assertions made by Wikipedia for these medical conditions (P<.05 for all 9).
Conclusion: Most Wikipedia articles representing the 10 most costly medical conditions in the United States contain many errors when checked against standard peer-reviewed sources. Caution should be used when using Wikipedia to answer questions regarding patient care.
Since its 2001 launch, Wikipedia (
http://www.wikipedia.org/) has become the most popular general reference site on the Internet, ranking 6th globally based on Internet traffic.
1 As of March 2014, it contained more than 31 million articles in 285 languages.
2 Wikipedia's prominence has been made possible by its fundamental design as a
wiki, or
collaborative database, allowing all users the ability to add, delete, and edit information at will. However, it is this very feature that has raised concern in the medical community regarding the reliability of the information it contains.
Despite these concerns, Wikipedia has become a popular source of health care information,
3 with 47% to 70% of physicians and medical students admitting to using it as a reference.
4-6 In actuality, these figures may be higher because some researchers suspect its use is underreported.
7 Although the effect of Wikipedia's information on medical decision making is unclear, it almost certainly has an influence.
Wikipedia has several mechanisms in place to deal with unverifiable information and vandalism.
8 Because of the frequency of editing and revisions, most instances of vandalism only exist for a few days after being identified, with half of the corrections being posted less than 3 minutes after being identified.
9 One study found that some corrections were made almost instantaneously in 42% of cases.
10 There is a push on Wikipedia to have statements backed by references and unverifiable statements being called out to readers.
11 Haigh
12 observed that, in general, medically related articles on Wikipedia are accompanied by a sufficient amount of reputable citations.
To evaluate Wikipedia's accuracy, we compared Wikipedia articles on the 10 most costly medical conditions in the United States with recognized peer-reviewed sources.
A few studies
12,25-27 have compared Wikipedia articles with standard peer-reviewed sources and have shown it to be roughly equivalent to these sources. The most notable study, by Giles,
25 compared Wikipedia with the
Encyclopedia Britannica. Other authors
12,26,27 have compared Wikipedia with textbooks and national databases and showed comparable results. In contrast, other researchers
28-30 have determined that Wikipedia is unsuitable as a reference for drugs. Except for psychiatric conditions,
26 scientific research has never, to our knowledge, focused on Wikipedia's content on prevalent medical conditions. A recent study by Azer
31 concluded that Wikipedia is not a reliable information source for medical students in gastroenterology and hepatology.
The present study demonstrated that most Wikipedia articles on the 10 most costly conditions in the United States contained assertions that are inconsistent with peer-reviewed sources. Because our standard was the peer-reviewed published literature, it can be argued that these assertions on Wikipedia represent factual errors.
A perplexing finding in our study was that most of the dissimilar assertions found by the reviewers failed to demonstrate discordance. A reporting bias may have plausibly occurred: each article reviewer was either an internal medicine resident or a rotating intern physician at the time of the review and may not have believed that every assertion was worth reporting. For example, the diabetes mellitus Wikipedia article stated that it is a condition in “which a person has high blood sugar.” One reviewer might have accurately recorded this statement as an assertion, whereas another might have assumed the statement to be common knowledge and erroneously not recorded it as an assertion. These incongruent criteria for assertions may explain the difference found between reviewers.
Although 9 of 10 articles demonstrated discordance between Wikipedia articles and the peer-reviewed sources, the article on concussions did not. This finding may have occurred because Wikipedia has a number of different contributors to each article and the contributors to this particular article were more expert.
The present study had 5 main limitations. First, it did not address errors of omission, but rather was designed to detect assertional errors. It is possible that the Wikipedia article did not contain important information about a topic. However, we opted not to examine errors of omission because of the subjectivity involved with determining what should be included in a review article on a specific medical topic. Second, the present study would have been stronger if more than 2 reviewers were assigned to each article. A future study design could use additional reviewers with more varied specializations to strengthen its findings. Third, we used any peer-reviewed reference as a standard that included an initial search through a subscription-only service (UpToDate). Fourth, we used physicians-in-training rather than content experts as reviewers, which may have created a bias that the present study was not designed to measure. Lastly, we did not check the assertions in the peer-reviewed sources, a limitation that may prove important because peer-reviewed sources are often not in agreement. Future studies might also include how the convenience of Wikipedia may influence perception of the reliability of the information found.
Most Wikipedia articles for the 10 costliest conditions in the United States contain errors compared with standard peer-reviewed sources. Health care professionals, trainees, and patients should use caution when using Wikipedia to answer questions regarding patient care.
Our findings reinforce the idea that physicians and medical students who currently use Wikipedia as a medical reference should be discouraged from doing so because of the potential for errors.
Financial Disclosures: None reported.