To the Editor: 

In their May 2014 article, Hasty et al¹ reported that Wikipedia contains errors. This finding is based on 2 assumptions about knowledge dissemination in medicine that we question. The first is that the peer-reviewed literature is an accurate reflection of the raw data. The second is that nonspecialists can quickly access a source, such as Up-to-Date, and verify the truthfulness of complex issues. The authors assumed that third-year residents with no specific expertise could correctly ascertain the accuracy of claims made on Wikipedia by comparing them with statements in the peer-reviewed literature. 

Hasty and colleagues have not made their dataset public, so it is impossible to confirm the veracity of their conclusions. This is disappointing because, in this day and age, publicly available data can easily be posted online to facilitate reanalysis and discussion. The authors did share with us a small subset of their dataset on major depressive disorder. We closely examined 2 statements from Wikipedia that the researchers identified as inaccurate. To illustrate the problematic nature of these findings, we discuss them in-depth in the following paragraphs. 

This statement was coded as being contradictory to the peer-reviewed literature, with one coder noting, “No, fluoxetine is approved for age 8 and older.” This statement further illustrates the problems with the study methods. The US Food and Drug Administration (FDA) approval only means that there is a statistically significant difference between the medication and placebo, and a closer look at the data and methods is necessary if one wishes to understand the clinical significance of the results. In 1 of the 2 studies submitted to the FDA for fluoxetine's approval, in addition to a commonly practiced placebo run-in phase, there was a fairly unique medication run-in phase to ensure that only children who were medication responders were allowed into the study. Yet, even with this advantage, for the prospectively defined primary outcome measure, 65% of the children taking Prozac had a beneficial response compared with 53% of the patients taking placebo, a result that was not statistically significant. It was only by looking at other measures that clinical significance was found; on the patient- and parent-rated scales there was no advantage to Prozac, but on 1 of the clinician-rated scales there was a slight advantage to Prozac. Although Russell Katz, MD, of the FDA wrote, “one could argue that this post hoc choice of primary outcome is inappropriate,” the FDA accepted the post hoc change and approved Prozac for children.³ 

In fact, there is perhaps no more contentious and complex issue in medicine right now than the use of antidepressants in young children. Even in adults there is an ongoing debate about the efficacy of antidepressants.⁴ In the case of antidepressants there is evidence of selective reporting,⁵ ghostwritten papers, and a well-documented difference between the published and unpublished clinical trial data.⁶ Several years ago, an editorial in The Lancet summarized the peer-reviewed research on pediatric antidepressant use as “confusion, manipulation, and institutional failure.”⁷ In reference to the pediatric use of antidepressants, Healy referred to the difference between the published data and the actual raw clinical trial data as the “greatest known divide in all of medicine.”⁸ 

In 2004, a study in the BMJ examined the methods and reporting of the published trials of the newer antidepressants in children younger than 18 years.⁹ According to these researchers, the drugs offered only a modest benefit over placebo but had significant risks. In their conclusions they noted that: “Antidepressant drugs cannot confidently be recommended as a treatment option for childhood depression.” These researchers also noted that in the original Prozac pediatric trials submitted to the FDA, Prozac did not show efficacy over placebo on the primary end points. 

The coders also found fault with this statement and mentioned that combination therapy should be used. Again, this is an area rife with debate. For instance, the European Medicines Agency recommends that, for children and adolescents aged 8 years or older, medication should only be used if the patient is unresponsive to psychological therapy. The European Medicines Agency does not recommend medication for children younger than 8 years.¹⁰ The recent Treatment for Adolescents With Depression Study (TADS) is often cited as evidence for the superiority of fluoxetine plus cognitive behavioral therapy, but support for this therapy comes from the unblinded arm of the study. Some researchers have stated that it is a failed trial of Prozac: “TADS found no statistical advantage of fluoxetine over placebo on the primary end point, the children's depression rating scale (CDRS-R; P=0.10), but this was not mentioned in the abstract.”¹¹ 

It seems problematic to conclude that statements made in Wikipedia are wrong based on peer-reviewed literature. The peer-reviewed literature is filled with ambiguity, different viewpoints, and debate. As the editors of Nature stated, “scientists understand that peer review per se provides only a minimal assurance of quality, and that the public conception of peer review as a stamp of authentication is far from the truth.”¹² 

Much of the problem with the medical literature stems from a failure of pharmaceutical companies to release data. The current debate about Tamiflu is just one example.¹³ In general, seasoned readers of the clinical trial literature assume that the beneficial effects of a trial medication are exaggerated and that the adverse effects are downplayed. Only a naive reader would assume a published clinical trial portrays a true picture of all of the data. Vioxx is another example of a medication, in which there was a substantial disconnect between the published literature and the actual clinical trial data, yet it had been approved by the FDA.¹⁴ Many of the leaders in medical publishing and academic medicine are now calling for researchers to freely share data so that others can analyze it (eg, the AllTrials campaign, http://www.alltrials.net/). Several pharmaceutical companies are now agreeing to share their data in principle.¹⁵ 

It seems to us that trial data selected for publication should be looked upon skeptically unless all of the data are freely available. This critical viewpoint seems to be especially important when interpreting the study by Hasty et al¹ because they claimed that a widely used website has made misleading statements, but the authors do not identify the actual statements. We found problems with their coding by examining just a small subset of statements on major depressive disorder—whether the same problems would apply to other diseases they examined (eg, heart disease, hypertension, diabetes) is unknown. To verify their conclusions would require that the entire dataset be available, increasingly an ideal within most scientific disciplines.¹⁶ Ironically, the lack of access to their data is in direct contrast to Wikipedia, where the debate about any given statement is freely available for all to see. 

The purpose of this letter is not to debate the use of antidepressants in children but simply to point out that, because there is a healthy debate in the medical literature about these medications, it is hard to summarize the veracity of complex statements in an Excel file with simple yes or no answers. It is not surprising that some information in Wikipedia and in the peer-reviewed literature is controversial, biased, or even wrong. However, the methods and conclusions of Hasty et al¹ suggest that physicians and researchers are not sufficiently skeptical of the medical literature. This lack of skepticism has profound implications for public health, much more so than potentially erroneous statements made in Wikipedia.