It was interesting to read the article by Prinsen and colleagues regarding osteopathic manipulative treatment (OMT) of patients with low back pain in the February 2014 issue of
The Journal of the American Osteopathic Association.
1 The authors are to be commended for seeking to implement a more pragmatic and efficient alternative to the conventional randomized controlled trial for efficacy of OMT. However, in using the American Osteopathic Association Clinical Assessment Program (AOA-CAP) data, the authors introduced several biases and limitations into their methodology. Consequently, their conclusions were inconsistent with current best evidence on OMT in the management of chronic low back pain.
The most critical methodological questions raised by the authors following their study were whether a visual analog scale (VAS) score for pain may be too insensitive to reveal the potentially statistically significant advantages of managing low back pain with OMT and whether the potentially confounding effects of medication use for low back pain can be adequately controlled in a randomized controlled trial. Apparently, the authors were unaware of the results of the OSTEOPAThic Health outcomes In Chronic low back pain (OSTEOPATHIC) Trial, which squarely address each question. The OSTEOPATHIC Trial used a randomized, double-blind, sham-controlled design to assess OMT efficacy in 455 patients,
2 thereby making it the largest trial to address such questions. Therein, a VAS was used to demonstrate statistically significant and clinically relevant pain improvement with OMT in patients with chronic low back pain.
3 In fact, VAS pain scores were sufficiently robust to demonstrate a large OMT effect in the subgroup of patients with moderate to severe levels of baseline low back pain.
4
Beyond these statistically significant and clinically relevant pain reductions with OMT in the OSTEOPATHIC Trial, it was also shown that patients in the OMT group less frequently used prescription medication for low back pain than did patients in the sham OMT group, even after controlling for multiple potential confounders.
3 Notably, in the OSTEOPATHIC Trial (unlike the AOA-CAP program), medication for low back pain was independently prescribed by physicians who were not part of the investigative team and, therefore, were blinded to patient treatment allocation. The results of the OSTEOPATHIC Trial are consistent with representative, population-based data from the National Ambulatory Medical Care Survey indicating that osteopathic physicians manage low back pain by prescribing medications, particularly nonsteroidal anti-inflammatory drugs, less frequently than allopathic physicians.
5
There were several other methodological issues in the authors' study that deserve further comment. First, given the large number of graduates of colleges of osteopathic medicine who have entered residency programs approved by the Accreditation Council for Graduate Medical Education,
6 family medicine residencies approved by the AOA are not likely to be truly representative of osteopathic medical care. Second, because the participation rate of invited family medicine residencies was not reported, selection bias may exist even within the subgroup of AOA-approved residencies. Third, because the randomization process for selecting patient medical records within residency programs was not strictly overseen by the investigators, it cannot simply be assumed to have been validly performed. Fourth, only 55% of eligible patients had both initial and final VAS scores for low back pain. It is unclear why imputation for missing data was not attempted to include the remaining proportion of study patients. Fifth, physicians at the participating residency programs potentially provided both prescription medication and OMT for low back pain, each in an unblinded fashion. This methodology raises serious questions about internal validity, particularly regarding the fundamental relationships among VAS pain scores, prescribing of analgesic medication, and provision of OMT.
Finally, the rationale for the AOA recommendation of abstracting 20 medical records per residency program to ensure adequate sample size and statistical power is not described and likely does not adequately address the clustering of observations within practice-based research networks. Parenthetically, such clustering is aggravated by the fact that only 27 family medicine residencies were needed to acquire 1013 medical records, rather than the 51 residencies that would have been needed with strict adherence to the 20-record limit. Failure to recognize clustering causes dramatically under-powered studies. Subsequent failure to adequately adjust for such clustering yields unrealistically precise treatment effects,
7 which inflate the type 1 error rate and bring statistically significant results into question. Unfortunately, the variance inflation factor needed to correct for clustering relating to OMT within clinics can be quite high, as recently demonstrated by the Consortium for Collaborative Osteopathic Research Development–Practice-Based Research Network (CONCORD-PBRN).
8 For example, in comparison with the sample size for a population-based study of individual patients, 20- to 40-fold increases in sample size may be needed to validly assess the use of various OMT techniques among patients within a small group of networked clinics.
In summary, despite the consistency of the authors' results with those of a randomized controlled trial by Andersson and colleagues,
9 both study results are at odds with current best evidence. The OSTEOPATHIC Trial has clearly shown that OMT provides statistically significant and clinically relevant pain improvement in patients with chronic low back pain as further manifested by the decreased need for prescription rescue medication.