Abstract
Context: Randomized controlled trials (RCTs) are considered the standard for establishing practice guidelines; however, they are expensive and time-consuming, and often the generalizability of the results is limited.
Objectives: To conduct an observational study using the findings of the American Osteopathic Association's Clinical Assessment Program (AOA-CAP) low back pain module, and to compare these findings with those of a major back pain–related RCT to determine the validity and generalizability of this pseudoexperimental model.
Methods: Data were abstracted from the AOA-CAP for Residencies platform from April 1, 2006, through October 5, 2007, with a diagnosis code consistent with low back pain. Process and outcome measures were compared after segregating a similar patient population to an RCT that compared “osteopathic spinal manipulation” with standard care.
Results: A total of 1013 medical records were abstracted and entered into the AOA-CAP low back pain module. Mean (standard deviation [SD]) age was 44.7 (15.9) years, and body mass index was 29.6 (8.1). The eligible patients comprised 415 men (41.0%) and 598 women (59.0%), and common comorbid disease was found in 69 patients (6.8%). Activities of daily living were limited in 402 patients (42.4%), whereas 546 (57.6%) had no limitations. Previous exacerbations of low back pain occurred in 653 patients (65.9%). Most patients had no sensory or proprioception deficit (729 [87.7%]), and motor function was normal in 636 patients (74.5%). Normal ankle and knee reflexes were found in 744 of 814 (91.4%) and 755 of 829 (89.0%) patients, respectively. Osteopathic manipulative treatment (OMT) was performed on the lumbar spine (576 patients [56.9%]), thoracic spine (411 [40.6%]), sacrum/pelvis (440 [43.4%]), rib (261 [25.8%]), and lower extremity (256 [25.3%]). A segregated patient cohort (n=539) showed statistically significant differences between patients who received OMT and those who did not with the use of analgesics, steroids, spinal injections, straight-leg raising, and days off or limited work duties.
Conclusion: The observational findings of the present study, which suggest that analgesic medication use is lower in patients who receive OMT, align with previous findings of RCTs and support the generalizability of these findings.
Methods of quantifying health care are increasingly important as both public and private insurers move to link payment to improved clinical outcomes.
1 The key to any value-based initiative's success is understanding which diagnostic or treatment regimen affords the best outcome for patients at the lowest cost. The current value-based initiatives—such as the Affordable Care Act and other government programs—tie payments to the degree to how much a physician's actions meet evidence-based standards or processes of care when making certain diagnoses.
2 As clinical outcomes increasingly form the basis for reimbursement levels, the evidence associated with desired outcomes is poorly developed, representing a health care paradox: high-value outcomes become tied to outcomes with limited evidence. As a result, it will be obligatory for researchers to develop additional methods for establishing practice guidelines to effectively care for populations until more detailed studies can be carried out.
Randomized controlled trials (RCTs) remain the standard for research experiments and the collection of evidence in clinical practice.
3 The findings of RCTs form the evidence base and standard processes of care in health care. For example, RCTs by White and Green
4 and Zimmerman and Hohlfield
5 demonstrated the values of angiotensin-converting enzyme inhibitors and angiotensin-receptor blockers for the treatment of patients with left ventricular systolic dysfunction and of aspirin for the treatment of patients with acute myocardial infarction, respectively. Potential challenges in conducting RCTs are ethical concerns, high cost, limited generalizability, and considerable time investment.
6
Nonexperimental study designs, such as pseudoexperimental, observational, or retrospective study designs, have many benefits, including lower cost, use of existing populations, improved generalizability, and more timely study completion.
7 In a pseudoexperimental study design, investigators have little or no control over the allocation of the treatments or other factors being studied. Pseudoexperimental studies lack randomization for treatment and control groups, which can lead to bias when drawing conclusions for the 2 different groups. Although there are techniques for diminishing or eliminating potential bias, the flawed interpretation of these study designs and the incorrect application of the techniques can lead to erroneous conclusions.
8
The American Osteopathic Association Clinical Assessment Program (AOA-CAP), developed in 2000, is a registry to support quality improvement in osteopathic residency programs in family medicine and internal medicine.
9 The AOA-CAP is designed to evaluate how often physicians use evidence-based processes and how this use affects intermediate outcomes across 8 clinical entities or modules. In 2003, participation in the program was mandated for any family practice residency accredited by the American College of Osteopathic Family Physicians and, in 2005, for any internal medicine residency accredited by the American College of Osteopathic Internists.
10 In 2011, participation in the program became voluntary for family practice residencies. Since its inception, AOA-CAP has been monitored by the AOA Bureau of Osteopathic Clinical Education and Research (BOCER). In 2005, under the direction of the BOCER, the AOA-CAP was directed to develop a module that would move beyond a simple characterization of residency practice performance. It would function as a practice-based research network (PBRN), which the Agency for Healthcare Research and Quality defines as “a group of ambulatory practices devoted principally to the primary care of patients.… Typically, PBRNs draw on the experience and insight of practicing clinicians to identify and frame research questions whose answers can improve the practice of clinical care.”
11 By linking these questions with rigorous research methods, a PBRN can produce research findings that are immediately relevant to the clinician and, in theory, more easily assimilated into everyday practice.
12,13
The goal of the present observational study was to examine the findings from the AOA-CAP and to evaluate associations between OMT, pain perception, work absenteeism, and medication use for patients with low back pain—in other words, to determine the association of osteopathic manipulative treatment (OMT) with the reduction of pain and decrease in morbidity-related factors. To compare data obtained using experimental (RCT) and pseudoexperimental (PBRN) designs, we compared the present study's observational findings with the findings of a previously reported RCT by Andersson et al.
14 We hypothesized that findings collected from this registry-based system will be consistent with findings of the RCT and prove the validity and generalizability of this pseudoexperimental model. If this is the case, the present study will provide the groundwork for additional PBRN studies, which would allow physicians and researchers a simpler way to add to the evidence base for many clinical conditions.
We provided descriptive statistics—including demographics, comorbid disease, physical findings, and pain levels—for all patients in the AOA-CAP low back pain module.
Analysis was completed using SAS software, version 9 (SAS Institute Inc). Data were summarized by the mean, median, and percentage distribution in the study population. To determine whether the sample population was consistent with a normal distribution, a χ2 test for goodness of fit was performed. Comparisons between the measure sets (ie, AOA-CAP data from the present study and Andersson et al data) were made using a paired t test. A P value of <.05 was considered statistically significant.
Our findings revealed that OMT was associated with reduced use of all analgesic medications and nonopioid analgesics, as well as an association with reduced missed or restricted-duty days at work. Our study found no statistically significant difference for the use of muscle relaxants between the OMT group and the standard-care group. The OMT-treated group showed improvements in straight-leg raising. Andersson et al
14 reported that there was a significant reduction in nonsteroidal anti-inflammatory drug and muscle relaxant use in patients receiving OMT. However, Andersson et al
14 found no significant difference in straight-leg raising between the first and final visit and at the final visit. The commonly used outcome of pain reduction as measured by visual analog score was not significantly different between OMT and conventional treatment in the present study. These findings are consistent with those of Andersson et al,
14 which showed no statistically significant difference in visual analog scale score between the first and final visit.
Moreover, back pain is often a disease of chronic pain altering the acute peripheral pain to supraspinal central nervous system pain. Henry et al
15 reviewed numerous studies that showed functional and morphologic changes associated with chronic back pain. The reorganization of sensory pain perception suggests that the standard of the visual analog score may be insensitive to the potential improvement associated with the application of OMT. Supraspinal central pain responds less well to opioid analgesia compared with peripheral pain.
16 Taken together, the findings of the present study and those of Andersson et al suggest that traditional primary endpoints, such as pain perception, may not reveal significant advantages of OMT for low back pain. Indeed, more studies need to be conducted to test whether the benefits of OMT as a treatment modality for low back pain can be confirmed by means of more unconventional measures, such as medication use, medication compliance, hospital admission times, and employee absenteeism.
We must note the potentially confounding effect of analgesic medications. If the patients who received OMT used less medication, then it could be reasonable to infer that those patients were in less pain, thus needing to take less medication. However, the group that did not receive OMT had similar pain scores but used more analgesic medication, leading to the possibility that if both groups had similar pain medication use, there would have been a significant difference in the pain scores. An RCT designed to test this theory may not be ethically feasible because the dosage of medication would need to be standardized between both groups. This process would entail medicating patients who did not need it, denying medication to those who did, or withdrawing patients from the study if their need for medication became too great. Thus, an experimental model other than an RCT would likely be required to obtain more data on this question.
The use of a patient registry to compare clinical practice outcomes with those of an RCT is a novel approach. Although both studies come to the same conclusion, they use very different methodologies, with large differences in generalizability and inference. The RCT by Andersson et al
14 started with 1193 patients; exclusion criteria, pain issues, lack of consent, medical problems, withdrawals, and loss to follow-up reduced the number for randomization to 155. The AOA-CAP study started with 1013 patients, and this number was reduced to 539 by applying exclusion criteria similar to those used in the RCT. With such a large loss of patients from the cohort in the RCT, the generalizability of the findings to large populations is suspect. Although fewer patients met the exclusion criteria in the present study (41.7% vs 87% in the RCT), the challenge in interpreting the findings from the AOA-CAP study is due to the selection bias that may exist in patients chosen for intervention. Although the instructions asked the residency program contact person to randomly select patients who had a diagnosis of low back pain, it was possible that programs selected patients who had the best outcome or who were chosen to highlight some other aspect of their care, such as fulfilling a quota to perform OMT on a certain number of patients.
The 2013 Patient-Centered Outcomes Research Institute methodology report
17 stated that “well-designed observational studies have been extremely valuable as a complement to RCTs, helping to determine under what circumstances and to which patients the findings of RCTs apply.” Using a patient registry system such as the AOA-CAP provides an opportunity for observational studies to be performed for several purposes: (1) to validate/confirm the results of an RCT, (2) to evaluate the manner in which the results of RCTs are being translated into patient care, and (3) to generate new research questions. However, as with all study designs, observational, registry-based studies have their weaknesses—namely, with data quality and bias, especially selection bias. The strengths of a registry-based study are that some questions may be answered quickly because the data have already been gathered and that there is a greater likelihood that the findings will be more broadly applicable because registries are based on real-world clinical practice.
Combining the PBRN study design with a relatively low-risk intervention, such as OMT, in a way that improves clinical outcomes is an excellent example of applying the evidence from an RCT to improve patient care and simultaneously increasing, and hopefully clarifying, the evidence in the literature about traditionally osteopathic-type interventions. A PBRN would also allow researchers to observe the effects of OMT through evaluating alternative outcome measures, in this case by the decreased use of medications and decreased numbers of lost or modified-duty work days. Osteopathic physicians should support the development of a PBRN, which would provide clinicians an opportunity to study the efficacy of OMT in a more cost- and time-efficient manner.