Abstract
Context: Randomized controlled trials (RCTs) are considered the standard for establishing practice guidelines; however, they are expensive and time-consuming, and often the generalizability of the results is limited.
Objectives: To conduct an observational study using the findings of the American Osteopathic Association's Clinical Assessment Program (AOA-CAP) low back pain module, and to compare these findings with those of a major back pain–related RCT to determine the validity and generalizability of this pseudoexperimental model.
Methods: Data were abstracted from the AOA-CAP for Residencies platform from April 1, 2006, through October 5, 2007, with a diagnosis code consistent with low back pain. Process and outcome measures were compared after segregating a similar patient population to an RCT that compared “osteopathic spinal manipulation” with standard care.
Results: A total of 1013 medical records were abstracted and entered into the AOA-CAP low back pain module. Mean (standard deviation [SD]) age was 44.7 (15.9) years, and body mass index was 29.6 (8.1). The eligible patients comprised 415 men (41.0%) and 598 women (59.0%), and common comorbid disease was found in 69 patients (6.8%). Activities of daily living were limited in 402 patients (42.4%), whereas 546 (57.6%) had no limitations. Previous exacerbations of low back pain occurred in 653 patients (65.9%). Most patients had no sensory or proprioception deficit (729 [87.7%]), and motor function was normal in 636 patients (74.5%). Normal ankle and knee reflexes were found in 744 of 814 (91.4%) and 755 of 829 (89.0%) patients, respectively. Osteopathic manipulative treatment (OMT) was performed on the lumbar spine (576 patients [56.9%]), thoracic spine (411 [40.6%]), sacrum/pelvis (440 [43.4%]), rib (261 [25.8%]), and lower extremity (256 [25.3%]). A segregated patient cohort (n=539) showed statistically significant differences between patients who received OMT and those who did not with the use of analgesics, steroids, spinal injections, straight-leg raising, and days off or limited work duties.
Conclusion: The observational findings of the present study, which suggest that analgesic medication use is lower in patients who receive OMT, align with previous findings of RCTs and support the generalizability of these findings.
Methods of quantifying health care are increasingly important as both public and private insurers move to link payment to improved clinical outcomes.
1 The key to any value-based initiative's success is understanding which diagnostic or treatment regimen affords the best outcome for patients at the lowest cost. The current value-based initiatives—such as the Affordable Care Act and other government programs—tie payments to the degree to how much a physician's actions meet evidence-based standards or processes of care when making certain diagnoses.
2 As clinical outcomes increasingly form the basis for reimbursement levels, the evidence associated with desired outcomes is poorly developed, representing a health care paradox: high-value outcomes become tied to outcomes with limited evidence. As a result, it will be obligatory for researchers to develop additional methods for establishing practice guidelines to effectively care for populations until more detailed studies can be carried out.
Randomized controlled trials (RCTs) remain the standard for research experiments and the collection of evidence in clinical practice.
3 The findings of RCTs form the evidence base and standard processes of care in health care. For example, RCTs by White and Green
4 and Zimmerman and Hohlfield
5 demonstrated the values of angiotensin-converting enzyme inhibitors and angiotensin-receptor blockers for the treatment of patients with left ventricular systolic dysfunction and of aspirin for the treatment of patients with acute myocardial infarction, respectively. Potential challenges in conducting RCTs are ethical concerns, high cost, limited generalizability, and considerable time investment.
6
Nonexperimental study designs, such as pseudoexperimental, observational, or retrospective study designs, have many benefits, including lower cost, use of existing populations, improved generalizability, and more timely study completion.
7 In a pseudoexperimental study design, investigators have little or no control over the allocation of the treatments or other factors being studied. Pseudoexperimental studies lack randomization for treatment and control groups, which can lead to bias when drawing conclusions for the 2 different groups. Although there are techniques for diminishing or eliminating potential bias, the flawed interpretation of these study designs and the incorrect application of the techniques can lead to erroneous conclusions.
8
The American Osteopathic Association Clinical Assessment Program (AOA-CAP), developed in 2000, is a registry to support quality improvement in osteopathic residency programs in family medicine and internal medicine.
9 The AOA-CAP is designed to evaluate how often physicians use evidence-based processes and how this use affects intermediate outcomes across 8 clinical entities or modules. In 2003, participation in the program was mandated for any family practice residency accredited by the American College of Osteopathic Family Physicians and, in 2005, for any internal medicine residency accredited by the American College of Osteopathic Internists.
10 In 2011, participation in the program became voluntary for family practice residencies. Since its inception, AOA-CAP has been monitored by the AOA Bureau of Osteopathic Clinical Education and Research (BOCER). In 2005, under the direction of the BOCER, the AOA-CAP was directed to develop a module that would move beyond a simple characterization of residency practice performance. It would function as a practice-based research network (PBRN), which the Agency for Healthcare Research and Quality defines as “a group of ambulatory practices devoted principally to the primary care of patients.… Typically, PBRNs draw on the experience and insight of practicing clinicians to identify and frame research questions whose answers can improve the practice of clinical care.”
11 By linking these questions with rigorous research methods, a PBRN can produce research findings that are immediately relevant to the clinician and, in theory, more easily assimilated into everyday practice.
12,13
The goal of the present observational study was to examine the findings from the AOA-CAP and to evaluate associations between OMT, pain perception, work absenteeism, and medication use for patients with low back pain—in other words, to determine the association of osteopathic manipulative treatment (OMT) with the reduction of pain and decrease in morbidity-related factors. To compare data obtained using experimental (RCT) and pseudoexperimental (PBRN) designs, we compared the present study's observational findings with the findings of a previously reported RCT by Andersson et al.
14 We hypothesized that findings collected from this registry-based system will be consistent with findings of the RCT and prove the validity and generalizability of this pseudoexperimental model. If this is the case, the present study will provide the groundwork for additional PBRN studies, which would allow physicians and researchers a simpler way to add to the evidence base for many clinical conditions.
The AOA-CAP is a Web-based primary care registry currently used in AOA-approved family medicine and internal medicine residencies. Residency programs randomly selected medical records of patients who received a diagnosis of low back pain (according to
International Classification of Diseases, Ninth Revision) for abstraction. Although an informational packet was provided to residency programs in an effort to standardize the abstraction, there was no strict oversight of how residency directors randomized medical record selection. Medical record exclusion criteria included an age younger than 17 years and chronic low back pain of less than 12 weeks' duration that was occurring at the time of abstraction. Patient data were entered using a standard set of directions developed by the BOCER, which focused on process and outcome measures (
Figure 1). The AOA Department of Research and BOCER recommended that 20 medical records be abstracted from each residency, which was intended to ensure an adequate sample for quality assessment. The AOA provided the data from the AOA-CAP.
Data were abstracted from the medical records of family medicine residencies that participated in AOA-CAP between April 1, 2006, through October 5, 2007. We contacted residency directors via e-mail. If they agreed to participate, they submitted their institutional data to the AOA-CAP database. To evaluate the association between OMT and patient improvement compared with an RCT by Andersson et al that evaluated “osteopathic spinal manipulation” vs standard allopathic care in patients with chronic and subchronic back pain,
14 we limited the analysis to patients with no related comorbidity (ie, neurologic disease, spondylolisthesis, and connective tissue disease) and no impairments in deep tendon reflexes or motor or sensory functions. We evaluated specific process and outcome measures from the registry, notably those of OMT on pain and medication use (analgesics, anti-inflammatory agents, muscle relaxants) in patients with a diagnosis of low back pain. We were able to evaluate the outcomes only for patients who had abstracted information available.
We provided descriptive statistics—including demographics, comorbid disease, physical findings, and pain levels—for all patients in the AOA-CAP low back pain module.
Analysis was completed using SAS software, version 9 (SAS Institute Inc). Data were summarized by the mean, median, and percentage distribution in the study population. To determine whether the sample population was consistent with a normal distribution, a χ2 test for goodness of fit was performed. Comparisons between the measure sets (ie, AOA-CAP data from the present study and Andersson et al data) were made using a paired t test. A P value of <.05 was considered statistically significant.
A total of 1013 medical records from 27 family medicine residencies were abstracted and entered into the AOA-CAP low back pain module. Many residency directors submitted more than the recommended number of 20 medical records. Mean (standard deviation [SD]) age was 44.7 (15.9) years, and body mass index was 29.6 (8.1). Patients comprised 415 men (41.0%) and 598 women (59.0%) and had the following relevant comorbid conditions: neurologic disease (91 [9.0%]), spondylolisthesis (65 [6.4%]), and connective tissue disease (50 [4.9%]) (
Table 1).
Table 2 depicts the baseline activities of daily living and the frequency of previous low back pain episodes. Of the 948 patients with available information, 546 patients (57.6%) had no limitations to daily activity, whereas 402 (42.4%) had at least some limitation. On review of 991 medical records, 338 patients (34.1%) had no history of acute low back pain, whereas 653 (65.9%) had at least 1 reported episode per year.
Table 3 depicts the 3 functions—the baseline sensory or proprioception, the motor, and the deep tendon reflex—of the patients. Of the 831 patients with available information, 729 (87.7%) had no sensory or proprioceptive deficit, and 102 (12.3%) had a deficit. Motor function was assessed as normal for 636 of 854 patients (74.5%), whereas 218 (25.5%) had a motor impairment. Normal ankle and knee reflexes were reported in 744 of 814 (91.4%) and 755 of 829 (89.0%) patients, respectively.
Of the patients who received at least 1 regional session of OMT for low back pain, 576 were treated in the lumbar region (56.9%), 411 (40.6%) in the thoracic spine, 440 (43.4%) in the sacrum/pelvis, 261 (25.8%) in the rib, and 256 (25.3%) in the lower extremity (
Table 4). The 209 patients who received no OMT for low back pain are also shown adjacent to the average number of OMT sessions patients received for each body region.
To better compare data from the present study with that of the RCT by Andersson et al,
14 we identified a cohort of 539 patients (
Figure 2). Using this cohort, we compared data from patients receiving OMT with those who did not receive OMT (
Table 5 and
Table 6). We compared AOA-CAP patients who were treated with OMT alongside those who were not, noting the relationship between frequently prescribed medical treatments for low back pain and the likelihood that those patients received OMT. There is a statistically significant (
P<.05) difference between patients who received or did not receive OMT and oral steroids (
P=.0481), opioid (
P=.0001), nonopioid analgesics (
P=.0199), and spinal injections (
P=.0052) (
Table 5). Straight-leg raising (
P=.0092), days off of work (
P=.0001), and days with limited work duties (
P=.0001) were all significantly different between patients who did and did not receive OMT (
Table 6).
By confining the present study to AOA-CAP data, we were able to screen the data of many more patients than Andersson et al
14: 1013 patients vs 155 patients. Data from the present study aligned with those of Andersson et al
14 as follows: there was no statistically significant difference in mean (SD) visual analog–scale scores for the OMT group vs the standard-care group (2.9 [2.2] vs 2.8 [2.0] and 3.2 [2.3] vs 2.6 [2.4], respectively), and both studies found decreased use of analgesics in the use of nonsteroidal medication (63% vs 72.7% [
P=.019] and 24.3% vs 54.3% [
P<.001], respectively).
Our findings revealed that OMT was associated with reduced use of all analgesic medications and nonopioid analgesics, as well as an association with reduced missed or restricted-duty days at work. Our study found no statistically significant difference for the use of muscle relaxants between the OMT group and the standard-care group. The OMT-treated group showed improvements in straight-leg raising. Andersson et al
14 reported that there was a significant reduction in nonsteroidal anti-inflammatory drug and muscle relaxant use in patients receiving OMT. However, Andersson et al
14 found no significant difference in straight-leg raising between the first and final visit and at the final visit. The commonly used outcome of pain reduction as measured by visual analog score was not significantly different between OMT and conventional treatment in the present study. These findings are consistent with those of Andersson et al,
14 which showed no statistically significant difference in visual analog scale score between the first and final visit.
Moreover, back pain is often a disease of chronic pain altering the acute peripheral pain to supraspinal central nervous system pain. Henry et al
15 reviewed numerous studies that showed functional and morphologic changes associated with chronic back pain. The reorganization of sensory pain perception suggests that the standard of the visual analog score may be insensitive to the potential improvement associated with the application of OMT. Supraspinal central pain responds less well to opioid analgesia compared with peripheral pain.
16 Taken together, the findings of the present study and those of Andersson et al suggest that traditional primary endpoints, such as pain perception, may not reveal significant advantages of OMT for low back pain. Indeed, more studies need to be conducted to test whether the benefits of OMT as a treatment modality for low back pain can be confirmed by means of more unconventional measures, such as medication use, medication compliance, hospital admission times, and employee absenteeism.
We must note the potentially confounding effect of analgesic medications. If the patients who received OMT used less medication, then it could be reasonable to infer that those patients were in less pain, thus needing to take less medication. However, the group that did not receive OMT had similar pain scores but used more analgesic medication, leading to the possibility that if both groups had similar pain medication use, there would have been a significant difference in the pain scores. An RCT designed to test this theory may not be ethically feasible because the dosage of medication would need to be standardized between both groups. This process would entail medicating patients who did not need it, denying medication to those who did, or withdrawing patients from the study if their need for medication became too great. Thus, an experimental model other than an RCT would likely be required to obtain more data on this question.
The use of a patient registry to compare clinical practice outcomes with those of an RCT is a novel approach. Although both studies come to the same conclusion, they use very different methodologies, with large differences in generalizability and inference. The RCT by Andersson et al
14 started with 1193 patients; exclusion criteria, pain issues, lack of consent, medical problems, withdrawals, and loss to follow-up reduced the number for randomization to 155. The AOA-CAP study started with 1013 patients, and this number was reduced to 539 by applying exclusion criteria similar to those used in the RCT. With such a large loss of patients from the cohort in the RCT, the generalizability of the findings to large populations is suspect. Although fewer patients met the exclusion criteria in the present study (41.7% vs 87% in the RCT), the challenge in interpreting the findings from the AOA-CAP study is due to the selection bias that may exist in patients chosen for intervention. Although the instructions asked the residency program contact person to randomly select patients who had a diagnosis of low back pain, it was possible that programs selected patients who had the best outcome or who were chosen to highlight some other aspect of their care, such as fulfilling a quota to perform OMT on a certain number of patients.
The 2013 Patient-Centered Outcomes Research Institute methodology report
17 stated that “well-designed observational studies have been extremely valuable as a complement to RCTs, helping to determine under what circumstances and to which patients the findings of RCTs apply.” Using a patient registry system such as the AOA-CAP provides an opportunity for observational studies to be performed for several purposes: (1) to validate/confirm the results of an RCT, (2) to evaluate the manner in which the results of RCTs are being translated into patient care, and (3) to generate new research questions. However, as with all study designs, observational, registry-based studies have their weaknesses—namely, with data quality and bias, especially selection bias. The strengths of a registry-based study are that some questions may be answered quickly because the data have already been gathered and that there is a greater likelihood that the findings will be more broadly applicable because registries are based on real-world clinical practice.
Combining the PBRN study design with a relatively low-risk intervention, such as OMT, in a way that improves clinical outcomes is an excellent example of applying the evidence from an RCT to improve patient care and simultaneously increasing, and hopefully clarifying, the evidence in the literature about traditionally osteopathic-type interventions. A PBRN would also allow researchers to observe the effects of OMT through evaluating alternative outcome measures, in this case by the decreased use of medications and decreased numbers of lost or modified-duty work days. Osteopathic physicians should support the development of a PBRN, which would provide clinicians an opportunity to study the efficacy of OMT in a more cost- and time-efficient manner.
Many factors limit the degree to which the results of an RCT can be compared with those of a registry database such as the AOA-CAP. These include the potential confounding factors of medication use and selection bias, as previously discussed, as well as differences in generalizability and missing or undocumented data. The RCT and the analysis from the AOA-CAP data were performed separately, and thus it was impossible to compare and accommodate for all confounding factors. That our findings are similar to those of the Andersson et al
14 study suggests that both approaches are valuable.
Bias can also be introduced into a retrospective or observational study by loss to follow-up or failure to fully document a patient's condition. Of note, only 55% of patients had both initial and final visual analog scale scores recorded, raising concern regarding the outcomes of those patients with missing data. It is possible that patients who had OMT were less likely to request pain-related medications from the physician or be more likely to return to work sooner. Caution should be used when interpreting these data because of missing data and potential biases.
Our findings suggest that analgesic medication use is lower in patients who receive OMT, which aligns with previous findings of an RCT and supports the generalizability of these findings. In light of the cost and time savings of the present observational study design, PBRN studies may be particularly useful as we move into an era of cost-effective, evidence-based medicine.
The AOA-CAP is supported by the American Osteopathic Association and the American Osteopathic Foundation. We thank J. Justin McCormick, PhD, and Bethany Heinlein for their expert editorial review.