Sage Journals: Discover world-class research

Abstract

Background

Acupuncture and yoga have both been shown to be effective in chronic pain. Underrepresented populations have poorer pain outcomes with less access to effective pain care.

Objective

To assess the feasibility of bundling group acupuncture with yoga therapy for chronic neck, back or osteoarthritis pain in safety net settings.

Methods

This was a feasibility pilot in Bronx and Harlem primary care community health centers. Participants with chronic neck, back or osteoarthritis pain received acupuncture and yoga therapy over a 10-week period. Participants received 10 weekly acupuncture treatments in group setting; with Yoga therapy sessions beginning immediately following the 3^rd session. Primary outcome was pain interference and pain intensity on the Brief Pain Inventory (BPI); Outcomes were measured at baseline, 10-week close of intervention, and 24-week follow-up.

Results

93 patients were determined to be eligible and completed the baseline interview. The majority of participants were non-White and Medicaid recipients. 78 (84%) completed the intervention and 10-week survey, and 58 (62%) completed the 24-week post intervention survey. Participants received an average number of 6.5 acupuncture sessions (out of a possible 10), and 4 yoga sessions (out of a possible 8) over the 10-week intervention. Patients showed statistically significant improvements in pain at the close of the intervention and at a somewhat lesser rate, at 24-weeks post intervention. Challenges included telephone outreach and site coordination integrating acupuncture with yoga therapy. The trial also had to be stopped early due to the COVID-19 pandemic.

Conclusions

Bundling acupuncture therapy and yoga therapy is feasible for an underrepresented population with chronic pain in urban community health centers with preliminary indications of acceptability and benefit to participants.

Keywords

acupuncture therapy yoga therapy nonpharmacologic pain care underserved setting

Background

Estimates of chronic pain in the US adult population have ranged from 10-40% in relevant large surveys,^1-4 with a documented association of impaired physical and psychological functioning^5-7 and lost work productivity.^8,9 Minority populations differ both in the prevalence and outcomes of chronic pain;^10-13 wherein race/ethnicity and socioeconomic factors influence access to pain care.^13-16 Opioids continue to be used for chronic pain with opioid addiction, and diversion and deaths remaining an ongoing epidemic in the US.¹⁷ By the end of 2020, accelerated by the COVID-19 pandemic, opioid overdose deaths increased by over 35% from 2019.¹⁸ Opioid fatalities are associated with lower socioeconomic status,¹⁹ making access to non-opioid and nonpharmacologic pain care options a priority in underserved communities. Following the Consolidated Standards of Reporting Trials (CONSORT) guidelines,²⁰ this paper reports on a project examining the feasibility of a novel approach to chronic pain that combines group acupuncture therapy with yoga therapy delivered in the primary care setting to an underserved patient population.²¹

Acupuncture therapy is effective for chronic pain conditions,²² including chronic low back pain (cLBP),^23,24 neck pain,²⁴ shoulder pain and knee pain from osteoarthritis.^25-28 A large individual patient data meta-analysis (39 trials) of 20,827 patients with chronic pain found acupuncture to be significantly better than sham treatment or usual care, with only a 15% loss in treatment effect after 1 year.²⁵ Patients with more severe pain at baseline improved more from acupuncture treatment than those with lower levels of pain, compared to sham or non-acupuncture controls.²⁹ Acupuncture therapy is supported or recommended as part of comprehensive pain care by the U.S. Agency for Healthcare Research and Quality (AHRQ),³⁰ the U.S Food and Drug Administration (FDA),³¹ the National Institutes for Health (NIH)³² and the Joint Commission (TJC).³³ A retrospective claims-based study found initial visits to chiropractors, physical therapists or acupuncturists for new onset LBP substantially decreased early and long-term use of opioids.³⁴ Active military service members who accessed acupuncture for chronic pain had reduced risk of long-term adverse outcomes.³⁵

Acupuncture therapy - which includes palpation, the manual techniques Tui na and/or Gua sha, needling body points and needling auricular acupuncture points, and adhesive application of ear seeds for extended auricular therapy--is feasible and effective specifically for underrepresented populations at risk for health outcome disparities.³⁶ However, cost and access to individual acupuncture treatment continues to pose a barrier to widespread implementation in this patient population where insurance reimbursement rates remain low, and out-of-pocket expenses forbidding.^37,38 To address these issues, group acupuncture, which is less costly, is being offered in many settings across the U.S. Evidence indicates that acupuncture delivered in a group setting is beneficial for chronic pain.^37,39,40 Our ‘Acupuncture Approaches to Decrease Disparities in Outcomes of Pain Treatment (AADDOPT-2)’ trial compared 12 sessions of individual practitioner-patient acupuncture to group acupuncture (n = 706)³⁹ in the primary care setting in an underserved population demonstrated chronic pain reduction and improved function at 12 weeks in both arms. Qualitative studies to date show that acupuncture treatment in a group setting is highly acceptable to patients^41-43 and that the group setting, community-based locations, and low cost of this model help eliminate some of the barriers to access.⁴⁰

Yoga therapy is an emerging healthcare profession⁴⁴ that includes the therapeutic application of attention/meditation (dhyana) and other mental practices, controlled breathing (pranayama), movement (vinyasa), physical postures (asanas), and applied philosophy/lifestyle.¹⁷ Yoga therapy sessions differ from yoga classes in that they are delivered one-on-one or in small groups, include a thorough client intake, an individualized plan of care and ongoing assessment of progress.⁴⁵ Certified yoga therapists have training as yoga teachers (200+ hours) followed by additional specialized training (800+ hours) in clinical conditions, therapeutics, and integrative care.

Yoga practices demonstrate benefit for pain as well as pain-associated function⁴⁶ and disability,⁴⁷ in musculoskeletal pain disorders including back, neck, osteoarthritis, rheumatoid arthritis and fibromyalgia.^48-50 Yoga has also been studied in underrepresented community settings,^51,52 and was found to be non-inferior to physical therapy for cLBP in such patients.⁵² The NIH recognizes the evidence for yoga,³² and the AHRQ and American College of Physicians (ACP) recommend yoga for cLBP.^30,53

In AADDOPT-2³⁹ we noted a natural progression for participants being treated with acupuncture therapy to begin to engage in more movement as their pain improved, and for acupuncturists to encourage activity and steps in movement recovery. Multiple studies and reviews have shown movement helps to reduce pain, improve function, and reduce frequency and intensity of chronic pain flare-ups.⁵⁴ We hypothesized that bundling acupuncture with an effective movement therapy may be an optimal progression for recovery.²¹ Yoga therapy was chosen for this combined intervention for its contribution of movement practices, as well as its breathwork, mindfulness, and relaxation techniques that are well-aligned with acupuncture therapy and Chinese medicine principles, and relevant to chronic pain management. Broadly speaking, the model leveraged acupuncture to benefit function and reduce pain, together with yoga to provide a guided, therapeutic approach to movement; thus augmenting pain management and recovery with movement in a safe and sustainable way. No previous study has evaluated the combination of acupuncture and yoga therapy in patients with chronic pain, even though combining movement therapy with therapies like acupuncture has been an established model of rehabilitative care.⁵⁵ Both therapies are effective for chronic pain; both have been shown to be feasible in underrepresented populations.^39,51,52 Given the need for patient activation in the context of chronic pain, yoga therapy combined with acupuncture therapy may offer a significant synergy for chronic pain management and recovery. We designed a pilot study to assess if it is feasible to provide these care options for this population at their primary care sites in the context of a study that could assess outcomes.

Methods

Study Design

The study used a repeated measures quasi experimental design comparing baseline measures to those at 10 weeks and 24 weeks from baseline. To optimize feasibility and acceptability to patients and participating health centers while still generating meaningful outcome data, we did not randomize within the practices, but instead offered the intervention to all patients who met eligibility criteria. To collect pre-acupuncture assessments of pain, we included a 10-day intake run-in period prior to the initial acupuncture session that ensures treatment within a time frame that is consistent with typical time to appointments for many consultations (see Table 1 below). Study was approved by all IRBs, specifically those of Albert Einstein College of Medicine, Montefiore Medical Center; Institute for Family Health; and Icahn School of Medicine at Mount Sinai.

Table 1.

Outcome Measures Data Collection Timeline.

Measure	Enrollment	Run-in 1	Treatment Initiation	Week 10	Week 24
BPI	X	X	X	X	X
CES-D	X			X	X
Pain Free Days	X	X	X	X	X
PGIC				X	X
ACE	X			X	X
PROMIS 10	X			X	X
PROMIS Satisfaction	X			X	X
Demographics	X

BPI = Brief Pain Index; CES-D = Center for Epidemiological Studies - Depression Scale; PGIC = Patient Global Impression of Change; ACE = Altarum Consumer Engagement; PROMIS = Patient-Reported Outcomes Measurement Information System.

Study Participants and Eligibility

Outpatients with chronic pain were receiving primary care at multiple Institute for Family Health (IFH) community health center sites in Manhattan and the Bronx, and two Montefiore Medical Group (MMG) sites in the Bronx: The Family Health Center and Williamsbridge Family Practice Center. Eligibility for participants included:

• 21 years of age or older

• Chronic pain (3 months or more in duration) due to a qualifying diagnosis of back pain, neck pain, and/or osteoarthritis.

• Ability to provide consent in English or Spanish

• Reliable phone contact

• Availability for up to 10 weekly consecutive treatments and follow-up data collection up to 24 weeks.

Exclusion criteria were

• Recent acupuncture treatment or yoga instruction/therapy in the 6 months prior to recruitment

• Pregnancy

• Severe psychiatric problems as assessed by the study team (e.g., chronic interpersonal problems, cognitive impairment or active psychosis that is uncontrolled by medication that precludes the ability to provide informed consent or complete the survey instruments).

Recruitment

Primary care providers at the IFH and MMG sites made referrals to the study team by paper referral or electronic medical record in-basket. Referring providers obtained verbal permission from the patient for a Clinical Research Coordinator (CRC) to contact them. The CRC confirmed patient eligibility and consent orally over the phone to be reviewed and confirmed in person. Participants did not receive an incentive to attend treatment sessions but did receive modest incentives to complete the research interviews. Participants were provided $25 for completing the enrollment interview, $5 for completing the Run-in 1 interview (and $5 for subsequent run-in interviews, if necessary), and $15 for completing the treatment initiation interview before the first intervention session. After the intervention, participants were provided $25 and $30 for completing study interviews at Week 10 and Week 24, respectively (Table 1 shows the interval of assessments).

Intervention

IFH patients received the intervention at the Family Health Center of Harlem, while Montefiore patients received the intervention at the two Bronx MMG sites. Treatment consisted of ten consecutive weekly group acupuncture therapy treatments described in detail in a previous design paper²¹ and intervention manualization.⁵⁶ Yoga therapy treatments were started at week 3 and consisted of eight consecutive yoga therapy sessions that occurred immediately after acupuncture, in an adjacent room.

Usual Care

During the entire study, patients continued to receive usual care for management of chronic pain as provided by primary care providers. Usual care may include referral for specialty consultation or physical therapy, or use of medication, but does not typically include acupuncture or yoga services. We anticipated that patients would vary substantially in the duration and severity of pain and the specifics of the approach to pain management.

Acupuncture Therapy Component

During the intervention phase, each participant was given acupuncture therapy in a group setting (seated in a stationary chair) over 10 consecutive weeks. Direct practitioner/patient session time typically lasted 30-40 minutes. Session time could reach up to 60 minutes, to include time for the participants to arrive and settle in, as well as rest time after treatment. Sessions were shortened to as little as 15 minutes if participants were late to their scheduled treatment time. Three consecutive weekly sessions of acupuncture were given before yoga therapy was introduced. This allowed time for acupuncture therapy to reduce chronic pain severity, readying the participant to begin an active phase of movement and recovery provided by yoga therapy. Thereafter, patient appointments were arranged so that acupuncture treatment immediately preceded yoga therapy treatment on the same day.

Yoga Therapy Component

At week 2, a 20-25-minute yoga intake was conducted, following acupuncture therapy, to assess and inform a yoga plan of care. At week 3, that plan of care introduced the yoga therapy following the acupuncture therapy, so that week’s 3 through 10 were combined sequential interventions, as stated above. Participants’ yoga therapy sessions lasted 30-35 minutes, either individually or in dyads. To foster self-efficacy and sustained benefit, participants were further directed to practice at home via simple instructions both verbally and with handouts in patient’s preferred language of English or Spanish. Participants were guided in the yoga philosophy of mindful awareness and non-harming in order to avoid exacerbation of symptoms and ensure optimal safety in home practice.

Therapy Manualizations

Complex therapy interventions are developed for research using a consensus process also called ‘manualization’.^57,58 Our acupuncture therapy was based on the pragmatic intervention manualization used in our AADDOPT-2 trial^21,39,56 that allows for individualizing treatment from a pre-determined set of options adaptable to individual participant needs and in a group acupuncture setting. As acupuncture is rarely given alone, we intentionally incorporated detailed aspects of acupuncture practice to better reflect acupuncture care in real-world settings.⁵⁶ These included discussions with the participant on the history and nature of the chronic pain conditions as well as hands-on assessment and intervention. Approaches included palpation of regions of interest relative to the site of a chronic pain presentation, assessing for temperature, pain, responsivity to touch, Tui na to invigorate the body surface and regions of pain and Gua sha for inflammation and persistent or recurring focal pain. Acupuncture point options, and associated ah shi or trigger points, could be needled to obtain the de qi response with acupuncture needles then allowed to rest in place if elected. The strength of the de qi response was gauged to the patient. Either needle treatment or pressure (with affixed Vaccaria seeds) in the ear was included. A consensus-built pool of commonly used traditional acupuncture points was included for the study’s specific chronic pain problems.

A yoga therapy manualization process was then modeled on the above experience and resulted in a consensus approach for chronic pain in an underserved multi-morbid patient population. Yoga positions (asanas) from the low back pain trial⁵⁹ were incorporated along with specific asanas used in trials for upper body/neck pain⁶⁰ and for knee Osteoarthritis pain trials.⁶¹ To balance the individualization of yoga practices that yoga therapy provides, along with the structure necessary for research study and replication, a list of practices most relevant to the clinical population was agreed on. Yoga therapists were free to choose from the selected practices within each category, beginning with breathing practices and adding new practices at each session as appropriate to the patients’ readiness and ability. These included multiple options in the following categories of stabilizing poses, mobilizing poses, breathing practices, relaxation, mental practices, and applied philosophy. All practices could be tailored for individual needs and preferences, using yoga props (blocks, straps, pillows), chairs, the wall, or general pose variations. During and at the end of the study we encouraged participants to utilize the practices learned as part of self-management options on their own and in the context of accessing a public yoga class and modifying practices to their own needs.

Outcome Measures

Feasibility was assessed by examining study participant recruitment, compliance of the participants with the treatment protocol, and completion of pain outcome instruments. This was measured through referrals to the study from providers, acupuncture and yoga treatment attendance, and percentage assessments completed. Threshold for referrals was identified as 300; acupuncture session average threshold was 5 sessions, and yoga session average threshold was identified at 3. Minimum percentage survey completion 10-week interview (our primary outcome) was 80%. Minimum thresholds were based on our previous studies.^36,37,39 The assessment of the pain outcome measures was also an integral part of our study design to assess feasibility for a larger pragmatic effectiveness trial. The primary treatment outcome was pain interference and pain intensity. Secondary treatment outcomes were pain free days, depression, functional status, patient activation, and pain medication utilization.²¹ These measures were used during the pre-intervention phase, during which patients were receiving usual care only, and compared to the period after patients receive the combined acupuncture and yoga sessions. Data was collected for 10 days before acupuncture and yoga therapy, at 10 weeks post intervention initiation and at 24 weeks following the end of treatment (see Table 1, below). The multiple pre-measurement points over a ten-day run-in period allowed us to document and monitor any variable patterns of pain pre-intervention. Patient socio-demographic measures were taken at enrollment. Adverse events associated with acupuncture treatment and/or yoga sessions, both serious (deaths, illnesses leading to hospitalization) and minor, were tracked and reviewed monthly. Any serious adverse effects were to be reviewed by a designated Data Safety Monitor.

Primary Treatment Outcome Measures

The Brief Pain Inventory: Short Form (BPI)⁶² provided the BPI subscales on Pain Severity and Pain Interference. The BPI is a nine-item measure that asks patients to indicate how their pain influences function: select aspects of their everyday life including mood, walking, sleep and their ability to work over the past 24 hours, as well as the level and intensity of pain. The measure was adapted for use by phone, modifying a question which asks participants to refer to a diagram of the body. Intervention responsiveness was defined as a 30% or greater improvement from baseline in pain interference and pain severity in keeping with a commonly used threshold for clinically important change in pain.⁶³

Secondary Treatment Outcome Measures

Secondary measures included Center for Epidemiological Studies - Depression Scale (CES-D),⁶⁴ the Altarum Consumer Engagement (ACE),⁶⁵ the Patient-Reported Outcomes Measurement Information System (PROMIS 10) measure,^66,67 the PROMIS Satisfaction with Participation in Social Roles measure,⁶⁸ the Patient Global Impression of Change (PGIC),⁶⁹ and medication utilization. Pain Free Days was also collected, which is a self-report measure that has been used in previous pain research,³⁷ indicating the number of pain free days in the previous 2 weeks.

Data Analysis

Sample Size Calculations

Using an alpha of .05, two tailed, a power of .80, and a clinically meaningful change defined as a 30% improvement on the BPI, we estimated that we needed approximately 134 participants treated through the GAPYOGA trial. To account for attrition we based our assessment on the AADDOPT-2 trial where we had complete data at 12 weeks for 84% of the participants. Therefore, for this trial we planned to recruit 150 participants. That said, on March 13, 2020, due to the onset of the COVID pandemic, recruitment and intervention for our study was terminated, with an enrollment of 93; as a result, our power was reduced to 74%.

Quantitative Analysis

Prior to any univariate or multivariate analysis, all data was reviewed to make sure values were in range and that outliers did not reflect typographical or data entry problems. Missing responses for any index measure were imputed using the average values from completed measures within the same index. Data was described univariate at baseline, 10 weeks and 24 weeks to assess loss to follow-up. Each primary and secondary outcome, pain interference and pain severity, were examined separately using repeated measures t-tests.

Results

Retention, Feasibility and Engagement

There were 467 referrals received from medical providers over the recruitment period. Of these, 93 patients were determined to be eligible and completed the baseline interview. Seventy-eight (84%) of the 93 participants completed the intervention and 10-week interview. Fifty-eight (62% from baseline) completed the 24-week post intervention survey. There was no statistically significant difference in socio-demographics between those participants lost to follow up (either through attrition or the impact of the pandemic) vs study completers (data not shown). Participants received an average number of 6.5 acupuncture sessions (out of a possible 10), and 4 yoga sessions (out of a possible 8) over the 10-week intervention. At baseline, participants’ average age was 54 years, with the majority of participants being female (Table 2). A significant proportion (47.3%) of participants identified as Black or Hispanic (44.1%). Medicaid insurance coverage represented the majority of participants (63.4%), identifying the research population as underserved. At Week-10 and Week-24 the proportions remain similar despite the loss of participants. Participants had high acceptance as reported in our previous trial and our qualitative study.^39,43 While not measured by survey, referring primary care providers reported enthusiasm that was also reflected in their level of referral.

Table 2.

Demographics at Baseline, 10 Weeks and 24 Weeks.

		Baseline		Week 10		Week 24
DEMOGRAPHICS		N	%	N	%	N	%
	Total	93	100.0	78	100.0	58	100.0
Age	(Average, range)	54.0	29-85	53.5	29-85	54.8	29-85
Patient Sex	Male	16	17.2	13	16.7	10	17.2
Patient Sex	Female	77	82.8	65	83.3	48	82.8
Born in US	No	43	46.2	35	44.9	24	41.4
	Yes	48	51.6	41	52.6	32	55.2
	Missing	2	2.2	2	2.6	2	3.4
Marital Status	Married/Living with Partner	30	32.3	25	32.1	20	34.4
	Divorced/Separated	12	13.0	9	11.6	5	8.6
	Widowed	9	9.7	7	9.0	5	8.6
	Single/Never Married	36	38.7	31	39.8	23	39.6
	Missing	6	6.5	6	7.7	5	8.6
Race/Ethnicity	Non- Hispanic Black/African American	40	43.0	33	42.3	25	43.1
	Hispanic	41	44.1	34	43.6	23	39.7
	Non- Hispanic White	4	4.3	3	3.8	3	5.2
	Non- Hispanic Multiracial	2	2.2	2	2.6	2	3.4
	Non- Hispanic Other	5	5.4	5	6.4	4	6.9
	Missing	1	1.1	1	1.3	1	1.7
Work Status	Work Full-time	17	18.3	14	17.9	10	17.2
	Working Part-time	8	8.6	6	7.7	4	6.9
	Unemployed	20	21.5	18	23.1	13	22.4
	Retired	19	20.4	16	20.5	14	24.1
	Unable to work due to disability	22	23.7	18	23.1	12	20.7
	Homemaker	3	3.2	2	2.6	1	1.7
	Other	3	3.2	3	3.8	3	5.2
	Missing	1	1.1	1	1.3	1	1.7
Insurance	Medicaid (Managed Care)	59	63.4	48	61.5	35	60.3
	Private (including HMO)	16	17.2	14	17.9	13	22.4
	No Insurance/Self Pay	1	1.1	1	1.3	1	1.7
	Other	10	10.8	9	11.5	6	10.3
	Missing	7	7.5	6	7.7	3	5.2
Income	Less than $20,000	22	23.7	18	23.1	12	20.7
	$20,000-$39,000	9	9.7	8	10.3	5	8.6
	$40,000 or more	5	5.4	5	7.6	4	6.9
	Don’t Know/Refused	52	55.9	42	53.8	32	55.2
	Missing	5	5.4	5	6.4	5	8.6

Primary Outcomes

Both pain interference and pain severity show statistically significant improvement from baseline to 10-weeks (mean difference -2.1, P < .01) and baseline to 24-weeks (mean difference −1.62, P < .01) (Figure 1). Improvement was strongest from baseline to 10-weeks, at the close of the intervention. While improvement diminished somewhat at 24-week post intervention, significant improvement was maintained. In regard to pain responsiveness, using a 30% decrease as the benchmark for clinically meaningful improvement, at the 10-week end of intervention period, over one-third of the patients showed a 30% or greater improvement in both pain interference (37.5%) and pain severity (38.7%) (See Figure 2). At the 24-week follow-up, this proportion decreased to 25% for pain interference and 16% for pain severity.

Figure 1.

Mean Changes in Primary and Secondary Outcomes. * Significant improvement baseline to midpoint (10 weeks) and baseline to endpoint (24 weeks); ** Significant improvement baseline to midpoint; *** Significant baseline to endpoint (all assessed at P < .05).

Figure 2.

Proportion of Patients improving by 30% or Better.

Secondary Outcomes

Pain-free days and general pain showed a similar pattern of greatest improvement at 10-weeks with a reduction of effect at 24-weeks but still maintenance of significant improvement from baseline. Average pain and patient impression of physical change (PCIG Physical) showed significant improvement from baseline to the 10-week term of the intervention. However, by 24-weeks these improvements nullified. The patient impression of mental change (PCIG Mental), and engagement (ACE inform) showed no significant improvement at 10-weeks, but at the 24-week follow-up these showed significant improvements from baseline. The PROMIS 10 and PROMIS Satisfaction showed no significant change over the study period.

Adverse Events

There were no serious events during the course of this study. There were four minor adverse events: dizziness during an acupuncture session, retained needle from acupuncture (but discovered by yoga therapist), worsening of leg pain attributed to acupuncture by the participant, and dizziness and weakness after intervention attributed by participant to intervention. Our experience is in line with the safety literature for acupuncture therapy^70,71 and yoga.^72,73

Discussion

Combining and individualizing beneficial therapies as part of chronic pain rehabilitation is an established feature of comprehensive, multidisciplinary pain care.^74-76 In our study we aimed to leverage the decreased pain from acupuncture and then build upon it together with yoga therapy - addressing not only the chronic pain but also initial barriers to movement that chronic pain can entail. Bundling acupuncture therapy and yoga therapy is feasible for underrepresented population with chronic pain in urban community health centers with indications of acceptability and benefit to participants. Participants completed an average of 6.5 acupuncture and 4 yoga sessions, above our average thresholds, again demonstrating engagement and commitment to the intervention in this patient population. Further supporting feasibility in primary care, we found that providers at these safety net health centers readily referred patients with chronic pain for this intervention. Referrals to the study numbered 467, above our threshold. Significant improvement was seen in participants’ pain score at the end of the intervention, especially in pain interference. In addition, though the degree of pain improvement declined at 24 weeks, there was only minimal ‘decay’, with statistical significance continuing for pain interference and pain free days. This improvement was evident despite the likelihood that the challenges of the pandemic in NYC—including social isolation, increased anxiety and financial strain—might have been expected to have a negative impact on outcomes.

A qualitative study done with these participants also confirmed feasibility and acceptability.⁴³ Participants reported through the study that the combined therapy helped their pain as well as improved their psychological well-being. Additional themes that emerged as well included a transformative healing process and the importance of the relationship with the acupuncture and yoga providers.

Comprehensive pain care strategies include not only implementation of effective modalities¹⁷ but aim to maximize their benefit through optimal combinations and bundling of care.^77,78 Yoga and acupuncture therapy have inherent similarities. Both provide patients with counsel on self-care, e.g., breathing techniques to mitigate pain. Acupuncture therapy encourages participant movement as part of chronic pain recovery; Yoga not only offers specific movement strategies for pain management, but also utilizes non-movement practices that can impact pain processing, emotions associated with pain, and the impact of pain on daily life e.g., breathing, visualization techniques, and relaxation practices.

The distinction between a yoga class and yoga therapy is important to highlight here. Yoga therapy involved selection of specific movements important for strengthening and improving range of motion, maintaining gains, and promoting participant activation. Yoga therapy tailors movement to each person’s needs which can be delivered individually or in small groups and designed to accommodate differently-abled participants who may not feel safe or included in a general yoga class. The therapeutic application of yoga practices is not new, but clarification of yoga therapy as a healthcare profession in Western contexts has emerged in recent decades.

Yoga therapy instructors have advanced training to work with clinical populations and are most likely to provide safe and appropriate yoga practices for patients with chronic pain. While their services may initially be more expensive than standard yoga classes, the practices can then be continued independently at home. Our yoga intervention was not intended to create a reliance on expensive yoga services, but to empower patients to use yoga in daily life as self-care. There was an emphasis on incorporating small simple practices into existing routines. Prior research has demonstrated long-term effects from a yoga intervention beyond the treatment period when development of home practice is emphasized.⁶¹ Yoga therapy is spreading rapidly worldwide and promises to add significantly to the long-term picture for comprehensive pain care.

Limitations

Factors associated with implementing bundled therapies at health centers that care for underserved populations are discussed in a separate paper.⁷⁹ Study challenges included difficulty reaching lower socioeconomic participants by telephone during recruitment, and site coordination to integrate group acupuncture with yoga therapy. In addition, the COVID pandemic caused the study to be stopped early before full recruitment could be achieved. The pandemic may also have added to difficulties in obtaining follow-up assessments at 24-weeks because of the overall disruption in participants’ lives during that time. Our study did not have a control group and so we could not assess whether the addition of yoga to acupuncture definitively added to treatment effect, i.e., provided synergy. Lastly, we were not able to assess feasibility associated with cost. However, even in the face of these limitations we did see significant improvement in our primary outcome measures. Lessons learned bundling care will help to inform the design and methodology of a larger pragmatic trial.

Conclusion

Access to nonpharmacologic pain care is minimal in low-resource medical centers; medical providers are seeking effective pain care options in light of the opioid crisis and welcome the ability to refer for effective nonpharmacologic options. Group acupuncture therapy was effective in our previous trial and was highly acceptable to patient participants as well as medical providers who referred patients.³⁹ Yoga therapy is also beneficial for chronic pain. Combining these therapies also appeared to be highly acceptable to both referring providers and patient participants. In addition, bundling these therapies at a participant’s primary care site may offer significant access benefit for patients. Further study in combining acupuncture and yoga therapy for chronic pain is warranted.

The data that support the findings of this study are available from the corresponding author upon reasonable request.

Footnotes

Author Contributions

AN, BJA, MDM, BK were investigators on AADDOPT-2 group acupuncture trial and part of original GAPYOGA design team; DMM was a research acupuncturist for AADDOPT-2; RT, SM, EW, DMM and MM joined team in transition to the pilot. SM worked with yoga team to develop yoga therapy manual. PM joined team as site co-PI for Montefiore. AN, RT, BK are responsible for article draft; each author reviewed and edited article.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Private Philanthropy. Funders have no role in study design or data collection, management, analysis, or interpretation, writing of report or any study activities.

Ethical Statement

Clinicaltrial

Clinicaltrials.gov # NCT04296344.

ORCID iDs

Raymond Teets

Arya Nielsen

Steffany Moonaz

Belinda J Anderson

Eve Walter

Mirta Milanes

Benjamin Kligler

References

Institute of Medicine, Committee on Advancing Pain Research, Care and Education . Relieving Pain in America: A Blueprint for Transforming Prevention, Care, Education, and Research. The National Academies Collection: Reports Funded by National Institutes of Health. Washington, DC: National Academies Press (US); 2011.

Blackwell

Villarroel

da Silva

JBG

DATED . Tables of Summary Health Statistics for U.S. Adults: National Health Interview Survey. National Center for Health Statisitics; Table A-5a-c, pages 1-9. National center for health statistics; 2016. https://ftp.cdc.gov/pub/Health_Statistics/NCHS/NHIS/SHS/2016_SHS_Table_A-5.pdf. Accessed December 17, 2022.

Dahlhamer

Lucas

Zelaya

, et al. Prevalence of chronic pain and high-impact chronic pain among adults - United States, 2016. MMWR Morb Mortal Wkly Rep. 2018;67(36):1001-1006. doi:10.15585/mmwr.mm6736a2

Zimmer

Zajacova

. Persistent, consistent, and extensive: The trend of increasing pain prevalence in older Americans. J Gerontol B Psychol Sci Soc Sci. 2020;75(2):436-447. doi:10.1093/geronb/gbx162

Gerrits

van Oppen

Leone

van Marwijk

van der Horst

Penninx

. Pain, not chronic disease, is associated with the recurrence of depressive and anxiety disorders. BMC Psychiatr. 2014;14:187. doi:10.1186/1471-244x-14-187

Vos

Abajobir

Abate

, et al. Global, regional, and national incidence, prevalence, and years lived with disability for 310 diseases and injuries, 1990-2015: A systematic analysis for the global burden of disease study 2015. Lancet. 2016;388(10053):1545-1602. doi:10.1016/S0140-6736(16)31678-6.

Global, regional, and national incidence, prevalence, and years lived with disability for 354 diseases and injuries for 195 countries and territories, 1990-2017: A systematic analysis for the global burden of disease study 2017. Lancet. 2018;392(10159):1789-1858. doi:10.1016/s0140-6736(18)32279-7

Goetzel

Hawkins

Ozminkowski

Wang

. The health and productivity cost burden of the “top 10” physical and mental health conditions affecting six large U.S. employers in 1999. J Occup Environ Med. 2003;45(1):5-14.

Allen

Hines

Pazdernik

Konecny

Breitenbach

. Four-year review of presenteeism data among employees of a large United States health care system: A retrospective prevalence study. Hum Resour Health. 2018;16(1):59. doi:10.1186/s12960-018-0321-9

10.

Blackwell

Lucas

Clarke

. Summary health statistics for U.S. Adults: national health interview survey, 2012. Vital health stat. 2014;10(260). https://www.cdc.gov/nchs/data/series/sr_10/sr10_260.pdf. Accessed December 17, 2022.

11.

Johannes

Zhou

Johnston

Dworkin

. The prevalence of chronic pain in United States adults: Results of an Internet-based survey. J Pain. 2010;11(11):1230-1239. doi:10.1016/j.jpain.2010.07.002.

12.

Nguyen

Ugarte

Fuller

Haas

Portenoy

. Access to care for chronic pain: Racial and ethnic differences. J Pain. 2005;6(5):301-314. doi:10.1016/j.jpain.2004.12.008

13.

Todd

Deaton

D’Adamo

Goe

. Ethnicity and analgesic practice. Ann Emerg Med. 2000;35(1):11-16.

14.

Tait

Chibnall

. Racial/ethnic disparities in the assessment and treatment of pain: Psychosocial perspectives. Am Psychol. 2014;69(2):131-141. doi:10.1037/a0035204

15.

Anderson

Green

Payne

. Racial and ethnic disparities in pain: Causes and consequences of unequal care. J Pain. 2009;10(12):1187-1204. doi:10.1016/j.jpain.2009.10.002

16.

Mossey

. Defining racial and ethnic disparities in pain management. Clin Orthop Relat Res. 2011;469(7):1859-1870. doi:10.1007/s11999-011-1770-9

17.

Tick

Nielsen

Pelletier

, et al. Evidence-based nonpharmacologic strategies for comprehensive pain care: The consortium pain task force white paper. Explore. 2018;14(3):177-211. doi:10.1016/j.explore.2018.02.001

18.

Ahmad

Rossen

Sutton

. Provisional drug overdose death counts. Centers for disease control and prevention; national center for health statistics. National center for health statistics. https://www.cdc.gov/nchs/nvss/vsrr/drug-overdose-data.htm. Accessed February 15, 2023.

19.

Altekruse

Cosgrove

Altekruse

Jenkins

Blanco

. Socioeconomic risk factors for fatal opioid overdoses in the United States: Findings from the mortality disparities in American communities study (MDAC). PLoS One. 2020;15(1):e0227966. doi:10.1371/journal.pone.0227966

20.

Eldridge

Chan

Campbell

, et al. CONSORT 2010 statement: Extension to randomised pilot and feasibility trials. BMJ. 2016;355:i5239. doi:10.1136/bmj.i5239

21.

Nielsen

Teets

Moonaz

, et al. Group acupuncture therapy with yoga therapy for chronic neck, low back, and osteoarthritis pain in safety net setting for an underserved population: Design and rationale for a feasibility pilot. Glob Adv Health Med. 2020;9:2164956120964716. doi:10.1177/2164956120964716

22.

Yuan

Wang

Liu

, et al. Acupuncture for musculoskeletal pain: A meta-analysis and meta-regression of sham-controlled randomized clinical trials. Sci Rep. 2016;6:30675. doi:10.1038/srep30675

23.

Chou

Deyo

Friedly

, et al. Nonpharmacologic therapies for low back pain: A systematic review for an American college of physicians clinical practice guideline. Ann Intern Med. 2017;166(7):493-505. doi:10.7326/m16-2459

24.

Yuan

Guo

Liu

Sun

Zhang

. Traditional Chinese medicine for neck pain and low back pain: A systematic review and meta-analysis. PLoS One. 2015;10(2):e0117146. doi:10.1371/journal.pone.0117146

25.

Vickers

Vertosick

Lewith

, et al. Acupuncture for chronic pain: Update of an individual patient data meta-analysis. J Pain. 2018;19(5):455-474. doi:10.1016/j.jpain.2017.11.005

26.

Kwon

Pittler

Ernst

. Acupuncture for peripheral joint osteoarthritis: A systematic review and meta-analysis. Rheumatology (Oxford). 2006;45(11):1331-1337. doi:10.1093/rheumatology/kel207

27.

Lin

Huang

Zhu

Huang

Qin

Fan

. The effects of acupuncture on chronic knee pain due to osteoarthritis: A meta-analysis. J Bone Joint Surg Am. 2016;98(18):1578-1585. doi:10.2106/JBJS.15.00620

28.

Woods

Manca

Weatherly

, et al. Cost-effectiveness of adjunct non-pharmacological interventions for osteoarthritis of the knee. PLoS One. 2017;12(3):e0172749. doi:10.1371/journal.pone.0172749

29.

Witt

Vertosick

Foster

, et al. The effect of patient characteristics on acupuncture treatment outcomes: An individual patient data meta-analysis of 20,827 chronic pain patients in randomized controlled trials. Clin J Pain. 2019;35(5):428-434. doi:10.1097/ajp.0000000000000691

30.

Skelly

Chou

Dettori

, et al. Noninvasive nonpharmacological treatment for chronic pain: A systematic review update. Comparative Effectiveness Review No. 227. AHRQ Publication No. 20-EHC009. Rockville, MD: Agency for Healthcare Research and Quality; 2020. https://effectivehealthcare.ahrq.gov/products/noninvasive-nonpharm-pain-update/research. Accessed March 15, 2023.

31.

U.S. Food and Drug Administration . FDA Education Blueprint for Health Care Providers Involved in the Management or Support of Patients with Pain; 2017. https://www.fda.gov/drugs/news-events-human-drugs/fda-education-blueprint-health-care-providers-involved-management-or-support-patients-pain. Accessed June 15 2022.

32.

Nahin

Boineau

Khalsa

Stussman

Weber

. Evidence-based evaluation of complementary health approaches for pain management in the United States. Mayo Clin Proc. 2016;91(9):1292-1306. doi:10.1016/j.mayocp.2016.06.007

33.

The joint commission R3 report: requirement, rational, reference. Pain assessment and management standards for hospitals. The joint commission. https://www.jointcommission.org/-/media/tjc/documents/standards/r3-reports/r3_report_issue_11_2_11_19_rev.pdf. Accessed March 14, 2023.

34.

Kazis

Ameli

Rothendler

, et al. Observational retrospective study of the association of initial healthcare provider for new-onset low back pain with early and long-term opioid use. BMJ Open. 2019;9(9):e028633. doi:10.1136/bmjopen-2018-028633

35.

Meerwijk

Larson

Schmidt

, et al. Nonpharmacological treatment of army service members with chronic pain is associated with fewer adverse outcomes after transition to the veterans health administration. J Gen Intern Med. 2019;35(3):775-783. doi:10.1007/s11606-019-05450-4

36.

McKee

Kligler

Fletcher

, et al. Outcomes of acupuncture for chronic pain in urban primary care. J Am Board Fam Med. 2013;26(6):692-700. doi:10.3122/jabfm.2013.06.130003

37.

Kligler

Nielsen

Kohrrer

, et al. Acupuncture therapy in a group setting for chronic pain. Pain Med. 2018;19(2):393-403. doi:10.1093/pm/pnx134

38.

Bonakdar

Palanker

Sweeney

. Analysis of state insurance coverage for nonpharmacologic treatment of low back pain as recommended by the American college of physicians guidelines. Glob Adv Health Med. 2019;8:2164956119855629. doi:10.1177/2164956119855629.

39.

McKee

Nielsen

Anderson

, et al. Individual vs. Group delivery of acupuncture therapy for chronic musculoskeletal pain in urban primary care—a randomized trial. J Gen Intern Med. 2020;35(4):1227-1237. doi:10.1007/s11606-019-05583-6

40.

White

Richardson

Richmond

Freedman

Bevis

. Group acupuncture for knee pain: Evaluation of a cost-saving initiative in the health service. Acupunct Med. 2012;30(3):170-175. doi:10.1136/acupmed-2012-010151

41.

Kligler

Buonora

Gabison

Jacobs

Karasz

McKee

. “I Felt Like It Was God’s Hands Putting The Needles In”: A qualitative analysis of the experience of acupuncture for chronic pain in a low-income, ethnically diverse, and medically underserved patient population. J Altern Complement Med. 2015;21(11):713-719. doi:10.1089/acm.2014.0376

42.

Chuang

Hashai

Buonora

Gabison

Kligler

McKee

. “It’s Better in a Group Anyway”: Patient experiences of group and individual acupuncture. J Altern Complement Med. 2018;24(4):336-342. doi:10.1089/acm.2017.0262

43.

Jyung

Mah

Moonaz

, et al. “The pain left, I was off and running”: A qualitative analysis of group acupuncture and yoga therapy for chronic pain in a low-income and ethnically diverse population. J Integr Complement Med. 2022;28(4):328-338. doi:10.1089/jicm.2021.0192

44.

Santiago Perez

Pola

Crowe

Van Puymbroeck

Schmid

. The emerging healthcare professional: Certified yoga therapist (C-IAYT). Complement Ther Clin Pract. 2020;39:101147. doi:10.1016/j.ctcp.2020.101147

45.

Kepner

Devi

Le Page

Kraftsow

Lee

. The differences between yoga teacher training and yoga therapist training and the distinction between yoga teaching and yoga therapy. Int J Yoga Therap. 2014;24:7-21.

46.

Wieland

Skoetz

Pilkington

Vempati

D’Adamo

Berman

. Yoga treatment for chronic non-specific low back pain. Cochrane Database Syst Rev. 2017;2017(1):CD010671. doi:10.1002/14651858.CD010671.pub2

47.

Bussing

Ostermann

Ludtke

Michalsen

. Effects of yoga interventions on pain and pain-associated disability: A meta-analysis. J Pain. 2012;13(1):1-9. doi:10.1016/j.jpain.2011.10.001

48.

Jiang

Yuan

. Effects of yoga on patients with chronic nonspecific neck pain: A PRISMA systematic review and meta-analysis. Medicine (Baltimore). 2019;98(8):e14649. doi:10.1097/md.0000000000014649

49.

Ward

Stebbings

Cherkin

Baxter

. Yoga for functional ability, pain and psychosocial outcomes in musculoskeletal conditions: A systematic review and meta-analysis. Muscoskel Care. 2013;11(4):203-217. doi:10.1002/msc.1042

50.

Kan

Zhang

Yang

Wang

. The effects of yoga on pain, mobility, and quality of life in patients with knee osteoarthritis: A systematic review. Evid Based Complement Alternat Med. 2016;2016:6016532. doi:10.1155/2016/6016532

51.

Moscoso

Goese

Van Hyfte

, et al. The impact of yoga in medically underserved populations: A mixed-methods study. Complement Ther Med. 2019;43:201-207. doi:10.1016/j.ctim.2019.02.005

52.

Saper

Lemaster

Delitto

, et al. Yoga, physical therapy, or education for chronic low back pain: A randomized noninferiority trial. Ann Intern Med. 2017;167(2):85-94. doi:10.7326/m16-2579

53.

Qaseem

Wilt

McLean

Forciea

. Noninvasive treatments for acute, subacute, and chronic low back pain: A clinical practice guideline from the American college of physicians. Ann Intern Med. 2017;166(7):514-530. doi:10.7326/M16-2367

54.

Geneen

Moore

Clarke

Martin

Colvin

Smith

. Physical activity and exercise for chronic pain in adults: An overview of cochrane reviews. Cochrane Database Syst Rev 2017;24(4):CD011279. doi:10.1002/14651858.CD011279.pub3

55.

Phuphanich

Droessler

Altman

Eapen

. Movement-based therapies in rehabilitation. Phys Med Rehabil Clin N Am. 2020;31(4):577-591. doi:10.1016/j.pmr.2020.07.002

56.

Nielsen

Anderson

Citkovitz

, et al. Developing and employing a ‘responsive manualization’ in the ‘cupuncture approaches to decrease disparities in outcomes of pain treatment’ comparative effectiveness study. Acupunct Med. 2019;37(3):184-191. doi:10.1177/0964528419834015

57.

MacPherson

Schroer

. Acupuncture as a complex intervention for depression: A consensus method to develop a standardised treatment protocol for a randomised controlled trial. Complement Ther Med. 2007;15(2):92-100. doi:10.1016/j.ctim.2006.09.006

58.

Nielsen

Ocker

Majd

, et al. Acupuncture intervention protocol: Consensus process for a pragmatic randomized controlled trial of acupuncture for management of chronic low back pain in older adults, an NIH HEAL initiative funded project. Glob Adv Health Med. 2021;10:2164956121100709. doi:10.1177/21649561211007091

59.

Saper

Sherman

Delitto

, et al. Yoga teacher training Manual. Supplement to: yoga vs. physical therapy vs. education for chronic low back pain in predominantly minority populations: Study protocol for a randomized controlled trial. Trials. 2014;15:67. doi:10.1186/1745-6215-15-67

60.

Cramer

Lauche

Hohmann

, et al. Randomized-controlled trial comparing yoga and home-based exercise for chronic neck pain. Clin J Pain. 2013;29(3):216-223. doi:10.1097/AJP.0b013e318251026c

61.

Moonaz

Bingham

3rd Wissow

Bartlett

. Yoga in sedentary adults with arthritis: Effects of a randomized controlled pragmatic trial. J Rheumatol. 2015;42(7):1194-1202. doi:10.3899/jrheum.141129

62.

Tan

Jensen

Thornby

Shanti

. Validation of the brief pain inventory for chronic nonmalignant pain. J pain. 2004;5(2):133-137. doi:10.1016/j.jpain.2003.12.005

63.

Olsen

Bjerre

Hansen

Tendal

Hilden

Hrobjartsson

. Minimum clinically important differences in chronic pain vary considerably by baseline pain and methodological factors: Systematic review of empirical studies. J Clin Epidemiol. 2018;101:87-106.e2. doi:10.1016/j.jclinepi.2018.05.007

64.

Radloff

. The CES-D scale. Appl Psychol Meas. 1977;1(3):385-401. doi:10.1177/014662167700100306

65.

Duke

Lynch

Smith

Winstanley

. Validity of a new patient engagement measure: The Altarum consumer engagement (ACE) measure. Patient. 2015;8(6):559-568. doi:10.1007/s40271-015-0131-2

66.

Hays

Bjorner

Revicki

Spritzer

Cella

. Development of physical and mental health summary scores from the patient-reported outcomes measurement information system (PROMIS) global items. Qual Life Res. 2009;18(7):873-880. doi:10.1007/s11136-009-9496-9

67.

Pain interference: A brief guide to the PROMIS pain interference instruments. Paitent-reported outcomes measurement information system (PROMIS); healthmeasures.net. http://www.healthmeasures.net/images/PROMIS/manuals/PROMIS_Pain_Interference_Scoring_Manual.pdf. Accessed December 18, 2022.

68.

Hahn

Beaumont

Pilkonis

, et al. The PROMIS satisfaction with social participation measures demonstrated responsiveness in diverse clinical populations. J Clin Epidemiol. 2016;73:135-141. doi:10.1016/j.jclinepi.2015.08.034

69.

Dworkin

Turk

Wyrwich

, et al. Interpreting the clinical importance of treatment outcomes in chronic pain clinical trials: IMMPACT recommendations. J Pain. 2008;9(2):105-121. doi:10.1016/j.jpain.2007.09.005

70.

NIH consensus conference. Acupuncture. JAMA. 1998;280(17):1518-1524.

71.

MacPherson

Thomas

Walters

Fitter

. A prospective survey of adverse events and treatment reactions following 34,000 consultations with professional acupuncturists. Acupunct Med. 2001;19(2):93-102.

72.

Cramer

Ward

Saper

Fishbein

Dobos

Lauche

. The safety of yoga: A systematic review and meta-analysis of randomized controlled trials. Am J Epidemiol. 2015;182(4):281-293. doi:10.1093/aje/kwv071

73.

Cramer

Ostermann

Dobos

. Injuries and other adverse events associated with yoga practice: A systematic review of epidemiological studies. J Sci Med Sport. 2018;21(2):147-154. doi:10.1016/j.jsams.2017.08.026

74.

Bonica

. Management of Pain. Philadelphia, PA: Lippincott William and Wilkins; 1953.

75.

Bonica

Loeser

. Chapter 9: Multidisciplinary/interdisciplinary pain programs. In: Loeser

Chapman

Fordyce

, eds. The Management of Pain Vol 1. Philadelphia, PA: Lea and Febiger; 1990:197-210.

76.

Fishman

Ballantyne

Rathmell

Bonica

. Bonica’s Management of Pain. 4th ed. Philadelphia, PA: Lippincott, Williams and Wilkins; 2010. http://site.ebrary.com/id/10822842

77.

O’Connor

Eisenberg

Buring

, et al. Within-team patterns of communication and referral in multimodal treatment of chronic low back pain patients by an integrative care team. Glob Adv Health Med. Mar. 2015;4(2):36-45. doi:10.7453/gahmj.2014.076

78.

Mahrer

Gold

Luu

Herman

. A cost-analysis of an interdisciplinary pediatric chronic pain clinic. J Pain. 2018;19(2):158-165. doi:10.1016/j.jpain.2017.09.008

79.

Anderson

Meissner

Mah

, et al. Barriers and facilitators to implementing bundled acupuncture and yoga therapy to treat chronic pain in community healthcare settings: A feasibility pilot. J Altern Complement Med. 2021;27(6):496-505. doi:10.1089/acm.2020.0394

Group Acupuncture Therapy With Yoga Therapy for Chronic Neck,Low Back,and Osteoarthritis Pain in Safety Net Settings for an Underserved Population: A Feasibility Pilot Study

Abstract

Background

Objective

Methods

Results

Conclusions

Keywords

Background

Methods

Study Design

Study Participants and Eligibility

Recruitment

Intervention

Usual Care

Acupuncture Therapy Component

Yoga Therapy Component

Therapy Manualizations

Outcome Measures

Primary Treatment Outcome Measures

Secondary Treatment Outcome Measures

Data Analysis

Sample Size Calculations

Quantitative Analysis

Results

Retention, Feasibility and Engagement

Primary Outcomes

Secondary Outcomes

Adverse Events

Discussion

Limitations

Conclusion

Footnotes

Author Contributions

Declaration of Conflicting Interests

Funding

Ethical Statement

Clinicaltrial

ORCID iDs

References