Sage Journals: Discover world-class research

Abstract

Background

Assessing the use and effectiveness of complementary and integrative health (CIH) therapies via survey can be complicated given CIH therapies are used in various locations and formats, the dosing required to have an effect is unclear, the potential health and well-being outcomes are many, and describing CIH therapies can be challenging. Few surveys assessing CIH therapy use and effectiveness exist, and none sufficiently reflect these complexities.

Objective

In a large-scale Veterans Health Administration (VA) quality improvement effort, we developed the “Complementary and Integrative Health Therapy Patient Experience Survey”, a longitudinal, electronic patient self-administered survey to comprehensively assess CIH therapy use and outcomes.

Methods

We obtained guidance from the literature, subject matter experts, and Veteran patients who used CIH therapies in designing the survey. As a validity check, we completed cognitive testing and interviews with those patients. We conducted the survey (March 2021-April 2023), inviting 15,608 Veterans with chronic musculoskeletal pain with a recent CIH appointment or referral identified in VA electronic medical records (EMR) to participate. As a second validity check, we compared VA EMR data and patient self-reports of CIH therapy utilization a month after survey initiation and again at survey conclusion.

Results

The 64-item, electronic survey assesses CIH dosing (amount and timing), delivery format and location, provider location, and payor. It also assesses 7 patient-reported outcomes (pain, global mental health, global physical health, depression, quality of life, stress, and meaning/purpose in life), and 3 potential mediators (perceived health competency, healthcare engagement, and self-efficacy for managing diseases). The survey took 17 minutes on average to complete and had a baseline response rate of 45.3%. We found high degrees of concordance between self-reported and EMR data for all therapies except meditation.

Conclusions

Validly assessing patient-reported CIH therapy use and outcomes is complex, but possible.

Keywords

acupuncture yoga veterans integrative medicine meditation

Introduction

The use of complementary and integrative health (CIH) therapies, such as yoga, meditation/mindfulness, and Tai Chi, has proliferated throughout healthcare systems, businesses, at home, and communities. CIH therapies are available through health systems at in-person classes, electronic applications (apps), and via telehealth formats. Although CIH therapies appear to affect a range of outcomes,^1–13 the field has not coalesced around the dosing (eg, what amount of CIH therapies over what time period) required to have an effect, even more so when CIH therapies are often given in concert with other treatment therapies. Moreover, it is unclear whether patients’ self-reports or electronic appointment and billing data are more accurate depictions of patients’ CIH therapy use. Yet, it can be difficult to clearly write survey stand-alone (not verbally delivered) questions to elicit the details necessary to determine CIH therapy dosing (amount and timing), delivery format, and patient-reported outcomes.

The Veterans Health Administration (VA), the nation’s largest integrated healthcare system, is transforming to a Whole Health System of care.¹⁴ This incorporates CIH therapies as part of Veterans’ routine care in addition to allopathic care, reflecting a major shift from episodic, disease-centered care to engaging and empowering patients throughout their lives to take charge of their life and health. The VA encourages appropriate CIH therapy use not only at the VA, but also at-home, or in the community. As such, there is a need for the VA to assess not only patients’ use of CIH therapies it provides as a part of the Veterans health and well-being plan, both in and outside the VA, but also patients’ use of the therapies at their homes or communities, and outcomes from using those therapies.

Two well-known surveys of CIH therapy use are the “Self Assessment of Change” by Rittenbaugh et al (2011)¹⁵ and the Center for Disease Control and Prevention’s National Health Interview Survey (NHIS).¹⁶ The former assesses patient-reported outcomes with 16 items but does not assess the duration, timing, format, or location of the CIH therapy provided. The NHIS survey assesses CIH therapy use frequency and reasons for use. But does not assess dosing, therapy format and location, and several important CIH-relevant outcomes.

Therefore, we developed the Complementary and Integrative Health Therapy Patient Experience Survey as part of a large-scale quality improvement effort collaboratively conducted by researchers and operational partners to support the ongoing monitoring of CIH therapy provision and effectiveness in the VA. This longitudinal electronic patient survey was designed to comprehensively assess CIH therapy use details (eg, amount, timing, location, and format), and patient-reported health outcomes. This paper describes the survey development process, content, and validation.

Methods

Overview

We developed an electronic longitudinal Complementary and Integrative Health Patient Therapy Experience Survey to assess individuals’ use of 8 CIH therapies ((1) yoga, (2) meditation, (3) mindfulness, (4) Tai Chi, (5) Qigong, (6) acupuncture, (7) therapeutic massage, and (8) chiropractic care) and to examine 7 patient-reported health and wellness outcomes related to that use. The survey instrument was designed to be used among Veterans with chronic musculoskeletal pain but includes other health outcomes that are highly prevalent in the general population, so the instrument can be used for almost all populations. We report below on the steps we conducted from October 2018 to January 2021 to develop the survey, using the Consensus-Based Checklist for Reporting of Survey Studies (CROSS) as guidance.¹⁷ This included first reviewing the literature and engaging the 12-member Advisory Board of a VA-funded CIH therapy research study, which was comprised of nationally-known clinicians and researchers specializing in each of the 8 CIH therapies, to develop the survey. We also conducted cognitive and electronic survey format testing among Veteran patients utilizing CIH therapies in the VA. We then reviewed and iteratively modified the ordering of the survey items with the company administering the survey, given their expertise in electronic survey user-design. We also obtained input on pain measurement items from our colleagues in the NIH-DoD-VA Pain Management Collaboratory.¹⁸ We fielded the survey from March 2021 to April 2023 among a target sample of 15,608 Veterans with chronic musculoskeletal pain. In May 2021, we conducted validity checks by interviewing Veterans whose self-reports of CIH use did not match what we found in the EMR and administrative records (appointment and billing data; “EMR” for short) to determine if additional revisions to the survey wording were needed. These details are provided below. The VA Greater Los Angeles Institutional Review Board determined this to be a quality improvement project, not a research study.

Measuring CIH Therapy Use

We first reviewed the literature to examine how other surveys described CIH therapies, and assessed CIH therapy dosing (frequency and timing), provider locations (community-based or VA-based), formats (eg, app, in-person, telehealth), and payors (self, insurance, VA). We then posed potential survey wording to the subject matter experts on the Advisory Board and study team members (ST, SZ, ARE, KL, AC) interviewed each clinician/researcher pair for each CIH therapy. Experts recommended collapsing Tai Chi and Qigong and collapsing meditation and mindfulness to reduce respondent burden, because although distinctive, they were similar enough in practice and should have similar effects. (Other researchers using this survey can easily opt to ask about these therapies individually.) These experts were also instrumental in determining survey questions related to assessing Veterans’ perceptions of CIH delivery formats and determining an effective “dose” for each therapy (the amount of CIH used over a specific time period that might have an effect). Especially important was determining how to most effectively structure the questions given that the electronic survey would require using complex skip patterns. The subject matter experts also advised on the frequency that the survey should be given to best assess CIH use and its relationship to patient-reported outcomes longitudinally.

Measuring Patient-Reported Outcomes and Potential Mediators

We first reviewed the literature and consulted the same subject matter experts to determine the health and well-being outcomes that might most be affected by CIH therapies, the measures that best assessed those outcomes, and a few potential mediators of the relationship between CIH therapy use and the outcomes. Study team members (ST, SZ, ARE, AC, CD, BL, AK, BB, BK, AW) then used an iterative consensus process to generate a draft list of survey items, prioritizing measures that were frequently used, had higher validity and were brief. Subsequently, we also added a final measure based on the results of the respondent cognitive interviews described below.

Testing and Validating Content

We conducted 4 phases of testing. First, we gave the draft survey to 9 Veteran patients who had received CIH therapies and conducted cognitive interviews. We asked them to identify health and well-being constructs affected by CIH therapies and asked for their feedback on and comprehension of the survey items for clarity and completeness of capturing their experience. When doing so, we used the “think aloud” method of survey validity testing,¹⁹ which asks respondents to read the survey and talk out loud how they interpret it and how they might respond given their interpretation. This enabled us to determine which questions might be misunderstood or be too difficult to answer.

Second, we iteratively tested the phone and computer versions of the draft survey among our project team and 9 additional Veteran patients who had used CIH therapies. The electronic format had skip patterns that reduced respondent burden when asking about details of 8 therapies, but we did not want those skip patterns to be burdensome. As such, we gave the online and phone interview versions of the draft survey to patients and again conducted the “think aloud” cognitive interviews to get their input on skip pattern complexity and survey content, and made additional survey revisions accordingly. The 18 Veteran patients participating in this testing represented the broad range of Veterans in age, gender and race/ethnicity.

The third validity check was conducted a month after we initiated the survey. We compared the percentage of patients reporting CIH therapy use at their VA medical facility vs the percentage of patients having CIH therapy use recorded in the VA’s EMR to determine the degree of discordance, given they ideally should match. We then interviewed 30 of the 45 patients for whom we observed discrepancies to determine the reason for the discrepancies. When the full baseline sample was collected, we conducted that discordance validity check a second time.

Distributing the Survey

We conducted the survey from March 2021 to April 2023 using a contracted survey company. The sampling frame included 15,608 Veterans with chronic musculoskeletal pain identified as newly initiating CIH therapies in the VA EMR. Using the Dillman method,²⁰ the survey company sent both an email invitation and a hard copy mailed invitation. This invitation included a cover letter, a $2 bill incentive, a prepaid return envelope for patients wanting to send completed baseline surveys, and information for opting out from further mailings via a tollfree telephone number or return postcard. The cover letter came from the Executive Director of the VA’s Office of Patient Centered Care and Cultural Transformation (author BK), and described the survey’s purpose and length. It also told them they would receive 4 surveys over 6 months, provided a personalized link to the electronic survey, gave respondents a phone number to call if they preferred conducting the survey over the telephone, and noted that completing the survey was voluntary. Veterans who did not opt out and did not respond within a 6-week period were mailed a second survey with a prepaid envelope and cover letter.

Results

Survey Content- CIH Therapy Use

The results of our multi-pronged process to determine the best way to measure longitudinal CIH therapy use and outcomes from that use with an electronic patient survey resulted in the Complementary and Integrative Health Therapy Patient Experience Survey, containing 64 items and taking on average 17 minutes to complete. Table 1 shows the 6 items used to assess CIH therapy use definitions, formats, provider locations, payors, and use frequencies. It also shows the rationale for the survey wording. Additionally, it includes an item asking about use of 18 other therapies respondents might have used, such as exercise, nerve/steroid injections, and braces.

Table 1.

How the Survey Assesses Use of CIH Therapies - CIH Descriptions, Amount, Timing, Location, Format, Provider Type, Payor, and Survey Timing.

CIH Therapy Descriptions

Massage therapy - “ massage therapy, given by a clinician” (as opposed to partner or spa)

Acupuncture - “full body acupuncture, not ear acupuncture or battlefield acupuncture” (rationale: The latter is commonly offered in the VA.)

Yoga - “yoga for 15 minutes or more, either by using an app, taking an online or in-person class, or doing it on your own after some training, (We are not asking about just stretching or breathwork therapy. We are asking only about yoga).”

Meditation, mindfulness - “meditation, mindfulness or mantram repetition for 15 minutes of more, either by using an app, taking an online or in-person class, or doing it on your own after some training”

Tai chi or Qigong – “Tai chi or Qigong for 15 minutes of more, either by using an app, taking an online or in-person class, or doing it on your own after some training”

Chiropractic care – “chiropractic care”

Amount and timing of CIH therapy used

Survey first asks about any use – “did you do any of the following therapies or activities in the past 4 weeks or are you about to start doing any of them? Response options:

- Yes, I used it in the last 4 weeks

- Yes, I am about to start doing this

- No, I have not used it and am not just about to start using it

Rationale: The survey asks whether or not they are about to start using CIH, so the survey will skip over the question about the number of visits among those who have not yet used CIH.

Survey then asks 2 questions about the # of visits - “for this study we are looking for veterans that vary in their level of experience with <INSERT CIH > or have yet to receive this type of care, but are planning to in the future. Rationale: Concerned respondents would over-report.

- “In the past 4 weeks, about how many times, if any, did you get [CIH THERAPY]

Response options: 1 time, 2 times, 3 times, 4 times, 5 times, it was more than 5 times, not sure, but more than 5 times

- “In 4 weeks before that, about how many times, if any, did you get [CIH THERAPY]

Rationale: We determined a dose should be defined as use in the past 8 weeks, however it might be difficult for many people to remember how many sessions they had over an 8-week timeframe. As such, the survey breaks down that 8-week period and asks about the amount used over a month in 2 questions.

Also asks about “consistent” use: “Thinking about the past 6 months, which best describes how consistently you do “YOGA”, “TAI CHI OR QIGONG”, “MEDITATION or MINDFULNESS”> for 15 minutes or more each time.” response options: Not doing yet, not consistently doing it (A few times a month or less), somewhat consistently doing it (approximately once per week), consistently doing it (doing 2 or more times per week). Rationale: Given we were studying change in outcomes pre- and post-CIH therapy use, we wanted a sample that excluded respondents who had a level of proficiency in that therapy before baseline.

Provider location and CIH therapy format

For each [ACUPUNCTURE, MASSAGE THERAPY, CHIROPRACTIC] visit in the past 4 weeks, where did you get it? At the VA or in the community? Response options:

- I got____ [#] [RANGE: 0-100 FOR EACH] visits with a VA provider,

- I got____ [#] [RANGE: 0-100 FOR EACH] visits with a provider in the community,

- I’m not sure

For each [MEDITATION, MINDFULNESS, YOGA, TAI CHI, QIGONG] class or session you did in the past 4 weeks, where did you do it? Response options:

- I did____ [#] classes or sessions in-person with a VA provider.

- I did____ [#] classes or sessions at home guided by a VA provider.

- I did____ [#] classes or sessions in-person with a community provider.

- I did____ [#] classes or sessions at home guided by a community provider.

- I did____ [#] classes or sessions on my own, using a recording, video, or app.

- I’m not sure

Payor

Who paid for covered the <EACH CIH THERAPY>?

- The VA paid for or covered it.

- Other insurance paid for or covered it.

- I Had to pay for it myself.

- Not sure

Rationale: Cost is more of a barrier for some therapies than others. Understanding how patients are accessing therapies that are not typically covered by insurance is important to understand.

Table 1 also shows the suggested survey timing to capture longitudinal use and outcomes. That timing was related to the dose that the subject matter experts thought might have an effect on pain and pain-related conditions. They agreed that scientists had not yet coalesced around what might be an effective dose. However, when pressed, they said a minimum of five-eight sessions of meditation, yoga, Tai Chi, or Qigong lasting at least 15 minutes over an 8-week period might have an effect. Likewise, a minimum of one-two doses of acupuncture, therapeutic massage or chiropractic care in a 4-week period might have an effect. As such, the baseline survey asks separately about the past 4 weeks and the 4 weeks before that. Breaking up the 8-week time period into 2 time periods also might improve respondent recall, as some respondents might more easily remember what they did in chunks of months, instead of 2 months. The survey wording changed slightly for follow-up surveys, asking about use since the last survey instead of during the past 8 weeks for the 1-month and 3-month surveys.

Survey Content- Outcomes, Mediators and Covariates

Table 2 shows the 48 items used to measure 7 health and well-being outcomes: (1) pain (severity, chronicity, and interference); (2) global mental health; (3) depression; (4) stress; (5) global physical health; (6) quality of life; and (7) purpose in life. In the survey development phase, Veteran patients who had used CIH therapies suggested we add the last construct. The table also shows the 3 potential mediators (1) perceived health competency, (2) patient engagement in their healthcare, and (3) self-efficacy for managing chronic diseases) and 5 items measuring sociodemographic characteristics (1) education, (2) income, (3) employment, (4) housing, and (5) relationship status). The survey does not include a few common sociodemographic measures because they exist in the VA’s EMR (ie, age, gender, race and ethnicity, and rural vs urban residence).

Table 2.

Constructs and Measures Included in the Survey to Assess Patient Reported Outcomes and Potential Mediators.

Construct	Measures
Pain
Pain interference	Brief pain inventory (BPI)²¹ and PEG²² 7 items -“what number best describes how, during the past week, pain has interfered with your….”
	“General activity?
	“Mood?”
	“Walking ability?”
	“Normal work (includes both outside the home and housework)?”
	“Relations with other people?”
	“Sleep?”
	“Enjoyment of life?”
	Response options: 1-10 scale on interfered
	High impact chronic pain (HICP)²³; 1 item - “over the past 3 months, how often did pain limit your life or work activities?”
	Response options: “Never”, “some days”, “most days”, “every day”
	PMC3¹⁸ derived item - “are you currently not working or unable to work due to pain or a pain condition?
	Response options: “Yes, not working/unable to work”; “no, I am working/able to work”
Pain intensity	Brief pain inventory (BPI)²¹ and PEG²² 4 items; 1-10 scale
	“What number best describes….”
	“Your pain at its worst in the last week”.
	“Your pain at its least in the last week”.
	“Your pain on average in the past week”.
	“How much pain you have right now”.
Change in pain	Patients’ global impression of change (PGIC)²³; 2 items
	At baseline: “Since doing < INSERT EACH CIH ACTIVITY > recently, have you seen changes in your pain?”
	Response options: “My pain is much better now”; “my pain is slightly better now;” my pain is about the same now”; “my pain is slightly worse now”; and “my pain is much worse now”.
	At 1, 3, 6 month surveys: “Thinking back over the [time frame] and your use of any of these activities or therapies over this period, have you seen changes in your pain?
	Response options: (Same as above)
Pain chronicity	CDC chronic pain²⁴; 1-item
	“In the past 3 months, how often did you have pain?’
	Response options: “Never”; “at least some days”; “most days”; “every day”
Stress
Stress past month	Perceived stress Scale-4 (PSS-4)²⁵; 4-items
	“In the last month, how often have you felt…
	“ that you were unable to control the important things in your life?”
	“Confident about your ability to handle your personal problems?”
	“That things were going your way?”
	“Difficulties were piling up so high that you could not overcome them?”
	Response options: Never, almost never, sometimes, fairly often, very often
Change in stress	Patients’ global impression of change (PGIC)²³; 2 items
Change in stress	(Same as item measuring change in pain above)
Depression
Depression	Patient health Questionnaire-2 (PHQ2)²⁶; 2 items
	“ over the past 2 weeks, how often have you been bothered by any of the following problems?”
	“Little interest or pleasure in doing things”
	“Feeling down, depressed or hopeless”
	Response options: Not at all, several days, more than half the days, nearly every day
Change in depression	Patients’ global impression of change (PGIC)²³; 2 items
Change in depression	Same as item measuring change in pain above)
Global health
Global health, global mental health, global physical health	PROMIS10²⁷; 10 items:
	“In general, would you say your health is:”
	“In general, would you say your quality of life is:”
	“In general, how would you rate your physical health?”
	“In general, how would you rate your mental health, including your mood and your ability to think?”
	“In general, how would you rate your satisfaction with your social activities and relationships?”
	“In general, please rate how well you carry out your usual social activities and roles. (This includes activities at home, at work and in your community, and responsibilities as a parent, child, spouse, employee, friend, etc.)”
	Response options: Excellent, very good, good, fair, poor
	“To what extent are you able to carry out your everyday physical activities such as walking, climbing stairs, carrying groceries, or moving a chair?”
	Response options: Completely, mostly, moderately, a little, not at all
	“In the past week, how often have you been bothered by emotional problems such as feeling anxious, depressed or irritable?”
	Response options: Never, rarely, sometimes, often, always
	“In the past week, how would you rate your fatigue on average?”
	Response options: None, mild, moderate, severe, very severe
Change in global mental health, global physical health	Patients’ global impression of change (PGIC)²³; 2 items
Change in global mental health, global physical health	(Same as item measuring change in pain above)
Fatigue
Fatigue	PROMIS10²⁷; 1 item (noted above in global health section)
	“In the past week, how would you rate your fatigue on average?”
	Response options: None, mild, moderate, severe, very severe
Change in fatigue	Patients’ global impression of change (PGIC)²³; 2 items
Change in fatigue	(Same as item measuring change in pain above)
Quality of life
Quality of life	PROMIS10²⁷; 1 item (noted above in global health section)
	“In general, would you say your quality of life is:”
	Response options: Excellent, very good, good, fair, poor
Change in quality of life	Patients’ global impression of change (PGIC)²³; 2 items
Change in quality of life	(Same as item measuring change in pain above)
Purpose in life
Purpose in life	Life engagement test (LET)²⁸; 6 items
	“There is not enough purpose in my life.”
	“To me, the things I do are all worthwhile.”
	“Most of what I do seems trivial and unimportant to me.”
	“I value my activities a lot.”
	“I don’t care very much about the things I do.”
	“I have lots of reasons for living.”
	Response options: Strongly agree, agree, neutral, disagree, strongly disagree
Potential mediator – Health competency
Perceived health competency	Perceived health competency scale²⁹; 2 items
	“I am able to do things for my health as well as most other people.”
	“It is difficult for me to find effective solutions to the health problems that come my way.”
	Response options: Strongly agree, agree, neutral, disagree, strongly disagree
Potential mediator – Ownership of One’s own health
Ownership of One’s own health	Altarum consumer engagement measure³⁰; 5 items in 1 subscale
	“My health is my responsibility, not someone else’s.”
	“The most important thing that affects my health is my own actions.”
	“I can help prevent or reduce problems with my health.”
	“I can follow through on home medical treatments.”
	“When I have a question about my health, I find the answer.”
	Response options: “Strongly agree, agree, neither agree nor disagree, disagree, strongly disagree”
Potential mediator – Self-efficacy for managing chronic disease^31,32; 2 of 6 items in the scale
	“How confident do you feel that you can do the different tasks and activities needed to manage your health condition so as to reduce your need to see a doctor?”
	“How confident do you feel that you can do things other than just taking medication to reduce how much your illness affects your everyday life?”
	Response options: On a scale of 1-10, “not at all confident” to “Totally confident”

The full survey can be found in the Appendix.

Testing and Validating Survey Content

In both rounds of testing, the respondents appeared to understand the questions. The only substantial suggestion patients gave in the first round was to add a measure of purpose in life. The second round of testing showed the computer version took 16 minutes on average to complete and the mobile phone version took 18 minutes on average. It also showed the online questions and skip patterns were understandable and appropriate.

For the validity check conducted a month after survey initiation, we compared patients’ baseline self-reports of CIH therapies used at their VA medical facility vs what was recorded in the VA’s EMR to examine the degree of concordance between the 2. We found all but meditation had a high degree of concordance, so we called 45 Veterans with discordant information on meditation to determine the reason for the discordance. Of the 30 agreeing to speak to us, 47% (n = 14) reported using meditation at the VA but a week or 2 later than what was shown in the EMR, so their use was not showing in the EMR at first. The other half (53%; n = 16) said they did not use meditation at the VA when the EMR showed they did (4 said their survey was inaccurate, 8 said the EMR was inaccurate, 2 reported they signed up for meditation but ended up not using it, and 2 reported not being able to remember using it). The VA meditation clinical expert on the Advisory Board reported that many Veterans receive meditation as part of broader, multi-component VA programs and VA programs often provide meditation but call it something else (eg, “Stress and Relaxation”) to attract participants, so the patients might not have realized they received a meditation session. When we examined the medical records, we found many patients were indeed receiving programs in which some meditation was embedded, but those programs were not categorized as meditation classes (eg, they were wellness or pain relief classes).

Table 3 shows the results of our repeating that concordance examination among the baseline sample responders who completed the BPI pain measure. We again found high concordance for all CIH therapies but 1 (meditation), signifying the survey relatively accurately captured actual use. The VA EMR showed only 8.3% of patients had received a meditation class within a 4-week period, while 26.0% of patients reported receiving a meditation class at the VA on the baseline survey during that same 4-week period. For the remaining CIH therapies, the concordance ranged from 88.2% to 94.9%.

Table 3.

Degree of Concordance Between CIH Therapy Use Reported in the Baseline Survey versus the EMR (n = 6453).

CIH Therapy	% Reporting CIH Therapy Use in the Survey	% Reporting CIH Therapy Use in the EMR	% With Concordant Responses of CIH Use in EMR and Survey
Chiropractic care	24.0	28.2	88.2
Acupuncture	11.9	17.2	90.0
Massage therapy	6.4	4.7	94.1
Yoga	11.9	4.4	89.4
Tai chi/Qigong	6.3	5.3	94.9
Meditation/Mindfulness	26.0	8.3	73.3

Characteristics of the Baseline Sample

The survey’s response rate at baseline was 45.6% (7123/15,608). Table 4 shows a comparison of baseline survey responders (n = 6453) and non-responder (n = 9155), merging survey data with EMR data from the VA’s Corporate Data Warehouse on respondents’ age, gender, race/ethnicity, and rural/urban residence. We also examined a VA measure of disability, service connectedness; the more disabled a Veteran is, the less they have to pay for care. Survey completers were slightly more likely to be older, White, male, and slightly more likely to have some chronic physical health conditions (obesity, diabetes, and hypertension), while slightly less likely to have some mental health conditions (anxiety and depression).

Table 4.

Demographic Characteristics and Health Conditions of Baseline Survey Respondents and Nonrespondents.

	Responder (n = 6453)	Non-Responder (n = 9155)	P-Value^a
Characteristic	Mean (std^b)	Mean (std)	P-Value^a
Age	56.0 (13.9)	52.3 (14.7)	<.0001
	n (%)	n (%)
Race			<.0001
White	4430 (68.7%)	5856 (64%)
Black	1318 (20.4%)	2205 (24.1%)
Asian	95 (1.5%)	176 (1.9%)
NHOPI^c	74 (1.1%)	115 (1.3%)
AIAN^c	54 (.8%)	86 (.9%)
Multi-race	89 (1.4%)	128 (1.4%)
Unknown/Declined/Missing	393 (6.1%)	589 (6.4%)
Ethnicity			.62
Not Hispanic	5356 (83%)	7558 (82.6%)
Hispanic	651 (10.1%)	968 (10.6%)
Unknown/Declined/Missing	446 (6.9%)	629 (6.9%)
Gender			.027
Male	5109 (79.2%)	7111 (77.7%)
Female	1344 (20.8%)	2044 (22.3%)
Urban residence			.14
Yes	5510 (85.4%)	7919 (86.5%)
No	929 (14.4%)	1217 (13.3%)
Unknown	14 (.2%)	19 (.2%)
Degree of disability (“service connected”)			.029
Not disabled/zero %	898 (13.9%)	1388 (15.2%)
Somewhat disabled <50%	958 (14.8%)	1261 (13.8%)
Disabled >=50%	4597 (71.2%)	6506 (71.1%)
Obesity	3548 (55%)	4430 (48.4%)	<.0001
Diabetes	2007 (31.1%)	2402 (26.2%)	<.0001
Hypertension	3945 (61.1%)	5000 (54.6%)	<.0001
Insomnia	2731 (42.3%)	3692 (40.3%)	.013
Anxiety	3306 (51.2%)	4923 (53.8%)	.0018
PTSD	3234 (50.1%)	4678 (51.1%)	.23
Depression	3672 (56.9%)	5407 (59.1%)	.0075

^aP values were obtained using a two-sample t test for continuous variables (eg, age), chi-square tests for categorical variables with more than 2 categories (eg, race, ethnicity, gender, urban residence, service connected), and two-sample tests of proportions for dichotomous variables (obesity, diabetes, hypertension, insomnia, anxiety, PTSD, depression).

^bstd = standard deviation.

^cAIAN = American Indian or Alaska Native, NHOPI = Native Hawaiian or Other Pacific Islander.

Discussion

With the input from a large group of nationally known CIH therapy research and clinical subject matter experts, patients who used CIH therapies, and VA national clinical operations leaders, we collaboratively developed, tested and fielded a 17-minute, 64-item, electronic survey, the Complementary and Integrative Health Therapy Patient Experience Survey, to measure patients’ longitudinal use of 8 CIH therapies and 7 longitudinal patient reported outcomes related to that use. We tested and applied the survey in the population of VHA healthcare system users with chronic musculoskeletal pain. Given these therapies are now being used in a variety of formats in healthcare systems, as well as in patients’ homes and in their communities, this survey addresses the need to comprehensively assess the amount, timing, format, provider, payor, and location of the use of CIH therapies. Our validity check comparing CIH use in the VA EMR vs self-reported VA care showed few discrepancies, demonstrating a high degree of validity for all CIH therapies except meditation. For meditation, we found self-reports were much higher than what was in the EMR, likely because patients were receiving meditation as part of a larger program and not necessarily receiving meditation stand-alone therapies.

We faced some challenges in designing this survey. We struggled with how to best assess the frequency of CIH therapy use, which was difficult given the field has not yet agreed on the dose (what amount over what time) needed to have an effect for each CIH therapy, as noted above. Another complexity was determining which of many possible outcomes to assess, given CIH therapies appear to affect many. We narrowed it down to a range of commonly studied health and wellness patient reported outcomes. We also wanted to find the right balance between creating a lengthy burdensome survey and having a sufficient number of items to comprehensively capture use. Finally, determining the best user-centered design was especially difficult given the complex skip pattern required to keep the survey from being extraordinarily lengthy. Nevertheless, we observed a relatively good response rate of 45.6% at baseline. This response rate was similar to that of the 2022 NHANES survey of CIH therapy use among the general population (49.6%)³³ and much higher than other recent national surveys of Veterans’ healthcare and health status (23.5%).³⁴

This survey also has some limitations. First, we oriented it to persons with chronic pain and 7 pain-related conditions instead of the general population. As such, we could have assessed other health conditions. However, the outcomes we did assess affect the majority of the general population, so others researching the effects of CIH therapy use may also be able to utilize this survey instrument. Additionally, we limited the survey to 8 CIH therapies. However, we believe researchers can easily substitute other provider-delivered or self-guided CIH therapies. In spite of these issues, we feel confident in our survey due to our extensive development process involving many experts in CIH therapy delivery and research, experts in electronic survey design, and patients who receive CIH therapies. We also feel confident due to the results of our validity checks.

In conclusion, we developed a longitudinal, electronic survey that comprehensively and appropriately captures actual CIH therapy use and key health and wellness outcomes from patients’ perspectives. The survey can be used by any healthcare organization, or anyone interested in assessing CIH therapy use, and patient reported outcomes. Future research will use data from this survey to examine CIH therapy effectiveness.

Supplemental Material

Supplemental Material - Measuring Patient-Reported Use and Outcomes From Complementary and Integrative Health Therapies: Development of the Complementary and Integrative Health Therapy Patient Experience Survey

Supplemental Material for Measuring Patient-Reported Use and Outcomes From Complementary and Integrative Health Therapies: Development of the Complementary and Integrative Health Therapy Patient Experience Survey by Stephanie L Taylor, PhD, A Rani Elwy, PhD, Barbara G Bokhour, PhD, Scott S Coggeshall, Amy Cohen, PhD, Claudia Der-Martirosian, PhD, Taona Haderlein, PhD, Jolie Haun, PhD, Benjamin Kligler, MD, Alex T Kloehn, MA, Karl A Lorenz, MD, MSHS, Briana Lott, MPH, Marlena H Shin, JD, MPH, Tammy Schult, PhD, Joy Toyama, DrPH, Alison M Whitehead, MPH, Xiaoyi Zhang, MS, and Steven B Zeliadt, PhD in Global Advances in Integrative Medicine and Health.

Footnotes

Acknowledgments

The authors wish to thank the APPROACH Study Advisory Board members: Jeffery Dusek, PhD; Ansgar Furst, PhD, MSc; Christine Goertz, DC. PhD; David Kearney, MD; Anthony J. Lisi, DC.; Louise Mahoney; Juli Olson, DC, LA; Pam Pence; Friedhelm Sandbrink MD; Robert Saper, MD; Greg Serpa, PhD; Peter Wayne, PhD; Claudia Witt, MD. We also are grateful to Dr Jolie Haun’s research team who performed the cognitive and pilot testing of the survey: Lisa Ballistrea, Lisa Ballistrea Goff, Julie McMahon-Grenz, and Maisha Standifer.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Funding was provided by the Department of Veterans Affairs: Office of Patient-Centered Care and Cultural Transformation and Quality Enhancement Research Initiative (grant #PEC 16-354). The views in this paper are the views of the authors and do not represent the views of the Department of Veterans Affairs or the U.S. Government.

ORCID iDs

Stephanie L Taylor

Benjamin Kligler

Alison M Whitehead

Supplemental Material

Supplemental material for this article is available online.

References

Nahin

Boineau

Khalsa

Stussman

Weber

. Evidence-based evaluation of complementary health approaches for pain management in the United States. Mayo Clin Proc. 2016;91(9):1292-1306.

Polusny

Erbes

Thuras

, et al. Mindfulness-based stress reduction for posttraumatic stress disorder among Veterans: a randomized clinical trial. JAMA. 2015;314(5):456-465.

Chou

Deyo

Friedly

, et al. Nonpharmacologic therapies for low back pain: a systematic review for an American college of physicians clinical practice guideline. Ann Intern Med. 2017;166(7):493-505.

Morone

Greco

Moore

, et al. A mind-body program for older adults with chronic low back pain: a randomized clinical trial. JAMA Intern Med. 2016;176(3):329-337.

Skelly

Chou

Dettori

, et al. Noninvasive Nonpharmacological Treatment for Chronic Pain: A Systematic Review Update. Agency for Healthcare Research and Quality (US); 2020. https://www.ncbi.nlm.nih.gov/books/NBK556229/. Accessed July 31, 2020.

Cherkin

Herman

. Cognitive and mind-body therapies for chronic low back pain and neck pain: effectiveness and value. JAMA Intern Med. 2018;178(4):556-557.

Giannitrapani

Holliday

Miake-Lye

Hempel

Taylor

. Synthesizing the strength of the evidence of complementary and integrative health therapies for pain. Pain Med. 2019;20(9):1831-1840.

Donaldson

Neumark-Sztainer

Gaugler

, et al. Yoga practice among Veterans with and without chronic pain: a mixed methods study. Med Care. 2020;58(29 Suppl):S133-S141.

Solloway

Taylor

Shekelle

, et al. An evidence map of the effect of Tai Chi on health outcomes. Syst Rev. 2016;5(1):126.

10.

Hilton

Hempel

Ewing

, et al. Mindfulness meditation for chronic pain: systematic review and meta-analysis. Ann Behav Med. 2017;51(2):199-213.

11.

Paige

Miake-Lye

Booth

, et al. Association of spinal manipulative therapy with clinical benefit and harm for acute low back pain: systematic review and meta-analysis. JAMA. 2017;317(14):1451-1460.

12.

Department of Health and Human Services . Intra-agency pain research coordinating committee. National pain strategy - a comprehensive population health -level strategy for pain. NIH interagency pain research coordinating committee. https://www.iprcc.nih.gov/sites/default/files/HHSNational_Pain_Strategy_508C.pdf

13.

Qaseem

Wilt

McLean

Clinical Guidelines Committee of the American College of Physicians , et al.. Noninvasive treatments for acute, subacute, and chronic low back pain: a clinical practice guideline from the American college of physicians. Ann Intern Med. 2017;166(7):514-530.

14.

Kligler

. Whole health in the Veterans health administration. Glob Adv Health Med. 2022;11:2164957X221077214.

15.

Ritenbaugh

Nichter

et al. Developing a patient-centered outcome measure for complementary and alternative medicine therapies I: defining content and format. BMC Compl Alternative Med. 2011;11:135.

16.

Clarke

Black

Stussman

Barnes

Nahin

. Trends in the use of complementary health approaches among adults: United States, 2002-2012. Natl Health Stat Report 2015;10(79):1-16.

17.

Sharma

Minh Duc

Luu Lam Thang

, et al. A consensus-based checklist for reporting of survey studies (CROSS). J Gen Intern Med. 2021;36(10):3179-3187.

18.

Kerns

Brandt

Peduzzi

. NIH-DoD-VA pain management collaboratory. Pain Med. 2019;20(12):2336-2345.

19.

Fonteyn

Fisher

. Use of think aloud method to study nurses’ reasoning and decision making in clinical practice settings. J Neurosci Nurs. 1995;27(2):124-128.

20.

Dillman

Smyth

Christian

. Internet, Phone, Mail and Mixed-Mode Surveys: The Tailored Design Method. 4th ed. Hoboken, NJ: John Wiley; 2014.

21.

Cleeland

Ryan

. Pain assessment: global use of the brief pain inventory. Ann Acad Med Singapore. 1994;23(2):129-138.

22.

Krebs

Lorenz

Bair

, et al. Development and initial validation of the PEG, a three-item scale assessing pain intensity and interference. J Gen Intern Med. 2009;24(6):733-738.

23.

Zelaya

Dahlhamer

Lucas

Connor

. Chronic Pain and High-Impact Chronic Pain Among U.S. Adults, 2019. NCHS Data Brief, No 390. Hyattsville, MD: National Center for Health Statistics; 2020.

24.

Hurst

Bolton

. Assessing the clinical significance of change scores recorded on subjective outcome measures. J Manip Physiol Ther. 2004;27:26-35.

25.

PSS-4 Cohen

Kamarck

Mermelstein

. A global measure of perceived stress. Journal of Health and Social Behavior. 1983;24:385-396.

26.

Kroenke

Spitzer

Williams

. The patient health questionnaire-2: validity of a two-item depression screener. Med Care. 2003;41(11):1284-1292.

27.

Cella

Riley

Stone

, et al. The Patient-Reported Outcomes Measurement Information System (PROMIS) developed and tested its first wave of adult self-reported health outcome item banks: 2005-2008. J Clin Epidemiol. 2010;63(11):1179-1194.

28.

Scheier

Wrosch

Baum

et al. The life engagement test: assessing purpose in life. J Behav Med. 2006;29(3):291-298. doi:10.1007/s10865-005-9044-1

29.

Smith

Wallston

Smith

. The development and validation of the perceived health competence scale. Health Educ Res. 1995;10(1):51-64.

30.

Duke

Lynch

Smith

Winstanley

. Validity of a new patient engagement measure: the Altarum consumer engagement (ACE) measure. Patient. 2015;8(6):559-568.

31.

Lorig

Sobel

Ritter

Laurent

Hobbs

. Effect of a self-management program on patients with chronic disease. Effect Clin Pract. 2001;4(6):256-262.

32.

Ritter

Lorig

. The English and Spanish self-efficacy to manage chronic disease scale measures were validated using multiple studies. J Clin Epidemiol. 2014;67(11):1265-1273.

33.

Nahin

Rhee

Stussman

. Use of complementary health approaches overall and for pain management by US adults. JAMA. 2024;331:613-615. doi:10.1001/jama.2023.26775

34.

VHA’s 2022 Survey of Veteran Enrollees’ Health and Use of Health Care Findings Report. Prepared by Westat, Inc. under contract with Trilogy Federal, LLC for the Office of Strategic Planning and Analysis (OSPA), VHA Chief Strategy Office Veterans Health Administration (VHA), U.S. Department of Veterans Affairs (VA) https://www.va.gov/VHASTRATEGY/SOE2022/VASOE-FindingsReport-Final.pdf

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.31 MB