Evaluation of an Integrative Post-Traumatic Stress Disorder Treatment Program

Abstract

Objectives:

Post-traumatic stress disorder (PTSD) and combat-related stress can be refractory, pervasive, and have a devastating impact on those affected, their families, and society at large. Challenges dealing with symptoms may in turn make a servicemember more susceptible to problems, including alcohol abuse, interpersonal conflict, and occupational problems. An effective treatment strategy will address multifactorial issues by using a holistic multimodal approach. Back on Track is an intensive outpatient program utilizing a holistic philosophy and multimodal treatments to provide a whole systems approach for the treatment of combat-related stress reactions and PTSD in active duty servicemembers.

Design/Setting/Subjects:

An explanatory, sequential, mixed-methods program evaluation was conducted to assess the effectiveness of a PTSD and combat stress treatment program. Quantitative outcomes were collected and analyzed on 595 participants at pre- and postinterventions and 6-week follow-up and qualitative data were gathered through participant interviews.

Intervention:

The manualized program uses a multimodal, psychoeducational group therapy format with a holistic approach for treating combat stress, increasing resiliency, and assisting with reintegration. Rotating providers visit from other programs and services to deliver content in bio–psycho–social–spiritual domains, including didactic lectures on mindfulness and the relaxation response and daily sessions of yoga nidra and meditation.

Outcome measures:

The primary outcome measure was PTSD symptom severity assessed with the PTSD Checklist-Military Version (PCL-M). Secondary outcomes included self-efficacy, knowledge, use, and satisfaction. Quantitative data were contextualized with interview data.

Results:

Results demonstrated a highly statistically significant effect of the program when comparing within-subject PCL-M scores before and after program participation, signed rank S (N = 595) = −47,367, p < 0.001. This translates to a moderate effect size, Cohen's d (N = 595) = −0.55, 95% confidence interval = −0.62 to −0.47, and a mean decrease of 7 points on the PCL-M at postintervention, demonstrating response to treatment. There were significant increases in knowledge and self-efficacy and high levels of satisfaction with the program overall, content, materials, and delivery.

Conclusions:

The treatment program has served ∼800 servicemembers since inception and has since expanded to five installations. The provision of whole systems care where the approach is holistic, multimodal, and multidisciplinary may be a way forward for the successful treatment of PTSD and other debilitating behavioral health conditions in military contexts and beyond.

Introduction

Post-traumatic stress disorder (PTSD) is a debilitating condition that emerges after exposure to a traumatic event, characterized by four hallmark clusters of symptoms: reexperiencing, avoidance, negative cognitions and mood, and hyperarousal. ¹ These symptoms can be severe and pervasive and can have a devastating impact on those affected by the disorder as well as their families and society at large. While evidence-based first-line treatments of trauma-focused psychotherapy, eye movement desensitization and reprocessing, and medication address the underlying symptoms, dropouts and nonresponse rates are high ^2,3 perhaps because they fall short of treating the whole person. There is a gap in care that calls for a new approach to treat and measure the bio–psycho–social–spiritual domains of those with combat-related stress and PTSD, ⁴ especially in populations most vulnerable to trauma.

Military personnel are among the most at-risk populations for exposure to traumatic events and the development of PTSD ^5,6 with an estimated prevalence of PTSD at 23% for veterans returning from deployments in Afghanistan (Operation Enduring Freedom) and Iraq (Operation Iraqi Freedom). ⁷ Symptoms can affect normal daily activities and comorbidities typically accompany this condition, including depression or anxiety, drinking or drug problems, physical symptoms or chronic pain, employment problems, and relationship problems, including divorce. ⁸ Based on this body of research, the Back on Track program, a multimodal combat stress treatment program, was designed and stood up to address the comprehensive reintegration needs of combat-exposed servicemembers at Marine Corps Base Camp Lejeune, North Carolina (Camp Lejeune).

Program description

Back on Track is an intensive outpatient program utilizing a holistic philosophy and multimodal treatments to provide a whole systems approach for the treatment of combat-related stress reactions and PTSD in servicemembers at Camp Lejeune. The goal of Back on Track is to equip combat-exposed servicemembers with the knowledge, coping strategies, and psychosocial skills necessary to manage their symptoms, readjust, and reintegrate into garrison military and civilian environments. The guiding rationale of the program is that combat stress symptoms are expected reactions to abnormal situations, but that significant combat stress can result in disruptive symptoms (e.g., psychological and physiological arousal, depression, anxiety, dissociation, and physical symptoms). Challenges dealing with combat stress symptoms may in turn make a servicemember more susceptible to other problems, including substance abuse, interpersonal conflict, and occupational problems. The mix of therapies was designed to treat these multifactorial issues by treating the whole person. This multimodal approach included conventional psychotherapy and medication management; classes on physiology of stress, anger management, sleep hygiene, nutrition, problem-solving, family relationships, and financial planning; and daily sessions of complementary and integrative therapies, mindfulness and yoga nidra.

The program is multidisciplinary, in that 10 rotating providers visit from other programs and services on base to deliver content on topics guided by the program manual. Providers are licensed, credentialed, certified, or otherwise qualified by their respective credentialing and certifying agencies. Classes are held for a 2-week duration, for 8 hours per day, and accommodate an average of eight servicemembers per iteration. The manualized program is conducted in small cohorts to foster social support and normalizing effects. ⁹ This is particularly important in military populations where communities are small and there is a strong stigma associated with mental health diagnoses and treatment seeking. Servicemembers who were deployed to a combat area and were receiving usual care by a mental health provider or other medical personnel for readjustment challenges were eligible to be referred to the program.

In collaboration with Naval Hospital Camp Lejeune and Back on Track leadership and staff, researchers designed a program theory-driven evaluation with the main objective to assess if the goals, objectives, and outcomes of Back on Track were being achieved as intended.

Materials and Methods

Participants and procedure

The program evaluation utilized an explanatory sequential mixed-methods design where quantitative data were collected pre–post and qualitative interview data were collected as a follow-up to contextualize the survey data. ¹⁰ To operationalize the study goals and gain support from leadership and program staff, a program theory-driven logic model of the study was utilized. ¹¹ The logic model provided a snapshot diagram of program inputs, process variables (activities/participation), and outcome metrics identified in collaboration with program staff.

The outcome evaluation includes several populations: a large sample of program participants whose data were collected since program inception provided data on the primary outcome measure (N = 595), a smaller sample of participants whose data were collected during the 18-month evaluation period provided secondary outcome measures (N = 68 with a subset of N = 22 providing qualitative interviews), and additional program stakeholders with whom qualitative interviews were conducted (N = 11).

Participants

Quantitative analyses were conducted on a sample of 595 participants who attended Back on Track since inception and who completed pre- and postintervention surveys on the primary evaluation outcome, the PTSD Checklist-Military Version (PCL-M). Table 1 displays the demographics of this population. Secondary outcomes were available for analysis on a subset of 68 participants from the main sample of 595. These participants attended the program during the evaluation period and so the authors were able to collect additional outcome measures for this subset of the main sample.

Table 1.

Participant Characteristics for Primary Outcome of Post-traumatic Stress Disorder Checklist-Military Version (N = 595)

Demographics	N	%
Ethnicity/race
White American	463	80.1
Hispanic	45	7.8
African American	51	8.8
Asian American	10	1.7
Native American	4	0.7
Multiracial	5	0.9
Marital status
Married/engaged	367	63.16
Single	167	28.7
Divorced/separated	48	8.1
Pay grade
Junior enlisted	364	61.8
Senior enlisted	225	38.2
Deployments
Once	248	43.4
Twice	180	31.5
Three times	85	14.9
Four times	37	6.5
Five times or more	21	5.3
Branch
U.S. Marine Corps	541	0.91
U.S. Navy	54	0.09

Pay grade presented as Junior enlisted (E1–E4) and Senior enlisted, Officers, or Warrant Officers (E5–E9).

Measures

Quantitative measures

The primary outcome metric, change in PTSD symptom severity over time, was assessed using the PCL-M (range, 17–85; higher scores indicate more severe symptoms) at all assessment points. ¹² It has excellent internal consistency (Cronbach's α = 0.94–0.97), test-retest reliability (0.96), and concurrent validity. ¹³ The minimal clinically important difference (MCID) for self-reported PTSD symptom severity is a reduction of 10 or more points on the PCL. ¹⁴ Secondary outcome measures included items used to assess knowledge (17 items), self-efficacy (24 items), and satisfaction (17 items), which were created in collaboration with program staff to connect with program content and delivery.

Qualitative measures

Topics included in interviews with program staff and leadership included opinions about the program, what worked well, what did not, the content, participants, and areas for improvement to provide feedback for program improvement. Interviews with program participants explored topics of how they were referred to the program; their experiences with the classes, speakers, content, and delivery; the impacts the program made on participants; and whether they expected to use the knowledge and skills in the future.

Data analysis

Quantitative analysis

Using paired t tests and signed rank tests, the authors assessed whether the intervention would result in a lower PCL-M score (primary measure) with Cohen's d measure of 0.4 by the end of intervention training and the follow-up time point; a two-sided significance level of 0.05, with 90% power; and correlation of repeated measures of 0.5; the a priori sample size of 68 subjects was required to statistically power the analysis. All hypothesis tests were two-sided at the 0.05 level. Data were analyzed to determine if PCL-M scores on 595 participants changed from pre- to postintervention. For the primary outcome in the analysis, repeated-measures analysis of variance (ANOVA) was conducted to test group differences in baseline PCL-M scores for demographic characteristics.

Self-efficacy statements based on program content were assessed at pre- and postinterventions on 24 items. The Cochran–Mantel–Haenszel nonparametric test for categorical data was used to calculate the shift of endorsing self-efficacy from time 1 to time 2 between subjects.

Knowledge summary scores were obtained on 17 multiple-choice questions related to program content before and after the intervention. Expected use of skills and satisfaction with the program were assessed at postintervention.

Qualitative analysis

During the evaluation period, research staff visited the program at three time points to conduct qualitative interviews with participants. The authors interviewed 22 participants who completed Back on Track. Interviews were conducted on the ninth day of the 10-day program. The authors conducted an analysis to understand if the participants who were available and agreed to be interviewed during site visits were different from others who were not interviewed. Fisher's exact tests were conducted on demographic characteristics and there were no significant differences between interviewees and those who did not participate in an interview, but completed outcome assessments.

Qualitative analysis methods were based on rapid assessment processes. ¹⁵ This technique utilized iterative data collection and analyses conducted during debriefing sessions at the end of each day. Participant interviews were open-ended, semistructured, and followed an interview guide. Data from interview notes were compiled during debriefing sessions and entered into a database for further refinement of themes during each debriefing session. All participants who were available for a qualitative interview agreed to be interviewed. The authors investigated demographic characteristic differences using Fisher's exact nonparametric tests between those who were interviewed and those who were not.

All procedures involved with the study were reviewed and approved by the Institutional Review Board (IRB) that oversees research at Naval Hospital Camp Lejeune. This study was approved by the Naval Medical Center, Portsmouth, IRB.

Results

Effects of Back on Track program

PTSD symptoms (primary outcome)

Results demonstrated a highly statistically significant effect of the program on PTSD symptoms (PCL-M) when comparing within-subject scores before and after they participated in the program, signed rank S = −47,367, p < 0.001, N = 595. This translates to a moderate effect size, Cohen's d = −0.55, 95% confidence interval [CI] = −0.62 to −0.47, and a mean decrease of 7 points on the PCL-M, demonstrating treatment response (based on 5-point threshold) and approaching minimal clinically important significance (based on 10-point threshold) at postintervention. ¹⁶ Significant differences were found between those who reported a PTSD diagnosis and those who experienced combat-related stress, but were not diagnosed with PTSD (pretest PCL-M: F = 12.4, p < 0.001; post-test PCL-M: F = 22.4, p < 0.001). Results of repeated-measures ANOVA did not uncover any statistically significant subgroup differences in demographic characteristics in preintervention PCL-M scores.

Secondary outcomes

For self-efficacy, pre- to postshifts in all 24 categorical items were highly significant in a positive direction, mean difference (MD) = −22.25, standard deviation (SD) = 12.66, effect size d = 1.76, 95% CI 1.37–2.15; signed rank S = 1139.00, p < 0.001, N = 68. Based on the signed rank test, the change in knowledge was highly statistically significant when comparing within-subject pre- and postknowledge scores, S = 789.50, p < 0.001, N = 68. The mean difference in knowledge score from pre- to postintervention was MD = 1.96, SD = 1.87, effect size d = 1.05, 95% CI = 0.73–1.36.

In terms of participants' expected use of skills, an overwhelming majority noted that they expected to use the skills from this training in their everyday lives. Frequencies of ratings for expected use of specific elements of the program are displayed in Table 2. At the 6-week follow-up, the actual skills that participants utilized were assessed and those frequencies are also displayed in Table 2. The majority of participants were highly satisfied with the program overall and with its content and materials. Frequencies are displayed in Table 3.

Table 2.

Expected and Actual Use of Skills

Use of skills	%
Expectations of use
Expected use of skills	98.0
Stress management	93.0
Trigger identification	87.0
Sleep hygiene	82.0
Actual use
Stress management	96.0
Relaxation response	87.0
Mindfulness	79.0
Trigger identification	92.0
Sleep hygiene	80.0

Frequencies do not add to 100%.

Table 3.

Participant Satisfaction Ratings

Satisfaction	% Highly satisfied	% Satisfied	% Not satisfied
Program overall	88.0	12.0	0.0
Program content	66.0	34.0	0.0
Program materials	74.0	26.0	0.0

Participant interviews

Emerging themes from the participant interviews collectively support the holistic and multidimensional therapeutic effects of this program. Participants provided rich descriptions of the impacts experienced on their stress and depression symptoms, self-regulation, quality of life, social and family relationships, and gaining a general sense of ease.

Useful skills

Many skills were found to be useful by the program participants. Understanding the concepts of cognitive distortions and reframing were discussed as essential skills. One participant mentioned that the use of these skills helped to identify emotions or physical events that are triggers. This skill was the most frequently mentioned aspect of value in the program. Specifically, the issues of how to calm oneself before the situation escalates and how to immediately slow down and reassess were repeatedly emphasized as key skills that participants were looking to master. “If I could apply that in my daily life, I'd be a different person,” one participant said. Communication skills, yoga breathing, sleep hygiene, and mindfulness were also helpful skills that resonated with the group. The majority of respondents enjoyed the yoga sessions; however, for some, it was uncomfortable to move and stretch. These findings align with the survey data.

Positive social impacts

The group element was discussed as crucial to the impact of the program, with a participant noting, “The group aspect is very good, we fed off each other as tough topics came up.” Participants learned that others are having similar experiences, and one noted that it was refreshing to be with others going through the same thing. The interaction or having a group of people to relate to was emphasized as one of the best factors of the program.

Most significant changes

Participants were already noticing changes in their quality of life before completing the program. Participants' perspectives on their issues and quality of life were beginning to shift. One participant said, “Life seems more manageable in general.” Patience improved for many, and they reported less stress and anxiety and improvements in sleep. Better communication with family was a common experience.

Changes in behavior, such as sitting with the family at the dinner table and being able to spend time with children without feeling irritable, created a happier, more energized home environment. Improved relationships with their spouses and children were repeatedly mentioned by participants as a significant change since beginning the program. One father noted, “My 5-year-old son doesn't avoid ‘mean Dad!’” Some commented that their spouses also noticed their happier mood and their improved patience and communication skills. In addition to day-to-day changes, there seemed to be a shift in terms of beliefs about the future, as illuminated by one participant's comment, “Motivation is coming back a little more.” There was a general sentiment that the program was valuable and provided participants with the opportunity to learn new ways to manage their stressors and understand how they can enhance their quality of life.

Highly satisfied

Overall, participants spoke highly about the program. They received care quickly after referral and typically entered the next scheduled round of classes, typically between 2 days and 6 weeks after they were referred. They used phrases such as outstanding, love it, and great program that allows you to relate to others about combat.

Participants appreciated the opportunity to learn new tools and be exposed to new ways of dealing with their stressors. Cognitive-behavioral worksheets designed to help participants identify stressors and aid in appropriate responses were often mentioned as especially helpful. Participants conveyed a growing sense of comfort and open-mindedness that allowed them to express their feelings and approach their issues in a different way. The exercises learned, such as yoga and breathing, received high praise from participants. The handouts were said to be very helpful. Many commented that they will share the handouts with their spouses and families and will refer back to the class materials. Participants shared that the information was taught clearly and differed from other programs they experienced previously.

Participants noted that the speakers were prepared and knowledgeable and that their passion came through. Some stated that the speakers created an open environment where they felt comfortable asking questions and commenting about their own experiences. Participants liked that the classes were broken up into shorter sessions that were manageable and to the point without the use of death by PowerPoint slides.

Discussion

The main objective of this program evaluation was to ascertain if the goals, objectives, and outcomes of the Back on Track combat stress treatment program were being achieved as intended. Results demonstrated a highly statistically significant effect of this multimodal multidisciplinary program on PTSD symptoms at postintervention. While the change in pre–post PCL-M scores suggested that participants were responding to treatment, it did not reach the threshold for MCID. However, meaningful impact from patient perspectives is demonstrated in the satisfaction with outcomes and qualitative descriptions of functional status. Additionally, there were highly significant increases in self-efficacy, knowledge, and satisfaction with the program overall, content, materials, and delivery. Participants with combat stress had slightly better response to treatment than those who had been diagnosed with PTSD, supporting the program objective that early treatment may be the best mitigation strategy for this population.

The findings from this study on PTSD impact and satisfaction align with previous surveys of active duty and veteran populations seeking integrative therapies as an adjunctive or stand-alone treatment for PTSD. ¹⁷ The qualitative findings duplicate an earlier study that identified a key theme that arose from the care-seeking experiences of combat-exposed servicemembers, the desire for holistic health care. ¹⁸ There have been disappointing results in assessments of gold standard, cognitive-behavioral exposure-based therapies alone and evidence is mixed for efficacy of individual complementary and alternative medicine (CAM) therapies, yet significant reductions in symptoms of PTSD for veterans being treated with CAM and mind–body skills determined that integrative health models that include CAM are low risk, low cost, and acceptable to patients and providers. ¹⁹ This program evaluation extends these findings and fills a gap in literature by providing a study that assesses a holistic, multimodal whole systems approach to provision of care for PTSD and combat stress.

A limitation of this study is that the evidence is based on a cohort study where participants acted as their own controls. Future research may include study designs such as comparative effective studies where several PTSD treatment programs are compared head-to-head as issues of ethical research are a barrier to randomizing this vulnerable clinical population to no treatment. The key element for leaders may be the quick access to care that this program provides. Timing for access to care with this population is essential. When a servicemember is ready to seek help, Back on Track accommodates participants typically within the next cohort of classes.

The program utilizes a holistic philosophy of care addressing the participant's mind, body, social, and spiritual needs supported by multimodal treatments (conventional psychotherapy and pharmacology along with complementary and integrative medicine) that form the basis of the whole systems approach to provision of care. The multidisciplinary approach exposes participants to a myriad of other services on base that they may access after the program ends. The group treatment format supports normalization of stress responses and helps move participants out of social isolation. The psychoeducational approach targets those who experienced combat trauma by providing discussion of the etiology of PTSD so that participants understand underlying processes and the fundamental truth that their feelings and actions are generally normal reactions to traumatic stress. This practical, holistic, and patient-centered care establishes a foundation for treatment that complements protocol-based therapies that focus on coping and affect regulation skills.

Conclusions

The Back on Track program has served approximately 800 marines and sailors since its inception over 10 years ago. It is a well-known and respected program at Camp Lejeune, has demonstrated effectiveness, and has been implemented at four other military installations across the United States. With continued attention paid to treating a person holistically, using a whole systems approach, the unique needs of combat veterans with deployment-related stress can be addressed more effectively. Programs such as Back on Track hold promise for easing the burden on our nation's warfighters and add efficiency to a taxed but vital military mental health treatment system. Considering whole systems research where the approach is holistic, multimodal, and multidisciplinary may be a way forward for successful treatment of PTSD and other debilitating behavioral health conditions.