The Fallacy of Sham-Controlled Neurofeedback Trials: A Reply to Thibault and Colleagues (2018)

Abstract

Background: Sham-controlled neurofeedback (NFB) trials consistently find no separation on ADHD outcome measures leading many to conclude that NFB’s beneficial effects are due to placebo. Method: We deconstruct the NFB training methodology and findings of six sham-controlled trials that assessed for evidence of learning. Results: All six studies found no evidence NFB subjects learned to self-modulate the targeted electroencephalogram (EEG). Careful analyses revealed these studies’ training methodologies were antithetical to the established science of operant conditioning thereby preventing subjects from learning to self-modulate. These findings are in marked contrast to NFB studies whose methodology mirror the best practices of operant conditioning. Conclusion: The premise that NFB’s beneficial effects are due to placebo phenomenon is unproven as these studies compared two forms of false-feedback, not operant conditioning of the EEG. Because these studies are highly cited and considered the gold standard in scientific rigor, a reappraisal of the evidence is urgently needed.

This is now Thibault, Veissière, Olson, and Raz’s (2018) eighth publication making the same argument based on the consistent finding of no separation on any outcome measure when comparing so-called “genuine” neurofeedback (NFB) and sham feedback in sham-controlled trials (e.g., Thibault & Raz, 2017). The authors therefore assert that NFB operates as a placebo, all be it a powerful one, with effects commonly equivalent to optimized versions of established ADHD treatments (e.g., Pigott, 2017). In their current effort, the authors provide guidance how clinicians can ethically prescribe NFB “as a form of neurosuggestion therapy” (Thibault et al., 2018, p. 2). Our Guest Editorial deconstructs these sham-controlled studies demonstrating the fallacies of the authors’ argument. We also examine the evidence supporting neurosuggestion as a therapeutic intervention as well as that supporting NFB’s specificity, sustainability, and effectiveness when compared with stimulant medication (SM). Finally, we question why prescribe NFB as a placebo when with proper training clinicians can provide operant conditioning of the electroencephalogram (EEG) with proven sustained effects.

Learning Methodology Matters

Table 1 summarizes the methodology and findings from six sham-controlled trials treating ADHD. Although each study acknowledged NFB is based on operant learning, their methodology violated established learning science by using either automated or manually adjusted EEG reward thresholds to maintain an “about 80%” level of reward across sessions and subjects. This procedure is contrary to basic learning principles. First, operant conditioning targets a response followed by a stimulus-event to make the desired response occur more or less frequently and then plots the target response’s occurrence over time to document whether or not learning has occurred. In these studies, the target response was not consistently calculated, monitored, plotted, and presented to NFB subjects. Therefore, it is not known what response (if any) was conditioned. Second, the studies do not reference the effects of practice in the experimental process. Subjects in both groups engaged in the same set of behaviors during sessions (e.g., maintaining stillness and focus, reducing muscle and eye-movement artifacts, relaxation, posture, and breathing). If subjects did not engage in these practiced behaviors, their EEG data were riddled with artifact and worthless. Third, operant conditioning of the EEG requires that these core concepts are strictly adhered to demonstrating the operant behavior has been learned and such documentation of learning should occur before examining outcome measures of interest (Cannon, 2015).

Table 1.

Sham-Controlled NFB Studies.

Study citation	NFB training methodology	Key findings
Logemann, Lansbergen, van Os, Bocker, and Kenemans (2010)	“Feedback thresholds were automatically and dynamically adjusted every 30 s to keep power 80% of time above or below threshold (depending on whether feedback consisted of up or down training)” (p. 51).	1. Study terminated when there was no trend of an NFB effect in the interim analysis. 2. Found “NFB treatment did not seem to affect EEG” (p. 51). 3. Found “most participants thought they were in the sham group. For the treatment group, 10 out of 14 (71%) participants thought they received sham feedback. 10 out of 12 participants in the sham group thought they were in the sham group” (p. 51).
Lansbergen, van Dongen-Boomsma, Buitelaar, and Slaats-Willemse (2011)	“Reward threshold levels were automatically adjusted every 30 s so that the child was rewarded about 80% of the time (i.e., received positive feedback)” (p. 279).	1. Found “analyses revealed significant improvements of ADHD symptoms over time, but changes were similar for both Groups” (p. 275). 2. Found “75% of children and their parent(s) in the active neurofeedback group and 50% of children and their parent(s) in the placebo feedback group thought they received placebo feedback” (p. 275). 3. Based on these pilot results, the authors changed their NFB training methodology to have trainers adjust “manually the feedback parameters” for new subjects (p. 283).
Arnold et al. (2013) Collaborative Neurofeedback Group (2013)	“Reinforcement was provided for EEG theta–beta power ratio below a threshold that was set minute-to-minute by fuzzy logic based on the immediately preceding EEG” (p. 412). Auto-thresholding ensured subjects played videogames with full-control approximately 80% of the time.	1. Both groups showed significant improvement in ADHD symptoms but there was no NFB specific effect. 2. In a subsequent publication (Collaborative Neurofeedback Group, 2013), authors report, “the sham group (as well as active group) showed no obvious EEG changes in a simple pre–post measure of theta/beta ratio” (p. 5).
van Dongen-Boomsma, Vollebregt, Slaats-Willemse, and Buitelaar (2013) Vollebregt, van Dongen-Boomsma, Slaats-Willemse, and Buitelaar (2014b)	This is a continuation of Lansbergen et al. (2011). For newly enrolled subjects (n = 27) “reward threshold levels were manually adjusted so that the child was rewarded about 80% of the time (ie, received positive feedback), consequently the amount of reward remained at about the same level across sessions and across groups” (p. 823).	1. Authors combined subjects from Lansbergen et al. (n = 14) with 27 new subjects who had “trainers” manually readjust thresholds to maintain the same “about 80%” level of reward. 2. Found “while total ADHD symptoms improved over time for both groups, there was no significant treatment effect” (p. 821). 3. Although authors report that “guessing assignment was no better than chance level” (p. 821), in a subsequent article (Vollebregt et al., 2014b), the authors note, “most participants of NFB placebo-controlled RCTs conducted until now seem to experience the treatment as a placebo condition” (p. 2).
Vollebregt, van Dongen-Boomsma, Buitelaar, and Slaats-Willemse (2014a)	Same subjects/method as van Dongen-Boomsma et al. (2013).	1. Found “no significant treatment effect on any of the neurocognitive variables” (p. 460). 2. Pre–post EEG data were reported for only 10 of the 22 NFB subjects. Found more evidence of negative shaping of the EEG away from the reward targets than positive shaping.
Schönenberg et al. (2017a)	“Reward thresholds were automatically adjusted every 15 s to provide positive feedback about 80% of the time” (p. 677).	1. Found “self-reported ADHD symptoms decreased substantially for all treatment groups between pretreatment and the end of 6 month follow-up, independent of treatment condition” (p. 673). 2. Found “no significant effect of time or treatment-by-time interaction was observed” (p. 678) for the targeted EEG confirming there was no evidence NFB subjects learned to self-modulate.

Note. NFB = neurofeedback; EEG = electroencephalogram; RCT = randomized controlled trials.

In these studies, every reset of the EEG reward threshold delivered operant consequences to subjects’ brains antithetical to the goal of training. As Pigott and colleagues (2017) note

if the targeted EEG was strengthening, reinforcement was withdrawn and reset down to 80% thereby punishing participants for learning to self-modulate. Conversely, if the targeted EEG was decreasing, participants were reinforced up to 80% thereby rewarding them for decreasing its strength. (p. 897)

At every reset of the reward threshold, NFB subjects therefore were either rewarded for not learning to self-modulate the targeted EEG or administered a Type 2 punishment for the beginnings of success.

Given their flawed methodology, it is not surprising that all six studies found:

• No evidence NFB subjects learned to self-modulate the targeted EEG;

• No separation between NFB and sham feedback on any outcome measure; and

• When assessed, the vast majority (71% to 75%) of NFB subjects thought they received sham-feedback—correctly determining the NFB they received was often false.

Intriguingly, four of the studies also found significant improvement in both groups, leading Thibault and colleagues among many others to argue that these beneficial effects are due to placebo phenomena versus any specific effects from NFB. Two points in response below:

First, flawed methodology prevented NFB subjects from learning to self-modulate the targeted EEG and therefore no specific effects should be expected since each study compared two forms of false-feedback. Second, both groups participated in an active intervention. Ninaus and colleagues (2013) found multiple cortical regions of the brain are activated when blinded subjects were told to focus and try to control randomly moving bars during five 20-s rounds. In contrast, no such changes occurred when subjects were instructed to merely watch the moving bars. Subjects in sham-controlled trials are commonly instructed to sit still, focus, and use their brains to increase positive feedback. Similar cortical regions therefore likely underwent a vigorous workout during subjects’ 30+ sessions sitting still and trying to control that which was uncontrollable. This is hardly a “placebo” intervention as traditionally understood and likely only had positive effects because subjects were deceived into believing they had a 50% chance of receiving accurate EEG feedback. Transparency eliminated the brain activation found by Ninaus et al. as it likely would in all false-feedback trials.

Thibault and colleagues’ (2018) claim that NFB is a placebo is not supported by the referenced data. Their referenced studies compared two forms of false-feedback—not operant conditioning of the EEG. NFB has a 75+ year history of scientific inquiry documenting operant conditioning of the EEG in cats (e.g., Wyrwicka & Sterman, 1968), primates (e.g., Schafer & Moore, 2011), and people (e.g., Jasper & Shagass, 1941), including a 40-year history of research treating ADHD children (Lubar & Shouse, 1976; Shouse & Lubar, 1979). The authors though dismiss this extensive research history asserting that “Following the results from recent double-blind studies, we can now add EEG-nf for ADHD to this list of placebo therapies that masquerade under other biomedical labels” (p. 2). In contrast, it is our assessment that it is these double-blind studies themselves that are the masquerade since they did not compare operant conditioning of the EEG with a sham-control but rather two forms of false-feedback.

Bad Science Begets More Bad Science

In their introductions, each of these sham-controlled studies states something similar to “neurofeedback is based on the assumption that deviant brain activity patterns can be voluntarily modulated by operant learning strategies” (Schönenberg et al., 2017a, p. 674) and yet then used a methodology antithetical to operant learning. When we challenged Schönenberg and colleagues to either “acknowledge that their neurofeedback methodology violates the very essence of operant conditioning or explain the errors in our analysis” (Pigott et al., 2017, p. 897), these authors stated that they used a “previously established protocol” (Schönenberg et al., 2017b, p. 897) and then made additional points unrelated to our analysis.

This is the problem. Bad science begets more bad science until it is corrected. Each of these studies cited one or more of their predecessors and appears more focused on single/double/triple blinding and empirical rigor than ensuring competence in administering the independent variable, in this case operant conditioning of the EEG. True scientific rigor demands a higher level of adherence to learning principles when evaluating treatments based on operant conditioning.

Unfortunately, the impact factors of the journals publishing these six studies ranged from 2.5 to 11.6 placing them in the mid-to-top tier of behavioral health journals. These studies therefore have had a nefarious impact on the scientific literature as they are highly cited in research and review articles, meta-analyses, editorials, and authoritative practice guidelines (e.g., AACAP, 2011) as well as by insurance companies when denying coverage since these studies are presumed to demonstrate that NFB has no specific effects when rigorously evaluated and therefore does not meet evidence-based treatment standards. This contaminated scientific literature has harmed the public by limiting access to a treatment with a long history of using operant conditioning to improve lives by teaching children and adults how to self-modulate targeted neuronal activity.

Neurosuggestion, Specificity, and Comparative Effectiveness

Thibault and colleagues (2018) argue it is the efficacy of suggestion and the placebo effect that drives behavioral change from NFB—nothing specific to NFB itself—and if transparent, clinicians can ethically prescribe NFB as a placebo treatment “with an eye for amplifying the psychosocial mechanisms of suggestion rather than grasping at the elusive neural signatures many practitioners speciously assign as the cause of ADHD” (p. 709). To buttress their argument, the authors cite an unpublished, uncontrolled, open-label feasibility study they presented at a hypnosis conference (Veissière, Olson, & Raz, 2017). In this study, the authors used a decommissioned magnetic resonance imaging (MRI) machine as a prop with nine ADHD children. They told the children it was an inactive “brain machine” and the authors would “use it as a suggestion” to “help their brain heal itself.” While in the MRI, the authors “gave the children positive verbal suggestions to promote relaxation, focus, and confidence.” They report that in follow-up interviews, parents of two children “reported near complete remission of symptoms, and six reported improvements in areas such as confidence, self-control, and social skills” (p. 709) The authors then claim that “In essence, this study provided neurofeedback-like treatment, but instead of focusing on a specific physiological mechanism, we emphasized suggestion-based healing” (Thibault et al., 2018, p. 708, 709). Four points in response below:

First, besides the inherent potential for multiple biases in an unpublished, uncontrolled, open-label hypnosis study, we have no evidence of functional deficits or improvements in ADHD symptoms using standardized measures for such deficits in the children themselves. Instead, just post-treatment “qualitative” interviews conducted by the authors with the children’s parents of domains unrelated to ADHD’s core symptoms (e.g., “confidence, self-control, and social skills”). Furthermore, we have no data indicating the diagnoses were correct. One would assume an accurate differential diagnosis was conducted at some point in these children’s evaluation procedures; however, this is not clear given the lack of information available. Finally, this is a hypnosis feasibility study using an MRI machine as a prop, not “neurofeedback-like treatment.” It is hard to see how this study provides anything more than anecdotal support for a new experimental treatment.

Second, Thibault and colleagues ignore the evidence suggestive of NFB’s specificity and effectiveness in treating the “neural signatures” of ADHD. For example, in their double-blinded within-subject reversal design studies, Lubar and Shouse (Lubar & Shouse, 1976; Shouse & Lubar, 1979) demonstrated both (a) the functional relationship between the sensory motor rhythm (SMR) and manifestation of hyperkinetic behaviors and (b) that through real-time SMR feedback paired with operant conditioning, ADHD children could learn to self-regulate SMR with the resulting improvements or worsening in their hyperkinetic behaviors based on whether they were reinforced to increase or decrease SMR. In their clinical utility of EEG article, Loo and Barkley (2005) state, “To demonstrate that EEG changes are responsible for treatment effects, reporting of actual EEG changes and correlation with treatment outcome must be shown” (p. 72). Four studies have met this challenge by correlating the extent of NFB subjects’ learning to self-modulate the targeted EEG with treatment outcome and found that those subjects demonstrating the greatest learning experience the most improvement on ADHD outcome measures (Drechsler et al., 2007; Gevensleben et al., 2009; Janssen et al., 2016; Lubar, Swartwood, Swartwood, & O’Donnell, 1995). These findings provide further evidence that enhancing EEG self-regulation is the mechanism of change from NFB treatment versus “neurosuggestion” or other placebo effects. Furthermore, two randomized controlled trials (RCT) have compared electromyographic (EMG) biofeedback with NFB to control for both nonspecific effects and the effects of self-regulation training (Bakhshayesh, Hansch, Wyschkon, Rezai, & Esser, 2011; Strehl et al., 2017). Both of these studies found that subjects learned to self-modulate the targeted physiological mechanism (either EMG or EEG) with NFB demonstrating significant superiority over EMG in reducing ADHD symptoms and this despite the fact that EMG subjects demonstrated more pronounced learning to self-regulate. Finally, in one RCT SM combined with NFB was found superior in multiple outcome domains at the end of treatment and 6-month follow-up to SM combined with attention training that used the identical instructions and game sequences as NFB except that the feedback was not based on subjects’ EEG thereby suggesting a specific effect for NFB as an augmentation to SM (Li, Yang, Zhuo, & Wang, 2013). Lia and colleagues also found that the combined SM/NFB subjects used significantly lower doses of SM during follow-up and reported fewer adverse side effects.

Third, the authors fail to acknowledge that in eight head-to-head comparisons with SM (see Table 2), NFB resulted in essentially equivalent improvement in treating ADHD’s core symptoms (Duric, Assmus, Gundersen, & Elegen, 2012; Flisiak-Antonijczuk, Adamowska, Chładzińska-Kiejna, Kalinowski, & Adamowski, 2015; Fuchs, Birbaumer, Lutzenberger, Gruzelier, & Kaiser, 2003; Gelade et al., 2017; Meisel, Servera, Garcia-Banda, Cardo, & Moreno, 2013; Moreno-García, Meneres-Sancho, Camacho-Vara de Rey, & Servera, 2019; Rossiter, 2004; Rossiter & La Vaque, 1995). These eight studies comprised 581 subjects and in only one head-to-head comparison (n = 32) has SM been found superior to NFB (Ogrim & Hestad, 2013). These comparative effectiveness studies provide strong evidence that NFB is an evidence-based treatment for ADHD.

Table 2.

Studies Comparing NFB With SM in Treating ADHD’s Core Symptoms.

Study	Subjects/design	Key findings
Rossiter and La Vaque (1995)	46 ADHD children and adults matched by age (M = 12.8 years), IQ, gender, and ADHD subtype to receive either 20 NFB sessions based on standardized EEG protocols (n = 23) or SM (n = 23) based on patient or parent preference. Outcome measure was the TOVA.	1. Both the NFB and SM groups improved (p < .05) on measures of inattention, impulsivity, information processing, and variability, but did not differ (p > .3) on TOVA change scores. 2. The authors concluded, “The EEG biofeedback program is an effective alternative to stimulants and may be the treatment of choice when medication is ineffective, has side effects, or compliance is a problem” (p. 48).
Fuchs, Birbaumer, Lutzenberger, Gruzelier, and Kaiser (2003)	34 ADHD children ages 8 to 12 years were assigned based on parental preference to NFB (n = 22) or SM (n = 12). NFB consisted of 30 60-min sessions with sessions administered 3 times per week. The NFB protocol was either theta/beta or SMR training dependent the child’s subtype of ADHD. The doses for the SM group were adjusted during study based on need and ranged between 10 and 60 mg/day. Outcome measures were the TOVA, Attention Endurance Test, and parent- and teacher-rated CBRS.	1. Both groups showed significant improvement in each of the outcome measures with no significant differences between groups. 2. The authors concluded, “These findings suggest that neurofeedback was efficient in improving some of the behavioral concomitants of ADHD in children whose parents favored a nonpharmacological treatment” (p. 1).
Rossiter (2004)	62 ADHD children and adults ages 7-55 were matched to NFB (n = 31) or SM (n = 31) based on patient or parent preference. Patients were matched by (in order) age, sum of 4 baseline TOVA scores, IQ, gender, and ADHD subtype. The SM patients were titrated based on TOVA results and maintained on the dose that maximized TOVA scores. The NFB patients received either 40 sessions in office or 60 at home over 3 to 3.5 months based on standard protocols. Outcome measures were the TOVA for both groups and for the NFB group only the BASC and BADDS.	1. Both the NFB and SM groups had similar significant improvements in attention, impulsivity, and processing speed on the TOVA with no significant differences between groups. 2. The NFB group demonstrated statistically and clinically significant improvement on behavioral measures (BASC, ES = 1.16, and BADDS, ES = 1.59). 3. The author concluded that “confidence interval and nonequivalence null hypothesis testing confirmed that the neurofeedback program produced patient outcomes equivalent to those obtained with stimulant drugs” (p. 233).
Duric, Assmus, Gundersen, and Elegen (2012)	130 ADHD children and adolescents, ages 6 to 18 years, were randomly assigned to receive either (a) NFB, (b) SM, or (c) combined NFB/SM. After randomization, 39 dropped out (36 immediately after randomization), 13 from the NFB group, 15 from the SM group, 11 from the combined group resulting in 91 completing the study; NFB (n = 30), SM (n = 31), and combined (n = 30). The NFB group received 30 40-minute theta/beta sessions 3 times per week for 10 weeks. Outcome measures were the Inattention and Hyperactivity subscales of the parent-rated CMADBD-P.	1. The parents reported highly significant effects of the treatments in reducing the core symptoms of ADHD, but no significant differences between the treatment groups were observed. 2. Although not significant, the NFB group showed twice the level of pre–post change in attention compared with the other two treatments (3.1 vs. 1.1 and 1.5 for the means) and NFB’s effect size was larger than the other two treatments on both the Inattention and Hyperactivity subscales and total score measures. 3. The authors concluded, “NFB produced a significant improvement in the core symptoms of ADHD, which was equivalent to the effects produced by methylphenidate, based on parental reports. This supports the use of NFB as an alternative therapy for children and adolescents with ADHD” (p. 1).
Meisel, Servera, Garcia-Banda, Cardo, and Moreno (2013)	23 ADHD children, ages 7 to 14 years, were randomly assigned to receive either 40 theta/beta NFB (n = 12) or SM (n = 11). Outcome measures were behavioral rating scales completed by fathers, mothers, and teachers (ADHD RS-IV and ODDRS-IV) at baseline and post-treatment as well as 2- and 6-month follow-up of academic performance.	1. In both groups, there were similar significant reductions in ADHD functional impairment as rated by parents and in primary ADHD symptoms by parents and teachers. 2. Significant academic performance improvements were only detected in the NFB group. 3. NFB gains were maintained in both the 2- and 6-month follow-up assessment. 4. The authors concluded, “Our findings provide new evidence for the efficacy of Neurofeedback, and contribute to enlarge the range of non-pharmacological ADHD intervention choices” (p. 12).
Ogrim and Hestad (2013)	32 ADHD children, ages 7 to 16 years, were randomly assigned to receive either 30 sessions of QEEG-guided NFB (n = 16) or SM (n = 16). The 30 NFB sessions took place over 6 to 9 months. Outcome measures were parent and teacher Conners’ Rating Scales, BRIEF, CPT, QEEG and ERP.	1. SM was superior to NFB with a large effect size on the Conners’ Rating Scales and confirmed by other outcome measures. 2. The QEEG spectral power in the theta and beta bands did not change in either group. 3. In ERP, the P3 no-go component increased significantly in eight of 12 SM responder patients, but did not increase in nonresponders or the NF group. 3. The authors concluded, “Our study supports effects for stimulants, but not for NFB. Effects of NFB may require thorough patient selection, frequent training sessions, a system for excluding nonresponders, and active transfer training” (p. 448).
Flisiak-Antonijczuk, Adamowska, Chładzińska-Kiejna, Kalinowski, and Adamowski (2015)	115 ADHD children, ages 6 to 14 years, meeting similar criteria regarding the nature of ADHD were assigned to receive either 20 NFB sessions (n = 85) or MPH adjusted to their age (n = 30). Outcome measure was a structured interview of ADHD symptoms based on DSM-IV criteria.	1. Both treatments significantly reduced (p < .01) the number of attention deficit, hyperactivity and impulsiveness symptoms in subgroups with attention deficit prevalence and mixed type ADHD. 2. There were only four children with hyperactivity and impulsiveness prevalence and none in the MPH group so a comparison between treatments could not be made for this subtype of ADHD. 3. The authors concluded, “The NF method proved similarly effective to methylphenidate in reducing the number of symptoms in two types of ADHD: ADHD with the prevalence of attention deficit and in mixed type ADHD” (p. 31).
Gelade et al. (2016) Gelade et al. (2017) (6-month follow-up findings)	112 ADHD children, ages 7 to 13 years, were randomly assigned to receive either 30 sessions of theta/beta NFB (n = 39), 30 sessions of moderate to vigorous PA (n = 37), or optimally titrated SM (n = 36) over the course of 10 weeks. Optimal SM was determined via the same procedures as the MTA (i.e., double-blind placebo-controlled titration in which subjects received in random order 1-week each of 5 mg, 10 mg, 15 mg, and 20 mg SM along with 1-week of placebo). At the end of each week, parents and teachers completed rating scales of inattention and hyperactivity-impulsivity along with a side effects questionnaire. This information was used to determine optimal SM dosing for 4 weeks prior to administering the post treatment outcome measures. There was no similar involvement of parents and teachers in NFB and PA treatments. Outcome measures were parent and teacher ratings on the SDQ and SWAN.	1. All three treatments evidenced significant improvement on the parent-rated SDQ and SWAN Hyperactivity/Impulsivity scales (p < .001). 2. SM was superior to NFB and PA on the parent-rated SWAN Inattention scale (p < .001) and on all teacher-rated scales (p < .001). 3. Gelade et al. (2016) concluded, “optimally titrated methylphenidate is superior to neurofeedback and physical activity in decreasing ADHD symptoms in children with ADHD” (p. 1) 4. At 6-month follow-up, Gelade et al. (2017) reported, “Interestingly, teacher reports showed less inattention and hyperactivity/impulsivity at follow-up for NFB than PA (p = .004–.010), even after controlling for medication use (p = .013–.036). Our findings indicate that the superior results previously found for parent reports and neurocognitive outcome measures obtained with MPH compared to NFB and PA post intervention became smaller or non-significant at follow-up. Teacher reports suggested superior effects of NFB over PA” (p. 1). COMMENT: As the authors note, they followed a similar strategy as the MTA Cooperative study to determine optimal SM dosing. As Pigott’s (2017) analysis of the MTA study demonstrates, making parents and teachers integral to delivering SM and BT treatments biased the use of their ratings when compared to outcomes of treatments they were not involved with. Similarly in this study, it was the parent and teacher ratings that both identified the optimal SM dose and their subsequent ratings were then compared with parent/teacher ratings of NFB and PA subjects even though there is strong evidence that using parent/teacher ratings biases the report of outcomes favoring those treatments the parents and teachers were most involved in delivering. This is consistent with Gelade et al.’s (2017) 6-month results that showed “outcome measures obtained with MPH compared to NFB and PA post intervention became smaller or non-significant at follow-up” as parents and teachers were “less proximal” in their follow-up assessments to their prior role of providing these assessments weekly for 5 weeks to determine optimal SM dosing.
Moreno-García, Delgado-Pardo, Camacho-Vara de Rey, Meneres-Sancho, and Servera (2015); Moreno-García, Meneres-Sancho, Camacho-Vara de Rey, and Servera (2019) Pigott (2017)	59 ADHD children, ages 7 to 14 years, were randomly assigned to receive either 40 sessions of theta/beta NFB that was tailored based on learning curves (n = 21), BT that combined parent and teacher training along with 15 individualized cognitive therapy sessions for the child (n = 19) and protocol-driven pharmacology (PH; n = 19). Outcomes measures were parent and teacher ADHD RS-IV ratings, parent ADDES ratings, and IVA/CPT.	1. All three treatments evidenced a significant impact in reducing ADHD symptoms based on parent and teacher ratings as well as on IVA measures of attention and response control. 2. While the authors concluded that “From a global perspective, BT had the most extensive results, but PH had the greatest capacity to improve overall attention. NF was able to improve both control response and inattention” (Moreno-García et al., 2019, p. 1), Pigott (2017) documents how this conclusion for BT was based on the biased ratings of parents and teachers who were integral to delivering the package of BT treatments, but not the NFB and PH treatments. 3. Furthermore, as reported by Moreno-García et al. (2015), “Treatment differences observed in attentional variables in post-treatment are not maintained in follow-up phase” (p. 222) thereby indicating that the report that “PH had the greatest capacity to improve overall attention” was not maintained in the follow-up IVA/CPT assessment.
Li, Yang, Zhuo, and Wang (2013) (NFB/SM combination RCT)	40 ADHD children, ages 7 to 16, were randomly assigned to combined NFB and SM or SM combined with attention training that used the identical instructions and game sequences as NFB except the feedback was not based on subjects’ EEG. Subjects were assessed using multiple parameters at baseline, after 20 treatment sessions, after 40 treatment sessions, and at 6-month follow-up.	1. The study found that “compared to the control group, patients in the combination NFB/SM group had reduced ADHD symptoms and improved in related behavioural and brain function” (p. 1). 2. The combined SM/NFB subjects used significantly lower doses of SM during 6-month follow-up and reported fewer adverse side effects. 3. The authors concluded, “The combination of EEG feedback and methylphenidate treatment is more effective than methylphenidate alone. The combined therapy is especially suitable for children and adolescents with ADHD who insufficiently respond to single drug treatment or experience drug side effects” (p. 1).

Note. IQ = intelligence quotient; NFB = neurofeedback; EEG = electroencephalogram; SM = stimulant medication; TOVA = Test of Variables of Attention; SMR = sensory motor rhythm; CBRS = Conners’ Behavior Rating Scale; BASC = Behavior Assessment System for Children; BADDS = Brown Attention Deficit Disorder Scales; ES = effect size; CMADBD-P = Clinician’s Manual for the Assessment of Disruptive Behavior Disorders–Rating Scale for Parents; ADHD-RS-IV = ADHD Rating Scale-IV; ODDRS-IV = Oppositional defiant disorder rating scale based on DSM-IV; BRIEF = Behavior Rating Inventory for Executive Function; CPT = Continuous Performance Test; DSM = Diagnostic and statistical manual of mental disorders; PA = physical activity; SDQ = Strength and Difficulty Questionnaire; SWAN = Strengths and Weaknesses of ADHD symptoms and Normal Behavior Scale; MPH = methylphenidate; BT = behavior therapy; ADDES = Attention Deficit Disorder Evaluation Scale; IVA = Integrated Visual and Auditory; CPT = Continuous Performance Task; RCT = randomized controlled trials; QEEG = quantitative EEG; ERP = evoked response potential; MTA = multimodal treatment study of children with ADHD; PH = pharmacology.

Fourth, Thibault and colleagues fail to acknowledge the extensive evidence from NFB studies whose training methodology mirror the best practices of operant conditioning. These studies consistently find NFB subjects learn to self-modulate the targeted EEG, this learning is associated with improvements on a wide variety of ADHD outcome measures of interest, and both are sustained at follow-up (e.g., Leins et al., 2007; Strehl et al., 2017; Strehl et al., 2006) even up to 2 years later (Gani, Birbaumer, & Strehl, 2008). These findings demonstrating the sustained ability to self-modulate the targeted EEG during follow-up with ongoing symptomatic improvement are unlike anything in the placebo literature. Findings further buttressed by Doren et al.’s (2018) recent meta-analysis documenting the sustained effects on ADHD outcomes for NFB subjects in RCTs.

The NFB Field Shares the Blame

Although NFB’s origins are based in the science of learning, the field has been negligent at ensuring that clinicians, researchers, and device manufactures adhere to this science. Examples include the following:

• Monitoring within-session learning curves is not standard practice for NFB clinicians, and in fact it is our observation that most clinicians do not assess for evidence of learning.

• The vast majority of NFB studies do not assess for evidence of learning even though this is the presumed mechanism of change. It is only recently that this is required for publication in the industry-sponsored journal NeuroRegulation when authors claim to provide operant conditioning of the EEG.

• Virtually all device manufacturers include an auto-thresholding option despite Sherlin and colleagues (2011) clarion call that such systems violate learning science and “could effectively train in the opposite direction and result in an increase in aberrant and negative (EEG) behaviors” (p. 299). Unfortunately, this option is used by many, if not most, clinicians, particularly those who oversee multiple “NFB” sessions at a time.

As with any form of operant conditioning, there are learners and nonlearners. The same is true with NFB and it is learners who experience the most improvement on ADHD outcome measures. Given this fact, it is an indictment of the field that there are no studies comparing strategies to identify those practices that best promote subjects’ learning to self-modulate the targeted EEG. Consequently, there is no empirical guidance to determine which operant training methodologies are most effective in maximizing learning—and this in a field with a 75+ year history of basic and applied research.

Finally, the NFB field and its detractors continue to conduct research that violates behavioral principles, and both sides cite such substandard research when it supports their viewpoint. This practice must stop. Evidence of learning trumps all, and if there is no evidence of learning, operant conditioning of the EEG did not occur.

Conclusion

There is plenty of blame to go around, yet if the field is to evolve and progress, we must demand training methodologies that follow learning principles and proof that learning occurred from all who claim to perform NFB. Hence, our critique of Thibault and colleagues, and the sham-controlled studies on which their argument is based, is also a plea to NFB researchers and clinicians to demonstrate that their methods are consistent with the best practices in behavioral learning. If both sides can agree to this rigor, it will promote clarity and consistency in the NFB literature that is not present today and provide guidance to necessary steps for advancing it forward.

Most importantly, we hope our Guest Editorial conveys the truth that learning methodology matters. With this caveat, we strongly recommend operant conditioning of the EEG for the treatment of ADHD, either as a standalone treatment or augmentation to other evidence-based treatments. As for prescribing “neurosuggestion therapy” for the treatment of ADHD, more research is required since its underlying premise is unproven and evidentiary base anecdotal.