Abstract
The post-reinforcement pause (PRP) is an operant effect in which response latencies increase on trials following the receipt and consumption of reward. Human studies demonstrate analogous effects in electronic gambling machines that utilise random ratio reinforcement schedules. We sought to identify moderators of the human PRP effect, hypothesising that the magnitude of gamblers’ PRPs is moderated by the type of reinforcing outcome (genuine wins vs. losses-disguised-as-wins [LDWs] vs. free-spin bonus features) and individuals’ level of gambling immersion, a cognitive state linked to problem gambling. Experienced slot machine users (N = 53) played a real slot machine for 20 min. The dependent variable was defined as the time delay in the initiation of each bet (“Spin Initiation Latency”; SIL). Using 80% of trials, a linear model was fit regressing SIL on the independent variables (outcome type, immersion, and outcome-by-immersion interaction), and a larger group of covariates (participant ID, trial number, winnings, etc.) selected using double-robust LASSO-regularised regression. The previously unseen 20% of cases were used to validate the model. Positively reinforcing outcome types (wins, LDWs, bonus spins) showed significantly larger SILs than losses, indicating a PRP effect. Immersion did not predict response latencies, but win-by-immersion and LDW-by-immersion interactions indicated that pauses were greater among more immersed participants. The small number of free-spin bonus features showed similar trends that were not statistically significant. These results indicate that gamblers immersed in play remained sensitive to in-game reinforcement (contrary to a prevailing account), and provide guidance for researchers bridging laboratory research and real-world behaviour.
Introduction
The post-reinforcement pause (PRP) is an operant behavioural effect that was initially established in animal research. It describes a prolongation of the time to initiate the operant response following positive reinforcement delivery (compared with non-reinforced trials) on ratio-type reinforcement schedules (Ferster & Skinner, 1957; Schlinger et al., 2008). Manipulating a fixed ratio schedule, two classic experiments in pigeons found that PRPs increased linearly with ratio size (Felton & Lyon, 1966; Powell, 1968). These effects could not be explained by the animals’ levels of fatigue or satiety (Schlinger et al., 2008), implicating instead the processing of reward value. These non-human studies thus suggest that the PRP may reflect the subjective valuation of obtained rewards.
Recently, the PRP effect has shown considerable utility in research on the use of digital gambling products like slot machines and video poker. These devices’ fast-paced, continuous format has long been recognised as a real-world example of risky decision-making and reinforcement learning on random ratio schedules (Clark et al., 2013; Haw, 2008; Skinner, 1953). Once a given bet is completed (e.g., when a slot machine’s reels have stopped spinning and any celebratory audiovisual feedback has concluded), the latency to initiate the next spin—hereafter called “Spin Initiation Latencies” (SILs)—tend to be longer following positively-reinforced “winning” outcomes compared with non-reinforced “losing” outcomes, in line with the classic PRP effect (Chu et al., 2018; Delfabbro & Winefield, 1999; Dixon et al., 2014). A PRP effect has also been observed with loot boxes, a gambling-style mechanic for distributing randomised digital objects inside video games (Larche et al., 2021). The duration of SILs in gambling tasks correlates positively with the size of wins (Delfabbro & Winefield, 1999; Limbrick-Oldfield et al., 2022), and with prior practice on the slot machine (Ferrari et al., 2022). Moreover, in laboratory experiments that have employed a fully-neutral baseline outcome (which does not typically occur in authentic slot machines because of the loss of the wager), wins were seen to increase the SIL but in addition, loss outcomes shortened the SIL as a possible index of frustrative non-reward (Eben et al., 2020, 2023; Verbruggen et al., 2017). Thus, extant studies measuring SILs suggest the PRP is a cross-species indicator of reward processing that appears to span ratio-type reinforcement schedules, and which contains both appetitive and aversive responses.
Modern gambling products—including slot machines—have become increasingly sophisticated in recent decades (Newall et al., 2021; Schüll, 2012) and have introduced new forms of positive reinforcement that are different from the typical definition of wins wherein the payout exceeds the wager. One set of innovations concerns bonus features, of which the free-spin bonus feature is the most pervasive (Parke & Griffiths, 2006). These are highly stimulating and sustained events in which the gambler is awarded with a number of free plays in lieu of a guaranteed cash payout. Another form of reinforcement is the loss-disguised-as-a-win (LDW), which occurs when: (1) the cash payout is worth less than the amount bet and (2) the device plays celebratory audiovisual feedback nevertheless (Dixon et al., 2010). Although research on free-spin bonus features is scant (but see Taylor et al., 2017), considerable experimental evidence has shown that exposure to LDWs leads users to significantly overestimate how often they are winning (Barton et al., 2017; Myles, Carter, et al., 2023).
As the gulf widens between the programming capacities of slot machine developers and gambling researchers, it is increasingly unfeasible to create convincing facsimiles of modern gambling products. As a result, gambling researchers are moving to more naturalistic designs that use genuine slot machines, housed in a laboratory environment (Murch et al., 2017; Stewart et al., 2002). Although this approach engenders a more convincing gambling experience to research participants, the use of real slot machines requires measuring and statistically controlling for variables that randomly vary between testing sessions, such as net profit or loss (Ferrari et al., 2022; Limbrick-Oldfield et al., 2022; see the double-robust feature selection procedure described in Methods below). In addition, variables inherent to laboratory research must be modelled so that their influence on PRP magnitude does not confound observed results (e.g., trial number; Ferrari et al., 2022).
In addition to the potential influences that outcome type and these control variables may exert on PRPs during slot machine use, the impact of the gamblers’ cognitive state during play is not well understood. In one laboratory study, impulsivity and stress jointly impacted the PRP effect in a multi-stage decision-making task (Raio et al., 2020). We were specifically interested in whether the PRP effect depends on immersion in play, a state of high focus and absorption in gambling that may interfere with attention to external stimuli and goals. Immersion in gambling has been linked with individuals’ risk of experiencing gambling-related harms and Gambling Disorder across a variety of activities and research contexts (Murch & Clark, 2021; Rogier et al., 2021). The experience is typically reported as a feeling of losing track of time, forgetting about all things beyond the game, or experiencing a “trance-like” state that may include depersonalisation and derealisation.
A prominent qualitative account of immersion in gambling suggests that the experience is marked by an insensitivity to reinforcement (Schüll, 2012). This resonates with a wider narrative that argues winning money is not the goal of the activity but simply a necessity for continued gambling (Reith, 2007); indeed, larger wins might even interfere with the immersed state. From this perspective, we could hypothesise that the PRP effect will be blunted among more immersed gamblers. However, other evidence suggests the opposite may be true: our previous analyses using eye tracking found that self-reported immersion was correlated with a higher overall number of eye movements (saccades) and an increased time spent looking at the slot machine’s credit display compared with the spinning reels (Murch, Limbrick-Oldfield, et al., 2020). Unlike the spinning reels of the screen, the credit display is not a stimulating aspect of the game. It does, however, indicate how much further gambling is possible. We thus interpreted those effects as evidence that immersion is a hyper-attentive state of intense concentration and performance monitoring (Murch & Clark, 2021).
In this secondary analysis of data from the earlier eye-tracking project, we sought to investigate whether the cognitive experience of immersion in slot machine gambling moderates the PRP and whether this effect further varies with the nature of reinforcement. We hypothesise that:
SILs for winning spins on a real slot machine will be longer than SILs for losing spins, indicating a PRP effect,
PRPs will generalise to free-spin bonus features and LDW outcomes,
Immersion will affect SIL duration, and
Immersion will interact with outcome type in affecting SIL duration.
Methods
Participants
Eye tracking analyses from this sample have been reported in two previous publications (Kim et al., 2022; Murch, Limbrick-Oldfield, et al., 2020). Hypotheses in earlier studies were pre-registered, but the current study’s hypotheses were not.
Participants were experienced slot machine users living in or near Vancouver, Canada, who responded to listings on the community message board Craigslist.ca (N = 53; 32 men and 21 women; mean age = 33.53, SD = 12.30). Inclusion criteria required all participants to be at least 19 years old and have no neuropsychiatric diseases, ophthalmic diseases, psychotropic medication use, or neurological problems resulting from traumatic brain injury. Participants were excluded if they reported high-risk problem gambling (see “Questionnaires” section) or visual acuity requiring correction beyond ± 4 diopters. This experiment was approved by the host university’s Behavioural Research Ethics Board.
Questionnaires
We measured past-year gambling problems using the Problem Gambling Severity Index (PGSI), a widely adopted and validated instrument (Currie et al., 2013; Ferris & Wynne, 2001; Holtgraves, 2009; Williams & Volberg, 2014). The PGSI uses nine items to assess past-year experiences of symptoms and consequences related to excessive gambling. It is scored on a 4-point Likert-type scale ranging from “Never” (0) to “Almost always” (3), with scores greater than or equal to eight out of 27 indicating high-risk problem gambling.
Immersion during the gambling sessions was defined as the combined average of the 5-item modified Dissociation Questionnaire (e.g., “I felt like I was in a trance while playing the slot machine,” “I lost track of time while playing the slot machine”) (Diskin & Hodgins, 1999; Jacobs, 1988), and the 2-item Flow subscale of the Game Experience Questionnaire (e.g., “I felt completely absorbed”) (IJsselsteijn et al., 2013; Poels & de Kort, 2007). We have previously shown good internal consistency for this combined immersion questionnaire (Murch & Clark, 2019). Items were scored on a 5-point Likert-type scale ranging from “Very slightly or not at all (0),” to “Extremely (4),” averaged, and then standardised across participants.
Procedure
Participants provided signed consent, and completed the PGSI. They then gambled on “Buffalo Spirit” (Scientific Games Co., Las Vegas, NV), a popular slot machine that was widely available in local gambling venues at the time of the study. Participants were endowed with $40 CAD for their session and asked to continue betting until all funds were exhausted (n = 18) or until 20 min had elapsed (n = 35). To avoid confounding PRPs with decision-making processes related to selecting different betting strategies, participants were instructed to place bets using the popular “maxi-min” strategy that uses the maximum number of concurrent bets (40 pay-lines) at the minimum stake ($0.01 CAD) (Livingstone & Woolley, 2008; Templeton et al., 2015); a total value of $0.40 CAD per bet.
Participants initiated each “trial” (in the form of placing a bet on the slot machine) by pressing the spin button on the right-hand side of the fascia. The device’s reels spun for approximately 4–6 s (Chu et al., 2018) and stopped to reveal one of four possible outcomes: a loss, a win, a loss disguised as a win, or a free-spin bonus feature (which awarded 15 or more free spins that were treated as a single event). We note that the frequencies of these outcomes were highly unbalanced. In particular, the free-spin bonus features are rare; the median participant received one free-spin bonus feature, and the first quartile of participants had none (Table 1). Following each spin, any payout and its concomitant audiovisual feedback were presented. After any audiovisuals finished, the device was silent, waiting for the participant to initiate the next spin. This interval was coded as the SIL; there was no audiovisual accompaniment to losses, so the SIL was simply from the end of the reel spin to the next participants’ response.
Descriptive results.
Note: “LDWs” = Losses-Disguised-as-Wins. “Min.” = Minimum. “Max.” = Maximum. “Q1” and “Q3” refer to the 25th and 75th percentiles, respectively.
Immediately following the slot machine session, participants completed the immersion questionnaire. They were paid $20 CAD for participating and up to $20 extra for any profits accrued over the course of the gambling session.
Data processing
Each gambling session was recorded using a video capture of the slot machine screen. Then, using an image recognition programme developed in Python 2.7 using OpenCV (Intel, Santa Clara, CA), we processed the video files to create a trial-by-trial timeseries for each participant, which contained the outcome types (loss, LDW, win, or bonus feature) and corresponding durations, and the participants’ responses. The SILs were calculated in milliseconds from the offset of any audiovisual feedback to the onset of the participant’s next response, at a resolution determined by the video frame rate. The four binary outcome factors, participants’ immersion ratings, and the four immersion-by-outcome interaction terms were the independent variables entered at the final modelling stage described below.
To improve the model’s fit, we identified a group of potential covariates which included trial number (centred on 1), credits held prior to the current spin (centred on 4000, the total endowed amount), the amount of any winnings, and whether the trial occurred while the participant was in a state of profit prior (i.e., if their credit balance exceeded the initial endowment). These four covariates, their interaction terms (10), their squares (4), and individual participant factors (53) were entered into the double-robust feature selection procedure described next. Covariates surviving that procedure joined the independent variables as inputs in the final model.
Data analysis
Primary analyses were completed in the R statistical computing language using the glmnet package (Hastie et al., 2021; R Core Team, 2023). The data and annotated analysis script are publicly archived. 1 The distributions of all study variables were inspected. Visual inspection of boxplots representing the dependent variable, SIL, showed evidence of outlying values. This could indicate pauses taken by participants that were unrelated to the experiment per se (e.g., pausing play to ask questions about the task). It could also indicate that some participants employed a “button-mashing” strategy to ensure the fastest possible pace of play while making their behaviour insensitive to trial outcomes. We thus Winsorized the most extreme 2.5% of cases on each end of the distribution. After doing so, the resulting distribution of SIL scores had fewer outliers but remained positively skewed. For the purposes of later regression modelling, it was naturally log-transformed. The resulting distribution appeared symmetrical (though somewhat platykurtic; see archived analysis ledger; see Note 1).
Our modelling strategy employed the double-robust LASSO regression procedure, a method for covariate selection employed in the field of machine learning (Belloni et al., 2013; Tibshirani, 1996). This approach for selecting relevant covariates to be fed into a planned linear regression model is a variation on fixed effects regression procedures used in earlier research (Allison, 2012; Chu et al., 2018; Murch, Ferrari, et al., 2020). It seeks to balance the need for correctly-specified models against the propensity for regression models to capitalise on error variance. The participant factor was not modelled as a random effect because earlier validations of the double-robust LASSO method utilised only fixed factors (Belloni et al., 2013), although we note that treating participant ID as a random effect did not affect the results reported here (see Note 1).
First, the data were randomly split into a model training (80%; 7,478 trials) and a model validation (20%; 1,861 trials) sample that retained equal proportions of each outcome type. Then, the group of 71 potential covariates was systematically reduced using the training data by regressing the independent (outcome type) and dependent (SIL) variables on them. As LASSO regularisation removes irrelevant inputs by design, covariates that remained in either model were understood to have some relationship with either the independent or dependent variables in this study. For each LASSO model in this step, hyperparameter optimisation was performed using 10-fold cross-validation in the training sample. The selected λ values were those that resulted in the simplest model within 1 standard error of the value that minimised the mean-squared error of the cross-validation holdout data (Hastie et al., 2021).
Next, using the training dataset, the dependent variable (SIL) was regressed on the independent variables (outcome type, immersion, and immersion-by-outcome interaction terms) and the 56 remaining covariates (46 of which identified individual participant effects; see the online Supplementary Materials Table 1) using ordinary least squares. Model assumptions were checked by visually inspecting residual and Q–Q plots, as well as a correlation matrix of the predictors (see archived analyses). The model was then validated using the holdout dataset comprised of 20% of previously unseen cases. For additional details, see archived materials (see Note 1).
Results
Descriptive results are summarised in Table 1. The median participant completed 178 trials (“bets”). They received 35 reinforcing outcomes, of which 16 were genuine wins, 18 were LDWs, and one was a free-spin bonus feature. The median participant made 10.58% of bets while in a state of profit and finished the study with $15.50 remaining of the original $40.00 endowment. A minority of participants never saw a free-spin bonus feature (14/53) or a state of overall profit (15/53). Immersion scores ranged from 0.14 to 2.86 out of 4, with a median of 1.14. This suggests that slot machine sessions were typically rated as “a little” immersive.
Model validation
Figure 1 depicts the model’s predictive performance with the previously unseen data. Actual SIL values appear well-distributed across the full range of predictions made. The median SIL in the test group was 1.26 s (Min. = 0.40, Q1 = 0.90, Q3 = 1.91, Max. = 4.37). The exponentiated root-mean-squared error of model-predicted natural log SILs was 1.47 s.
Model predictions in the validation sample.
Model results
The fit of independent variables in the SIL model is presented in Table 2, and full model results are available in the Supplementary Materials Table 1. Wins, LDWs and free-spin bonus features all showed significant positive relationships with SIL duration (all p < .001; see Table 2 and Figure 2), consistent with a PRP effect. Immersion score did not predict SIL duration, b = −0.006, 95% CI [−0.03, 0.02], t(7,414) = −0.43, p = .665, but interacted significantly and positively with winning outcomes, b = 0.04, 95% CI [0.01, 0.07], t(7,414) = 2.31, p = .021, and LDW outcomes, b = 0.05, 95% CI [0.02, 0.08], t(7,414) = 3.41, p = .001; see Figure 2. The interaction between immersion and the rarer free-spin bonus features, though similar in magnitude to that of wins and LDWs (Figure 2), was not statistically significant, b = 0.08, 95% CI [−0.04, 0.21], t(7,414) = 1.28, p = .201. In the model, 50 of 56 model covariates were significantly related to SIL duration and included trial number, the number of credits won, and most participant ID factors (see Supplementary Materials Table 1).
Linear regression model: independent variables’ relationships with SIL.
Note: Covariates in this model are not shown, but are detailed in Supplementary Materials Table 1. Loss trials were set as the reference category to prevent complete separation in the model. “x” indicates interaction term. “LDWs” = Losses-Disguised-as-Wins. “Bonus” = Free-spin bonus feature.
SILs stratified by outcome type and immersion level.
Discussion
In this study, we explored whether positively reinforcing outcome types and self-reported immersion moderate consummatory pauses in the latency between spins—the PRP effect—during slot machine gambling, using an authentic gambling terminal situated in a laboratory environment. We found clear evidence of PRPs: trials involving any form of positive reinforcement were associated with significant increases in the SIL. Real wins, free-spin bonus features, and losses that were “disguised” as wins via win-concurrent audiovisual feedback all showed a reliable PRP effect. Although free-spin bonus features were quite infrequent in these data, Figure 2 shows that the largest PRP effects occurred for bonus features, followed by wins, followed by LDWs. This gradation is consistent with pupillary indicators of reward reported in our earlier eye-tracking experiment (Kim et al., 2022), as well as a separate study exploring PRPs and skin conductance responses in a video gaming context (Larche et al., 2021). Thus, these response latency effects on gambling initiation appear to be sensitive to reward valuation, as seen in the original literature in non-human animals. We suspect gamblers’ perception of reward is in fact closely linked to the celebratory audiovisual effects, which are most intense during free-spin bonuses and least jubilant following LDWs. This is supported by evidence that presenting distinct audiovisuals with a negative valence to LDWs diminished the PRP effect (Scarfe et al., 2021).
Our results also support the existence of cognitive influences on the PRP effect: consummatory pauses following wins and LDWs were significantly larger among participants who reported a greater sense of immersion in play. Pauses were also observed to a numerically similar degree following free-spin bonus features, but the rarity of those outcomes in this study severely diminished our power to detect an effect (see Figure 2). In any case, the significant interaction effects for wins and LDWs with immersion run contrary to the notion that immersed slot machine gamblers are rendered “insensitive” to positive reinforcement and trial-by-trial outcome value (Oakes et al., 2018; Schüll, 2012). From that perspective, one might expect a clear rhythm to the pace of slot machine gambling (e.g., one spin every 4 s; Chu et al., 2018). Instead, immersed gamblers show a greater adjustment in their pace of play as a function of reinforcing vs. losing outcomes. Thus, our data support an alternative account that gambling immersion represents a “zoned in” kind of focus associated with enhanced reactivity to positively reinforcing in-game outcomes (Murch, Limbrick-Oldfield, et al., 2020).
Limitations
Several limitations are of note. First, our hypotheses were exploratory in nature, and pre-registered replication is needed. Second, different slot machine outcomes occur with different frequencies, and there is some inherent correlation between financial factors and specific outcomes. In particular, the fact that most participants saw only one or two bonus features while a minority saw no bonus features is an important limitation on our estimate of these outcomes’ impact on the PRP. These unavoidable links confound reported coefficients to some extent, and experimental research could further clarify the magnitude of the effects reported here. Finally, by excluding people who reported a high risk of experiencing past-year gambling problems, our results reflect only a subset of the gambling population and may not generalise to at-risk or clinically relevant samples. Notably, however, the well-established relationship between problem gambling severity and experiences of immersion during gambling (Murch & Clark, 2021; Rogier et al., 2021) suggests that these effects may occur in high-risk populations and that the magnitude of these effects may in fact be larger in this group.
Conclusion
This exploratory study provides an approach to using human behaviour on real slot machines to corroborate effects classically observed in animal studies and simplified laboratory tasks. Our findings showed that positively reinforcing outcomes during slot machine gambling are associated with PRPs and that this behaviour is moderated by immersion, a cognitive state that is linked to problem gambling. These analyses illustrate the sensitivity of the human PRP effect in quasi-naturalistic settings (i.e., an authentic gambling product in the lab) and provide novel insights into the links between cognitive states and operant reward-based behaviour.
Our results raise an interesting question for future research and public policy: should celebratory audiovisual stimuli be made consistent with the objective receipt of monetary windfalls, to potentially diminish the erroneous perception of winning (see also Barton et al., 2017; Scarfe et al., 2021)? Positive audiovisual feedback for net-negative trials (including LDWs) is already prohibited in Australia’s states of Queensland and Tasmania (Livingstone et al., 2019; Myles, Bennett, et al., 2023). Alternatively, gamblers may feel that celebratory audiovisual stimuli are a source of entertainment that represent non-monetary rewards in their own right and that these devices should not be modified. Further research is needed to clarify gamblers’ perceptions and preferences to find effective harm prevention strategies that are desirable to the populations who are impacted. In addition to informing the nature of future design and regulation for digital gambling products, these results could help to translate between insights gathered from gambling activities, video games, and social casino apps, which each envision a different kind of reward resulting from their use (Willson & Leaver, 2016).
Supplemental Material
sj-pdf-1-qjp-10.1177_17470218241239054 – Supplemental material for Post-reinforcement pauses during slot machine gambling are moderated by immersion
Supplemental material, sj-pdf-1-qjp-10.1177_17470218241239054 for Post-reinforcement pauses during slot machine gambling are moderated by immersion by W. Spencer Murch, Mario A Ferrari and Luke Clark in Quarterly Journal of Experimental Psychology
Footnotes
Acknowledgements
The authors would like to thank Dr Mariya Cherkasova, Dr Jolande Fooken, Dr Eve Limbrick-Oldfield, Dr Miriam Spering, and Kent MacDonald for their support on the eye tracking and screen capture components of the study.
Declaration of conflicting interests
The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: W.S.M. received training and funding from the Centre for Gambling Research at UBC, a research laboratory jointly supported by the Government of British Columbia and the British Columbia Lottery Corporation (BCLC; a Canadian Crown Corporation). M.A.F. has received a speaker honorarium from the BCLC, a Canadian Crown Corporation. M.A.F. has not received any further direct or indirect payments from the gambling industry or groups substantially funded by gambling. LC is the Director of the Centre for Gambling Research at UBC, which is supported by funding from the Province of British Columbia and the British Columbia Lottery Corporation (BCLC), a Canadian Crown Corporation. The Province of BC government and the BCLC had no role in the preparation of this article and impose no constraints on publishing. LC has received travel expenses from Scientific Affairs (Germany), the International Center for Responsible Gaming (US), and the Institut fur Glucksspiel und Gesellschaft (Germany). He has received fees for academic services and consultancy from Scientific Affairs (Germany), the International Center for Responsible Gaming (US), GambleAware (UK), Gambling Research Australia, and Gambling Research Exchange Ontario (Canada). He has been remunerated for legal consultancy by the BCLC. He has not received any further direct or indirect payments from the gambling industry or groups substantially funded by gambling. LC receives an honorarium for his role as Co-Editor-in-Chief for International Gambling Studies from Taylor & Francis, and he has received royalties from Cambridge Cognition Ltd. relating to neurocognitive testing.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was funded by the Centre for Gambling Research at UBC, a Discovery Grant from the Natural Sciences and Engineering Research Council of Canada (NSERC; grant number RGPIN-201704069) awarded to L.C., and NSERC doctoral scholarships awarded to W.S.M. and M.A.F.
Data accessibility statement
Supplementary material
The supplementary material is available at qjep.sagepub.com.
Notes
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
