An evaluation of the hot-hand hypothesis in the NBA when accounting for shot difficulty

Abstract

The hot-hand hypothesis suggests that athletes can experience periods of exceptional performance where the success of their next action is influenced by previous successes. This study examines the hot-hand phenomenon in the context of the NBA, focusing on shot difficulty as a critical factor. Previous research has yielded conflicting results, often ignoring the varying difficulty of shots and the role of defensive pressure. Utilizing shot log data from the 2014-2015 NBA season, this paper introduces new models that account for shot difficulty with defender proximity. By employing XGBoost classifiers and difficulty-weighted metrics, we find that while traditional hot-hand effects are minimal, adjusting for shot difficulty reveals a reverse hot-hand effect: players tend to perform worse following streaks of successful but difficult shots. This research provides a nuanced understanding of the hot-hand phenomenon, highlighting the importance of considering shot difficulty in sports performance analysis. These findings have implications for coaching strategies and the psychological interpretation of performance streaks in competitive basketball.

Introduction

The hot-hand hypothesis, a concept important in sports psychology and culture, asserts that athletes can experience a brief period of time marked by above-average performance. Translating this broader idea to a basketball setting, the hypothesis posits that a player who has made a few shots in a row is more likely to make their next shot than one who has had less prior success. This idea gained prominence in the basketball world following research performed by Gilovich, Vallone, and Tversky. ¹ Their work involved looking at shooting records of the Boston Celtics and Philadelphia 76ers, as well as observing players in controlled environments. Following their studies, they concluded that streaks of successful shots, regardless of length, were statistically indistinguishable from those created via random chance. In essence, their work contradicted the so-called hot-hand hypothesis. ¹

Despite this, the premise of having a hot hand still exists among players, coaches, and spectators alike. Modern perceptions of the hot-hand hypothesis are often based on frameworks regarding the psychological and tactical aspects of sports. Indeed, this concept has been studied in several sports, sometimes with other terminologies such as “momentum” or “streakiness”. For example, in the setting of Grand Slam tennis, Meier et al. demonstrates the importance of psychological momentum shifts when analyzing player performance. ² Page and Coates take a social approach to their analysis of momentum in tennis and find that winning a set makes it more likely to win the next set in men's singles matches. ³ Examining frameworks of the hot-hand hypothesis that tackle the psychological and tactical aspects of the game can provide important context when discussing the associated statistical results. As a motivating factor, we turn to Morgulev who suggests bridging the gap between psychology and in-game statistics can be beneficial for understanding the hot-hand hypothesis. ⁴

To highlight this effect, we note that viewers often gravitate to players associated with incredible streaks of successful shots due to the excitement evoked from such events. Such streaks are then attributed to a player's in-game confidence and skill, thereby increasing viewers’ expectations for their next shots. This introduces a sort of positive feedback loop in which viewers witness many consecutively made shots and reinforce their own belief in the hot hand. However, this phenomenon extends past bystanders into active members of the game as well. Coaches and players alike often execute plays to get the ball in the hands of those who are on a hot streak. ⁵ This can be attributed to the belief that making a few shots in a row enhances the chances of making one's next shot and such an idea is integral to winning strategies in basketball.

Going beyond the psychological motivations behind in-game strategic changes, prior research has focused on different frameworks of evaluating the hot-hand hypothesis. Meier et al. examine team-based momentum shifts. ² In this, the framework deviates from a standard view of the hot-hand hypothesis and focuses on the impact external factors such as TV timeouts on team points scored during extended runs. On the other hand, Attali uses a singular shot framework and finds that even single successful shots can have adverse effects on a player's next shot due to increased shot distance. ⁶ Compared to that work, we extend our framework of interest to highlight the effects of having a hot hand when considering extended runs of individual play. In a sense, we combine the longer nature of Weimer, Steinert-Threlkeld, and Coltin, ⁷ while focusing specifically on player performance as in Attali. ⁶ With such a framework, we can apply statistical analysis to determine the validity of the hypothesis.

Additional research in the past has also focused on many different aspects of the hot-hand hypothesis, with different types of data being considered. Specifically, much of the existing literature focuses on how this phenomenon exists in free throw attempts. This is especially notable since studies have found that there are brief periods of increased accuracy in free throws following prior successful shots. ⁸ However, these effects are only observable over short time-frames and in general, for longer sequences of free throws, players are not subject to a hot hand. These results are validated even in the face of confounding variables that may affect shot performance due to the use of a multivariate framework.

Although the results reported by Arkes are solely for free throw attempts, similar conclusions related to labeling the hot-hand phenomenon as a fallacy have been found in related studies. ⁸ Lantis and Nesson corroborate the findings of a small hot hand regarding free throws. ⁹ Specifically, the effect is minute when considering only the previous free throw, but doubles when considering the past three or four free throws. This conclusion may not necessarily hold when considering longer shot streaks of free throws. Moreover, via regression analysis, they also claim that a made shot from the field does not increase the probability of making one's next shot. Interestingly, making a shot reduces the expected number of points during the team's next possession. ⁹

On the other hand, much research has been done in support of the opposite conclusion, which is that the hot-hand phenomenon is real and applicable to players in the NBA. Methodologies employed vary from mathematical models and game theory principles to Bayesian analysis. Some studies find a strong existence of players having a hot hand in both field goals and free throws through the use of simulations and equilibrium models in competitive environments. ¹⁰ However, they also find that enhanced performance may be short-lived due to a shifting of or increase in defensive strategies targeted towards the player with the hot hand. These results are replicated in studies that use a Bayes factor test to compare hidden Markov and constant models, but only for longer streaks of shots or extreme cases of streakiness. ¹¹ Other studies consider near-controlled shooting environments such as the NBA's Three Point Contest, in which participants face no defense and attempt only three-point spots from similar distances. Koehler and Conley ¹² find no evidence of the hot-hand hypothesis across players, while Miller and Sanjurjo ¹³ directly refute this claim. Lantis and Nesson somewhat side with the latter, where they find a hot-hand effect is present, but only when considering shots taken in the same location during the Three Point Contest. ¹⁴

While these results may be valid, it is important to consider that none take the difficulty of shots into account, and some even omit an analysis that includes defender data. In fact, few studies have taken shot difficulty into account, perhaps due to the lack of reliable and specific data needed to accurately measure shot difficulty. Furthermore, much of the data and methods used by professional teams have been made proprietary and/or remains impractical for usage in large-scale research, such as the methods used by Chang et al. ¹⁵ Introducing difficulty into the hot-hand hypothesis also adds a different layer of analysis that may not be amenable to most testing mechanisms. Regardless of these obstacles, regression analysis-based studies that control for player and game effects appear to show that the hot-hand hypothesis is valid when accounting for shot difficulty. ¹⁶ Although, this may be attributed to an effect described by Csapo et al., in which making more shots leads to taking more difficult shots while not experiencing any actual decrease in field goal percentage. ¹⁷

With respect to existing research, our proposed methodology deviates from the norm in five major ways. The most apparent difference is the incorporation of in-game shot difficulty as opposed to semi-controlled environments like free throws,^8,9 or the Three Point Contest.^12,13,14 In addition, our formulation of the hypothesis itself is based on conditional probabilities and hypothesis testing is used to determine whether differences in probabilities are statistically significant. This frequentist approach is inherently different from studies with a Bayesian perspective and those that employ regression analysis.^11,16 Third, unlike single-shot frameworks like the one proposed by Attali, we consider the effects found in longer shot streaks. ⁶ Fourth, we expand the realm of analysis to league-wide shots rather than narrowing the scope to a case-by-case basis for individual players as done by Csapo et al. ¹⁷ These two differences are key in understanding how trends are shaped over longer periods of gameplay across the league, which provides a more holistic overview of the hot-hand hypothesis. Finally, we use models that incorporate readily available shot characteristic data using intuitive feature engineering to quantify shot difficulty. Instead of using linear or logistic regression, we use an XGBoost classifier to enhance accuracy. This allows us to more clearly measure shot difficulty as compared to existing research which oftentimes uses only shot and defender distance as a proxy for difficulty.^6,18 With this methodology, we find evidence that the hot-hand hypothesis exists, but in reverse, and only when accounting for prior shot difficulty via point metrics. That is, we find that performing better in previous shot streaks as measured by difficulty-adjusted metrics correlates with worse performance in future shots.

Problem formulation

In this section, we provide some notation and then formulate standard and difficulty-adjusted hot-hand hypotheses (abbreviated as HHH). Consider an individual shot with outcome ${\mathrm{s}}_{\mathrm{m}}\in \{0,1\}$ , where 0 indicates a miss and 1 indicates a make. The standard hot-hand hypothesis is interested in finding the probability of making this $m^{th}$ shot given the outcome of the previous n shots $(s_{m-1},s_{m-2},\dots ,s_{m-n})$ with varying values of n. Let $S_n$ be the sequence of outcomes of the previous n shots and $\mathrm{\Sigma }(S_n)=k$ indicate that k of the previous n shots were successful. We also use the notation $S_n(i)$ to indicate the outcome of the $i^{th}$ shot in $S_n$ (i.e., $S_n(i)=s_{m-i}$ ). Each shot also has a number of points $p_i$ associated with it. Since we omit free throws from analysis, we have $p_i\in \{2,3\}$ . Let $\mathrm{\Pi }(S_n)$ denote the number of points scored in the previous n shots. Observe that for any $n\ge 1$ and $S_n$ , we have $0\le \mathrm{\Sigma }(S_n)\le n$ and $0\le \mathrm{\Pi }(S_n)\le 3n$ . The HHH can be formulated in different ways in terms of the outcomes present in $S_n$ . We formally write two of these forms below.

Hypothesis 1 (HHH1): The probability of making a shot given that the previous n shots all made is greater than the same given that at least one shot missed in the previous n shots. We represent this as

$$\begin{array}{c}P(s_m=1|\mathrm{\Sigma }(S_n)=n)>P(s_m=1|\mathrm{\Sigma }(S_n)<n).\end{array}$$

(1)

Hypothesis 2 (HHH2): The probability of making a shot increases as the streak of previous shots made increases. We represent this as

$$\begin{array}{c}P(s_m=1|\mathrm{\Sigma }(S_{n_1})=n_1)>P(s_m=1|\mathrm{\Sigma }(S_{n_2})=n_2)\mathrm{\forall }1\le n_2<n_1.\end{array}$$

(2)

Indeed, we can also study a generalization of Hypothesis 1 in which we consider the cases of making k out of n previous shots instead of just the case in which all n previous shots were successful. This introduces the following:

Hypothesis 3 (HHH3): The probability of making a shot increases as the number of made shots within a fixed-length previous shot sequence increases. We represent this as

$$\begin{array}{c}P(s_m=1|\mathrm{\Sigma }(S_n)=k_1)>P(s_m=1|\mathrm{\Sigma }(S_n)=k_2)\mathrm{\forall }0\le k_2<k_1\le n.\end{array}$$

(3)

Moving forward, we only consider Hypotheses 2 and 3 for the standard case of the hot-hand hypothesis since the latter encompasses Hypothesis 1. Of course, the main focus of this paper is the difficulty-adjusted hot-hand hypothesis. In this, we examine whether the difficulty of previous shots, as well as their outcomes, impact the outcome of an individual shot. We use the same notation as before, except introduce the shot difficulty of the $i^{th}$ shot to be $d\mathrm{\_}i$ . Note that $d_i\in [0,1]$ and a lower value indicates an easier shot. Let the average difficulty of the previous n shots be

$$\begin{array}{c}{\overline{D}}_n={\displaystyle \frac{1}{n}\sum _{i=1}^n{d}_{m-i}.}\end{array}$$

(4)

The easiest way to formulate the difficulty-adjusted hot-hand hypothesis is simply to use the standard version and condition based on the relation between the difficulty of the $m^{th}$ shot and the average difficulty of the previous n shots. That is, formulating Hypothesis 1 to account for difficulty would be represented as

$$P(s_m=1|\mathrm{\Sigma }(S_n)=n,d_m<{\overline{D}}_n)>P(s_m=1|\mathrm{\Sigma }(S_n)<n,d_m<{\overline{D}}_n),$$

$$P(s_m=1|\mathrm{\Sigma }(S_n)=n,d_m\ge {\overline{D}}_n)>P(s_m=1|\mathrm{\Sigma }(S_n)<n,d_m\ge {\overline{D}}_n).$$

However, this requires two sets of conditional probabilities to account for the fact that the difficulty of the $m^{th}$ shot can be either greater than or less than the average difficulty of the previous n shots. Sparsity in the data with respect to either one of these sets may lead to unreliable results. To counteract this, we observe that intuitively, making easier shots should have less of an impact on future shots than making harder shots. Thus, to rely on a single set of probabilities, we need only look toward a difficulty-weighted approach of shot success or points scored. Define difficulty-weighted shot success as

$$\begin{array}{c}DSS(S_n)=\sum _{i=1}^n(s_{m-i}\cdot d_{m-i})\end{array}$$

(5)

and

$$\begin{array}{c}DPTS(S_n)=\sum _{i=1}^n(s_{m-i}\cdot p_{m-i}\cdot d_{m-i}).\end{array}$$

(6)

These can be considered as weighted totals and we easily see that $0\le DSS(S_n)\le n$ and $0\le DPTS(S_n)\le 3n$ for any $n\ge 1$ . We can then formally write the difficulty-adjusted hot-hand hypothesis in terms of these metrics as follows.

Hypothesis 4 (DAHHH1): The probability of making a shot increases as the difficulty-weighted shot success of the previous n shots increases. We represent this as

$$\begin{array}{c}P(s_m=1|DSS(S_n)=\alpha )>P(s_m=1|DSS(S_n)=\beta )\mathrm{\forall }0\le \beta <\alpha \le n.\end{array}$$

(7)

Hypothesis 5 (DAHHH2): The probability of making a shot increases as the difficulty-weighted points scored of the previous n shots increases. We represent this as

$$\begin{array}{c}P(s_m=1|DPTS(S_n)=\alpha )>P(s_m=1|DPTS(S_n)=\beta )\mathrm{\forall }0\le \beta <\alpha \le 3n.\end{array}$$

(8)

With these formulations, especially those of Hypotheses 4 and 5, we note that the conditioning occurs purely on the (difficulty-adjusted) success of previous shots and does not consider current shot difficulty. As previously noted, prior research has found that outcomes of previous shots may lead to tactical shifts that affect future shot selection and types. ⁶ Conditioning on current shot difficulty alone may implicitly capture these changes, but would also be counterintuitive given the standard view of the hot-hand hypothesis as based on past outcomes. On the other hand, our formulations are consciously designed to explicitly focus on previous shot difficulty. With this, we can reconcile data-driven evaluations of the hot-hand hypothesis while staying true to the psychological principles that underlie the phenomenon. In the remainder of this paper, we will examine Hypotheses 2 through 5 using statistical methodologies and report whether or not they hold for players across the NBA.

Data and methodology

Data

We use three datasets to aid in our analysis of the hot-hand hypothesis. Our primary source consists of shot log data from the 2014–2015 NBA season, obtained from Kaggle. ¹⁹ This dataset contains information on over 128,000 individual shots taken by 281 unique players. Each row in the dataset holds information regarding both game data (i.e., date, opponent, location) and shot data. The latter is more important for assessing shot difficulty since it more closely captures shot attributes that affect outcomes. These include shot distance, time on the game and shot clocks, and distance from the closest defender. A more detailed view on these features, as well as additional features that are useful for estimating difficulty, can be found in Section 3.2.1.

In addition to the shot log dataset, we use 2014–2015 NBA “Per 100 Possessions” data sourced from Basketball Reference. ²⁰ This dataset contains information such as field goal percentage, rudimentary stats (e.g., points, assists, rebounds), as well as offensive and defensive ratings. For our analysis, we only use the offensive and defensive rating entries from this dataset. The former is used as a player effect to capture offensive quality and the latter measures defensive prowess of the closest defender for each shot in the original shot log dataset. As much as distance from the closest defender can affect a shot's difficulty, it stands to reason that the skill of said defender has a significant impact as well. Furthermore, we postulate that the height difference between a player and their defender may play a role in determining shot difficulty. As such, we draw player heights from a third dataset curated on Kaggle. ²¹ Although this dataset contains data across the NBA from 1996 to 2022, we consider reported player heights for solely the 2014–2015 season to maintain consistency with the other datasets in use.

Before using these datasets to estimate shot difficulty, we clean them to avoid discrepancies in player naming conventions and merge them while keeping only relevant features. As our baseline, we use the names as described in the NBA season dataset and convert them to lowercase while keeping any important punctuation. ²¹ We also rename some columns for clarity. Additional feature engineering is also performed and will be described in the following section.

Methodology

Feature engineering

Intuitively, certain game and shot characteristics contribute to either enhancing or detracting from a particular shot's difficulty. For example, from a viewer's standpoint, a shot farther away from the basket would appear to be more difficult than one much closer. Similarly, a shot taken closer to the end of the shot clock may be more difficult than one taken earlier, all other factors held constant. These intuitions motivate the creation of certain additional features and the removal of others. For clarity, these features and their descriptions can be found in Table 1.

Table 1.

Features in cleaned dataset with some relevance to estimating shot difficulty.

Feature	Description
SHOT_NUMBER	Index of the player's shot in a particular game
PERIOD	Quarter in which the shot was taken (overtime periods are denoted as 5, 6, 7, …)
GAME_CLOCK	Time remaining in the period, in seconds
SHOT_CLOCK	Time remaining on the shot clock, in seconds (if the shot clock is turned off, then this is the same as GAME_CLOCK)
TOT_TIME_LEFT	Total time left in the game, in seconds
DRIBBLES	Number of dribbles taken before the shot
DRIBBLES_BIN	Place DRIBBLES into one of the intervals $[0,1),[1,3),[3,6),[6,\mathrm{\infty })$ and use the associated index: 0, 1, 2, 3
DRIBBLES_EXP	Predicted shot percentage using exponential regression on binned values of DRIBBLES using intervals $[0,0],[1,1],[2,2],[3,4],[5,9],[10,\mathrm{\infty })$
DRIBBLES_POLY	Predicted shot percentage using cubic regression on binned values of DRIBBLES using intervals $[0,0],[1,1],[2,2],[3,4],[5,9],[10,\mathrm{\infty })$
DRIBBLES_BIN_EXP	Predicted shot percentage using exponential regression on percentages associated with values of DRIBBLES_BIN
SHOT_DIST	Player's distance from the basket when the shot was taken
SHOT_DIST_BIN	Place SHOT_DIST into one of the intervals $[0,4),[4,8),[8,12),[12,16),[16,24),[24,32),[32,\mathrm{\infty })$ and use the associated index: 0, 1, 2, …, 6
SHOT_DIST_POLY	Predicted shot percentage using cubic regression on binned values of SHOT_DIST using intervals $[4d,4d+4)$ for $0\le d\le 8$
SHOT_DIST_BIN_POLY	Predicted shot percentage using cubic regression on percentages associated with values of SHOT_DIST_BIN
CLOSE_DEF_DIST	Distance between the player who attempts the shot and the closest defender
CLOSE_DEF_DIST_BIN	Place CLOSE_DEF_DIST into one of the intervals $[0,2),[2,4),[4,6),[6,\mathrm{\infty })$ and use the associated index: 0, 1, 2, 3
CLOSE_DEF_DIST_EXP	Predicted shot percentage using exponential regression on binned values of CLOSE_DEF_DIST using intervals $[2d,2d+2)$ for $0\le d\le 14$
CLOSE_DEF_DIST_POLY	Predicted shot percentage using quadratic regression on binned values of CLOSE_DEF_DIST using intervals $[2d,2d+2)$ for $0\le d\le 14$
HEIGHT	Height of the player who attempts the shot, in centimeters
HEIGHT_DIFF	Difference in heights of the player who attempts the shot and the closest defender, in centimeters
OFF_RTG	Offensive rating of the player who attempts the shot
DEF_RTG	Defensive rating of the closest defender
RTG_DIFF	Difference between OFF_RTG and DEF_RTG
PTS_TYPE	Points associated with the shot (2 or 3)
HOME	Binary indicator for whether the player who attempts the shot is playing at home (1) or not (0)
FGM	Binary indicator for a successful shot (0 = miss; 1 = make)

In a similar vein, some features with values located in certain clusters may have a closely related impact on shot difficulty. For example, a “catch-and-shoot” shot, characterized as a shot taken without dribbling, may differ in difficulty from a shot taken after a single dribble. However, it is likely that shooting after taking a few dribbles may not be entirely dependent on the exact number of dribbles. To account for this, we introduce binned features, which assign a singular value to all data points that lie within pre-defined interval. Specifically, the intervals used are open on the right end so that for intervals $[x,y)$ and $[y,z)$ , the data point y is placed in the second bin. Values assigned to points in each bin start at 0 for the leftmost (least) bin and increase by 1 for each subsequent bin. We create three binned features based on number of dribbles, shot distance, and distance to the closest defender. For a description of these features, including the exact intervals used, see the features with suffix BIN in Table 1.

We also introduce regression-based features that map values of original features to an approximate experimental shot percentage. This is done only for the three features we previously binned (number of dribbles, shot distance, and distance to the closest defender). The creation of regression-based features starts by defining intervals for each original feature such that each interval contains an approximately equal number of data points. Then, for each interval, we compute its percentage of made shots. Now, we consider the reduced dataset where each interval is represented by its midpoint (independent variable) and its shot percentage (dependent variable). Observe that the number of points in this reduced dataset is equal to the number of intervals. We then perform either polynomial regression or exponential regression, or both, depending on the data shape. The degree for polynomial regression is chosen so that the coefficient of determination $R^2$ is maximized without overfitting. We find that the cubic model works best for the DRIBBLES and SHOT_DIST features, whereas a quadratic model works best for the CLOSE_DEF_DIST feature. Furthermore, the exponential model defined by $f(x)=a{e}^{bx}+c$ appears to be unsuitable for the SHOT_DIST feature. Finally, the regression-based features are computed by feeding the original feature values into the polynomial and exponential regression models and using their output. For a description of these features, including the exact intervals used, see the features with suffix POLY and EXP in Table 1.

We also introduce two additional features by combining the methodologies used above. Specifically, we compute an exponential-regression-based feature using the binned dribbles feature and a cubic-regression-based feature using the binned shot distance feature. The intervals used for each of these are simply formed by using the binned values so that regression computation is made easier. That is, the intervals used are $[0,1),[1,2),\dots$ . These two features are named DRIBBLES_BIN_EXP and SHOT_DIST_BIN_POLY in Table 1.

Estimating shot difficulty

Estimating shot difficulty is done in two phases: model selection and hyperparameter tuning. In the first phase, we consider three different models: logistic regression, polynomial (quadratic) logistic regression, and XGBoost. Each of these models is used to predict shot difficulty given specific player-related and shot-related features. The shot-related features are broadly split into four categories, stemming from the number of dribbles, shot distance, distance to the closest defender, and player height. The exact features list for each of these categories can be found in Table 2.

Table 2.

Shot-related categories and features for shot difficulty model training.

Category	Features
Number of Dribbles	DRIBBLES, DRIBBLES_BIN, DRIBBLES_EXP, DRIBBLES_POLY, DRIBBLES_BIN_EXP
Shot Distance	SHOT_DIST, SHOT_DIST_BIN, SHOT_DIST_POLY, SHOT_DIST_BIN_POLY
Closest Defender Distance	CLOSE_DEF_DIST, CLOSE_DEF_DIST_BIN, CLOSE_DEF_DIST_EXP, CLOSE_DEF_DIST_POLY
Height	HEIGHT, HEIGHT_DIFF

For each possible combination of features, the models are trained and run using k -fold cross validation where $k=10$ . Accuracy scores are computed and the features that provide the best score for each model are reported in Table 3. We see that the XGBoost model performs the best, albeit slightly. This concludes the first phase of estimating shot difficulty and we continue to the second phase with the XGBoost model, trained using the features DRIBBLES_POLY, SHOT_DIST, CLOSE_DEF_DIST_EXP, and HEIGHT_DIFF.

Table 3.

Best features and accuracy for un-tuned shot difficulty models.

Model	Dribbles Feature	Shot Distance Feature	Closest Defender Feature	Height Feature	Accuracy
Logistic Regression	DRIBBLES_EXP	SHOT_DIST_POLY	CLOSE_DEF_DIST_BIN	HEIGHT_DIFF	0.61614
Poly-Log Regression	DRIBBLES_BIN_EXP	SHOT_DIST_POLY	CLOSE_DEF_DIST	HEIGHT_DIFF	0.61760
XGBoost	DRIBBLES_POLY	SHOT_DIST	CLOSE_DEF_DIST_EXP	HEIGHT_DIFF	0.61899

The second phase consists of hyperparameter tuning. This is done by finding the optimal hyperparameter values from a range of potential candidates used in the XGBoost classifier model. Due to the large and continuous nature of the search space for some parameters, we opt for a randomized search method, rather than grid or manual search implementations. The use of randomized search also appears to be more effective than these other proposed methods. ²² The list and ranges of hyperparameters tested can be found in Table 4. In addition to the parameter list obtained from the first phase, we include the following game-related features: PERIOD, SHOT_CLOCK, HOME. Intuitively, these make sense to include since shots taken towards the end of the shot clock, or even towards the end of games, may tend to be more difficult due to increased pressure. Moreover, we acknowledge the impact of “home-court advantage” by including the HOME feature. In total, the model now has seven input features. We run the XGBoost classifier with these features using k -fold cross validation (for $k=5$ ) across 50 randomized sets of hyperparameters. We achieve an accuracy of 0.62068, which is an improvement from the previous baseline, and the optimal hyperparameter values are reported in Table 4.

Table 4.

Sets of hyperparameter values in XGBoost model for randomized search and their optimal value.

Hyperparameter	Value Set	Optimal Value
n_estimators	$[100,150]$	101
learning_rate	$[0.05,0.15]$	0.09714
max_depth	$\{3,4,5\}$	4
max_leaves	$[4,10]$	7
subsample	$\{0.6,0.7,0.8,0.9,1.0\}$	0.6
reg_alpha	$[0.5,0.75]$	0.52729
reg_lambda	$[1.5,2]$	1.99623

Curating shot result data

Using the trained XGBoost model in Section 3.2.2 with the hyperparameter values specified in Table 4, we compute shot difficulty scores for each individual shot in the existing cleaned shot log dataset. Scores are continuous and lie in the range $[0,1]$ , where higher scores indicate more difficult shots. These are then added to a new dataset that will be used in Section 3.2.4 to compute conditional probabilities related to the standard and difficulty-adjusted hot-hand hypotheses. In this new shot result dataset, we store the game ID, player name, and shot number of every shot. Furthermore, in order to determine validity of our hypothesis, we need to track the field goal percentage, average shot difficulty, and other related metrics for fixed sequences of shots. To do this, for any shot with shot number $m>1$ , we store the result of the $i^{th}$ previous shot, the number of made shots in the last n shots, and the result of the last n shots as a string sequence, where $1\le n\le 10,1\le i\le n$ . Moreover, for each shot, we also store the average difficulty of the last n shots, as well as the difficulty-adjusted shot success ( $DSS$ ) and difficulty-weighted points scored ( $DPTS$ ), as defined in Equations $5$ and $6$ , respectively. Table 5 contains a full list of the features in the shot result dataset and their descriptions.

Table 5.

All features in shot result dataset.

Feature	Description
GAME_ID	Unique ID of the game in which the shot was taken
PLAYER_NAME	Name of the player who took the shot
SHOT_NUMBER	Index of the player's shot in a particular game
PTS_TYPE	Points associated with the shot (2 or 3)
SHOT_DIFFICULTY	Estimated shot difficulty as predicted by tuned XGBoost classifer
FGM- $i$	Outcome (0 = miss; 1 = make) of the $i^{th}$ previous shot (for $1\le i\le 10$ )
TOT_FGM- $i$	Number of made shots in last i shots (for $1\le i\le 10$ )
PREV- $i$	Outcomes of last i shots (for $1\le i\le 10$ ), prefixed by ‘X’ for indexing purposes, in order of recency
D_AVG- $i$	Average difficulty of the last i shots (for $1\le i\le 10$ ) as defined by Equation $4$
DSS- $i$	$DSS$ of the last i shots (for $1\le i\le 10$ ) as defined by Equation $5$
DPTS- $i$	$DPTS$ of the last i shots (for $1\le i\le 10$ ) as defined by Equation $6$
FGM	Binary indicator for a successful shot (0 = miss; 1 = make)

Computing conditional probabilities

Recall that we are interested in determining the validity of Hypotheses 2 through 5. Since each of these can be represented via conditional probabilities, we find the necessary probabilities and later use an appropriate statistical test (Section 4) to examine whether differences in sets of these probabilities are significant. For each of these hypotheses, we refer to their corresponding equations (see Section 2 for Equations 2 to 8) to find the appropriate conditional probabilities. In the discrete case (i.e., standard hot-hand hypothesis), we gather a set of conditional probabilities directly by taking all shots that meet the given condition and finding the percentage of successful shots. Specifically, for Hypothesis 2, we need to find

$$P(s_m=1|\mathrm{\Sigma }(S_n)=n)$$

for all $1\le n\le 10$ . To do so, for each possible value of n, we take the subset of shots such that all of the previous n shots were successful (i.e., $\mathrm{\Sigma }(S_n)=n$ ). From this, we can easily calculate the percentage of successful shots, which gives the desired conditional probability. We do the same to calculate

$$P(s_m=1|\mathrm{\Sigma }(S_n)<n)$$

which represents the conditional probability of making a shot given that at least one shot in the last n was unsuccessful. These probabilities are visually shown in Figure 1.

Figure 1.

Probability of making a shot given that the previous n shots were all successful (circles) or there was at least one miss in the last n shots (X's), for each $1\le n\le 10$ .

The computation performed for Hypothesis 3 is similar, with the appropriate probability being computed as

$$P(s_m=1|\mathrm{\Sigma }(S_n)=k)$$

for each $1\le n\le 10$ and $0\le k\le n$ . This gives 10 plots which are shown across Figures 2 and 3.

Figure 2.

Probability of making a shot given k makes in the previous n shots, for each $1\le n\le 5$ .

Figure 3.

Probability of making a shot given k makes in the previous n shots, for each $6\le n\le 10$ .

Observe that for Hypothesis 4, we are interested in probabilities of the nature

$$P(s_m=1|DSS(S_n)=\alpha )$$

for $1\le n\le 10$ and $\alpha \in [0,n]$ . Since the values of $DSS(S_n)$ are continuous, it is clear that computing conditional probabilities directly as we did in the discrete case would be difficult and may not yield effective results. This logic also holds for use of the $DPTS$ metric. As such, for the formulations associated with the difficulty-adjusted hot-hand hypothesis, we compute probabilities via prediction mechanisms. For each value of n, we create a basic XGBoost classifier with hyperparameters max_depth = 3, max_leaves = 8, and learning_rate = 0.1. The $i^{th}$ classifier is then trained on 75% of the relevant data, which is the subset of shots whose shot number is at least $i+1$ (i.e., there are i previous shots). The classifier uses only one input feature depending on the corresponding metric and one output feature (FGM). When examining Hypothesis 4, the $i^{th}$ classifier ( $1\le i\le 10$ ) uses the input feature DSS- i, whereas DPTS- i is used for Hypothesis 5. Following the training process, probabilities are estimated by the classifier for metric values separated by 0.1 in the range $[0,n]$ for Hypothesis 4 and in the range $[0,3n]$ for Hypothesis 5. Relevant plots for these hypotheses are shown in Figures 4 through 7. Before plotting, a Savitsky-Golay filter was used on the estimated probability data using a window size of 7 and cubic polynomial for fitting purposes.

Figure 4.

Probability of making a shot given $DSS$ values in the previous n shots, for each $1\le n\le 5$ . Plots were smoothed using a cubic Savitzky-Golay filter.

One problem that naturally arises when considering regression frameworks is the consequence of the Frisch-Waugh-Lovell (FWL) Theorem. This states that including all determinants in a regression model reduces bias and leads to more accurate results as compared to a two-stage approach where predicted variables are used as explanatory variables. ²³ A natural concern would be to question whether the same is applicable to our methodology. We note that our framework of hypothesis testing based on conditional probabilities is purely non-parametric. The actual predictive model we use is used only for hypothesis testing and not for coefficient estimation like in regression models. As such, the FWL Theorem should not be problematic for our study.

Results

In order to validate the hypotheses at hand, we look at whether there is a statistically significant difference between the conditional probabilities found in Section 3.2.4. To do this, we use different statistical tests depending on the type of data we have. In the following subsections, we examine each formulation of the hot-hand hypothesis, in both its standard and difficulty-adjusted forms, to determine whether the outcomes of prior shots have an impact on future shots.

Standard hypotheses

We first analyze the probabilities associated with Hypothesis 2. Recall that this conjectures that the probability of making a shot increases as the number of consecutive previous made shots increases. Here, we treat $P(s_m|\mathrm{\Sigma }(S_n)=n)$ as an independent distribution for each n. We can then treat the samples taken for $s_m=1$ as proportions and determine whether the differences between consecutive probabilities are statistically significant. For this, we use one-sided two-proportion z-tests with significance level $\alpha =0.05$ . However, since we run multiple related z-tests where reducing the possibility of a Type I error is important, we adjust the significance level using Bonferroni correction. We run a total of 45 tests, which reduces the significance level to ${\alpha }_{adj}=\frac{\alpha }{45}\approx 0.00111$ . For each pair of indices $1\le i<j\le n=10$ , denote the proportions

$$\widehat{p_1}=P(s_m=1|\mathrm{\Sigma }(S_i)=i),$$

$$\widehat{p_2}=P(s_m=1|\mathrm{\Sigma }(S_j)=j),$$

each of which are taken from a population of $n_1$ and $n_2$ shots, respectively. Then, we calculate the z-test statistic as

$$\begin{array}{c}\hat{z}={\displaystyle \frac{\widehat{p_1}-\widehat{p_2}}{\sqrt{\hat{p}\cdot \hat{q}\cdot \left({\displaystyle \frac{1}{n_1}+{\displaystyle \frac{1}{n_2}}}\right)}},}\end{array}$$

(9)

where

$$\hat{p}={\displaystyle \frac{n_1\cdot \widehat{p_1}+n_2\cdot \widehat{p_2}}{n_1+n_2},\hat{q}=1-\hat{p}.}$$

The p -values corresponding to the z-statistic obtained for each $(i,j)$ pair (where $i<j$ ) are reported in Table 6. We do not calculate the p -values for $i>j$ since this would be equivalent to running a z-test on swapped streaks, which would result in testing for an increase in probability as the prior shot streak length decreases. This is counterintuitive for our purposes and we ignore these index pairs in the table by using N/A values. Although we report p-values for each pair, we are mainly interested in the values along the superdiagonal since these represent whether the difference in probabilities given fully successful prior shot streaks of consecutive lengths is significant. Observe that none of the values are less than ${\alpha }_{adj}$ , which leads to the conclusion that Hypothesis 2 is invalid.

Table 6.

p -values from z-tests for hypothesis 2 using streaks of made shots of length $i<j$ .

	$\mathit{j}$
$\mathit{i}$	1	2	3	4	5	6	7	8	9	10
1	N/A	0.7079	0.8670	0.9765	0.8517	0.3350	0.0717	0.1428	0.0432	0.1126
2	N/A	N/A	0.7487	0.9529	0.8103	0.3033	0.0674	0.1352	0.0411	0.1095
3	N/A	N/A	N/A	0.8664	0.7149	0.2474	0.0529	0.1214	0.0373	0.1036
4	N/A	N/A	N/A	N/A	0.4376	0.1341	0.0300	0.0896	0.0287	0.0889
5	N/A	N/A	N/A	N/A	N/A	0.1806	0.0408	0.1004	0.0316	0.0931
6	N/A	N/A	N/A	N/A	N/A	N/A	0.1526	0.1988	0.0598	0.1318
7	N/A	N/A	N/A	N/A	N/A	N/A	N/A	0.4463	0.1515	0.2250
8	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	0.1992	0.2589
9	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	0.5000
10	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A

For Hypothesis 3, we look at the probability of making a shot given that exactly k shots were successful in the previous n shots, where n varies from 1 to 10 and k varies from 0 to n. In this sense, for a fixed n, we treat $P(s_m|\mathrm{\Sigma }(S_n)=k)$ as an independent distribution for each k. Then, the samples taken for $s_m=1$ are proportions and we use a one-sided two-proportion z-test with significance level $\alpha =0.05$ to determine statistical significance between probabilities corresponding to consecutive values of k. Of course, since we run multiple related z-tests, we adjust the significance level using Bonferroni correction. For a particular n, there are $n+1$ possible values of k and we run a z-test on each pair of values $(i,j)$ such that $i<j$ . There are $N(n)=\frac{(n)(n+1)}{2}$ such pairs and thus, we run that many tests for a particular n. The resulting significance level used is ${\alpha }_{n,adj}=\frac{\alpha }{N(n)}$ . These adjusted significance levels are provided in Table 7. Then, for each fixed n and index pairs $0\le i<j\le n$ , we calculate the z-statistic as in Equation $9$ using

$$\widehat{p_1}=P(s_m=1|\mathrm{\Sigma }({\mathrm{S}}_{\mathrm{n}})=\mathrm{i}),$$

$$\widehat{p_2}=P(s_m=1|\mathrm{\Sigma }(S_n)=j).$$

Table 7.

Bonferroni-corrected significance levels for each shot streak length n using $\alpha =0.05$ for hypothesis 3.

$\mathit{n}$	Number of tests $\mathit{N}(\mathit{n})$	${\mathit{\alpha }}_{\mathit{n},\mathit{a}\mathit{d}\mathit{j}}=\frac{\mathit{\alpha }}{\mathit{N}(\mathit{n})}$
1	1	0.05000
2	3	0.01667
3	6	0.00833
4	10	0.00500
5	15	0.00333
6	21	0.00238
7	28	0.00179
8	36	0.00139
9	45	0.00111
10	55	0.00091

For prior shot sequences of any length $1\le n\le 10$ , we find that the p -values corresponding to the z-statistic obtained for each $(i,j)$ pair is greater than the significance level ${\alpha }_{n,adj}$ . For brevity, we provide only the p -values for the case of $n=10$ in Table 8. Interestingly, when considering the cases $n\in \{1,2\}$ (i.e., small shot streaks), there is a statistical significance between the probabilities when conditioning on no makes in the last n shots and when conditioning on n makes in the last n shots, but in the reverse direction of Hypothesis 3. That is, while the p-values obtained from the current tests, which test for an increase in conditional probability, are extraordinarily high, the reverse p -values are extremely low. This would statistically indicate that players tend to shoot better after missing their last few shots as compared to making them. These p-values are shown in Table 9 and Table 10. Similar to Table 6, for each of these tables, we only consider $i<j$ as well. With this, we conclude that Hypothesis 3 fails to hold in general.

Table 8.

p -values from z-tests for hypothesis 3 when $i<j$ shots were made in the last $n=10$ shots.

	$\mathit{j}$
$\mathit{i}$	0	1	2	3	4	5	6	7	8	9	10
0	N/A	0.665	0.569	0.586	0.603	0.635	0.603	0.599	0.650	0.650	0.171
1	N/A	N/A	0.248	0.271	0.307	0.388	0.309	0.307	0.452	0.514	0.094
2	N/A	N/A	N/A	0.581	0.672	0.811	0.667	0.631	0.753	0.655	0.119
3	N/A	N/A	N/A	N/A	0.624	0.821	0.617	0.573	0.729	0.637	0.114
4	N/A	N/A	N/A	N/A	N/A	0.754	0.501	0.477	0.681	0.615	0.109
5	N/A	N/A	N/A	N/A	N/A	N/A	0.264	0.292	0.572	0.572	0.100
6	N/A	N/A	N/A	N/A	N/A	N/A	N/A	0.477	0.678	0.615	0.109
7	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	0.679	0.619	0.110
8	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	0.543	0.097
9	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	0.105
10	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A

Table 9.

p -values from z-tests for hypothesis 3 when $i<j$ shots were made in the last $n=1$ shots.

	$\mathit{j}$
$\mathit{i}$	0	1
0	N/A	0.99981
1	N/A	N/A

Table 10.

p -values from z-tests for hypothesis 3 when $i<j$ shots were made in the last $n=2$ shots.

	$\mathit{j}$
$\mathit{i}$	0	1	2
0	N/A	0.80135	0.99352
1	N/A	N/A	0.97653
2	N/A	N/A	N/A

While we fail to find substantial evidence in favor of the hot-hand hypothesis, we find it prudent to address a potential concern regarding potential biases. As introduced by Miller and Sanjurjo, comparing conditional probabilities in the context of hot-hand studies often introduces a bias that makes it inherently harder to find evidence in support of the hypothesis. ²⁴ However, this bias features most prominently when conditioning on a previous streak of all makes or all misses. In contrast, our evaluation framework is more robust and instead considers different numbers of fixed, consecutive counts of makes and misses. With this, our results are adjusted according to the non-parametric robustness test suggested by Miller and Sanjurjo. ²⁴ Moreover, even when discounting Bonferroni correction, which admittedly makes tests more conservative, we find no evidence for the standard hot-hand hypothesis.

Difficulty-Adjusted hypotheses

Assessing the validity of the difficulty-adjusted hypotheses must be done differently than those of the standard form, mainly because there are significantly more data points and the desired probabilities are derived from a continuous independent variable. The former reason limits the practicality of multiple pairwise two-proportion z-tests, but the latter lends itself to an analysis of time-series data. Indeed, for a fixed n, we consider probabilities of the form

$$P(s_m=1|DSS(S_n)=\alpha )$$

for $\alpha \in [0,n]$ and a similar structured probability for the $DPTS$ metric (with $\alpha \in [0,3n]$ ). Using the method described in Section 3.2.4 for computationally finding these probabilities, we see that the data can be treated as time-series-like. However, instead of indexing data points over time snapshots as in exact time-series data, we index over increasing values of a difficulty-adjusted metric with equal spacing.

If one (or both) of the difficulty-adjusted hypotheses were to hold, we would expect there to be a change in the mean probability as the metric increases. In essence, we would expect there to be a unit root, and for the data to exhibit properties of non-stationarity. While looking at Figures 4–7, it seems apparent that the data is non-stationary for all n across both metrics $DSS$ and $DPTS$ . To confirm this intuition, we run an augmented Dickey-Fuller (ADF) test for each $1\le n\le 10$ using the probabilities obtained in Section 3.2.4. We use a standard significance level of $\alpha =0.05$ . The p -values associated with Hypotheses 4 ( $DSS$ ) and 5 ( $DPTS$ ) can be found in Table 11 and Table 12, respectively. We see that, in general, the data we obtained is non-stationary, which would imply that there is a significant difference in conditional probabilities over increasing metric values.

Figure 5.

Probability of making a shot given $DSS$ values in the previous n shots, for each $6\le n\le 10$ . Plots were smoothed using a cubic Savitzky-Golay filter.

Figure 6.

Probability of making a shot given $DPTS$ values in the previous n shots, for each $1\le n\le 5$ . Plots were smoothed using a cubic Savitzky-Golay filter.

Figure 7.

Probability of making a shot given $DPTS$ values in the previous n shots, for each $6\le n\le 10$ . Plots were smoothed using a cubic Savitzky-Golay filter.

Table 11.

p -values and stationarity via ADF tests for $DSS$ metric in hypothesis 4.

$\mathit{n}$	$\mathit{p}$ -value	Stationarity
1	0.11640	NOT STATIONARY
2	0.03699	STATIONARY
3	0.66244	NOT STATIONARY
4	0.99721	NOT STATIONARY
5	0.88314	NOT STATIONARY
6	0.94568	NOT STATIONARY
7	0.82892	NOT STATIONARY
8	0.25094	NOT STATIONARY
9	0.34833	NOT STATIONARY
10	0.04892	STATIONARY

Table 12.

p -values and stationarity via ADF tests for $DPTS$ metric in hypothesis 5.

$\mathit{n}$	$\mathit{p}$ -value	Stationarity
1	0.60703	NOT STATIONARY
2	0.99305	NOT STATIONARY
3	0.94491	NOT STATIONARY
4	0.93799	NOT STATIONARY
5	0.00033	STATIONARY
6	0.49953	NOT STATIONARY
7	0.79871	NOT STATIONARY
8	0.45386	NOT STATIONARY
9	0.08208	NOT STATIONARY
10	0.02853	STATIONARY

To fully understand the trend at hand, we consider the application of the Mann-Kendall test for monotonicity. The goal of using this test across both the $DSS$ and $DPTS$ metrics is to determine whether there is an increasing or decreasing trend related to conditional probabilities as the metric values increase. With the Mann-Kendall tests, we use a standard value of $\alpha =0.05$ . Before discussing the statistical results, we first note that a simple visual observation of Figures 4–7 shows an apparent decrease in probability for longer shot streaks (i.e., $n\ge 2$ ), with some appearing to have a bit of a spike for larger metric values. This informal finding is then corroborated by the p -values taken from the Mann-Kendall tests, as reported in Table 13 and Table 14. We also point out that many of the p -values are rounded off to zero, indicating a strong confidence in a monotonic decreasing trend, despite any probability spikes that may be visually noticeable. From this, we conclude that the difficulty-adjusted formulations of the hot-hand hypothesis are generally invalid.

Table 13.

p -values and monotonicity via mk tests for $DSS$ metric in hypothesis 4.

$\mathit{n}$	$\mathit{p}$ -value	Trend
1	1.00000	NO TREND
2	0.00016	DECREASING
3	0.00000	DECREASING
4	0.00000	DECREASING
5	0.00000	DECREASING
6	0.00000	DECREASING
7	0.18447	NO TREND
8	0.00000	DECREASING
9	0.08145	NO TREND
10	0.00048	DECREASING

Table 14.

p -values and monotonicity via mk tests for $DPTS$ metric in hypothesis 5.

$\mathit{n}$	$\mathit{p}$ -value	Trend
1	0.10902	NO TREND
2	0.00000	DECREASING
3	0.00000	DECREASING
4	0.00000	DECREASING
5	0.00000	DECREASING
6	0.00000	DECREASING
7	0.01347	DECREASING
8	0.00000	DECREASING
9	0.00000	DECREASING
10	0.01647	DECREASING

Unlike our concerns regarding potential biases associated with the results of Miller and Sanjurjo in the previous section, no such concerns are immediately relevant here. ²⁴ This is because we are making an implicit comparison of conditional probabilities rather than an explicit one. In essence, we care more about monotonicity trends and stationarity rather than the p-values associated with comparison-based hypothesis testing.

However, a valid potential concern is the lack of observations for larger shot streaks. While we have at least 50,000 observations each for shot streaks of length six or fewer, this number decreases significantly for shot streaks of length seven or more. In practice, this is expected as there is generally a limit on how many shots a player can take in a game. This does not pose an obstacle to our results for two reasons. First, our conclusions are over all streak lengths, rather than any specific one. Second, despite apparent fluctuations in the probabilities, the overwhelming trend is still decreasing.

Discussion

Based on these results, we note that the hot-hand hypothesis is true in reverse, but only when considering the difficulty of previous shots, rather than just concerning ourselves with the raw total number of shots made. Of course, this would mean the actual notion of the hot-hand hypothesis, and our formulations in Hypotheses 4 and 5, are invalid. Intuitively, it makes sense for there to be a difference in how we treat the standard and difficulty-adjusted cases since the former treats all shots equally, regardless of their distance to the basket, how tightly the player is being guarded, as well as other game-related characteristics. However, it is peculiar that even the difficulty-adjusted case fails to hold, and actually exhibits a result that at first glance, seems to be counterintuitive. As such, we must view having a “hot hand” as a bit misleading in the common sense of the term, since simply making a few open shots cannot necessitate this label. On the contrary, from our findings, it is clear that the hot hand may not even be existent after making a string of difficult shots in succession.

This result, although surprising, may be justified in part to increasing levels of defense played by opposing teams when facing players on a hot streak. Once a player makes several difficult shots, offensive gameplans shift to cater to their supposed “hot hand”, while defensive strategies become increasingly aggressive to limit the shooter's efficacy. ¹⁸ While this seems to hold across varying lengths of shot streaks, we tend to see a jump in probability when conditioning on large $DSS$ or $DPTS$ values. This may indicate that while the hot-hand hypothesis is invalid in most cases, streaks that comprise of shots taken (and made) with unparalleled difficulty, may in fact raise the probability of a future shot's success. With this, we see that the hot-hand hypotheses may very well be valid, but only in extreme cases. Such a result is not out of the ordinary, as this is essentially the difficulty-adjusted equivalent to the findings of Wetzels et al. ¹¹

We now consider some potential drawbacks to these results. In the case of the standard formulations for the hot-hand hypotheses, observe that there is one primary issue that may arise. This is the usage of pooled shot percentages as representatives for conditional probabilities. However, this was a conscious decision to avoid using shot percentages for individual players and then testing for statistical significance. Focusing on individual players was not the focus of this paper and results may still have been skewed by a lack of sufficient data for certain players who are not volume shooters.

Furthermore, we see that the standard formulation of Hypothesis 3 does not hold for shot sequences of length $n\ge 3$ . This finding is in line with those of Arkes, ⁸ as well as Aharoni and Sarig. ¹⁰ In essence, this means that if a player has taken many shots in a game, the results of those shots do not have a significant impact on the current shot. However, there is a slight shot memory that occurs with $n\le 2$ shots. Specifically, the probabilities associated with making a shot given the outcomes of previous shots are affected based on the number of shots made. However, making two previous shots is associated with a decrease in shot probability as compared to making none of those two shots. While a bit surprising, this does somewhat corroborate the findings of Lantis and Nesson and can perhaps be attributed to a short-term subconscious increase in pressure to make the shot, resulting in decreased performance. ⁹

When accounting for prior shot difficulty, we unequivocally see evidence that players do not experience a hot hand for both short and long previous shot streaks. Instead, they appear to be subject to a “cold hand”. This finding stays the same regardless of how we measure average performance over the previous n shots when including shot difficulty. Currently, we provide two such metrics ( $DSS$ and $DPTS$ ). Note that these differ from other research in the same vein since our metrics quantify in-game difficulty-adjusted performance rather than theoretical expectation. In this sense, the metrics we use are more closely aligned to the actual game situation. There are a few shot lengths for which the data appears to be stationary or in which there is no general monotonic decreasing trend. While this would normally rebuke our overall claims, these tend to either be for large n or cases where the p -value is close to the significance level. Thus, these deviations are likely anomalies. In any case, our results contradict those of Bocskocsky, Ezekowitz, and Stein ¹⁶ and lie more closely in line with those in Lantis and Nesson. ⁹

Unsurprisingly, our determination that there is no hot-hand effect when considering in-game shots goes against prior studies that find such an effect when considering controlled environments or shot patterns with significantly less pressure. In situations like the NBA Three Point Contest or regular free throws, where stakes are rather minimal, there may indeed be improved performance after a streak of makes.^8,9,14 However, the presence of in-game factors separates our results from these studies. Interestingly, our findings differ from those that were derived from an underlying psychological framework. While Meier et al. ² and Page and Coates ³ both determine a hot-hand effect exists in tennis, we attribute this to a differing evaluation criteria due to the inherent differences in sports. These studies employ a broader perspective and focus on winning sets, which occur over longer periods of time. On the other hand, we are focused on singular possessions and shot outcomes, both of which occur far more frequently.

In this sense, it is more reasonable to compare our results through a framework that is statistically motivated, has a granular scope, and incorporates shot difficulty. Although Attali does not explicitly model shot difficulty (outside of shot distance), their findings corroborate ours in the sense that a cold-hand effect is definitively visible. ⁸ Lantis and Nesson provide similar results with a regression framework. ⁹ Studies that employ a Bayesian framework tend to find the opposite.^11,16 Such a difference between frequentist models and Bayesian models can be attributed to the latter's sensitivity to small trends in data when accounting for past beliefs. Specifically, hot-hand effects may be unnoticeable statistically, but prominent enough for future and past shot probabilities to differ from a Bayesian point of view. Indeed, a lack of statistical significance is enough for frequentist methods to fail to reject the null hypothesis. However, unlike general frequentist models, we also employ stationarity and monotonicity tests to go beyond simple hypothesis testing and find evidence for a cold-hand effect.

Conclusion

Since the idea of the hot-hand hypothesis gained traction in the mid-1980s, there has been a decisive split in opinion between whether it is truly valid, or simply a fallacy motivated by emotional biases rather than statistical facts. Viewers and active participants in basketball games appear to look upon this hypothesis favorably, as surrounding conversation tends to overhype players during successful shot streaks. While this may be caused by inherent biases towards players known to be good shooters, a significant portion of research in the sports analytics field has been devoted to confirming the validity of the hot-hand hypothesis. On the contrary, an equal effort has been made to disprove the conjecture, either in its entirety, or with certain caveats related to the length of shot streaks or type of shots.

In this paper, we enhance the original view of the hot-hand hypothesis by considering metrics related to the difficulty of previous shots. After all, if the hypothesis should hold, it would stand to reason that making an array of tough shots should increase a player's confidence and perhaps contribute to a successful future shot more so than a few open layups. While shot difficulty can be quantified in several ways, we use basic machine learning models to learn from input features relevant to shot outcomes and predict shot difficulty as a probability. We achieve a competitive accuracy, indicating that our measure of shot difficulty is appropriate. We then define metrics related to difficulty-adjusted shot success and integrate this into our formulation of the problem in Hypotheses 4 and 5. Through this new framework, we find that these hypotheses are invalid in most cases, while also rejecting their counterparts under the standard non-difficulty-adjusted viewpoint.

This finding has two major implications to the sport of basketball. On the offensive side, understanding that making an unprecedented amount of difficult shots correlates to a better chance of making a future shot forces coaches to shift rotations and offensive playcalling accordingly. Bench players who abruptly enter a hot streak, in the difficulty-adjusted sense, may require minutes past their initial allotment, requiring coaches to tailor any existing gameplan to their needs. Psychologically, this could also provide a significant momentum shift in the game. However, in general, making a few difficult shots need not require a change in gameplans and coaches should move away from plays designed to feed players with a short-lived “hot hand”. On the other side of the ball, playing against someone on a hot streak may require a change in defensive assignments on the fly. Coaches need to face the tough decision of switching a better defender onto a player on a hot streak at the expense of removing them from a player who is perhaps better overall. Defenders who face players who make a bevy of tough shots may face some frustration, leading to further defensive lapses. As such, our results regarding the invalidity of the difficulty-adjusted hot-hand hypothesis may have wide-reaching implications on both sides of the game.

This is not to say that the debate over the hot-hand hypothesis has been resolved. Indeed, there are some additional points that can be addressed through future experiments and studies. First and foremost, an argument can be made for a different shot-indexing mechanism. Currently, when considering shot streaks, we restart the indexing at the start of every game. That is, all of a player's shots during a particular game are counted as part of the same shot streak. However, it is possible that “heat” resets every half, or perhaps even every quarter. Further analysis into whether the duration between shots positively or negatively impacts shot performance could improve the current framework we have. Similarly, examining longer shot streaks (i.e., those longer than 10 shots) may also help, although there would likely be a sparsity of data across a single season. To improve our estimation of shot difficulty, it may be beneficial to condition on current shot difficulty as past outcomes influence the types of shots players take in the future. In parallel, features can be added to the existing model related to player archetypes. While current research does include fixed player effects, these are based on raw advanced statistics, such as efficiency ratings. Allowing the model to treat players differently based on their role on their team may shed some light into how to better quantify shot difficulty. Furthering our current work through these means would enhance our current understanding of the implications of our results.