Operation of the Basketball Jump Shooting Accuracy Test: Intra- and inter-rater reliability of scoring procedures and floor and ceiling effects for test performance

Abstract

The purpose of this study was to determine the intra- and inter-rater reliability of scoring procedures used in the newly developed Basketball Jump Shooting Accuracy Test (BJSAT) and assess for floor and ceiling effects in test performance. Thirty-one semi-professional basketball athletes completed four trials of the BJSAT. The BJSAT contains one jump shot at eight different locations, equally distributed across two- and three-point shots. Intra-rater reliability was determined by assessing the level of agreement between scores live in-person and watching captured video by the same assessor. Inter-rater reliability was determined by examining the level of agreement between two assessors who separately scored the BJSAT while watching captured video. Descriptive statistics and Cohen’s kappa (κ) were calculated to quantify the intra- and inter-rater reliability of the BJSAT. Floor and ceiling effects in scoring outcomes were analyzed to evaluate the suitability of the BJSAT. Significance for the study was set at p <0.05. Intra-rater reliability demonstrated an almost perfect (κ = 0.85, p <0.01) agreement between scores (12.6 ± 2.5 vs. 13.1 ± 2.8). The agreement for inter-rater reliability was rated as substantial (12.3 ± 2.5 vs. 13.5 ± 2.9, κ = 0.70, p <0.01). Floor and ceiling effects were absent in the BJSAT indicating the assessment is suitable for semi-professional basketball athletes. The BJSAT is an assessment where one or multiple assessors can reliably score shooting performance for functions including player monitoring, to assess the efficacy of interventions aimed at improving skills and to assist with team selection across the season.