Brief experimental analysis (BEA) is a well-researched approach to conducting problem analysis, where potential interventions are pilot tested using a single-subject alternating treatment design. However, its brevity may lead to a high frequency of decision-making errors, particularly in situations where one tested condition is rarely optimal for students (i.e., the base rate). The current study explored the accuracy of a specific variant of BEA, skill versus performance deficit analysis (SPA), across different variations of the basic BEA design, score difference thresholds, and reading and math curriculum-based measurements (CBMs). Findings indicate that the ABAB design provides a reasonable control of such error rates when using reading CBM, whereas subtraction CBM required the use of an ABABAB design. Such error rates could not be controlled, regardless of design, when using multiplication CBM. Implications for best practice in the use of BEA are discussed.
AndersenM. N.DalyE. J.IIIYoungN. D. (2013). Examination of a one-trial brief experimental analysis to identify reading fluency interventions. Psychology in the Schools, 50, 403–414. doi:10.1002/pits.21682
2.
Bar-HillelM.FischoffB. (1981). When do base rates affect predictions?Journal of Personality and Social Psychology, 41, 671–680.
3.
BarlowD. H.HayesS. C. (1979). Alternating treatment design: One strategy for comparing the effects of two treatments in a single subject. Journal of Applied Behavior Analysis, 12, 199–210. doi:10.1901/jaba.1979.12-199
4.
BerganJ. R.KratochwillT. R. (1990). Behavioral Consultation and Therapy. New York: Plenum. doi:10.1007/978-1-4899-7288-0
5.
BurnsM. K.GanuzaZ. M.LondonR. M. (2009). Brief experimental analysis of written letter formation: Single-case demonstration. Journal of Behavioral Education, 18, 2–34.
6.
BurnsM. K.TaylorC. N.Warmbold-BrannK. L.PreastJ. L.HospJ. L.FordJ. W. (2017). Empirical synthesis of the effect of standard error of the measurement on decisions within brief experimental analyses of reading fluency. Psychology in the Schools, 54, 640–654.
7.
ChristT. J.MonaghenB. D.ZopluogluC.Van NormanE. R. (2012). Curriculum-based measurement of oral reading: Evaluation of growth estimates derived with pre-post assessment methods. Assessment for Effective Intervention, 38, 139–153. doi:10.1177/1534508412456417
8.
ChristT. J.SilberglittB. (2007). Estimates of the standard error of measurement for curriculum-based measures of oral reading fluency. School Psychology Review, 36, 130–146.
9.
ChristT. J.ZopluogluC.MonaghenB. D.Van NormanE. R. (2013). Curriculum-based measurement of oral reading: Multi-study evaluation of schedule, duration, and dataset quality on progress monitoring outcomes. Journal of School Psychology, 51, 19–57.
10.
CoddingR. S.BegliciS.GottesmanD.JohnsonM.KertA. S.LebeoufP. (2009). Selecting intervention strategies: Using brief experimental analysis for mathematics problems. Journal of Applied School Psychology, 25, 146–168.
11.
DalyE. J.IIIMartensB. K.DoolE. L.HintzeJ. M. (1998). Using brief functional analysis to select interventions for oral reading. Journal of Behavioral Education, 8, 203–218.
12.
DalyE. J.IIIWittJ. C.MartensB. K.DoolE. J. (1997). A model for conducting a functional analysis of academic performance problems. School Psychology Review, 26, 554–574.
13.
DawesJ. M.PoncyB. C.SolomonB. G.DuhonG. J. (2018). A comparison of three variations of cover, copy, and compare procedures to increase multiplication fact performance. Manuscript in preparation.
14.
DuhonG. J.NoellG. H.WittJ. C.FreelandJ. T.DufreneB. A.GilbertsonD. N. (2004). Identifying academic skill and performance deficits: The experimental analysis of brief assessments of academic skills. School Psychology Review, 33, 429–443.
15.
DuhonG. J.PoncyB. C. (2015). The effectiveness of two explicit timing interventions to increase subtraction fluency. Unpublished dataset.
EckertT. L.ArdoinS. P.DaiseyD. M.ScarolaM. D. (2000). Empirically evaluating the effectiveness of reading interventions: The use of brief experimental analysis and single case designs. Psychology in the Schools, 37, 463–473.
18.
FuchsD.MockD.MorganP. L.YoungC. L. (2003). Responsiveness-to-intervention: Definitions, evidence, and implications for the learning disabilities construct. Learning Disabilities Research & Practice, 18, 157–171.
19.
GilbertsonD.WittJ.DuhonG.DufreneB. (2008). Using brief assessments to select math fluency and on-task behavior interventions: An investigation of treatment utility. Education & Treatment of Children, 31, 167–181.
20.
HardingJ.WackerD. P.CooperL. J.MillardT.Jensen-KovalanP. (1994). Brief hierarchical assessment of potential treatment components with children in an outpatient clinic. Journal of Applied Behavior Analysis, 27, 291–300.
21.
JohnstonJ. M.PennypackerH. S. (2008). Strategies and tactics of behavioral research (3rd ed.). New York, NY: Routledge.
22.
JonesK. M.WickstromK. F. (2002). Done in sixty seconds: Further analysis of the brief assessment model for academic problems. School Psychology Review, 31, 554–568.
23.
JonesK. M.WickstromK. F.DalyE. J. (2009). Best practices in brief assessment of reading concerns. In ThomasA.GrimesJ. (Eds.), Best practices in school psychology (5th ed., pp. 1163–1180). Bethesda, MD: National Association of School Psychologists.
24.
KahnemanD.TverskyA. (1973). On the psychology of prediction. Psychological Review, 80, 237–251.
25.
KilgusS. P.MetheS. A.MagginD. M.TomasulaJ. L. (2014). Curriculum-based measurement of oral reading (R-CBM): A diagnostic test accuracy meta-analysis of evidence supporting use in universal screening. Journal of School Psychology, 52, 377–405.
26.
LittleS. G.Akin-LittleA. (2014). Academic assessment and intervention. New York, NY: Taylor & Francis.
27.
McComasJ. J.WackerD. P.CoopreL. J.AsmusJ. M.RichmanD.StonerB. (1996). Brief experimental analysis of stimulus prompts for accurate responding on academic tasks in an outpatient clinic. Journal of Applied Behavior Analysis, 29, 397–401. doi:10.1901/jaba.1996.29-397
28.
MeehlP. E.RosenA. R. (1955). Antecedent probability and the efficiency of psychometric signs, patterns, or cutting scores. Psychological Bulletin, 52, 194–216.
29.
MercerS. H.HarpoleL. L.MitchellR. R.McLemoreC.HardyC. (2012). The impact of probe variability on brief experimental analysis of reading skills. School Psychology Quarterly, 27, 223–235.
30.
NoellG. H.FreelandJ. T.WittJ. C.GansleK. A. (2001). Using brief assessments to identify effective interventions for individual students. Journal of School Psychology, 39, 335–355.
31.
ParkerD. C.DickeyB. N.BurnsM. K.McMasterK. L. (2012). An application of brief experimental analysis with early writing. Journal of Behavioral Education, 21, 329–349.
32.
PoncyB. C.SkinnerC. H.AxtellP. K. (2005). An investigation of the reliability and standard error of measurement of words read correctly per minute using curriculum-based measurement. Journal of Psychoeducational Assessment, 23, 326–338.
33.
ReisenerC. D.DufreneB. A.ClarkC. R.OlmiD. J.TingstromD. H. (2016). Selecting effective interventions to increase math computation fluency via brief experimental analysis. Psychology in the Schools, 53, 39–57.
34.
SeethalerP. M.FuchsL. S. (2011). Using curriculum-based measurement to monitor kindergarteners’ mathematics development. Assessment for Effective Intervention, 36, 219–229.
35.
ShadishW. R.CookT. D.CampbellD. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Belmont, CA: Wadsworth, Cengage Learning.
36.
ShapiroE. S.EdwardsL.ZigmondN. (2005). Progress monitoring of mathematics among students with learning disabilities. Assessment for Effective Intervention, 30, 15–32.
37.
SkinnerC. H. (1998). Preventing academic skill deficits. In WatsonT. S.GreshamF. M. (Eds.), Handbook of child behavior therapy (pp. 61–79). New York, NY: Springer.
38.
StuebingK. K.FletcherJ. M.Branum-MartinL.FrancisD. J. (2012). Evaluation of the technical adequacy of three methods for identifying specific learning disabilities based on cognitive discrepancies. School Psychology Review, 41, 3–22.
39.
TillyW. D.III. (2002). Best practices in school psychology as a problem-solving enterprise. In ThomasA.GrimesJ. (Eds.), Best practices in school psychology (4th ed., pp. 21–36). Bethesda, MD: National Association of School Psychologists.
40.
VanDerHeydenA. M. (2011). Technical adequacy of response to intervention decisions. Exceptional Children, 77, 335–350.
41.
VanDerHeydenA. M.WittJ. C. (2005). Quantifying context in assessment: Capturing the effect of base rates on teacher referral and a problem-solving model of identification. School Psychology Review, 34, 161–183.
42.
VanDerHeydenA. M.WittJ. C.GilbertsonD. (2007). A multi-year evaluation of the effects of response to intervention (RTI) model on identification of children for special education. Journal of School Psychology, 45, 225–256.
43.
VanDerHeydenA. M.WittJ. C.NaquinG. (2003). Development and validation of a process for screening referrals to special education. School Psychology Review, 32, 204–227.
44.
Van NormanE. R.ChristT. J. (2016). Curriculum-based measurement of reading: Accuracy of recommendations from three-point decision rules. School Psychology Review, 45, 296–309.
45.
Van NormanE. R.ChristT. J.NewellK. W. (2017). Curriculum-based measurement of reading progress monitoring: The importance of growth magnitude and goal setting in decision making. School Psychology Review, 46, 320–328.
46.
Van NormanE. R.KlingbeilD. A.NelsonP. (2017). Posttest probabilities: An empirical demonstration of their use in evaluating the performance of universal screening measures across settings. School Psychology Review, 46, 349–362.
47.
What Works Clearinghouse. (2017). Procedures and standards handbook version 4.0. Washington, DC: Institute for Education Sciences.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
