A Comparison of Reliability Measures for Continuous and Discontinuous Recording Methods: Inflated Agreement Scores With Partial Interval Recording and Momentary Time Sampling for Duration Events
Restricted accessResearch articleFirst published online July, 2011
A Comparison of Reliability Measures for Continuous and Discontinuous Recording Methods: Inflated Agreement Scores With Partial Interval Recording and Momentary Time Sampling for Duration Events
The authors evaluated the extent to which interobserver agreement (IOA) scores, using the block-by-block method for events scored with continuous duration recording (CDR), were higher when the data from the same sessions were converted to discontinuous methods. Sessions with IOA scores of 89% or less with CDR were rescored using 10-s partial interval recording (PIR) and 10-s momentary time sampling (MTS). Results indicated that IOA scores for 10-s PIR and 10-s MTS were consistently higher than IOA scores based on CDR for the same sessions. Specifically, 10-s MTS provided higher overestimations for low-duration events, whereas 10-s PIR produced higher overestimations for moderate- and high-duration events. Implications for researchers and clinicians are briefly discussed.
Birkimer, J.C., & Brown, J.H. ( 1979). A graphical judgmental aid which summarizes obtained and change reliability data and helps assess the believability of experimental effects. Journal of Applied Behavior Analysis, 12, 523-533.
2.
Carroll, R.A., Rapp, J.T., Colby-Dirksen, A.M., & Lindenberg, A.M. (2009). Detecting changes in simulated events II: Using variations of momentary time-sampling to measure changes in duration events. Behavioral Interventions, 24, 137-155.
Green, S.B., McCoy, J.F., Burns, K.P., & Smith, A.C. ( 1982). Accuracy of observational data with whole interval, partial interval, and momentary time-sampling recording techniques. Journal of Behavioral Assessment, 4, 103-118.
5.
Hanley, G.P., Cammilleri, A.P., Tiger, J.H., & Ingvarsson, E.T. (2007). A method for describing preschoolers’ activity preferences. Journal of Applied Behavior Analysis, 40, 603-618.
6.
Hawkins, R.P., & Dotson, V.A. ( 1975). Reliability scores that delude: An Alice in wonderland trip through the misleading characteristics of interobserver agreement scores in interval recording. In E. Ramp & G. Semb (Eds.), Behavioral analysis: Areas of research and application pp. 359-376. Englewood Cliffs, NJ: Prentice Hall.
7.
Johnston, J.M., & Pennypacker, H.S. (1993). Strategies and tactics of behavior research (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum.
8.
Kazdin, A.E. ( 1977). Artifact, bias, and complexity of assessment: The ABCs of reliability. Journal of Applied Behavior Analysis, 10, 141-150.
9.
Lerman, D.C., Tetreault, A., Hovanetz, A., Bellaci, E., Miller, J., & Karp, H., . . . Toupard, A. (2010). Applying signal-detection theory to the study of observer accuracy and bias in behavioral assessment. Journal of Applied Behavior Analysis, 43, 195-213.
10.
Meany-Daboul, M.G., Roscoe, E.M., Bourrett, J.C., & Ahearn, W.H. ( 2007). A comparison of momentary time sampling and partial interval recording for evaluating functional relations. Journal of Applied Behavior Analysis, 40, 501-514.
