The purpose of this study was to examine the comparability of counts of embedded instruction learning trials when different methods of viewing and recording direct behavioral observations were used. In 13 classrooms, while videotaping embedded instruction implementation for a larger randomized controlled efficacy trial was occurring, teachers’ implementation of trials was coded in situ using pencil-and-paper methods. Videos were later coded using computer-assisted methods. Dependent-samples t tests, Pearson product-moment correlation coefficients, and additional score agreement calculations were conducted. Statistically significant differences were found in the estimates of trial frequency. Correlational analyses showed positive and strong relationships between the coding methods. Coding agreement was higher across the entire observation versus during 10-min continuous event blocks. In situ coding took significantly less time than video coding. Results provide empirical evidence for the advantages and disadvantages of common viewing and recording methods for quantifying behavior as part of systematic observation systems.
BernalM. E.GibsonD. M.WilliamsD. E.PessesD. I. (1971). A device for recording automatic audio tape recording. Journal of Applied Behavior Analysis, 4(2), 151–156. https://doi.org/10.1901/jaba.1971.4-151
2.
CohenE.RozenblatR. (2015). Applied, behavioral, analytic and. . . technological: A current literature review on the use of technologies in behavior analysis. European Journal of Behavior Analysis, 16(2), 178–187. https://doi.org/10.1080/15021149.2015.1085720
3.
FarrellA. D. (1991). Computer and behavioral assessment: Current applications, future possibilities and obstacles to routine use. Behavioral Assessment, 13(2), 159–179.
4.
GridleyN.BaywaterT. J.HutchingsJ. M. (2018). Comparing live and video observation to assess early parent–child interactions in the home. Journal of Child and Family Studies, 27(6), 1818–1829. https://doi.org/10.1007/s10826-018-1039-y
5.
HornE.LieberJ.LiS.SandallS.SchwartzI. (2000). Supporting young children’s IEP goals in inclusive settings through embedded learning opportunities. Topics in Early Childhood Special Education, 20(4), 208–223. https://doi.org/10.1177/027112140002000402
6.
JohnsonS. M.BolstadO. D. (1975). Reactivity to home observation: A comparison of audio recorded behavior with observers present or absent. Journal of Applied Behavior Analysis, 8(2), 181–185. https://doi.org/10.1901/jaba.1975.8-181
7.
KentR. N.O’LearyK. D.DietzA.DiamentC. (1979). Comparison of observational recordings in vivo, via mirror, and via television. Journal of Applied Behavior Analysis, 12(4), 517–522. https://doi.org/10.1901/jaba.1979.12-517
8.
KohlerF. W.StrainP. S.HoysonM.JamiesonB. (1997). Merging naturalistic teaching and peer-based strategies to address the IEP objectives of preschoolers with autism: An examination of structural and child behavior outcomes. Focus on Autism and Other Developmental Disabilities, 12(4), 196–206. https://doi.org/10.1177/108835769701200402
9.
LaneJ. D.LedfordJ. R. (2014). Using interval-based systems to measure behavior in early childhood special education and early intervention. Topics in Early Childhood Special Education, 34(2), 1–11. https://doi.org/10.1177/0271121414524063
10.
MartinM. (2019). Comparing in situ and video coding to quantify embedded instruction complete learning trials [Unpublished master’s thesis]. University of Florida.
11.
MudfordO. C.MartinN. T.HuiJ. K.TaylorS. A. (2009). Assessing observer accuracy in continuous recording of rate and duration: Three algorithms compared. Journal of Applied Behavior Analysis, 42(3), 527–539. https://doi.org/10.1901/jaba.2009.42-527
12.
Noldus Information Technology. (2015). The Observer XT (Version 12.5) [Computer software].
13.
PowellJ.MartindaleD.KulpS.MartindaleA.BaumanR. (1977). Taking a closer look: Time sampling and measurement error. Journal of Applied Behavior Analysis, 10(2), 325–332. https://doi.org/10.1901/jaba.1977.10-325
14.
RappJ. T.CarrollR. A.StangelandL.SwansonG.HigginsW. J. (2011). A comparison of reliability measures for continuous and discontinuous recording methods: Inflated agreement scores with partial recording and momentary time sampling for duration events. Behavior Modification, 35(4), 389–402. https://doi.org/10.1177/0145445511405512
15.
SkinnerB. F. (1953). Science and human behavior. Macmillan.
16.
SnyderP.AlginaJ.HemmeterM. L.McLeanM. (2015). Impact of professional development on preschool teachers’ use of embedded-instruction practices: An efficacy trial of tools for teachers [Abstract]. https://ies.ed.gov/funding/grantsearch/details.asp?ID=1619
17.
SnyderP.BishopC.HemmeterM. L.ReichowB.McLeanM.CrowR., & Embedded Instruction for Early Learning Project. (2017). EIOS: Embedded instruction for early learning observation system-revised (Version 1.1) [Manual and training videos, Unpublished instrument]. Anita Zucker Center for Excellence in Early Childhood Studies, University of Florida, Gainesville, FL.
18.
SnyderP.McLaughlinT.SandallS.McLeanM.HemmeterM. L.CrowR.ScottC., & Embedded Instruction for Early Learning Project. (2016). LTRS: Learning target rating scale (Version 2.3) [Manual, Unpublished instrument]. Anita Zucker Center for Excellence in Early Childhood Studies, University of Florida, Gainesville, FL.
TappJ.TichaR.KryzerE.GustafsonM.GunnarM. R.SymonsF. J. (2006). Comparing observational software with paper and pencil for time-sampled data: A field test of Interval Manager (INTMAN). Behavior Research Methods, 38, 165–169. https://doi.org/10.3758/BF03192763
21.
WesselD. (2015). The potential of computer-assisted direct observation apps. International Journal of Interactive Mobile Technologies, 9(1), 31–40. https://doi.org/10.3991/ijim.v9i1.4205
22.
WoleryM.HemmeterM. L. (2011). Classroom instruction: Background assumptions, and challenges. Journal of Early Intervention, 33(4), 371–380. https://doi.org/10.1177/1053815111429119
23.
YoderP. J.LloydB. P.SymonsF. J. (2018). Observational measurement of behavior (2nd ed.). Springer.
