Twenty-seven special education resource teachers were taught to use a standard trend estimation procedure as an aid in the visual analysis of their students' charts. Interrater agreement with reliability judges increased from .56 without trend estimation to .78 after training, and teachers became more certain of their judgments. In addition, consistent differences in certainty between teachers familiar and unfamiliar with the pupil, as represented by the graphed data, suggest that prior knowledge of student performance does have an effect on visual inference. Training in the use of the trend estimation technique also resulted in a shift in the criteria used in making judgments about graphed data. The increased reliability associated with the introduction of trend estimation suggests that the use of a judgmental aid may be important to visual inferences made by practitioners.
Get full access to this article
View all access options for this article.
References
1.
Bailey, D.B. (1984). Effects of lines of progress and semi-logarithmic charts on ratings of charted data. Journal of Applied Behavior Analysis, 17, 359–365.
2.
Barlow, D.H., & Hersen, M. (1984). Single case experimental designs: Strategies for studying behavior change (2nd ed.). New York: Peigamon.
3.
Campbell, D.T., & Stanley, J.C. (1963). Experimental and quasi-experimental designs for research. Boston: Houghton Mifflin.
4.
Carver, R.P. (1978). The case against statistical significance testing. Harvard Educational Review, 48, 378–399.
5.
Casey, A., Deno, S., Marston, D., & Skiba, R. (1988). Experimental teaching: Changing teachers' beliefs about effective instructional practices. Manuscript submitted for publication.
6.
Center, B., Skiba, R., & Casey, A. (1985/1986). A methodology for the quantitative synthesis of intra-subject design research. The Journal of Special Education, 19, 387–400.
Deno, S.L. (1986). Formative evaluation of individual student programs: A new role for school psychologists. School Psychology Review, 15, 358–374.
9.
DeProspero, A., & Cohen, S. (1979). Inconsistent visual analysis of intrasubject data. Journal of Applied Behavior Analysis, 12, 573–579.
10.
Edgington, E.S. (1980). Validity of randomization tests for one-subject experiments. Journal of Educational Statistics, 5, 235–251.
11.
Fuchs, L., Deno, S., & Mirkin, P. (1984). The effects of frequent, curriculum-based measurement and evaluation on pedagogy, student achievement, and student awareness of learning. American Educational Research Journal, 21, 449–460.
12.
Furlong, M.J., & Wampold, B.E. (1982). Intervention effects and relative variation as dimensions in experts' use of visual inference. Journal of Applied Behavior Analysis, 15, 415–421.
Glass, G.V., Willson, V.L., & Gottman, J.M. (1975). Design and analysis of time series experiments. Boulder: University of Colorado.
15.
Gorsuch, R.L. (1983). Three methods of analyzing limited time series (N of 1) data. Behavioral Assessment, 5, 141–154.
16.
Gottman, J.M. (1981). Time-series analysis: A comprehensive introduction for social scientists. New York: Cambridge University Press.
17.
Gottman, J.M., & Glass, G.V. (1978). Analysis of interrupted time-series experiments. In T.R. Kratochwill (Ed.), Single subject research: Strategies for evaluating change (pp. 197-235). New York: Academic Press.
18.
Green, S.B., McCoy, J.F., Burns, K.P., & Smith, A.C. (1982). Accuracy of observational data with whole interval, partial interval, and momentary time-sampling recording techniques. Journal of Behavioral Assessment, 4, 103–118.
19.
Greer, D. (1983). Contingencies of the science and technology of teaching and prebehavioristic research practices in education. Educational Researcher, 12, 3–9.
20.
Hartmann, D.P. (1974). Forcing square pegs into round holes: Some comments on “An analysis-of-variance model for the intra subject replication design.”Journal of Applied Behavior Analysis, 10, 151–166.
21.
Hinkle, D.E., Wiersma, W., & Jurs, S.G. (1979). Applied statistics for the behavioral sciences. Chicago: Rand McNally.
22.
Horne, G.P., Yang, M.C.K., & Ware, W.B. (1982). Time series analysis for single-subject designs. Psychological Bulletin, 91, 178–189.
23.
Huitema, B.E. (1986). Autocorrelation in behavior modification data: Wherefore art thou? In A.D. Poling & R.W. Fuqua (Eds.), Research & methods in applied behavior analysis: Issues and advances. New York: Plenum Press.
24.
Jenkins, J.R., Mayhall, W.F., Peschka, C.M., & Jenkins, L.M. (1974). Comparing small group and tutorial instruction in resource rooms. Exceptional Children, 40, 245–250.
25.
Jones, R.R., Weinrott, M.R., & Vaught, R.S. (1978). Effects of serial dependency on the agreement between visual and statistical inference. Journal of Applied Behavior Analysis, 11, 277–283.
26.
Kazdin, A.E. (1982). Single-case research designs: Methods for clinical and applied settings. New York: Oxford.
27.
Kirk, R.E. (1982). Experimental design: Procedures for the behavioral sciences (2nd ed.). Monterey, CA: Brooks/Cole.
28.
Knapp, T.J. (1983). Behavior analysts' visual appraisal of behavior change in graphic display. Behavioral Assessment, 5, 155–164.
29.
Koenig, C.H. (1973). Charting the future course of behavior. Dissertation Abstracts International, 33, 4200A. (University Microfilms No. 72-1850).
30.
Kratochwill, T.R., & Brody, G.H. (1978). Single subject designs: A perspective on the controversy over employing statistical inference and implications for research and training in behavior modification. Behavior Modification, 2, 291–307.
31.
Marston, D., Mirkin, P.K., & Deno, S.L. (1984). Curriculum-based measurement: Alternative to traditional referral, screening, and identification. The Journal of Special Education, 18, 109–118.
32.
McNemar, Q. (1975). Psychological statistics (5th ed.). New York: Wiley.
33.
Meichenbaum, D., & Burland, S. (1979). Cognitive behavior modification with children. School Psychology Digest, 8, 426–433.
34.
Michael, J. (1974). Statistical inference for individual organism research: Mixed blessing or curse?Journal of Applied Behavior Analysis, 7, 647–653.
35.
Parsonson, B.S., & Baer, D.M. (1978). The analysis and presentation of graphic data. In T.R. Kratochwill (Ed.), Single subject research: Strategies for evaluating change. New York: Academic Press.
36.
Powell, J., Martindale, A., Kulp, S., Martindale, A., & Bauman, R. (1977). Taking a closer look: Time sampling and measurement error. Journal of Applied Behavior Analysis, 10, 325–332.
37.
Revusky, S.H. (1976). Some statistical treatments compatible with individual organism methodology. Journal of the Experimental Analysis of Behavior, 10, 319–330.
38.
Shine, L.C., & Bower, S.M. (1971). A one-way analysis of variance for single-subject designs. Educational and Psychological Measurement, 31, 105–113.
39.
Skiba, R.J., Deno, S.L., Marston, D., & Wesson, C. (1986). Characteristics of time-series data collected through curriculum-based reading measurement. Diagnostique, 12, 3–15.
40.
Wampold, B.E., & Furlong, M.J. (1981a). The heuristics of visual inference. Behavioral Assessment, 3, 79–82.
41.
Wampold, B.E., & Furlong, M.J. (1981b). Randomization tests in single-subject designs: Illustrative examples. Journal of Behavioral Assessment, 3, 329–341.
42.
White, O.R. (1971). Pragmatic approaches to progress in the single case. Dissertation Abstracts International, 32, 5078-A. (University Microfilms No. 72-8618).
