We have been using critical incidents to research the design and evaluation of computer-mediated collaborative and communication technologies for use in K-12 education. A variety of methods have generated a number of critical incidents identified during classroom evaluations. To enhance our analysis and understanding of these incidents, we developed a Web-based forum called the collaborative critical incident tool (CCIT). Users (teachers) and researchers collaboratively post, rate incident criticality, and elaborate on critical incidents through sustained dialog, contributing to the understanding of underlying conditions, causes, and implications. In this paper we describe the tool and its impact on evaluation.
Get full access to this article
View all access options for this article.
References
1.
AnderssonB.NilssonS. (1964). Studies in the reliability and validity of the critical incident technique. Journal of Applied Psychology, 48(6), 398–403.
2.
BowersJ. (1994). The work to make a network work: Studying CSCW in action. In Proceeding of CSCW 1994: Computer-Supported Cooperative Work (pp. 287–298). New York, NY: Association of Computing Machinery.
3.
CarrollJ. M.Koenemann-BelliveauJ.RossonM. B.SingleyM. K. (1993), Critical incidents and critical themes in empirical usability evaluation. In Proceedings of the BCSHCI1993 People and Computers VIII (pp. 279–292). Cambridge, England: Cambridge University Press.
4.
CarrollJ. M.NealeD. C.IsenhourP. L. (2000). The collaborative critical incident tool: Supporting reflection and evaluation in a web community. Manuscript submitted for publication.
5.
del GaldoE. M.WilligesR. C.WilligesB. H.WixonD. R. (1987). A critical incident evaluation tool for software documentation. In MarkL. S.WarmJ. S.HustonR. L. (Eds.), Ergonomics and Human Factors(pp. 253–258). New York, NY: Springer-Berlag.
6.
FivarsG. (1973). The critical incident technique: A bibliography. Palo Alto, CA: American Institutes tor Research.
7.
FlanaganJ. C. (1954). The critical incident technique. Psychological bulletin, 5(4), 327–358.
8.
GreenbergS. (Ed.). (1991). Computer supported cooperative work and groupware. London: Academic Press.
9.
GrudinJ. (1989). Why groupware applications fail: Problems in design and evaluation. Office: Technology and People, 4(3), 245–264.
10.
GrudinJ, (1990), Groupware and cooperative work: Problems and prospects. In LauralB. (Ed.), The Art of Human Computer Interface Design. Reading, MA: Addison-Wesley.
11.
HartsonH. R.CastilloJ. C. (1998). Remote evaluation for post-deployment usability improvement. In Proceedings of the Working Conference on Advanced Visual Interface AVI1998 (pp. 22–29). L'Aquila, Italy.
12.
KoenemannJ.CarrollJ. M.ShafferC. A.RossonM. B.AbramsM. (1998). Designing collaborative applications for classroom use: The LiNC Project. In DruinA. (Ed.), The design of children's technology (pp. 99–122). San Francisco: Morgan-Kaufmann.
13.
MaloneT. B. (1990). Human factors and human error. In Proceedings of the Human Factors Society 34th Annual Meeting (pp. 651–654). Santa Monica, CA: Human Factors Society.
14.
MiesterD. (1985). Behavioral Analysis and Measurement Methods. New York: Wiley.
15.
MorrisM. E.PlantT. A.HughesP. T. (1992). CoOpLab: Practical experiences with evaluating a multi-user system. In Proceeding of the HCI 1992 Conference on People and Computers (pp. 355–368). Huddersfield, UK: Cambridge University Press.
16.
NealeD. C.CarrollJ. M. (1999). Multi-faceted evaluation for complex, distributed activities. In Proceedings of CSCL1999 Computer Supported Cooperative Learning (pp. 425–433). Mahwah, N. J.: Lawrence Erlbaum.
ShattuckL. G.WoodsD. D. (1994). The critical incident technique: 40 years later. In Proceedings of the Human Factors and Ergonomics Society 38th Annual Meeting (pp. 1080–1084). Santa Monica, CA: Human Factors and Ergonomics Society.
19.
SproullL.KieslerS. (1991), Connections: New ways of working in the networked organization. London: MIT Press.
20.
WrightP.MonkA. F. (1989). Evaluation for design. In Proceedings of BCSHCI1989 People and Computers V (pp. 345–358). London: Springer-Verlag.
