Data logging has been a standard, but under utilized, software evaluation technique for single-user systems. Large volumes of objective data can be collected automatically and unobtrusively. This data, however, is usually in the form of low-level system events, making it difficult to analyze and interpret meaningfully. In this paper we extend traditional logging approaches to collaborative multi-user (groupware) systems. We also show how data captured at a higher level of abstraction can characterize user-system interaction more meaningfully. Lastly, we show how higher-level data abstracted from logging can be more effectively combined with data from other usability methods.
Get full access to this article
View all access options for this article.
References
1.
CastilloJ. C.HartsonH. R.HixD. (1997). Remote usability evaluation at a glance (Tech. Report TR-97-12). Blacksburg, VA: Virginia Tech.
2.
GoodM. (1985). The use of logged data in the design of a new text editor. In Proceeding of ACM CHI1985 Conference on Human Factors in Computing Systems (pp. 93–97). New York: Association of Computing Machinery.
3.
HammontreeM. L.HendricksonJ. J.HensleyB. W. (1992). Integrated data capture and analysis tools for research and testing on graphical user interfaces. In Proceedings of ACM CHI1992 Conference on Human factors in Computing Systems (pp. 431–432). New York: Association of Computing Machinery.
4.
HilbertD. M.RedmilesD. F. (1998). An approach to large-scale collection of application usage data over the internet. In Proceedings of 20th International Conference on Software Engineering (pp. 136–145). Los Alamitos, CA: IEEE Computer Society.
5.
IsenhourP. L.CarrollJ. M.NealeD. C.RossonM. B.DunlapD. R. (2000). The virtual school: An integrated collaborative environment for the classroom. Manuscript submitted for publication.
6.
IsenhourP. L.RossonM. B.CarrollJ. M., (2000). Supporting Asynchronous Collaboration and Late Joining in Java Groupware. Manuscript submitted for publication.
7.
KoenemannJ.CarrollJ. M.ShafferC. A.RossonM. B.AbramsM. (1999). Designing collaborative applications for classroom use; The LiNC Project, in DruinA. (Ed.) The design of children's technology, San Francisco; Morgan-Kaufmann.
8.
NealeD. C.CarrollJ. M. (1999). Multi-faceted evaluation for complex, distributed activities. In Proceedings of CSCL1999 Computer Supported Cooperative Learning (pp. 425–433). Mahwah, N. J.: Lawrence Erlbaum.
NieminenM.KasviJ. J. J.PulkkisA.VartiainenM. (1995), Interactive task support on the shop floor: Observations on the usability of the interactive task support system and differences in orientation and hands-on training use. In Proceedings of the HCI1995 Conference on People and Computers (pp. 79–93). Huddersfield, UK: Cambridge University Press.
SiochiA. C.HixD. (1991). A study of computer-supported user interface evaluation using maximal repeating pattern analysis. In Proceedings of ACM CHI1991 Conference on Human Factors in Computing Systems (pp. 301–305). New York: Association of Computing Machinery.
13.
WrightP.MonkA. F. (1989) Evaluation for Design. People and ComputersV, 345–358.
14.
YoderE.McCrackenD.AkscynR. (1984). Instrumenting a human-computer interface for development and evaluation. In Proceedings of IF1P INTERACT1984 Human-Computer Interaction (pp. 907–912), North-HollandElsevier Science Publishers.
