A major determinant of the success of educational intranet sites is their usability. In addition to measures of task performance and navigation behavior, psychometric instruments are used to evaluate usability and the quality of human-computer interaction more generally. However, there is a lack of validated instruments for the evaluation of sites. The aim of the present study was to establish the psychometric properties of scales for measuring the quality of interaction with educational intranets. In two experiments, undergraduates (n = 81 and n = 109, respectively) performed an information retrieval task using different versions of the same university intranet site. Psychometric properties of the instruments were analyzed. The quality of the scales is evaluated, recommendations for their use are given, and further research into the scales is outlined.
Get full access to this article
View all access options for this article.
References
1.
AhujaJ., & WebsterJ. (2001). Perceived disorientation: An examination of a new measure to assess web design effectiveness. Interacting with Computers, 14, 5–29.
2.
BagozziR. P.DavisF. D., & WarshawP. R. (1992). Development and test of a theory of technological learning and usage. Human Relations, 45, 659–686.
3.
BrookeJ. (1996). SUS: A “quick and dirty” usability scale. In JordanP.ThomasB.WeerdmeesterB., & McClellandI. (Eds.), Usability evaluation in industry (pp. 189–194). London: Taylor & Francis.
4.
BrowningP., & WilliamsJ. (1997). Using the Internet in teaching and learning: A U.K. perspective. Computers & Geosciences, 23, 549–557.
5.
CampbellC. S., & MaglioP. P. (1999). Facilitating navigation in information spaces: Road-signs on the World Wide Web. International Journal of Human-Computer Studies, 50, 309–327.
6.
ChenH.WigandR., & NilanM. (1999). Optimal experience of web activities. Computers in Human Behavior, 15, 585–608.
7.
ChinJ. P.DiehlV. A., & NormanK. (1988). Development of an instrument measuring user satisfaction of the human-computer interface. In Conference on human factors in computing systems (pp. 213–218). New York: Association for Computing Machinery.
8.
CohenJ. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum.
9.
ConklinJ. (1987). Hypertext: An introduction and survey. IEEE Computer, 20, 17–41.
10.
DavisF. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly/September, 13, 318–340.
11.
DavisF. (1993). User acceptance of information technology: System characteristics, user perceptions and behavioral impacts. International Journal of Human-Computer Studies, 45, 19–45.
12.
DavisS., & WiedenbeckS. (2001). The mediating effects of intrinsic motivation, ease of use and usefulness perceptions on performance in first-time and subsequent computer users. Interacting with Computers, 13, 549–580.
13.
KirakowskiJ. (1996). The software usability measurement inventory: Background and usage. In JordanP.ThomasB.WeerdmeesterB., & McClellandI. (Eds.), Usability evaluation in industry (pp. 169–177). London: Taylor & Francis.
14.
KirakowskiJ., & CierlikB. (1998). Measuring the usability of web sites. In Proceedings of the 42nd Human Factors and Ergonomics Society annual conference (pp. 424–428). Santa Monica, CA: Human Factors and Ergonomics Society.
15.
LarsonK., & CzerwinskiM. (1998). Web page design: Implications of memory, structure and scent for information retrieval. Proceedings of CHI 1998 (pp. 25–32). Los Angeles, CA: Association for Computing Machinery.
16.
LeeM., & TedderM. (2004). Introducing expanding hypertext based on working memory capacity and the feeling of disorientation: Tailored communication through effective hypertext design. Journal of Educational Computing Research, 30, 171–195.
17.
LewisJ. (2002). Psychometric evaluation of the PSSUQ using data from five years of usability testing. International Journal of Human-Computer Interaction, 14, 463–388.
18.
LindgaardG., & DudekC. (2003). What is this evasive beast called user satisfaction?Interacting with Computers, 15, 429–452.
19.
MonetaG., & CsikszentmihalyiM. (1996). The effect of perceived challenges and skills on the quality of subjective experience. Journal of Personality, 64, 275–310.
20.
Murgolo-PooreM.PittL., & EwingM. (2002). Intranet effectiveness: A public relations paper-and-pencil checklist. Public Relations Review, 28, 113–123.
21.
NielsenJ. (1999). Intranet portals: The corporate information infrastructure. [Column posted on the World Wide Web.] Mountain View, CA: Author. Retrieved August 25, 2004 from the World Wide Web: http://www.useit.com/alertbox/990404.html
22.
NielsenJ. (2001). The 10 best intranet designs of 2001. [Column posted on the World Wide Web.] Mountain View, CA: Author. Retrieved August 25, 2004 from the World Wide Web: http://www.useit.com/alertbox/20011125.html
23.
NielsenJ. (2005). The canonical intranet homepage. [Column posted on the World Wide Web.] Mountain View, CA: Author. Retrieved May 24, 2005 from the World Wide Web: http://www.useit.com/alertbox/20050523.html
24.
OtterM., & JohnsonH. (2000). Lost in hyperspace: Metrics and mental models. Interacting with Computers, 13, 1–40.
25.
RosenfeldL., & MorvilleP. (2002). Information architecture for the World Wide Web (2nd ed.). Sebastopol, CA: O'Reilly.
26.
StalidisG.PrentzaA.VlachosI.MaglaveraS., & KoutsourisD. (2001). Medical support system for continuation of care based on XML web technology. International Journal of Medical Informatics, 64, 385–400.
27.
ThongJ.HongW., & TamK.-Y. (20902). Understanding user acceptance of digital libraries: What are the roles of interface characteristics, organizational context, and individual differences?International Journal of Human-Computer Studies, 59, 545–567.
28.
TractinskyN.KatzA. S., & IkarD. (2000). What is beautiful is usable. Interacting with Computers, 13, 127–145.
29.
van SchaikP., & LingJ. (2003a). Using online surveys to measure three key constructs of the quality of human-computer interaction in Web sites: Psychometric properties and implications. International Journal of Human-Computer Studies, 59, 545–567.
30.
van SchaikP., & LingJ. (2003b). The effect of link colour on information retrieval in educational intranet use. Computers in Human Behavior, 19, 553–564.
31.
WebbE.JonesA.BarkerP., & van SchaikP. (2004). Towards a classification of e-dialogue learning characteristics in students in Higher Education. Innovations in Education and Teaching International, 41, 93–103.
