The present study reports a reliability generalization (RG) meta-analysis of subscale and total scale scores on the Web-administered LibQUAL+™ protocol. Data were provided by 18,161 participantsfrom 43 universitiesin the United Statesand Canada. Results indicate that score reliabilities were remarkably invariant across campuses and different user groups.
Get full access to this article
View all access options for this article.
References
1.
Arnau, R. C. , Thompson, R. L., & Cook, C. (2001). Do different response formats change the latent structure of responses? An empirical investigation using taxonometric analysis. Educational and Psychological Measurement, 61, 23-44.
2.
Capraro, M. M. , Capraro, R. M., & Henson, R. K. (2001). Measurement error of scores on the Mathematics Anxiety Rating Scale across studies. Educational and Psychological Measurement, 61, 373-386.
3.
Caruso, J. C. (2000). Reliability generalization of the NEO personality scales. Educational and Psychological Measurement, 60, 236-254.
4.
Caruso, J. C. , & Edwards, S. (in press). Reliability generalization of the Junior Eysenck Personality Questionnaire. Personality and Individual Differences.
5.
Caruso, J. C. , Witkiewitz, K., Belcourt-Dittloff, A., & Gottlieb, J. (2001). Reliability of scores from the Eysenck Personality Questionnaire: A reliability generalization (RG) study. Educational and Psychological Measurement, 61, 675-682.
6.
Cook, C. , & Heath, F. (2001). Users’ perceptions of library service quality: A “LibQUAL+™” qualitative study. Library Trends, 49, 548-584.
7.
Cook, C. , Heath, F., & Thompson, B. (2001). Users’hierarchical perspectives on library service quality: A “LibQUAL+™” study. College and Research Libraries, 62, 147-153.
8.
Cook, C. , Heath, F., & Thompson, B. (2002). Score norms for improving library service quality: A LibQUAL+™ study. portal: Libraries and the Academy, 2, 13-26.
9.
Cook, C. , Heath, F., Thompson, B., & Thompson, R. L. (2001a). The search for new measures: The ARL “LibQUAL+™” study—A preliminary report. portal: Libraries and the Academy, 1, 103-112.
10.
Cook, C. , Heath, F., Thompson, R. L., & Thompson, B. (2001b). Score reliability in Webor Internet-based surveys: Unnumbered graphic rating scales versus Likert-type scales. Educational and Psychological Measurement, 61, 697-706.
11.
Cook, C. , & Thompson, B. (2001). Psychometric properties of scores from the Web-based LibQUAL+™ study of perceptions of library service quality. Library Trends, 49, 585-604.
12.
Courville, T. , & Thompson, B. (2001). Use of structure coefficients in published multiple regression articles: β isnot enough. Educational and Psychological Measurement, 61, 229-248.
13.
Crocker, L. , & Algina, J. (1986). Introduction to classical and modern test theory. New York: Holt, Rinehart & Winston.
14.
Heath, F. , Cook, C., Kyrillidou, M., & Thompson, B. (2002). ARL Index and other validity correlates of LibQUAL+™ scores. portal: Libraries and the Academy, 2, 27-42.
15.
Helms, J. E. (1999). Another meta-analysis of the White Racial Identity Attitude Scale’s Cronbach alphas: Implications for validity. Measurement and Evaluation in Counseling and Development, 32, 122-137.
16.
Henson, R. K. (2001). Understanding internal consistency reliability estimates: A conceptual primer on coefficient alpha. Measurement and Evaluation in Counseling and Development, 34, 177-189.
17.
Henson, R. K. , Kogan, L. R., & Vacha-Haase, T. (2001). A reliability generalization study of the Teacher Efficacy Scale and related instruments. Educational and Psychological Measurement, 61, 404-420.
18.
Hunter, J. E. , & Schmidt, F. L. (1990). Methods of meta-analysis: Correcting error and bias in research findings. Newbury Park, CA: Sage.
19.
Joint Committee on Standardsfor Educational Evaluation . (1994). The program evaluation standards: How to assess evaluations of educational programs (2nd ed.). Thousand Oaks, CA: Sage.
20.
Knapp, T. , & Sawilowsky, S. (2001). Constructive criticisms of methodological and editorial practices. Journal of Experimental Education, 70, 65-79.
21.
Reinhardt, B. (1996). Factorsaffecting coefficient alpha: A mini Monte Carlo study. In B. Thompson (Ed.), Advances in social science methodology (Vol. 4, pp. 3-20). Greenwich, CT: JAI Press.
22.
Sawilowsky, S. S. (2000). Psychometrics vs. datametrics: Comment on Vacha-Haase’s “reliability generalization” method and some EPM editorial policies. Educational and Psychological Measurement, 60, 157-173.
23.
Schmidt, F. L. , & Hunter, J. E. (1977). Development of a general solution to the problem of validity generalization. Journal of Applied Psychology, 62, 529-540.
24.
Thompson, B. (2001). Significance, effect sizes, stepwise methods, and other issues: Strong argumentsmove the field. Journal of Experimental Education, 70, 80-93.
25.
Thompson, B. (Ed.). (in press). A reader on score reliability: Reliability, reliability induction, and reliability generalization. Thousand Oaks, CA: Sage.
26.
Thompson, B. , Cook, C., & Heath, F. (2000). The LibQUAL+™ gap measurement model: The bad, the ugly, and the good of gap measurement. Performance Measurement and Metrics, 1, 165-178.
27.
Thompson, B. , Cook, C., & Heath, F. (2001). How many dimensions does it take to measure users’perceptions of libraries? A “LibQUAL+™” study. portal: Libraries and the Academy, 1, 129-138.
28.
Thompson, B. , & Vacha-Haase, T. (2000). Psychometrics is datametrics: The test is not reliable. Educational and Psychological Measurement, 60, 174-195.
29.
Vacha-Haase, T. (1998). Reliability generalization: Exploring variance in measurement error affecting score reliability across studies. Educational and Psychological Measurement, 58, 6-20.
30.
Vacha-Haase, T. , Kogan, L., Tani, C. R., & Woodall, R. A. (2001). Reliability generalization: Exploring reliability coefficients of MMPI clinical scales scores. Educational and Psychological Measurement, 61, 45-59.
31.
Vacha-Haase, T. , Kogan, L. R., & Thompson, B. (2000). Sample compositions and variabilities in published studies versus those in test manuals: Validity of score reliability inductions. Educational and Psychological Measurement, 60, 509-522.
32.
Vacha-Haase, T. , Tani, C. R., Kogan, L. R., Woodall, R. A., & Thompson, B. (2001). Reliability generalization: Exploring reliability variations on MMPI validity scale scores. Assessment, 8, 391-401.
33.
Viswesvaran, C. , & Ones, D. (2000). Measurement error in “Big Five Factors” personality assessment: Reliability generalization across studies and measures. Educational and Psychological Measurement, 60, 224-235.
34.
Whittington, D. (1998). How well do researchers report their measures? An evaluation of measurement in published educational research. Educational and Psychological Measurement, 58, 21-37.
35.
Wilkinson, L. , & American Psychological Association Task Force on Statistical Inference. (1999). Statistical methods in psychology journals: Guidelines and explanations. American Psychologist, 54, 594-604. print available from the APA home page at http://www.apa.org/journals/amp/amp548594.html)
36.
Yin, P. , & Fan, X. (2000). Assessing the reliability of Beck Depression Inventory scores: Reliability generalization across studies. Educational and Psychological Measurement, 60, 201-223.
