Validity and Reliability Reporting Practices in the Field of Health Education and Behavior

Abstract

Health education and behavior researchers and practitioners often develop, adapt, or adopt surveys/scales to quantify and measure cognitive, behavioral, emotional, and psychosocial characteristics. To ensure the integrity of data collected from these scales, it is vital that psychometric properties (i.e., validity and reliability) be assessed. The purpose of this investigation was to (a) determine the frequency with which published articles appearing in health education and behavior journals report the psychometric properties of the scales/subscales employed and (b) outline the methods used to determine the reliability and validity of the scores produced. The results reported herein are based on a final sample of 967 published articles, spanning seven prominent health education and behavior journals between 2007 and 2010. Of the 967 articles examined, an exceedingly high percentage failed to report any validity (ranging from 40% to 93%) or reliability (ranging from 35% to 80%) statistics in their articles. For health education/behavior practitioners and researchers to maximize the utility and applicability of their findings, they must evaluate the psychometric properties of the instrument employed, a practice that is currently underrepresented in the literature. By not ensuring the instruments employed in a given study were able to produce accurate and consistent scores, researchers cannot be certain they actually measured the behaviors and/or constructs reported.

Keywords

health education health behavior psychometric properties validity reliability

Get full access to this article

View all access options for this article.

References

American Psychological Association, American Educational Research Association, & National Council on Measurement in Education. (1999). Standards for educational and psychological testing. Washington, DC: Author.

Anastasi

Urbina

(1997). Psychological testing (7th ed.). Upper Saddle River, NJ: Prentice Hall.

Barry

A. E.

Chaney

E. H.

Stellefson

M. L.

Chaney

J. D.

(2011). So you want to develop a survey: Practical recommendations for scale development. American Journal of Health Studies, 26, 97-105.

Churchill

G. A.

(1979, February). A paradigm for developing better measures of marketing constructs. Journal of Marketing Research, XVI, 64-73.

Clark

L. A.

Watson

(1995). Constructing validity: Basic issues in objective scale development. Psychological Assessment, 7, 309-319.

Cronbach

L. J.

Meehl

P. E.

(1955). Construct validity in psychological test. Psychological Bulletin, 52, 281-302.

DeVellis

R. F.

(2003). Scale development: Theory and applications. Thousand Oaks, CA: Sage.

Eason

(1991). Why generalizability theory yields better results than classical tests theory: A primer with concrete examples. In Thompson

(Ed.), Advances in educational research: Substantive findings, methodological developments (Vol. 1, pp. 83-98). Greenwich, CT: JAI Press.

Glover

E. D.

(2004). A new health education paradigm: Uncommon thoughts about common matters. American Journal of Health Education, 35, 260-271.

10.

Goodwin

L. D.

(2002). Changing conception of measurement validity: An update on the new Standards. Journal of Nursing Education, 41, 100-106.

11.

Goodwin

L. D.

Goodwin

W. L.

(1999). Measurement myths and misconceptions. School Psychology Quarterly, 14, 408-427.

12.

Guion

R. M.

(1980). On Trinitarian doctrines of validity. Professional Psychology, 11, 385-398.

13.

Hall

B. W.

Ward

A. W.

Comer

C. B.

(1988). Published educational research: An empirical study of its quality. Journal of Educational Research, 81, 182-189.

14.

Hogan

T. P.

Agnello

(2004). An empirical study of reporting practices concerning measurement validity. Education and Psychological Measurement, 64, 802-812.

15.

Huck

S. W.

(2004). Reading statistics and research (4th ed.). Boston, MA: Pearson.

16.

Joint Committee for Standards on Educational Evaluation. (1994). The program evaluation standards: How to assess evaluations of educational programs. Newbury Park, CA: Sage.

17.

McDermott

R. J.

(2000). Health education research: Evolution or revolution (or maybe both?). Journal of Health Education, 31, 264-271.

18.

Nunnally

J. C.

(1982). Reliability of measurement . In Mitzel

H. E.

(Ed.), Encyclopedia of educational research (pp. 1589-1601). New York, NY: Free Press.

19.

Qualls

A. L.

Moss

A. D.

(1996). The degree of congruence between test standards and test documentation within journal publications. Educational and Psychological Measurement, 56, 209-214.

20.

Rowley

G. L.

(1976). The reliability of observational measures. American Educational Research Journal, 13, 51-59.

21.

Smith

M. L.

(2009). Instrumentation and psychometric property reporting in current health education literature. American Journal of Health Studies, 24, 232-239.

22.

Thompson

(1990). ALPHAMAX: A program that maximizes coefficient alpha by selective item deletion. Educational and Psychological Measurement, 50, 585-589.

23.

Thompson

(1992). Two and on-half decades of leadership in measurement and evaluation. Journal of Counseling and Development, 70, 434-438.

24.

Thompson

(1994, April). Common methodology mistakes in dissertations, revisited. Paper presented at the annual meeting of the American Educational Research Association, New Orleans. (ERIC Document Reproduction Services No. ED 368 771)

25.

Thompson

(1999). Five methodological errors in educational research: A pantheon of statistical significance and other faux pas. In Advances in social science methodology (Vol. 5, pp. 23-86). Greenwich, CT: JAI Press.

26.

Thompson

(2003). Score reliability: Contemporary thinking on reliability issues. Thousand Oaks, CA: Sage.

27.

Torabi

M. R.

(2004). Optimistically looking ahead: Closing remarks by incoming president of the academy. American Journal of Health Behavior, 28, 569-571.

28.

Vacha-Haase

Ness

Nilsson

Reetz

(1999). Practices regarding reporting of reliability coefficients: A review of three journals. Journal of Experimental Education, 67, 335-341.

29.

Ward

A. W.

Hall

B. W.

Schramm

C. F.

(1975). Evaluation of published educational research: A national survey. American Educational Research Journal, 12, 109-128.

30.

Wilkinson

, & The APA Task Force on Statistical Inference. (1999). Statistical methods in psychology journals: Guidelines and explanations. American Psychologist, 54, 594-604.