Because self-report is a valuable and often irreplaceable source of information about emotions, it is important to learn how measurement properties change across different demographic samples. Previous findings of mean differences in the level or variability of trait emotions for men versus women or across ethnic groups suggest that there may be differences in the internal consistency of the same measure when applied to different groups. Research groups using the Differential Emotions Scale (DES) have also begun to use the DES or derivative measures in children as young as 5 years old, raising questions about the psychometric performance of the measure across the life span. The present study uses reliability generalization to synthesize findings from 30 samples of raw data involving a total of 2,407 participants. Results indicate that higher socioeconomic status is associated with increased internal consistency in emotion scales. Age and majority ethnic status also were positively associated with internal consistency on a few emotion scales, whereas gender appeared unrelated to reliability of self-reported emotion.
Get full access to this article
View all access options for this article.
References
1.
Allen, J.G., & Haccoun, D.M. (1976). Sex differences in emotionality: A multidimensional approach. Human Relations, 29, 711-722.
2.
Anastasi, A., & Urbina, S. (1997). Psychological testing (7th ed.). New York: Macmillan.
3.
Bagby, R.M., Parker, J.D.A., & Taylor, G.J. (1994). The twenty-item Toronto Alexithymia Scale: I. Item selection and cross-validation of the factor structure. Journal of Psychosomatic Research, 38, 23-32.
4.
Bellah, R. (1986). Habits of the heart. Berkeley: University of California Press.
5.
Boyle, G.J. (1984). Reliability and validity of Izard’s Differential Emotions Scale. Personality and Individual Differences, 5, 747-750.
6.
Bretherton, I., & Beeghly, M. (1982). Talking about internal states: The acquisition of an explicit theory of mind. Developmental Psychology, 18, 906-921.
7.
Bretherton, I., Fritz, J., Zahn Waxler, C., & Ridgeway, D. (1986). Learning to talk about emotions: A functionalist perspective. Child Development, 57, 529-548.
8.
Brody, L.R., & Hall, J.A. (1993). Gender and emotion. In M. Lewis & J.M. Haviland (Eds.), Handbook of emotions (pp. 447-460). New York: Guilford.
9.
Brody, L.R., & Hall, J.A. (2000). Gender, emotion, and expression. In M. Lewis & J.M. Haviland (Eds.), Handbook of emotions (pp. 338-349). New York: Guilford.
10.
Buchner, A., Faul, F., & Erdfelder, E. (1996). G•Power: A priori, post-hoc, and compromise power analyses for the Macintosh (Version 2.1.1). Trier, Germany: University of Trier.
11.
Carey, T.C., Carey, M.P., & Kelley, M.L. (1997). Differential emotions theory: Relative contribution of emotion, cognition, and behavior to the prediction of depressive symptomatology in non-referred adolescents. Journal of Clinical Psychology, 53, 25-34.
12.
Carey, T.C., Finch, A.J., & Carey, M.P. (1991). Relation between differential emotions and depression in emotionally disturbed children and adolescents. Journal of Consulting and Clinical Psychology, 59, 594-597.
13.
Carstensen, L.L., & Charles, S.T. (1998). Emotion in the second half of life. Current Directions in Psychological Science, 7, 144-149.
14.
Carstensen, L.L., Pasupathi, M., Mayr, U., & Nesselroade, J.R. (2000). Emotional experience in everyday life across the adult life span. Journal of Personality & Social Psychology, 79, 644-655.
15.
Caruso, J.C. (2000). Reliability generalization of the NEO Personality Scales. Educational & Psychological Measurement, 60, 236-254.
16.
Caruso, J.C., & Edwards, S. (2001). Reliability generalization of the Junior Eysenck Personality Questionnaire. Personality & Individual Differences, 31, 173-184.
17.
Caruso, J.C., Witkiewitz, K., Belcourt-Dittloff, A., & Gottlieb, J.D. (2001). Reliability of scores from the Eysenck Personality Questionnaire: Areliability generalization study. Educational & Psychological Measurement, 61, 675-689.
18.
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum.
19.
Consedine, N.S., & Magai, C. (in press). The uncharted waters of emotion: Ethnicity, trait emotion, and emotion expression in older adults. Journal of Cross-Cultural Gerontology.
20.
Fujita, F., Diener, E., & Sandvik, E. (1991). Gender differences in negative affect and well-being: The case for emotional intensity. Journal of Personality & Social Psychology, 61, 427-434.
21.
Gross, J.J., et al. (1997). Emotion and aging: Experience, expression, and control. Psychology & Aging, 12, 590-599.
22.
Hall, J.A. (1978). Gender effects in decoding nonverbal cues. Psychological Bulletin, 85, 845-857.
23.
Hernandez, D.J. (1997). Child development and the social demography of childhood. Child Development, 68, 149-169.
24.
Houck, C.K., Biskin, D.S., & Regetz, J. (1973). A comparison of urban and rural reliability estimates for the Boehm Basic Concept Test. Psychology in the Schools, 10, 430-432.
25.
Izard, C.E. (1971). The face of emotion. New York: Appleton-Century-Crofts.
26.
Izard, C.E. (1972). Patterns of emotions. San Diego, CA: Academic Press.
27.
Izard, C.E. (1991). The psychology of emotions. New York: Plenum.
28.
Izard, C., Fine, S., Schultz, D., Mostow, A., Ackerman, B., & Youngstrom, E. (2001). Emotion knowledge as a predictor of social behavior and academic competence in children at risk. Psychological Science, 12, 18-23.
29.
Izard, C.E., Libero, D.Z., Putnam, P., & Haynes, O.M. (1993). Stability of emotion experiences and their relations to traits of personality. Journal of Personality and Social Psychology, 64, 847-860.
30.
Jenkins, J.M., & Oatley, K. (1996). Emotional episodes and emotionality through the life span. In C. Magai & S.H. McFadden (Eds.), Handbook of emotion, adult development, and aging (pp. 421-441). San Diego, CA: Academic Press.
31.
Kashani, J.H., Suarez, L., Allan, W.D., & Reid, J.C. (1997). Hopelessness in inpatient youths: A closer look at behavior, emotional expression, and social support. Journal of the American Academy of Child and Adolescent Psychiatry, 36, 1625-1631.
32.
Lawton, M.P., Kleban, M.H., & Dean, J. (1993). Affect and age: Cross-sectional comparisons of structure and prevalence. Psychology and Aging, 8, 165-175.
33.
Lazarus, R.S. (1991). Emotion and adaptation. New York: Oxford University Press.
34.
Lewis, M. (1993). Self-conscious emotions: Embarrassment, pride, shame, and guilt. In M. Lewis & J.M. Haviland (Eds.), Handbook of emotions (pp. 563-574). New York: Guilford.
35.
Maziade, M. (1984). Infant temperament: SES and gender differences and reliability of measurement in a large Quebec sample. Merrill-Palmer Quarterly, 30, 213-226.
36.
McNair, D.M., Lorr, M., & Droppleman, L.F. (1992). POMS manual: Profile of mood states. San Diego, CA: EdITS.
37.
Messick, S. (1995). Validity of psychological assessment: Validation of inferences from persons’ responses and performances as scientific inquiry into score meaning. American Psychologist, 50, 741-749.
38.
Plutchik, R. (1993). Emotions and their vicissitudes: Emotions and psychopathology. In M. Lewis & J.M. Haviland (Eds.), Handbook of emotions (pp. 53-66). New York: Guilford.
39.
Rehm, L.P. (1978). Mood, pleasant events, and unpleasant events: Two pilot studies. Journal of Consulting & Clinical Psychology, 46, 854-859.
40.
Rosenthal, R. (1991). Meta-analytic procedures for social research (Rev. ed., Vol. 6). Newbury Park, CA: Sage.
41.
Sattler, J.M. (1998). Clinical and forensic interviewing of children and families: Guidelines for the mental health, education, pediatric, and child maltreatment fields. San Diego, CA: Author.
42.
Shweder, R.A. (1993). The cultural psychology of the emotions. In M. Lewis & J.M. Haviland (Eds.), Handbook of emotions (pp. 417-433). New York: Guilford.
43.
Stapley, J.C., & Haviland, J.M. (1989). Beyond depression: Gender differences in normal adolescents’ emotional experiences. Sex Roles, 20, 295-308.
44.
Steiger, J.H. (1980). Tests for comparing elements of a correlation matrix. Psychological Bulletin, 87, 245-251.
45.
Stevens, J.P. (2001). Applied multivariate statistics for the social sciences (4th ed.). Hillsdale, NJ: Erlbaum.
46.
Tangney, J.P. (1990). Assessing individual differences in proneness to shame and guilt: Development of the Self-Conscious Affect and Attribution Inventory. Journal of Personality & Social Psychology, 59, 102-111.
47.
Tangney, J.P., & Fischer, K.W. (Eds.). (1995). Self-conscious emotions: The psychology of shame, guilt, embarrassment, and pride. New York: Guilford.
48.
Thayer, R.E. (2001). Calm energy: How people regulate mood with food and exercise. New York: Oxford University Press.
49.
Thompson, B., & Vacha-Haase, T. (2000). Psychometrics is datametrics: The test is not reliable. Educational & Psychological Measurement, 60, 174-195.
50.
Vacha-Haase, T. (1998). Reliability generalization: Exploring variance in measurement error affecting score reliability across studies. Educational & Psychological Measurement, 58, 6-20.
Vacha-Haase, T., Kogan, L.R., & Thompson, B. (2000). Sample compositions and variabilities in published studies versus those in test manuals: Validity of score reliability inductions. Educational & Psychological Measurement, 60, 509-522.
53.
Vacha-Haase, T., Ness, C., Nilsson, J., & Reetz, D. (1999). Practices regarding reporting of reliability coefficients: A review of three journals. Journal of Experimental Education, 67, 335-341.
54.
Wampler, B.E., da Silva, R., & Moore, M. (1989). Gram•mat•ik™ Mac: Powerful writing improvement (Version 1.0). San Francisco: Reference Software International.
55.
Watson, D. (2000). Mood and temperament. New York: Guilford.
56.
Watson, D., & Clark, L.A. (1994). The PANAS-X: Manual for the positive and negative affect schedule—Expanded form. Cedar Rapids: University of Iowa.
57.
Youngstrom, E.A., Frazier, T.W., & Butt, Z.A. (2001). The ugly emotions: Evidence for a third emotional dimension of dominance. Manuscript submitted for publication.
58.
Youngstrom, E.A., & Goodman, S. (2001). Children’s perceptions of other people: Mentalistic versus behavioristic descriptions of peers and adults. Developmental Science, 4, 164-173.
