Although researchers have documented multiple antecedent and outcome correlates of the self-awareness construct, relatively little work has focused on placing the construct within a nomological network. Using item response theory (IRT) analyses of differential item functioning (DIF), the authors compared the relationships between observed and latent performance domains on a 360° assessment for high- and low-self-aware individuals. The reliability of discrepancy-defined self-awareness was assessed and deemed adequate, and indices of graded response model fit to 360° data were recorded. As an initial step toward defining this discrepancy-defined construct within a larger nomological network, extant constructs of personal warmth, self-promoting, and self-monitoring were identified as potential contributors to self-awareness using the DIF investigative framework.
American Educational Research Association, American Psychological Association, & National Council on Measurement in Education (Joint Committee). (1999). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.
2.
Atwater, L.E., Ostroff, C., Yammarino, F.J., & Fleenor, J.W. (1998). Self-other agreement: Does it really matter?Personnel Psychology, 51, 577-598.
3.
Atwater, L.E., Waldman, D., Ostroff, C., Robie, C., & Johnson, K.M. (2005). Self-other agreement: Comparing its relationship with performance in the U.S. and Europe. International Journal of Selection & Assessment, 13, 25-40.
4.
Atwater, L.E., & Yammarino, F.J. (1992). Does self-other agreement on leadership perceptions moderate the validity of leadership and performance predictions?Personnel Psychology, 45, 141-164.
5.
Atwater, L.E., & Yammarino, F.J. (1997). Self-other rating agreement: A review and model. Research in Personnel andHuman Resource Management, 15, 121-174.
6.
Baker, F.B. (1995). EQUATE 2.1: Computer program for equating two metrics in item response theory [Computer program]. Madison : University of Wisconsin, Laboratory of Experimental Design.
7.
Barr, M.A., & Raju, N.S. (2003). IRT-based assessments of rater effects in multiple-source feedback instruments. Organizational Research Methods, 6, 15-43.
8.
Bass, B.M., & Yammarino, F.J. (1991). Congruence of self and others' leadership ratings of naval officers for understanding successful performance. Applied Psychology: An International Review, 40, 437-454.
9.
Blackman, M.C., & Funder, D.C. (1998). The effect of information on consensus and accuracy in personality judgment. Journal of Experimental Social Psychology , 34, 164-181.
10.
Candell, G.L., & Drasgow, F. (1988). An iterative procedure for linking metrics and assessing item bias in item response theory. Applied Psychological Measurement, 17, 39-52.
11.
Church, A.H. (1997). Do you see what I see? An exploration of congruence in ratings from multiple perspectives. Journal of Applied Social Psychology, 27, 983-1020.
12.
Cleveland, J.N., Murphy, K.R., & Williams, R.E. (1989). Multiple uses of performance appraisal: Prevalence and correlates. Journal of Applied Psychology , 74, 130-135.
13.
Cortina, J.M. (1993). What is coefficient alpha? An examination of theory and applications. Journal of Applied Psychology, 78, 98-104.
14.
Craig, S.B., & Kaiser, R.B. (2003). Applying item response theory to multisource performance ratings: What are the consequences of violating the independent observations assumption?Organizational Research Methods, 6, 44-60.
Dunnette, M.D. (1993). My hammer or your hammer?Human Resource Management, 32, 373-384.
17.
Edwards, J.R. (2001). Ten difference score myths. Organizational Research Methods, 4, 265-287.
18.
Ellis, B.B., Becker, P., & Kimmel, H.D. (1993). An item response theory evaluation of an English version of the Trier Personality Inventory (TPI). Journal of Cross-Cultural Psychology, 24, 133-148.
19.
Fleenor, J., McCauley, C., & Brutus, S. (1996). Self-other rating agreement and leader effectiveness . Leadership Quarterly, 7, 487-506.
20.
Fletcher, C., & Baldry, C. (2000). A study of individual differences and self-awareness in the context of multi-source feedback. Journal of Occupational and Organizational Psychology, 73, 303-319.
21.
Gardner, R.C., & Neufeld, R.W. (1988). Use of the simple change score in correlational analyses. Educational and Psychological Measurement, 47, 849-864.
22.
Greguras, G.J., & Robie, C. (1998). A new look at within-source interrater reliability of 360-degree feedback ratings. Journal of Applied Psychology , 83, 960-968.
23.
Hambleton, R.K., Swaminathan, H., & Rogers, H.J. (1991). Fundamentals of item response theory (Vol. 2). Newbury Park, CA: Sage.
24.
Hulin, C.L. (1987). A psychometric theory of evaluations of item and scale translations. Journal of Cross-Cultural Psychology, 18, 115-142.
Johns, G. (1981). Difference score measures of organizational behavior variables: A critique. Organizational Behavior and Human Performance , 27, 443-463.
27.
Kenny, D.A. (1988). Interpersonal perception: A social relations analysis . Journal of Social and Personal Relationships, 5, 247-261.
28.
Kenny, D.A. (1994). Interpersonal perception: A social relations analysis . New York: Guilford.
29.
Kruger, J., & Dunning, D. (1999). Unskilled and unaware of it: How difficulties in recognizing one's own incompetence leads to inflated self-assessments. Journal of Personality and Social Psychology, 77, 1121-1134.
30.
Laffitte, L.J. (1998). Measurement equivalence of a 360° feedback assessment with confirmatory factor analysis and item response theory. Unpublished doctoral dissertation, Illinois Institute of Technology, Chicago.
31.
Lawler, E.E., III. (1967). Management performance as seen from above, below, and within. Journal of Applied Psychology, 51, 247-253.
32.
Lawler, E.E., III. (1987). The design of effective reward systems. In J. W. Lorsch (Ed.), Handbook of organizational behavior (pp. 289-327). Englewood Cliffs, NJ: Prentice Hall.
33.
Lord, F.M. (1956). The measurement of growth. Educational and Psychological Measurement, 16, 421-437.
Penny, J.A. (2001). Differential item functioning in an international 360-degree assessment: Evidence of gender stereotype, environmental complexity, and organizational contingency. European Journal of Work & OrganizationalPsychology, 10, 245-271.
39.
Penny, J.A. (2003). Exploring differential item functioning in a 360-degree assessment: Rater source and method of delivery. Organizational Research Methods, 6, 61-79.
40.
Pollack, D.M., & Pollack, L.J. (1996). Using 360° feedback in performance appraisal. Public Personnel Management, 25, 507-528.
41.
Raju, N. (2001). DFITPS6: A FORTRAN program for calculating DIF/DTF [Computer program]Chicago: Illinois Institute of Technology.
42.
Raju, N., Van der Linden, W., & Fleer, P. (1995). An IRT-based internal measure of test bias with applications for differential item functioning. Applied Psychological Measurement, 19, 353-368.
43.
Robie, C., Zickar, M.J., & Schmit, M.J. (2001). Measurement equivalence between applicant and incumbent groups: An IRT analysis of personality scales. Human Performance, 14, 187-207.
44.
Rogosa, D.R., & Willett, J.B. (1983). Demonstrating the reliability of the difference score in the measurement of change. Journal of Educational Measurement, 20, 335-343.
45.
Ryan, A.M., Horvath, M., Ployhart, R.E., Schmitt, N., & Slade, L.A. (2000). Hypothesizing differential item functioning in global employee opinion surveys. Personnel Psychology, 53, 531-562.
46.
Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores. Psychometrika Monograph, 17.
47.
Samejima, F. (1983). Some methods and approaches of estimating the operating characteristics of discrete item responses. In H. Wainer & S. Messick (Eds.), Principals of modern psychological measurement: A Festschrift for Frederic M. Lord (pp. 159-182). Hillsdale, NJ: Lawrence Erlbaum.
48.
Sharkey, L.D. (1999). Changing organizational culture through leadership development: A case in leadership transformation. Organization Development Journal, 17, 29-73.
49.
Spain, J.S., Eaton, L.G., & Funder, D.C. (2000). Perspectives on personality: The relative accuracy of self versus others for the prediction of emotion and behavior. Journal of Personality, 68, 837-867.
50.
Stark, S. (2001). MODFIT: A computer program for model-data fit . Unpublished manuscript, University of Illinois, Urbana—Champaign.
51.
Weissenberg, P., & Kavanagh, M.J. (1972). The independence of initiating structure and consideration: A review of the evidence. Personnel Psychology, 25, 119-130.
52.
Zickar, M.J., & Robie, C. (1999). Modeling faking good on personality items: An item-level analysis. Journal of Applied Psychology, 84, 551-563.
53.
Zimmerman, D.W. (1995). A note on interpretation of formulas for the reliability of differences. Journal of Educational Measurement, 31, 143-147.
