Cluster Analyzing Profile Data Confounded with Interrater Differences: A Comparison of Profile Association Measures

Abstract

Seven association measures were compared for their effectiveness in relating and clustering data profiles confounded with interrater differences. Among these association indices were three distance measures, two measures of angular separation, and two measures of profile overlap. The objects of analysis were 50 jobs that had been rated by differ ent analysts on the items comprising the Occupa tion Analysis Inventory (OAI). Factor scores based on the OAI job ratings provided the profile data. Each of the seven association measures was applied to all pairwise combinations of factor-score profiles in the job sample, and the seven resultant 50 x 50 job proximity matrices were each subjected to hier archical cluster analysis. The job proximity ma trices and cluster structures based on the different association measures were then compared, re spectively, with a criterion proximity matrix and a criterion cluster structure. In relation to these two criteria, the angular measures (product-moment correlation and cosine) performed better than the distance and overlap measures. The results demon strate the importance of the choice of a profile as sociation measure in cluster analysis. The research er should be especially cautious when clustering en tities that have been rated by different judges. Un der such circumstances, it might be advisable to cluster analyze a data set using more than one as sociation measure and then to compare the alterna tive solutions for clarity and stability.

Get full access to this article

View all access options for this article.

References

Anderberg, M.R. Cluster analysis for applications. New York: Academic Press, 1973.

Archer, W.B. Computation of group job descriptions from occupational survey data (PRL-TR-66-12 . Lackland Air Force Base, TX: Personnel Research Laboratory, Aerospace Medical Division, December 1966. (NTIS No. AD 653 543)

Baker, R.F. , & Young, F.W. Interactive scaling with individual subjects II: An improved system (Report No. 32. Chapel Hill: University of North Carolina, L. L. Thurstone Psychometric Laboratory, April 1974.

Boese, R.R. , & Cunningham, J.W. Systematically derived dimensions of human work . JSAS Catalog of Selected Documents in Psychology, 1976, 6, 57. (Ms. No. 1270)

Borgen, F.H. , & Weiss, D.J. Cluster analysis and counseling research. Journal of Counseling Psychology, 1971,18, 583-591.

Computer Sciences Corporation. Program description manual—CODAP—computerized occupational data analysis programs . El Segundo, CA: Computer Sciences Corporation, September 1969.

Cronbach, L.J. , & Gleser, G.C. Assessing similarity between profiles. Psychological Bulletin , 1953, 50, 456-473.

Cunningham, J.W. , Tuttle, T.C. , Floyd, J.R. , & Bates, J.A. The development of the Occupation Analysis Inventory: An "ergometric" approach to an educational problem. JSAS Catalog of Selected Documents in Psychology, 1974, 4, 144. (Ms. No. 803)

Horn, J.L. An empirical comparison of methods for estimating factor scores. Educational and Psychological Measurement, 1965, 25, 313-322.

10.

Morsh, J.E. Identification of job types in the personnel career field (PRL-TR-65-9 . Lackland Air Force Base, TX: Personnel Research Laboratory, Aerospace Medical Division, April 1965. (NTIS No. AD 622 433)

11.

Nunnally, J.C. Psychometric theory (2nd ed.). New York: McGraw-Hill, 1978.

12.

Oats-Hills Consultants, Inc. CODAP/360: User's guide. Washington, DC: Author, 1970. (prepared for Task Analysis Branch, Headquarters, U. S. Marine Corps).

13.

Osgood, C.E. , & Suci, G.J. A measure of relation determined by both mean differences and profile information . Psychological Bulletin, 1952 , 49, 251-262.

14.

Phalen, W.J. , & Christal, R.E. Comprehensive occupational data analysis programs (CODAP), group membership (GRMBRS/GRPMBR), and automated diagramming (DIAGRM) programs (AFHRL-TR-73-5. Lackland Air Force Base, TX: Personnel Research Division, Air Force Human Resources Laboratory , April 1973. (NTIS No. AD 767 199)

15.

Riccobono, J.A. , & Cunningham, J.W. Work dimensions derived through systematic job analysis: A replicated study of the Occupation Analysis Inventory. JSAS Catalog of Selected Documents in Psychology, 1974, 4, 146. (Ms. No. 807)

16.

Riccobono, J.A. , Cunningham, J.W. , & Boese, R.R. Clusters of occupations based on systematically derived work dimensions: An exploratory study. JSAS Catalog of Selected Documents in Psychology , 1975, 5, 352. (Ms. No. 1150)

17.

Skinner, H.A. Differentiating the contribution of elevation, scatter, and shape in profile similarity. Educational and Psychological Measurement , 1978, 38, 297-308.

18.

U. S. Department of Labor, Bureau of Employment Security. Dictionary of occupational titles: Occupational classification (Vol. 2, 3rd ed.). Washington, DC: U. S. Government Printing Office, 1965.

19.

Ward, J.H. , Jr. Hierarchical grouping to maximize payoff (WADD-TN-61-29. Lackland Air Force Base, TX: Personnel Research Laboratory, Wright Air Development Division, March 1961. (NTIS No. AD 261 750)

20.

Ward, J.H. , Jr. Hierarchical grouping to optimize an objective function. Journal of the American Statistical Association, 1963, 58, 236-244.

21.

Ward, J.H. , Jr., & Hook, M.E. Application of a hierarchical grouping procedure to a problem of grouping profiles. Educational and Psychological Measurement, 1963, 23, 69-76.

22.

Winer, B.J. Statistical principles in experimental design (2nd ed.). New York: McGraw-Hill, 1971.