Objective: The aim of this work was to quantify the interrater reliability of a set of scales that assess repetition, posture, and force as used on site when examining industrial work. Background: Interrater reliability of observational assessment methods can vary depending on the definition of the methods and situations in which they are used. Method: In several industries, 846 jobs were assessed using pairs of analysts to rate the repetition, force, and posture of the upper limbs. Twelve analysts with varying experience levels participated. Results: Using an interclass correlation coefficient (ICC), force and repetition had reliability values of .60 and .71 before and .82 and .87 after discussion, respectively. After discussion, peak posture ratings had ICCs of .60 to .83. ICCs for average posture ratings ranged from .31 to .51 initially to .55 to .67 in final ratings. Less experienced analysts changed their initial ratings more than did the senior investigators. Conclusion: The high interrater reliability of the repetition and force metrics indicates that a single analyst is appropriate for basic job assessment. Posture ratings benefit greatly from a two-analyst system. Average postures should be assessed across a full range of the scale for interrater reliability assessment. Analyst pairs should be rotated to avoid forming biases. Application: For basic assessments of forceful exertions and repetitive motions, a single analyst can be used, reducing the resource requirements for both industry and large epidemiological studies.
Get full access to this article
View all access options for this article.
References
1.
American Conference of General Industrial Hygienists. (2004). ACGIH 2004 TLVs® and BEIs® (Publication No. 0104). Cincinnati, OH: ACGIH Worldwide.
2.
Armstrong, T. J., & Chaffin, D. B. (1979). Carpal tunnel syndrome and selected personal attributes. Journal of Occupational Medicine, 21, 481–486.
3.
Armstrong, T. J., Fine, L. J., Goldstein, S. A., Lifshitz, Y. R., & Silverstein, B. A. (1987). Ergonomics considerations in hand and wrist tendinitis. Journal of Hand Surgery [Am], 12, 830–837.
4.
Atroshi, I., Gummesson, C., Johnsson, R., Ornstein, E., Ranstam, J., & Rosen, I. (1999). Prevalence of carpal tunnel syndrome in a general population. Journal of the American Medical Association, 282, 153–158.
5.
Baluyut, R., Genaidy, A.M., Davis, L. S., Shell, R. L., & Simmons, R. L. (1995). Use of visual perception in estimating static postural stresses: Magnitudes and sources of errors. Ergonomics, 38, 1841–1850.
6.
Bernhardt J., Bate P.J., & Matyas, T. A. (1998). Accuracy of observational kinematic assessment of upper-limb movements. Physical Therapy, 78 (3), 259–70.
7.
Corlett, E. N., Madeley, S. J., & Manenica, I. (1979). Posture targetting: A technique for recording working postures. Ergonomics, 22, 357–366.
8.
de Krom, M. C., Kester, A. D., Knipschild, P. G., & Spaans, F. (1990). Risk factors for carpal tunnel syndrome. American Journal of Epidemiology, 132, 1102–1110.
9.
DeLooze, M. P., Toussaint, H. M., Ensick, J., & Mangnus, C. (1994). The validity of visual observation to assess posture in a laboratory simulated, manual material handling task. Ergonomics, 37, 1335–1343.
10.
Denis, D., Lortie, M., & Bruxelles, M. (2002). Impact of observers' experience and training on reliability of observations for a manual handling task. Ergonomics, 45, 441–454.
11.
Gelfand, D. M., & Hartman, D. P. (1975). Child behavior analysis and therapy.New York: Pergamon.
12.
Janson, H., & Olsson, U. (2001). A measure of agreement for interval or nominal multivariate observations. Educational and Psychological Measurement, 61, 277–289.
13.
Karhu, O., Kansi, P., & Kuorinka, I. (1977). Correcting working postures in industry: A practical method for analysis. Applied Ergonomics, 8, 199–201.
14.
Kilbom, A. (1994). Assessment of physical exposure in relation to work-related musculoskeletal disorders - What information can be obtained from systematic observations?Scandinavian Journal of Work, Environment, and Health, 20, 30–45.
15.
Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33, 159–174.
16.
Latko, W. A., Armstrong, T. J., Foulke, J. A., Herrin, G. D., Rabourn, R. A., & Ulin, S. S. (1997). Development and evaluation of an observational method for assessing repetition in hand tasks. American Industrial Hygiene Association Journal, 58, 278–285.
17.
Leclerc, A., Franchi, P., Cristofari, M.F., Delemotte, B., Mereau, P., Teyssier-Cotte, et al. (1998). Carpal tunnel syndrome and work organization in repetitive work: Across sectional study in France: Study Group on Repetitive Work. Occupational and Environmental Medicine, 55 (3), 180–187.
18.
Leskinen, T., Hall, C., Rauas, S., Ulin, S., Tonnes, M., Viikari-Juntura, E., et al. (1997). Validation of portable ergonomic observation (PEO) method using optoelectronic and video recordings. Applied Ergonomics, 28, 75–83.
19.
Li, G., & Buckle, P. (1999). Current techniques for assessing physical exposure to work-related musculoskeletal risks, with emphasis on posture-based methods. Ergonomics, 42, 674–695.
20.
Lowe, B. D. (2004a). Accuracy and validity of observational estimates of shoulder and elbow posture. Applied Ergonomics, 35, 151–171.
21.
Lowe, B. D. (2004b). Accuracy and validity of observational estimates of wrist and forearm posture. Ergonomics, 47, 527–554.
22.
Moore, J.S., & Garg, A. (1995). The strain index: A proposed method to analyze jobs for risk of distal upper extremity disorders. American Industrial Hygiene Association Journal, 56, 443–58.
23.
Rempel, D. M., & Diao, E. (2004). Entrapment neuropathies: Pathophysiology and pathogenesis. Journal of Electromyography and Kinesiology, 14 (1), 71–75.
24.
Roquelaure, Y., Raimbeau, G., Dano, C., Martin, Y.H., Pelier-Cady, M. C., Mechali, S., et al. (2000). Occupational risk factors for radial tunnel syndrome in industrial workers. Scandinavian Journal of Work, Environment, and Health, 26, 507–513.
25.
Silverstein, B. A., Fine, L. J., & Armstrong, T. J. (1987). Occupational factors and carpal tunnel syndrome. American Journal of Industrial Medicine, 11, 343–358.
26.
van der Beek, A., van Gaalen, L. C., & Frings-Dresen, H. W. (1992). Working postures and activities of lorry drivers: A reliability study of on-site observation and recordings on a pocket computer. Applied Ergonomics, 23, 331–336.
