Sage Journals: Discover world-class research

Abstract

Objective

Trade-offs between productivity, physical workload (PWL), and mental workload (MWL) were studied when integrating collaborative robots (cobots) into existing manual work by optimizing the allocation of tasks.

Background

As cobots become more widely introduced in the workplace and their capabilities greatly improved, there is a need to consider how they can best help their human partners.

Methods

A theoretical data-driven analysis was conducted using the O*NET Content Model to evaluate 16 selected jobs for associated work context, skills, and constraints. Associated work activities were ranked by potential for substitution by a cobot. PWL and MWL were estimated using variables from the O*Net database that represent variables for the Strain Index and NASA-TLX. An algorithm was developed to optimize work activity assignment to cobots and human workers according to their most suited abilities.

Results

Human workload for some jobs decreased while workload for some jobs increased after cobots were reassigned tasks, and residual human capacity was used to perform job activities designated the most important to increase productivity. The human workload for other jobs remained unchanged.

Conclusions

The changes in human workload from the introduction of cobots may not always be beneficial for the human worker unless trade-offs are considered.

Application: The framework of this study may be applied to existing jobs to identify the relationship between productivity and worker tolerances that integrate cobots into specific tasks.

Keywords

collaborative robot O*Net database productivity physical workload mental workload

INTRODUCTION

Robots have demonstrated tremendous utility as both independent and collaborative physical assistants in activities ranging from making deliveries in hospitals (Mutlu & Forlizzi, 2008; Niechwiadowicz & Khan, 2008) to working as assembly-line workers in factories (Akella et al., 1999; Jarrasse et al., 2014; Sauppé & Mutlu, 2015). Recent research into human-robot teaming, however, reveals a tremendously powerful yet unexplored potential for robots to not only serve as substitutes for people in repetitive, hazardous, or demanding tasks but to augment and complement human cognitive and physical capabilities, thereby enabling people to engage in work activities that were previously unsafe, unhealthy, mentally challenging, or unfeasible (Michalos et al., 2015; Pearce et al., 2018). A deeper understanding of how robots can augment, as opposed to artificially replicate, human cognitive and physical work capabilities, can open new frontiers, for example, performing manual handling tasks while avoiding musculoskeletal injuries (Barondess, et al., 2001), enabling multitasking in cognitively demanding jobs, or expanding employment opportunities for individuals with disabilities or age-related decline, thus reshaping human work across various populations.

Research on the allocation of manufacturing tasks to human and robot operators has mainly considered collision avoidance, while more recent work has considered cognitive and physical ergonomics (Michalos et al., 2014, 2015). Farber et al. (Faber et al., 2016) described an approach for considering ergonomics and human movement capability to ascertain the optimal assembly sequences for human-robot collaboration. This approach was not applied to actual tasks as not all influencing factors they identified could be quantified. Task allocation between collaborating human and robot teams has been considered based on a linear resource utilization model of workload (Tsarouchi et al., 2016). The concept of employing intelligent assist devices for reducing physical stress in manufacturing operations was introduced by Kruger et al. (Krüger et al., 2006). Robotic task sharing for the purpose of relieving the human operator from excessive physical demands of work, as well as for reducing human errors, was previously considered from an ergonomics perspective by Ogorodnikova (Ogorodnikova, 2008).

Previous work by Pearce et al. (2018) included a computational method to jointly consider performance and ergonomic characteristics, particularly makespan (i.e., production time) and physical strain, in human-robot teaming. This work addressed the limitations of prior work that solely focused on the production time to allocate work to multi-robot teams (Koes et al., 2005) or human-robot teams (Ponda et al., 2010) or that aimed to minimize the ergonomic impact on human workers through the introduction of a collaborative robot (Faber et al., 2016). This method included a process for modeling work elements, task constraints, production time, physical strain, and worker skills and a bi-objective optimization formulation using a mixed-integer linear program (MILP). The bi-objective formulation offered the ability to explore different weights for each objective and find task-allocation solutions that are Pareto optimal (Censor, 1977). Pearce et al. (2018) found that there were trade-offs in assigning different priorities to time and ergonomics, including completion time, total strain, and worker idle time. A simplified bi-objective approach by Schoen et al. (2020) assisted engineers in constructing human-robot collaborative task plans. While the interface and approach showed clear benefits for integrating optimization in the task planning process, the formulation of human and robot cost measures were left mostly to the discretion of the engineer, who may or may not have expertise in ergonomics.

Work activities are suitable for automation at varying levels. Activities that rely on interpersonal relationships, interactions with work teams, supervision, resolving conflicts, interacting with customers, or creativity may be more suitable for human workers. Alternatively, activities that offer challenging environmental conditions, hazardous job exposures, awkward body positioning, unchallenging repetitious work, or stressful time pressures may be more suitable for automation. Robots similarly have task performance and work-context constraints, including movement speed, dexterous manipulation, or reach limitations; inability to use certain tools or machinery designed for human use; and lacking appropriate end-effectors to perform work elements, such as applying the tape, tying knots, or placing packaging materials in tight spaces.

This study explores potential trade-offs between different jobs that are currently performed exclusively by human workers where a collaborative robotic assistant can potentially help reduce physical or mental workload (MWL) through the allocation of one or more tasks. To explore a wide variety of jobs, data were utilized from the Occupational Information Network (O*NET) system, developed by the U.S. Department of Labor and provides information on 1110 occupations within the U.S. economy (O*NET, 2020). A systematic review by Cifuentes et al. (2010) identified 28 studies that used O*NET to estimate work exposures in relation to health or safety outcomes. Evanoff et al. (2014) concluded that job exposure matrices based on O*Net can be used as a measure of workplace physical exposures for some studies of musculoskeletal disorders.

One purpose of the O*NET system is to describe the critical tasks for each occupation in the database. We utilized the O*NET database to conduct a data-driven investigation into possible human-robot task allocations for existing manual jobs based on optimizing productivity and human physical and mental tolerances. We define tolerance as the maximum workload to avoid harm while performing work activities. This includes physical and mental strain, discomfort, or injuries. We first formulated a translation from O*NET parameters to represent measures of human workload. Then, we quantified the match between parameters to capabilities of each agent (human and robot). These were combined in a multi-objective optimization that allocates subtasks to individual agents. We report on the quantitative and qualitative results of this analysis and offer commentary on how this approach can be used in the construction of human-robot collaborative programs.

METHODS

We first defined jobs and work activities in the context of the O*NET database. The methods used in the calculation of PWL and MWL are introduced and compared against parameters obtained from O*NET online and evaluated. Based on the O*NET Content Model, activities for selected jobs were evaluated for work context, skills, and constraints and ranked by suitability for automating or relying on human workers. We extracted the associated work activities, detailed work activities, and work context for every job identified. Each work context was similarly scored based on its potential for physical and environmental strain, discomfort, or injuries. Using these scores, the PWL and MWL for each job were calculated. Work activities were ranked based on their human-robot collaboration potential, and tolerances were evaluated in terms of given weights to PWL and MWL. According to cobot ability and the optimization algorithm, we determined which work activities were to be completed by the cobot. Critically, residual human capacity gained by allocating a task away from the human to the cobot was then reallocated to perform the activities of the job rated most important in O*Net. This assumed that the most important work activities were most critical to improving productivity. After work activities were allocated, the PWL and MWL were recalculated and compared with the pre-optimization PWL and MWL to determine if there is an improvement after optimization. The process is summarized in Figure 1.

Figure 1.

High-level summary of the process used in our approach.

Human Work Qualification

Representative hand intensive jobs currently performed by human workers selected from different industries in the O*NET database were evaluated for PWL and MWL. A job consists of one or more work activities performed during a workday, possibly in repeating cycles (Radwin et al., 1994). The O*NET work context database categorizes the physical and cognitive factors that influence how people do their work and the constraints under which a task can be performed. These factors include aspects of ergonomics such as environmental conditions (e.g., noise, thermal, and vibration), job hazards (e.g., radiation, falls, and diseases), and body positioning (e.g., sitting, standing, bending, twisting, repetitive motions). It also offers occupational data and including information on skills, abilities, knowledge, work activities, and interests associated with these occupations (Peterson et al., 2001). A key aspect that is missing from the O*NET Content Model are quantifications of the likelihood of fatigue or injury from job hazards and the degree of injury associated with their physical or mental workloads. Our task-analysis approach adapted current occupational ergonomics and workload models into the process. Instruments for quantifying PWL and MWL were selected due to corresponding O*NET Work Output Factors.

The strain index (SI) is an observational tool for assessing jobs for risk of work-related musculoskeletal disorders of the distal upper extremities (Moore & Garg, 1995). The SI was selected for evaluating the PWL due to its similarity with parameters obtainable from the O*NET database. There are six SI risk factors, including intensity of exertion (IE), duration of exertion (DE), efforts per minute (EM), hand/wrist posture (HWP), speed of work (SW), and duration of task per day (DD). For each task and each hand, these risk factors are assessed by assigning them to a category. Each of the factors is rated and their product is classified into three categories to predict job safety, with “low” (SI ≤ 3.0), “medium” (3.0 < SI < 7), and “high” (SI ≥ 7).

The NASA-Task Load Index (NASA-TLX) is a multi-dimensional scale designed to obtain workload estimates from one or more operators while they are performing a task or immediately afterward (Hart, 2006). It is the most applied MWL assessment tool and has been widely used in civil and military aviation, driving, nuclear, power plant control room operations, and air traffic control (Cao et al., 2009; Endsley & Rodgers, 1997; Hart, 2006). The NASA-TLX consists of six subscales to measure operator task performance, including mental demand (MD), physical demand (PD), temporal demand (TD), effort (E), performance (P), and frustration level (FL). Each parameter is rated from very low to very high and should be obtained from one or more operators while they are performing a task or immediately afterward.

Since the SI and NASA-TLX measures were not directly attainable from O*NET Online, representative dimensions for each of these tolerance scales were considered. Parameters from O*NET corresponding to the SI and NASA-TLX criteria are provided in Table 1. These criteria scores were used as PWL and MWL parameters. These parameters were ascertained by the O*NET Data Collection Program, which provides several hundred ratings, based on responses from sampled workers to O*NET questionnaires.

TABLE 1:

SI and NASA-TLX parameters corresponding to O*NET work output factors

Dimension	Criteria from O*Net	Description of dimension	Description in O*NET online
SI
Intensity of exertion	Static strength	An estimate of the strength required to perform the task	The ability to exert maximum muscle force to lift, push, pull, or carry objects
Duration of exertion	Spend time using hand to handle	The physiological and biomechanical stresses related to how long an exertion is maintained	How much does this job require using your hands to handle, control, or feel objects, tools or controls?
Efforts per minute	Spend time making repetitive motions	The number of exertions per minute	How much does this job require making repetitive motions?
Hand/wrist posture	Handling and moving objects	An estimate of the position of the hand	Using hands and arms in handling, installing, positioning, and moving materials, and manipulating things
Speed of work	Time pressure	An estimate of how fast the worker is working	How often does this job require the worker to meet strict deadlines?
Duration per day	Duration of typical work week	An estimate of hours of work	Number of hours typically worked in 1 week

NASA-TLX
Mental demand	Attention to detail	How mentally demanding was the task?	Job requires being careful about detail and thorough in completing work tasks
Physical demand	Performing general physical activities	How physically demanding was the task?	Performing physical activities that require considerable use of your arms and legs and moving your whole body, such as climbing, lifting, balancing, walking, stooping, and handling of materials
Temporal demand	Time pressure	How hurried or rushed was the pace of the task?	How often does this job require the worker to meet strict deadlines?
Effort	Achievement/effort	How hard did you have to work to accomplish your level of performance?	Job requires establishing and maintaining personally challenging achievement goals and exerting effort toward mastering tasks
Performance	Consequence of error	How successful were you in accomplishing what you were asked to do?	How serious would the result usually be if the worker made a mistake that was not readily correctable?
Frustration level	Stress tolerance	How insecure, discouraged, irritated, stressed, and annoyed were you?	Job requires accepting criticism and dealing calmly and effectively with high stress situations

Jobs were arbitrarily selected representing hand intensive work for a varying range of automation, where O*NET provided sufficient data to conduct the analyses. The degree of automation from O*Net is a scale from 0 to 100 where 0 is “not at all automated” and 100 is “completely automated.” We selected specific jobs where the level of automation level was not high, indicating the potential to benefit from a robotic assistant. The 16 jobs are shown in Table 2.

TABLE 2:

O*NET occupations studied

Job No	O*NET Occupation Code	Job Title	Description	Degree of Automation
1	51–2011.00	Aircraft structure, surface, rigging and systems assemblers	Assemble, fit, fasten, and install parts of airplanes, space vehicles, or missiles, such as tails, wings, fuselage, bulkheads, stabilizers, landing gear, rigging and control equipment, or heating and ventilating systems	29
2	53–6051.08	Freight and cargo inspectors	Inspect the handling, storage, and stowing of freight and cargoes	29
3	51–6041.00	Shoe and leather workers and repairers	Construct, decorate, or repair leather and leather-like products, such as luggage, shoes, and saddles	7
4	51–2093.00	Timing device assemblers and adjusters	Perform precision assembling, adjusting, or calibrating, within narrow tolerances, of timing devices such as digital clocks or timing devices with electrical or electronic components	23
5	47–3011.00	Helpers—brickmasons, blockmasons, stonemasons, and tile and marble setters	Help brickmasons, blockmasons, stonemasons, or tile and marble setters by performing duties requiring less skill. Duties include using, supplying or holding materials or tools, and cleaning work area and equipment	26
6	17–2111.03	Product safety engineers	Develop and conduct tests to evaluate product safety levels and recommend measures to reduce or eliminate hazards	25
7	51–9071.06	Gem and diamond workers	Fabricate, finish, or evaluate the quality of gems and diamonds used in jewelry or industrial tools	39
8	51–4071.00	Foundry mold and coremakers	Make or form wax or sand cores or molds used in the production of metal castings in foundries	47
9	51–8091.00	Chemical plant and system operators	Control or operate entire chemical processes or system of machines	66
10	51–2021.00	Coil winders, tapers, and finishers	Wind wire coils used in electrical components, such as resistors and transformers, and in electrical equipment and instruments, such as field cores, bobbins, armature cores, electrical motors, generators, and control equipment	16
11	51–9192.00	Cleaning, washing, and metal pickling equipment operators and tenders	Operate or tend machines to wash or clean products, such as barrels or kegs, glass items, tin plate, food, pulp, coal, plastic, or rubber, to remove impurities	36
12	49–2091.00	Avionics technicians	Install, inspect, test, adjust, or repair avionics equipment, such as radar, radio, navigation, and missile control systems in aircraft or space vehicles	19
13	51–3022.00	Meat, poultry, and fish cutters and trimmers	Use hand or hand tools to perform routine cutting and trimming of meat, poultry, and seafood	47
14	51–6011.00	Laundry and dry-cleaning workers	Operate or tend washing or dry-cleaning machines to wash or dry-clean industrial or household articles, such as cloth garments, suede, leather, furs, blankets, draperies, linens, rugs, and carpets. Includes spotters and dyers of these articles	57
15	51–9111.00	Packaging and filling machine operators and tenders	Operate or tend machines to prepare industrial or consumer products for storage or shipment. Includes cannery workers who pack food products	39
16	51–9199.01	Recycling and reclamation workers	Prepare and sort materials or products for recycling. Identify and remove hazardous substances. Dismantle components of products such as appliances	27

Note. All data are from O*NET database. All jobs in the following will be replaced by serial numbers.

The PWL and MWL scores calculated for each parameter of the job are listed in Table 3. To aggregate the variables, we averaged the O*NET questions for a given index and then normalized that range of ratings for each parameter of PWL and MWL. The results are shown in Table 3. Since parameter ratings from O*NET range from 0 to 100 and all scores were normalized to facilitate subsequent calculations and comparisons. Each SI variable rating was matched to the corresponding O*Net multiplier value and the PWL was calculated from the product of the six multipliers (Moore & Garg, 1995). MWL was calculated using the sum of six variables and similarly normalized to a 0 to 20 scale for each parameter (Hart, 2006). Since individual factors in the SI are multiplied, they were similarly multiplied in our analysis. Similarly, since individual factors in the NASA-TLX are summed, they were summed in our analysis.

TABLE 3:

Normalized PWL and MWL scores for each parameter of selected jobs From O*NET Online

Job Number
	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15	16
PWL
Static strength	5.68	5.08	4.60	3.16	8.08	2.20	2.92	7.00	5.32	5.08	5.44	4.60	4.84	4.60	5.92	6.16
Spend time making repetitive motions	2.35	1.20	2.00	1.05	1.95	0.95	2.25	2.30	1.75	2.78	2.03	1.65	2.85	2.45	2.45	2.40
Handling and moving objects	2.40	1.74	2.06	2.44	2.72	1.68	2.20	2.64	2.26	2.40	2.56	2.28	2.60	2.38	2.40	2.54
Time pressure	1.63	1.70	1.74	1.48	1.86	1.75	1.68	1.77	1.78	1.79	1.77	1.89	1.90	1.73	1.86	1.63
Duration of typical work week	1.23	1.08	1.03	0.56	0.90	1.29	1.28	1.03	1.16	0.96	1.05	1.23	1.24	0.80	1.00	1.30
Spend time using hand to handle	2.93	1.38	2.78	1.70	2.48	1.35	2.78	2.75	2.33	2.88	2.30	2.53	2.78	2.38	2.73	2.53
Total PWL	187.10	26.65	93.80	11.46	177.56	10.68	85.92	212.06	101.23	167.58	120.55	101.17	234.01	88.17	176.43	200.92

MWL
Attention to detail	19.2	11	12.6	15.4	12.4	13	19.2	11	12.6	15.4	12.4	13	19.2	11	12.6	15.4
Performing general physical activities	17	13.6	14	13	9.6	14	17	13.6	14	13	9.6	14	17	13.6	14	13
Time pressure	15	9.8	14.8	11.6	6.2	9.8	15	9.8	14.8	11.6	6.2	9.8	15	9.8	14.8	11.6
Achievement/effort	17.6	7.2	9.6	12.4	3.8	10.6	17.6	7.2	9.6	12.4	3.8	10.6	17.6	7.2	9.6	12.4
Consequence of error	15	15.2	17.2	11.6	12.2	11.4	15	15.2	17.2	11.6	12.2	11.4	15	15.2	17.2	11.6
Stress tolerance	18.2	7.4	15	14.2	11.4	15.2	18.2	7.4	15	14.2	11.4	15.2	18.2	7.4	15	14.2

Total MWL	83.6	81.2	67.2	61.2	82.6	81.4	67.6	84.8	88.2	90.6	85	92.6	90.4	70.4	80.2	75

To select work activities to reassign to a cobot, the contribution of each work activity for the total task was calculated. O*NET provides a description of work activities for each job, and each job has an associated set of abilities required to perform it. Since O*NET does not provide associations between work activities themselves and abilities, these associations were generated for the human based on the descriptions provided. These abilities also need to be matched with the robot description below to determine whether the robot is good at them. Then, the PWL and MWL levels for each work activity were calculated based on the quantified importance of the work activity for each job and the required abilities, to assign high levels of workload tasks to the cobot.

Robot Work Qualification

Human-robot teaming can potentially leverage the strength and endurance of robots with the flexibility and decision-making capabilities of human operators. Before deciding which work activities to assign to cobots, it is important to understand their ability to perform the task. A decision tree was developed based on reported robot capabilities (Figure 2). The goal of this decision tree was to return a singular, binary choice of whether a robot would be capable of executing the work activity. This decision would then be used in the allocation process to filter tasks that the robot could perform for possible selection. Because of the generality of some of the work activities, these judgments were also made in the general sense for the current analysis. Our decision process was guided by both technical feasibility, as demonstrated by state-of-the-art systems, and the general practices and capabilities a small to medium-sized enterprise would be able to implement and/or maintain such a system. To utilize this decision tree, the work activity is evaluated by each question in the tree. Each response leads to either another question, or a High or Low rating, at which point the decision process concludes. A High rating means that robot can perform the task, while a Low rating means that the robot cannot. Details about the task layout, available systems, and robot capabilities have a great bearing on the robot’s capability to perform a work activity. The current analysis makes these assessments in the general and abstract sense, but it is also possible to use this decision tree to generate a more customized and accurate result in specific cases where these properties are more clearly defined.

Figure 2.

Decision tree for evaluating robot ability to perform a work activity. This tree considers an individual work activity from the O*NET database as input and returns either a high (robot can perform the activity) or low (robot cannot perform the activity) decision rating.

The first decision in assessing robot abilities was whether the work activity required a non-static process. If the process can effectively be done as a single set of unchanging actions, a robot could be utilized to perform that activity. For example, Job 5’s Mix substances or compounds needed for work activities could be done with a robot that simply mixes a container at the press of a button. In contrast, Job 4’s Disassemble equipment for maintenance or repair requires the agent to react to the current state of the activity, and potentially respond to different states of the equipment. For activities where a static process can suffice, they are rated High, while those that cannot be completed in such a way continue down the tree.

The second decision was whether a feasible algorithm existed to handle the logic behind the non-static work activity. Such algorithms may include problem space search, neural networks, and machine learning (Andrychowicz et al., 2020; Thomas et al., 2018) or planning (Pearce et al., 2018; Schoen et al., 2020). Work activities for which there is no feasible algorithm (e.g., Job 6’s Researching Product Safety) are rated as Low, while work activities that have such an algorithm (e.g., Job 9’s Monitor equipment fluid levels) continue down the tree.

The third decision assessed whether sensing was required to execute the work activity. Sensing has come a long way, especially in the realms of computer vision. Numerous methods are available to handle identifying objects (Bhargava & Bansal, 2018; Guo et al., 2014; Myers et al., 2014), but usually these systems need to be heavily customized and remain somewhat fragile (Bhargava & Bansal, 2018). We note however that a wide range of input types that might need to be sensed, including vision, sound, or touch. If the process could be configured in a way to avoid requiring sensing (e.g., Job 12’s Analyze test or performance data to assess equipment operation), the work activity capability is rated High. In contrast, work activities that may require some sort of sensing, such as Job 13’s Inspect food products, must be further considered.

The final decision considered cases where sensing was required and assessed whether there was a way of modeling the relevant sensory feeds. Kadir et al. (2018) identified a primary challenge of cobots to be their inability to modify their behavior to handle variations in the workspace. The ability to modify their behavior in this way is dependent on their understanding of what those variations mean for the state of the workspace, and how to structure future work. There are some existing solutions that attempt to solve parts of this problem. For instance, deep learning for fault detection (Iqbal et al., 2019), human safety responsive control strategies (Michalos et al., 2015), computer vision for PCB inspection (Mar et al., 2011), or language and conversation systems (Leite et al., 2016; Robb et al., 2019) have been considered. Computer vision applications, when feasible, have shown efficacy. For instance, CoSTAR (Paxton et al., 2017) allows engineers to construct manufacturing tasks that rely on object recognition. For other sensory inputs (e.g., voice), the current successes with modeling still trail human levels of comprehension. For work activities with well-modeled sensory feeds such that they can reason sufficiently about the environment, (e.g., Job 2’s Measure product or material dimensions), they are rated High. If such modeling is not currently possible, or is impractical (e.g., Job 16’s Repair production equipment or tools), they are rated Low.

It is important to note that while some jobs contain similar or identical work activities as others, each work activity might end up with a different rating, depending on the assumptions associated with the job. For example, Job 7 has the work activity Operate Grinding Equipment, much like Job 16. This work activity is rated differently between the two, since in the context of the first, the grinding is likely done on a small stone or custom art piece, which is also likely the end product. Since there is a large amount of variability in the grinding targets, the sensory environment is poorly modeled. In contrast, the grinding in Job 16 is likely for destructive purposes, and could be done with a much simpler process, for which, a limited understanding of this sensory environment would be possible. Thus, Job 7’s version of this work activity table is rated Low, while Job 16’s is rated High. A final decision for each Job’s work activities is included in Table 5.

Optimization Approach

Our method ranks the importance and workload contribution of work activity scores obtained from O*NET. The job sets and parameters are outlined in Table 4.

TABLE 4:

Job sets and parameters specified for the optimization approach

Formulation	Description
Sets
J	a set of jobs to be scheduled
A	a set of work activities composing each job
W	a set of workers (human or robotic) who will be assigned work activities
I	a set of work activity importance scores from O*NET online
D	a set of descriptor importance from O*NET online
P	a set of physical parameter scores from O*NET online
M	a set of mental parameter scores from O*NET online
N	a set of number of work activities
Parameters
$I_{J, A}$	Values for the work activity importance scores I of work activity A in job J
$D_{J, A}$	Values for descriptor importance D of work activity A in job J
$\sum_{1}^{N}$	Values for the sum of a value from work activity 1 to work activity N
$\prod_{1}^{6}$	Values for the product of the six physical parameters from O*NET online
$\sum_{1}^{6}$	Values for the sum of the six mental parameters from O*NET online
$P = {IE, DE, EM, HWP, SW, DD}$	a set of physical parameter scores P from ONET online (there are six parameters, IE, DE, EM, HWP*, SW, and DD, respectively)
$M = {MD, PD, TD, E, F, FL}$	a set of mental parameter scores M getting from O*NET online (there are six parameters, MD, PS, TD, E, F, FL, respectively)
Tolerances
$PWL = \prod_{i = 1}^{6} P_{i}$	a measure of the physical workload of a job
$MWL = \sum_{i = 1}^{6} M_{i}$	a measure of the mental workload of a job
= $\frac{I_{J, A} \times D_{J, A}}{\sum_{1}^{N} I_{J, A} \times D_{J, A}}$	Normalized contribution is the contribution ratio of each work activity to the whole job
$PW L_{i} = \prod_{i = 1}^{6} P_{i} \times \frac{I_{J, A} \times D_{J, A}}{\sum_{1}^{N} I_{J, A} \times D_{J, A}}$	PWL_i for work activity x_i (needed to measure the PWL of each work activity in a job)
$MW L_{i} = \sum_{1}^{6} M_{i} \times \frac{I_{J, A} \times D_{J, A}}{\sum_{1}^{N} I_{J, A} \times D_{J, A}}$	MWL_i for work activity x_i (needed to measure the MWL of each work activity in a job)
$T I_{i} = α P W L_{i} + (1 - α) M W L_{i}$	Tolerance index TI_i for activity x_i (α is the weight coefficient between physical and mental workload)

The following assumptions are introduced for an optimal assignment policy:

For each activity $x_{i} \in W, i = 1, ..., N$ , the Tolerance is defined as the weighted sum of PWL and MWL, and Tolerance index belongs to either a high or a low workload category, denoted as W_H and W_L, respectively, based on a threshold β, that is,

$x_{i} \in W_{H} if T I_{i} \geq β$ otherwise $x_{i} \in W_{L}$ .

For an activity $x_{i} \in R, i = 1, ..., N$ , the robot compatibility can be grouped into two categories: high and low, denoted as R_H and R_L according to the robot work activity decision tree (Figure 2), such that only work activities with High ratings are considered highly compatible.

No strain index exists for robots, that is, $T I_{i} = 0 i f x_{i} \in R,$ .

Due to the orders of magnitude difference between PWL and MWL, a coefficient is needed to convert the weights between PWL and MWL. Hence, α represents the proportion of PWL in overall job tolerances. In contrast, 1-α, which presents the proportion of MWL in the overall job tolerances.

The total number of activities that the robot can be assigned in the process is limited by K, where $K \leq N$ .

Consider the following objective: The total tolerance index of a job is defined

T I = \sum_{i = 1}^{N} T I_{i} {(1 - r)}_{i}

Where

r_{i}

is the replacement sign for activity

x_{i}

, which is defined as

r_{i} = {\begin{matrix} 1 & if x_{i} \in R \\ 0 & if x_{i} \in W \end{matrix}

It is preferred that the cobot with high compatibility can replace human operations with a high tolerance score. In other words, the search space can be reduced by limiting activities $x_{i} \in {W_{H} \cap R_{H}}$ . Thus, $R \subseteq {W_{H} \cap R_{H}}$ . In addition, not all activities in such a group can be replaced since the total number of replaced activities is constrained by K. Then, the optimization objective is to select activities that can be replaced by robots so that the overall strain index of the whole process will be minimized.

m i n \sum_{i = 1}^{N} (1 - r_{i}) (α P W L_{i} + (1 - α) M W L_{i})

s . t . \sum_{i = 1}^{N} r_{i} \leq K

x_{i} = {0,1}

\begin{array}{l} x_{i} \in {\begin{matrix} R & if r_{i} = 1 \\ W & if r_{i} = 0 \end{matrix} \\ x_{i} \in {\begin{matrix} W_{H} & if T I_{i} \geq β \\ W_{L} & if T I_{i} < β \end{matrix} \end{array}

x_{i} \in {W_{H} \cap R_{H}}

Quantitative Results

In this section, we apply our approach to the selected jobs from the O*NET database and explore changes in PWL and MWL when different work activities are reassigned to cobots. The contribution and importance of each work activity were calculated in addition to the PWL and MWL for each work activity as shown in Table 5. The maximum number of work activities that were reassigned per job was no more than two activities since the algorithm converges beyond two assignments. Residual human capacity of the human worker after cobot task assignment was used to perform the activity that was ranked greatest in importance in O*NET among the work activities in a job to increase productivity. Therefore, the PWL and MWL for additional work activities changed accordingly while the PWL and MWL for the remaining work activities remained the same. Correspondingly, the new PWL and MWL for the whole job were calculated.

TABLE 5:

Description and contribution of each work activity (In descending order of the normalized contribution). Each job is listed to the 16th work activity based on the descending order of normalized contributions, with the rest of the activities omitted. The normalized contribution of unlisted work activities is smaller than others, and these work activities are not selected for optimization.

	Work Activity	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15	16
	Total
Job 1 aircraft structure, surface, rigging, and systems assemblers (27 work activities)
Robot work activity rating		Low	High	High	High	High	High	High	High	High	High	High	Low	Low	Low	High	Low
Normalized contribution	1	0.047	0.043	0.041	0.041	0.041	0.041	0.040	0.040	0.039	0.038	0.038	0.038	0.038	0.038	0.038	0.038
Normalized importance	1	0.038	0.037	0.043	0.043	0.043	0.043	0.037	0.037	0.036	0.037	0.037	0.037	0.037	0.037	0.037	0.038
PWL	187.10	8.72	8.10	7.75	7.68	7.68	7.68	7.49	7.54	7.21	7.10	7.10	7.10	7.10	7.10	7.10	7.20
MWL	83.60	3.90	3.62	3.46	3.43	3.43	3.43	3.35	3.37	3.22	3.17	3.17	3.17	3.17	3.17	3.17	3.22
Job 2 freight and cargo inspectors (16 work activities)
Robot work activity rating		High	Low	Low	Low	Low	Low	Low	High	Low	Low	High	Low	Low	High	High	High
Normalized contribution	1	0.086	0.086	0.086	0.086	0.083	0.083	0.081	0.079	0.077	0.067	0.045	0.043	0.031	0.027	0.020	0.020
Normalized importance	1	0.073	0.073	0.073	0.073	0.075	0.065	0.072	0.075	0.072	0.072	0.064	0.034	0.065	0.046	0.035	0.035
PWL	26.65	2.29	2.29	2.29	2.29	2.21	2.22	2.16	2.10	2.04	1.78	1.20	1.15	0.83	0.73	0.53	0.53
MWL	81.20	6.97	6.97	6.97	6.97	6.72	6.78	6.58	6.40	6.23	5.43	3.65	3.52	2.53	2.23	1.62	1.62
Job 3 shoe and leather workers and repairers (25 work activities)
Robot work activity rating		High	High	High	Low	Low	Low	Low	High	High	Low	Low	High	Low	Low	Low	High
Normalized contribution	1	0.052	0.052	0.051	0.051	0.051	0.051	0.050	0.050	0.050	0.045	0.042	0.041	0.040	0.039	0.038	0.035
Normalized importance	1	0.045	0.045	0.045	0.045	0.045	0.045	0.045	0.045	0.045	0.038	0.036	0.042	0.038	0.036	0.045	0.042
PWL	93.80	4.92	4.92	4.79	4.79	4.79	4.79	4.65	4.65	4.65	4.17	3.90	3.82	3.73	3.69	3.56	3.287
MWL	67.20	3.52	3.52	3.43	3.43	3.43	3.43	3.33	3.33	3.33	2.99	2.79	2.73	2.67	2.64	2.55	2.36
Job 4 timing device assemblers and adjusters (10 work activities)
Robot work activity rating		Low	High	Low	High	Low	High	Low	Low	High	High
Normalized contribution	1	0.121	0.121	0.116	0.108	0.108	0.108	0.095	0.093	0.090	0.039
Normalized importance	1	0.107	0.107	0.107	0.100	0.100	0.100	0.094	0.100	0.094	0.094
PWL	11.46	1.388	1.388	1.328	1.241	1.241	1.241	1.093	1.063	1.030	0.444
MWL	61.20	7.41	7.41	7.09	6.63	6.63	6.63	5.84	5.68	5.50	2.37
Job 5 blockmasons, stonemasons, and tile and marble setters (12 work activities)
Robot work activity rating		High	Low	High	High	Low	Low	Low	High	Low	Low	Low	Low
Normalized contribution	1	0.101	0.093	0.093	0.090	0.089	0.084	0.084	0.084	0.079	0.079	0.072	0.053
Normalized importance	1	0.092	0.092	0.092	0.092	0.082	0.082	0.082	0.082	0.082	0.082	0.078	0.061
PWL	177.56	17.88	16.46	16.46	15.90	15.80	14.93	14.93	14.93	14.05	14.05	12.77	9.41
MWL	82.60	8.32	7.66	7.66	7.39	7.35	6.94	6.94	6.94	6.53	6.53	5.94	4.38
Job 6 product safety engineers (6 work activities)
Robot work activity rating		Low	Low	Low	High	Low	Low
Normalized contribution	1	0.208	0.200	0.176	0.162	0.152	0.101
Normalized importance	1	0.20	0.198	0.181	0.16	0.16	0.10
PWL	10.68	2.23	2.14	1.88	1.73	1.63	1.08
MWL	81.40	16.97	16.32	14.32	13.16	12.39	8.25
Job 7 gem and diamond workers (21 work activities)
Robot work activity rating		Low	Low	Low	High	High	Low	Low	High	Low	High	Low	Low	Low	High	Low	Low
Normalized contribution	1	0.064	0.062	0.036	0.058	0.058	0.055	0.054	0.053	0.053	0.053	0.053	0.053	0.052	0.048	0.044	0.044
Normalized importance	1	0.057	0.056	0.032	0.049	0.049	0.049	0.049	0.049	0.049	0.049	0.049	0.049	0.053	0.049	0.051	0.051
PWL	85.92	5.49	5.29	3.07	5.02	5.02	4.72	4.63	4.51	4.51	4.51	4.51	4.51	4.47	4.12	3.78	3.78
MWL	67.60	4.32	4.16	2.41	3.95	3.95	3.71	3.64	3.55	3.55	3.55	3.55	3.55	3.51	3.24	2.97	2.97
Job 8 foundry mold and coremakers (A total of 11 work activities)
Robot work activity rating		High	High	High	High	Low	High	High	High	High	High	Low
Normalized contribution	1	0.103	0.103	0.103	0.101	0.100	0.096	0.093	0.093	0.093	0.084	0.030
Normalized importance	1	0.095	0.095	0.095	0.095	0.094	0.094	0.095	0.095	0.095	0.095	0.050
PWL	212.06	21.875	21.875	21.875	21.389	21.128	20.328	19.769	19.769	19.769	17.824	6.458
MWL	84.80	8.75	8.75	8.75	8.55	8.45	8.13	7.91	7.91	7.91	7.13	2.58
Job 9 chemical plant and system operators (16 work activities)
Robot work activity rating		High	High	High	High	High	Low	High	High	Low	High	Low	High	Low	Low	High	Low
Normalized contribution	1	0.099	0.093	0.073	0.073	0.073	0.070	0.046	0.052	0.064	0.064	0.060	0.049	0.049	0.048	0.047	0.042
Normalized importance	1	0.080	0.080	0.072	0.072	0.072	0.064	0.057	0.057	0.061	0.069	0.059	0.059	0.047	0.057	0.047	0.047
PWL	101.23	10.00	9.42	7.37	7.37	7.37	7.06	4.67	5.26	6.46	6.51	6.04	4.96	4.94	4.84	4.72	4.22
MWL	88.20	8.71	8.21	6.42	6.42	6.42	6.15	4.07	4.59	5.63	5.67	5.26	4.32	4.31	4.22	4.11	3.68
Job 10 coil winders, tapers, and finishers (16 work activities)
Robot work activity rating		High	High	High	High	High	High	High	High	High	Low	High	Low	Low	Low	Low	High
Normalized contribution	1	0.087	0.087	0.087	0.085	0.077	0.076	0.075	0.062	0.062	0.061	0.061	0.052	0.040	0.035	0.031	0.024
Normalized importance	1	0.074	0.074	0.074	0.082	0.074	0.074	0.069	0.062	0.062	0.074	0.074	0.061	0.036	0.036	0.036	0.037
PWL	167.58	14.53	14.53	14.53	14.18	12.89	12.72	12.55	10.44	10.44	10.25	10.25	8.77	6.64	5.80	5.14	3.94
MWL	90.60	7.85	7.85	7.85	7.67	6.97	6.88	6.78	5.64	5.64	5.54	5.54	4.74	3.59	3.13	2.78	2.13
Job 11 cleaning, washing, and metal pickling equipment operators and tenders (14 work activities)
Robot work activity rating		High	High	Low	High	High	High	High	High	High	Low	High	High	High	High
Normalized contribution	1	0.082	0.077	0.077	0.075	0.075	0.073	0.073	0.073	0.071	0.071	0.071	0.069	0.064	0.049
Normalized importance	1	0.077	0.071	0.077	0.069	0.069	0.071	0.071	0.071	0.071	0.069	0.069	0.069	0.071	0.076
PWL	120.55	9.86	9.32	9.29	9.08	9.08	8.79	8.79	8.79	8.52	8.56	8.56	8.31	7.73	5.87
MWL	85.00	6.96	6.57	6.55	6.40	6.40	6.20	6.20	6.20	6.01	6.04	6.04	5.86	5.45	4.14
Job 12 avionics technicians (13 work activities)
Robot work activity rating		High	Low	High	High	Low	Low	High	Low	Low	Low	Low	Low	High
Normalized contribution	1	0.100	0.097	0.090	0.088	0.082	0.082	0.077	0.070	0.070	0.066	0.066	0.062	0.051
Normalized importance	1	0.092	0.086	0.086	0.082	0.090	0.073	0.072	0.070	0.090	0.070	0.064	0.060	0.065
PWL	101.17	10.07	9.77	9.15	8.88	8.26	8.28	7.82	7.03	7.12	6.70	6.69	6.23	5.15
MWL	92.60	9.22	8.94	8.38	8.13	7.56	7.58	7.16	6.44	6.52	6.13	6.12	5.70	4.71
Job 13 meat, poultry, and fish cutters and trimmers (8 work activities)
Robot work activity rating		High	Low	High	High	Low	Low	Low	Low
Normalized contribution	1	0.139	0.135	0.135	0.132	0.130	0.126	0.090	0.112
Normalized importance	1	0.130	0.130	0.130	0.130	0.130	0.131	0.094	0.125
PWL	234.01	32.60	31.70	31.70	30.80	30.51	29.37	21.03	26.29
MWL	90.4	12.59	12.25	12.25	11.90	11.78	11.35	8.13	10.16
Job 14 laundry and dry-cleaning workers (21 work activities)
Robot work activity rating		Low	High	Low	Low	Low	Low	High	High	High	Low	High	Low	Low	Low	Low	Low
Normalized contribution	1	0.060	0.060	0.060	0.058	0.058	0.057	0.050	0.050	0.050	0.050	0.050	0.046	0.046	0.046	0.045	0.044
Normalized importance	1	0.051	0.051	0.053	0.053	0.053	0.045	0.053	0.053	0.053	0.053	0.053	0.053	0.053	0.053	0.051	0.051
PWL	88.17	5.28	5.28	5.27	5.09	5.12	5.05	4.41	4.41	4.41	4.41	4.41	4.02	4.02	4.02	3.97	3.84
MWL	70.40	4.21	4.21	4.21	4.07	4.09	4.03	3.52	3.52	3.52	3.52	3.52	3.21	3.21	3.21	3.17	3.07
Job 15 packaging and filling machine operators and tenders (21 work activities)
Robot work activity rating		High	High	Low	High	High	Low	Low	Low	Low	High	High	High	High	Low	High	Low
Normalized contribution	1	0.058	0.057	0.052	0.052	0.051	0.051	0.050	0.050	0.049	0.049	0.048	0.048	0.048	0.048	0.048	0.046
Normalized importance	1	0.050	0.050	0.050	0.050	0.050	0.050	0.052	0.050	0.052	0.044	0.050	0.050	0.046	0.050	0.050	0.047
PWL	176.43	10.27	10.03	9.21	9.21	8.96	9.02	8.87	8.75	8.61	8.59	8.48	8.48	8.48	8.48	8.48	8.14
MWL	80.20	4.67	4.56	4.19	4.19	4.07	4.10	4.03	3.98	3.92	3.90	3.85	3.85	3.85	3.85	3.85	3.70
Job 16 recycling and reclamation workers (14 work activities)
Robot work activity rating		High	High	High	Low	High	Low	Low	Low	Low	Low	Low	High	Low	Low
Normalized contribution	1	0.084	0.084	0.084	0.082	0.082	0.082	0.082	0.069	0.069	0.069	0.067	0.050	0.050	0.050
Normalized importance	1	0.084	0.084	0.084	0.082	0.082	0.082	0.082	0.069	0.069	0.069	0.067	0.050	0.050	0.050
PWL	200.92	16.81	16.81	16.81	16.39	16.39	16.39	16.39	13.83	13.83	13.83	13.41	10.00	10.00	10.00
MWL	75.00	6.28	6.28	6.28	6.12	6.12	6.12	6.12	5.16	5.16	5.16	5.01	3.73	3.73	3.73

The PWL and MWL changes for each job before and after optimally assigning tasks to the cobot and human worker are shown in Figure 3. Similar trends were observed for Jobs 2, 3, 5, 9, and 14, indicating that PWL and MWL decreased after optimization. Conversely, PWL and MWL increased post-optimization for Jobs 1, 4, 6, 7, 10, 11, 12, 13, and 15. It is worth noting that the PWL and MWL remained the same pre- and post-optimization for Jobs 8 and 16.

Figure 3.

The PWL and MWL for each job before and after optimization. The squares ■ represent PWL and MWL for the original job and the circles • represent PWL and MWL for the optimized job. When there is only one black square, the original and optimized values are coincident. WA is the serial number of work activities that are replaced during the optimization process in each job. RA is the serial number of work activity that is reassigned to the human in each job.

DISCUSSION

Jobs That Benefit From a Cobot Assistant

Some jobs showed both reductions in PWL and MWL after assigning work activities to the cobot. The remaining work activities were redistributed to the human worker according to their order of importance, as quantified in the O*NET database. If these work activities were of high importance but imposed little demand on workload, the human worker benefited. On the contrary, if jobs where work activities were of high importance but imposed greater demands, the human worker had greater workload.

Considering the work scenario, if work activities within a job cannot be completed in parallel by robots and humans working together, it is uncertain whether the work will benefit from collaboration. Although PWL and MWL may be improved when the human worker is idle, it does not necessarily improve productivity. If more work activities can be performed in parallel, the human worker will have residual capacity to perform other work activities. When the reassignment of work activities was of high importance, but the workload requirements were low, reassignment often resulted in improved workload while increasing productivity. Cobots excel either for mostly physical or mostly mental tasks. Therefore, if the cobot was capable of performing major work activities that had high levels of PWL and MWL and the human worker was reassigned to these work activities after the cobot has completed them, the optimizer would have more options to assign work activities to the human worker and may find solutions that are more desirable.

Consider Job 14 (Laundry and Dry-Cleaning Workers) where the robot was allocated work activities 2 (Operate garment treatment equipment) and 7 (Lubricate production equipment), and the extended human work activity was 3 (Sew clothing or other articles). This allocation is reasonable given that both robot activities are ones that could be programmed with either static programs or dynamic ones with minimal sensing. In comparison, the extended human work activity is one that requires careful custom work that is likely impossible to create general programs for a cobot. Furthermore, it is likely that if given additional time, this allocation would result in improved performance. A similar pattern occurred for other jobs, such as Job 2 (Freight and Cargo Inspectors), Job 3 (Shoe and Leather Workers and Repairers), and Job 16 (Recycling and Reclamation Workers).

In general, physical work activities that were of high importance benefited most from a cobot assistant. Cobots are good at exerting static strength for repetitive, prolonged, or continuous tasks, and they are better suited to work with a high level of physical workload. By contrast, humans are better at mental abilities rather than physical abilities, such as problem sensing, quality control analysis, and problem-solving. If a work activity required greater physical capabilities, the more likely it was assigned to a cobot. At the same time, if the work activity was also among the most important of all work activities, then the work activity might be assigned to the robot to benefit the whole job, as cobots are better suited to perform physical activities for long periods of time.

Jobs That do not Benefit From a Cobot Assistant

We have also identified scenarios that yielded undesirable outcomes when a robotic assistant was introduced. After assigning some work activities that meet the requirements for a cobot, the human worker was left with idle capacity that could be filled with additional work. In our algorithm, we assumed that capacity would be filled with the most important activity that the robot is incapable of performing, presumably to increase the overall output. If the activity that was selected was particularly demanding, it can have negative effects on PWL or MWL, depending on the work activity’s abilities.

An example of this situation can be seen in Job 4 (Timing Device Assemblers and Adjusters). The robot was assigned work activities 2 (Reshape small metal components for precision assembly) and 4 (Apply lubricants or coolants to workpieces), while work activity 1 (Disassemble equipment for maintenance or repair) for the human workers was extended. While having a robot assisting likely reduced some workload for the human, the human’s reassignment of the highly important work activity 1, which requires a great deal of dexterity and arm-hand steadiness, can be physically demanding when repeated for extended periods of time. This resulted in an overall increased workload, despite the assistance of the robot. Other Jobs 7 (Gem and Diamond Workers) and 12 (Avionics Technicians) presented a similar allocation pattern. Alternatively, the robot’s assistance sometimes failed to offset the cost of the additional human workload in cases where the extended work activity was relatively trivial, but nevertheless straining when repeated. This type of pattern was seen in Jobs 1 (Aircraft Structure, Surfaces, Rigging, and Systems Assemblers), 13 (Meat, Poultry, and Fish Cutters and Trimmers), and 15 (Packaging and Filling Machine Operators and Tenders).

There is no doubt that cobots promise many benefits, including low-cost automation and flexibility in small-batch production. However, these results indicated that according to the order of importance of residual work activities, when work activities redistributed to humans had less ability requirements and higher workload, it led to an increase in the overall workload. When cobots are introduced to help improve productivity, it therefore is also necessary to consider the effect of the assignment of human surplus capacity.

PWL versus MWL

For several cases in this study, we saw that the tolerances before and after optimization had different outcomes. There were some cases where the PWL increased and the MWL decreased after optimization, or vice versa. The tolerances of overall work presented variations when PWL and MWL were weighted differently. When that happens, it is important that the weights for PLW and MWL are considered in a manner that allows for minimal tolerances. Although PWL was much greater than MWL in some cases due to the calculation methods of PWL and MWL, the degree of change before and after optimization should be compared in practice.

Work activities that required different abilities had PWL and MWL scores that varied. Specifically, the tolerances for pre- and post-optimization remained unchanged when the ability and workload required by the redistributed work activities coincided with the original work activity, and tolerances persisted regardless of the weight of PWL and MWL. In this case, while the cobot improved productivity, the workload remained unchanged before and after optimization. The variable TI combines a weighted value of PWL and MWL to obtain the total tolerance value. Although this summation is a theoretical construct, there is precedence for considering their combination. Young et al. (2015) conclude that the interaction between MWL and PWL are interdependent in the quantification of workload limits that specify when workers are approaching or exceeding their performance tolerances. Marras and Hancock (2014) contend that the mind and body cannot be separated in ergonomics. PWL is in fact a component of MWL in the NASA-TLX. Basahel and Ajovalasit (2010) found that light physical activity could compensate for the performance decrements imposed by mental underload.

Limitations and Future Directions

Although we used parameters from O*NET online to approximate PWL and MWL, the parameters were not direct measures, which means they are similar, but not identical. It is likely that for a specific application, actual measures of PWL and MWL from that context need to be collected. Future research will apply the SI and NASA-TLX to actual jobs. The strain index was selected as a multi-factor quantitative prototypical measure of physical workload for the distal upper limb because it has been widely used and studied for validity (Spielholz, et al., 2008). Given the theoretical nature of the current study, it is not possible to include individual differences or other complementary factors. Similarly, the NASA-TLX was selected because of its widespread usage and the correspondence between O*NET on-line variables and these measures of PWL and MWL. Although we used parameters from O*Net online to approximate PWL and MWL, these parameters were not direct measures, which means they are similar, but not identical. We anticipate that future applications of this work will require actual measures of PWL and MWL by the employing the SI and NASA-TLX instruments to actual industrial tasks.

As with the SI and NASA-TLX, resulting quantities under completely different conditions and factors can be equivalent. In the algorithm, for each activity in a job, we calculated the PWL and MWL, and the activities with the highest workload and meeting the conditions for the assignment of the cobots were assigned to the cobots. Hence, whether the different conditions had the same result or not, in the algorithm, the activities which meet the conditions for the assignment were assigned to the cobot. The selection of the 16 jobs were not random, but the outcome of the analysis is significant by the knowledge of the possibility that PWL or MWL can change either way with the introduction of robotic assistants for performing specific tasks. This approach does not account for actual cobots performing actual tasks. The cobot’s capability rating for each work activity is the result of careful literature review and expert questioning, but in reality, cobot capabilities are highly dependent on the specifics of the work environment and task requirements While job context was taken into account when assessing work activities, considerable variation is likely across instances of the same job. Moreover, product variation may lead to changes in the requirements not captured in a general work activity description. Therefore, the applicability of our general approach is uncertain for specific job contexts without that information, but it is possible to reassess the robot’s capability—as well as human PWL and MWL measures—in these specific contexts and redo the allocation with our method, thus producing more accurate and customized assessments when more of this information of the jobs is known. We also do not consider any costs for using the cobot in the optimization objective function. Cost factors, such as wear and tear, energy consumption, worker training, and implementation difficulty would be important to consider for future research. These hypothetical conditions that could provide physical or mental loads do not necessarily mean that when a task that is allocated to the robot because of an unsuitable tolerance value, may actually be hazardous to the health of the operator.

The potentially complex relationships between work activities were not taken into account. We assumed that each work activity was independent and can be operated in parallel, but in practice, we need to consider cases where there are temporal or spatial interactions between work activities, or when single work activities could be themselves collaborative processes. In addition, we did not consider situations where changing the workspace or work cell to allow human and robot collaboration could have downstream effects on the workload. These downstream effects could be across related jobs in the same workspace and impact subsequent workspace and activities in a production line. Given the higher productivity of the optimized job, other jobs dependent on the output from the job in question could be impacted negatively if trying to adjust to that change in productivity. This could be addressed by considering the entire ecosystem of jobs in a given site and considering how the outputs of one may affect the others.

The variable K (the number of jobs the robot could be assigned) was limited to two in this study in order to reduce the number of variables and enable comparisons across jobs. Due to the convergence of diminishing results trend given our residual calculation and the specific abilities available, even if the K values selected are different, the variation trend of PWL and MWL scores is similar. Therefore, a representative case of K=2 is selected for discussion. In a job where work activities are chained together, thereby requiring prior execution of other work activities, choosing a low K may minimize agent handovers or context switches. In other cases, the choice of K may have to be adjusted to a higher value to fully leverage the cobot, such as when work activities can be performed in parallel. A more extensive consideration of K could be investigated in the future.

CONCLUSIONS

In this paper, we demonstrate an approach using the O*NET Online database to consider productivity and human tolerances simultaneously for integrating a collaborative robot into a manufacturing process. We present a variation of the tolerances as a measure of the sum of physical and mental workload of each work activity in a job and use its value to evaluate the change of workload pre- and post-optimization. We discuss trade-offs on whether humans use redundant abilities to work with cobots after assigning some work activities to the cobot. The evidence from this study suggests that productivity and the changes in human tolerances should be considered when involving cobots. Larger studies with longer follow-up are needed to be the next step that identifies the relationship between productivity and worker tolerances in real-world factory work, which presents collaborative simulation using cobots.

Footnotes

Acknowledgments

This research received support from the National Science Foundation (Award No. 1928425).

Key Points

An approach using the O*NET Online database for considering productivity and human tolerances together for integrating a collaborative robot into a manufacturing process was developed.

A theoretical framework was built to compare the trade-offs between PWL and MWL after assigning work activities to the cobot.

The changes in human workload through the introduction of cobots are not always beneficial and the productivity and the changes of human workload need to be considered in the production involving cobots.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by Directorate for Engineering (1928425).

ORCID iDs

Bilge Mutlu

Robert G. Radwin

Li Liu is a PhD candidate in industrial engineering at the Northeastern University (Shenyang, China). She was a visiting student in the Department of Industrial and Systems Engineering, University of Wisconsin-Madison. She has BS and MS degrees in industrial engineering from the Northeastern University.

Andrew J. Schoen is a PhD student in computer sciences at the University of Wisconsin-Madison. He received BS degrees in psychology and biology and a MS in computer sciences from the University of Wisconsin-Madison.

Curt Henrichs is a graduate student in computer sciences at the University of Wisconsin-Madison. He has a BS in computer engineering from the Milwaukee School of Engineering.

Yajun Zhang is pursuing his PhD in control science and engineering at Southeast University, Nanjing (Jiangsu, China). He is a visiting student in the Department of Industrial and Systems Engineering, University of Wisconsin-Madison.

Jingshan Li is a professor in industrial and systems engineering at the University of Wisconsin-Madison. He received a BS, degree from Tsinghua University, MS degree from Chinese Academy of Sciences, and PhD from the University of Michigan.

Bilge Mutlu is the Sheldon B. and Marianne S. Lubar Professor of Computer Science at the University of Wisconsin-Madison. He received a BID degree from the Middle East Technical University, an MS degree from Istanbul University, and MDes and PhD degrees from the Carnegie Mellon University.

Robert G. Radwin is Duane H. and Dorothy M. Bluemke Professor in the College of Engineering, University of Wisconsin-Madison. He has a BS from New York University and MS and PhD degrees from the University of Michigan.

References

Akella

Peshkin

Colgate

E. D.

Wannasuphoprasit

Nagesh

Wells

Peacock

(1999). May). Cobots for the automobile assembly line. In Proceedings 1999 IEEE International Conference on Robotics and Automation (Cat. No. 99CH36288C), Detroit, MI, USA, 10–15 May, 1999 (1, pp. 728–733). IEEE.

Andrychowicz

O. M.

Baker

Chociej

Józefowicz

McGrew

Pachocki

Petron

Plappert

Powell

Ray

Schneider

Sidor

Tobin

Welinder

Weng

Zaremba

(2020). Learning dexterous in-hand manipulation. The International Journal of Robotics Research, 39(1), 3–20. https://doi.org/10.1177/0278364919887447

Barondess

J. A.

Cullen

M. R.

deLateur

B. J.

Deyo

R. A.

Donaldson

K. S.

Drury

C. G.

, et al. (2001). Musculoskeletal disorders and the workplace: low back and upper extremities. Washington, D.C.: National Academy of Sciences.

Basahel

A. M.

Ajovalasit

(2010). Effects of interaction between physical and mental workload on human performance. In Anderson

(Ed.), Contemporary Ergonomics and Human Factors, Institute of Ergonomics and Human Factors (pp. 598-601). Taylor Francis.

Bhargava

Bansal

(2018). Fruits and vegetables quality evaluation using computer vision: A review. Journal of King Saud University-Computer and Information Sciences.

Cao

Chintamani

K. K.

Pandya

A. K.

Ellis

R. D.

(2009). NASA TLX: Software for assessing subjective mental workload. Behavior Research Methods, 41(1), 113–117. https://doi.org/10.3758/BRM.41.1.113

Censor

(1977). Pareto optimality in multiobjective problems. Applied Mathematics and Optimization, 4(1), 41-59.

Cifuentes

Boyer

Lombardi

D. A.

Punnett

(2010). Use of O*NET as a job exposure matrix: A literature review. American Journal of Industrial Medicine, 53(9), 898–914. https://doi.org/10.1002/ajim.20846

Endsley

M. R.

Rodgers

M. D.

(1997). Distribution of attention, situation awareness, and workload in a passive air traffic control task: Implications for operational errors and automation. Air Traffic Control Quarterly, 6(1), 25. https://doi.org/10.2514/atcq.6.1.21

10.

Evanoff

Zeringue

Franzblau

Dale

A. M.

(2014). Using job-title-based physical exposures from O*NET in an epidemiological study of carpal tunnel syndrome. Human Factors: The Journal of the Human Factors and Ergonomics Society, 56(1), 166–177. https://doi.org/10.1177/00187208134965

11.

Faber

Kuz

Mertens

Schlick

C.M.

(2016). Model-Based Evaluation of Cooperative Assembly Processes in Human-Robot Collaboration. In Advances in Ergonomics of Manufacturing: Managing the Enterprise of the Future (pp. 101-112). Cham: Springer.

12.

Guo

Bennamoun

Sohel

Wan

(2014). 3D object recognition in cluttered scenes with local surface features: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 36(11), 2270–2287. https://doi.org/10.1109/TPAMI.2014.2316828

13.

Hart

S. G.

(2006). Nasa-task load index (nas-tlx); 20 years later. Proceedings of the Human Factors & Ergonomics Society Annual Meeting, 50(4661), 904–908. https://doi.org/10.1177/154193120605000909

14.

Iqbal

Maniak

Doctor

Karyotis

(2019). Fault detection and isolation in industrial processes using deep learning approaches. IEEE Transactions on Industrial Informatics, 15(5), 3077–3084. https://doi.org/10.1109/TII.2019.2902274

15.

Jarrassé

Sanguineti

Burdet

(2014). Slaves no longer: Review on role assignment for human-robot joint motor action. Adaptive Behavior, 22(1), 70–82. https://doi.org/10.1177/1059712313481044.

16.

Kadir

B. A.

Broberg

Souza da Conceição

(2018). Designing human-robot collaborations in industry 4.0: Explorative case studies. In DS 92: Proceedings of the DESIGN 2018 15th International Design Conference, (pp. 601–610). https://doi.org/10.21278/idc.2018.0319

17.

Koes

Nourbakhsh

Sycara

Koes

Sycara

Nourbakhsh

Jennings

N. R.

(2005, July). Heterogeneous multirobot coordination with spatial and temporal constraints. In AAAI (Vol. 5, pp. 1292–1297).

18.

Krüger

Bernhardt

Surdilovic

Spur

(2006). Intelligent assist systems for flexible assembly. CIRP Annals-Manufacturing Technology, 55(1), 29-32.

19.

Leite

Pereira

Funkhouser

Lehman

J. F.

(2016). October). Semi-situated learning of verbal and nonverbal content for repeated human-robot interaction. In Proceedings of the 18th ACM International Conference on Multimodal Interaction, Tokyo Japan, 12–16 November, 2016 (pp. 13–20).

20.

Mar

N. S. S.

Yarlagadda

P. K. D. V.

Fookes

(2011). Design and development of automatic visual inspection system for PCB manufacturing. Robotics and computer-integrated manufacturing, 27(5), 949–962.https://doi.org/10.1016/j.rcim.2011.03.007

21.

Marras

W. S.

Hancock

P. A.

(2014). Putting mind and body back together: A human-systems approach to the integration of the physical and cognitive dimensions of task design and operations. Applied Ergonomics, 45(1), 55–60. https://doi.org/10.1016/j.apergo.2013.03.025

22.

Michalos

Makris

Spiliotopoulos

Misios

Tsarouchi

Chryssolouris

(2014). ROBO-PARTNER: Seamless Human-Robot cooperation for intelligent, flexible and safe operations in the assembly factories of the future. Procedia CIRP, 23, 71-76.

23.

Michalos

Makris

Tsarouchi

Guasch

Kontovrakis

Chryssolouris

(2015). Design considerations for safe human-robot collaborative workplaces. Procedia CIrP, 37, 248–253. https://doi.org/10.1016/j.procir.2015.08.014

24.

Moore

J. S.

Garg

(1995). The strain index: A proposed method to analyze jobs for risk of distal upper extremity disorders. American Industrial Hygiene Association Journal, 56(5), 443-458. https://doi.org/10.1080/15428119591016863.

25.

Mutlu

Forlizzi

(2008). March). Robots in organizations: The role of workflow, social, and environmental factors in human-robot interaction. In 2008 3rd ACM/IEEE International Conference on Human-Robot Interaction (HRI), Amsterdam, The Netherlands, March 12–15, 2008 (pp. 287–294). IEEE.

26.

Myers

Kanazawa

Fermuller

Aloimonos

(2014). Affordance of object parts from geometric features. In Workshop on vision meets cognition. CVPR (9).

27.

Niechwiadowicz

Khan

(2008). Robot based logistics system for hospitals-survey. In IDT Workshop on Interesting Results In Computer Science and Engineering.

28.

Ogorodnikova

(2008). Human Weaknesses and strengths in collaboration with robots. Periodica Polytechnica. Engineering. Mechanical Engineering, 52(1), 25.

29.

O*NET . (2020). National Center for O*NET Development. The O*NET® Content Model. O*NET Resource Center. https://www.onetcenter.org/content.html.

30.

Paxton

Hundt

Jonathan

Guerin

Hager

G. D.

(2017). CoSTAR: Instructing collaborative robots with behavior trees and vision. In IEEE international conference on robotics and automation (ICRA) (pp. 564-571). Singapore: IEEE.

31.

Pearce

Mutlu

Shah

Radwin

(2018). Optimizing makespan and ergonomics in integrating collaborative robots into manufacturing processes. IEEE Transactions on Automation Science and Engineering, 15(4), 1772–1784. https://doi.org/10.1109/TASE.2018.2789820

32.

Peterson

N. G.

Mumford

M. D.

Borman

W. C.

Jeanneret

P. R.

Fleishman

E. A.

Levin

K. Y.

Campion

M. A.

Mayfield

M. S.

Morgeson

F. P.

Pearlman

Gowing

M. K.

Lancaster

A. R.

Silver

M. B.

Dye

D. M.

(2001). Understanding work using the occupational information network (O*net): Implications for practice and research. Personnel Psychology, 54(2), 451–492. https://doi.org/10.1111/j.1744-6570.2001.tb00100.x

33.

Ponda

Choi

H.L.

How

(2010). Predictive planning for heterogeneous human-robot teams. In AIAA Infotech @ Aerospace 2010 (p. 3349).

34.

Radwin

R. G.

Lin

M. L.

Yen

T. Y.

(1994). Rapid communication. exposure assessment of biomechanical stress in repetitive manual work using frequency-weighted filters. Ergonomics, 37(12), 1984–1998. https://doi.org/10.1080/00140139408964962

35.

Robb

D. A.

Lopes

Padilla

Laskov

Chiyah Garcia

F. J.

Liu

Patron

(2019). June). Exploring interaction with remote autonomous systems using conversational agents. In Proceedings of the 2019 on Designing Interactive Systems Conference, Diego CA USA, 23–28 June, 2019 (pp. 1543–1556).

36.

Sauppé

Mutlu

(2015). April). The social impact of a robot co-worker in industrial settings. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, (pp. 3613–3622).

37.

Schoen

Henrichs

Strohkirch

Mutlu

(2020). October). Authr: A task authoring environment for human-robot teams. In Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology, USA, 20–23 October, 2020 (pp. 1194–1208).

38.

Spielholz

Bao

Howard

Silverstein

Fan

Smith

Salazar

(2008). Reliability and validity assessment of the hand activity level threshold limit value and strain index using expert ratings of mono-task jobs. Journal of Occupational and Environmental Hygiene, 5(4), 250–257. https://doi.org/10.1080/15459620801922211

39.

Thomas

Chien

Tamar

Ojea

J. A.

Abbeel

(2018). May). Learning robotic assembly from cad. In IEEE International conference on robotics and automation (ICRA), (pp. 1–9).

40.

Tsarouchi

Makris

Chryssolouris

(2016). On a human and dual-arm robot task planning method. Procedia CIRP, 57, 551-555.

41.

Young

M. S.

Brookhuis

K. A.

Wickens

C. D.

Hancock

P. A.

(2015). State of science: Mental workload in ergonomics. Ergonomics, 58(1), 1–17. https://doi.org/10.1080/00140139.2014.956151

Human Robot Collaboration for Enhancing Work Activities

Abstract

Objective

Background

Methods

Results

Conclusions

Keywords

INTRODUCTION

METHODS

Human Work Qualification

Robot Work Qualification

Optimization Approach

Quantitative Results

DISCUSSION

Jobs That Benefit From a Cobot Assistant

Jobs That do not Benefit From a Cobot Assistant

PWL versus MWL

Limitations and Future Directions

CONCLUSIONS

Footnotes

Acknowledgments

Key Points

Funding

ORCID iDs

References