A dynamic dispatching control system for processing workshop based on multi-agent and value matching

Abstract

In the current manufacturing environment, orders began to show trends in small batches and multiple varieties. Production workshops are expected to be more flexible and agile. This paper proposes a dynamic control system for the processing workshop based on the multi-agent architecture. The workshop layer is divided into multiple “processing cells” according to function. Machine agents and part agents are designed to achieve distributed control. In this paper, disturbances caused by random tasks to production are considered. A dynamic task dispatching mechanism based on the scrolling time window is investigated. When dispatching events occur, the machine agents and part agents determine the connection value between them through negotiation. An arbitration agent is designed to implement the dispatching through value matching. Finally, an actual experiment is carried out to verify the effectiveness of this dynamic control system.

Introduction

In today’s modern manufacturing environments, customer requirements have become personalized and dynamic. ¹ New task arrives continuously to the workshop with uncertain time, random quantities and unknown priorities. Tasks in the workshop begin to show trends in small batches and multiple varieties. In this new manufacturing scenario, the processing workshops are the first to be affected and are expected to be more flexible and agile. ²

A processing workshop mainly consists of different types of processing equipment, presenting a discrete distribution. Unlike the assembly line, this kind of workshop needs to handle a variety of disturbances. In addition to the small probability disturbance events such as equipment failures considered in several references,^3,4 the following two disturbances occur more frequently in the virtual environment: (1) The pre-processing preparation works (e.g. fixture and cutting tool replacement, and CNC program confirmation) are implemented by staffs, and the completion time is difficult to confirm; (2) During the processing of tasks, it is often necessary to replace the cutting tools that have reached their service life. These two disturbances cause the processing time to be unfixed and hard to predict accurately.

Due to the unstable processing time and the randomness of tasks, it is challenging to implement planned scheduling arranged on the premise of theoretical assumptions in actual production. In the new manufacturing situation, the workshop control system needs to dispatch tasks in real-time based on actual conditions. However, different from the closed-loop dynamic control system,^5,6 when the workshop control system makes a dispatching decision, it cannot know the result of the dispatching in real time. It will take a long time to know whether the decision is appropriate.

In order to make the production workshop better adapt to the new manufacturing environment, distributed artificial intelligence (DAI) was proposed. ⁷ This idea is to permit the decisional entities to work together to react quickly and dynamically. The multi-agent system (MAS) is the leading research idea in DAI, which can design and implement DAI structure to solve the problems of complexities, uncertainties, and dynamics. ⁸ Researchers applied this system architecture to workshops and eventually formed the multi-agent manufacturing system (MAMS). ⁹

In MAMS, devices and software functions are abstracted into different agents. ¹⁰ They are endowed with decision-making ability and can achieve complex goals through cooperation. ¹¹ Krothapalli and Deshmukh ¹² defined two typical agent types: machine agent (MA) and part agent (PA). Wong et al. ¹³ achieved a new contract network mechanism for dynamic control by using MAs and PAs. In addition to abstracting entities into agents, more and more functional agents are investigated, such as supervisor agent (SA) and database agent (DA).^14,15 Autonomy and negotiation are fundamental features to promote workshop operations in MAMS. ¹⁶ Duffie ¹⁷ proposed the heterarchical control architecture based on MAMS. Because of its strong flexibility, adaptability and stability, this structure is the main approach to realize MAMS.^11,18

Although in the 1990s, scholars proposed the concept of part agent and machine agent, the research of MAMS has long been in a theoretical state. Under the technology conditions of that time, it is challenging to realize real-time mapping of the physical layer and software layer in practice. As technology advances, the controllers of the machines start using external interfaces, which can be used to obtain data from controllers and control the machines in real-time. Simultaneously, new information collection techniques are also gradually applied in the workshop. For example, Barenji et al. ¹⁹ proposed an approach to achieve an agent-based control system based on RFID (radio frequency identification) technology. Parts can be monitored in this system. Zhang et al. ²⁰ proposed an approach that used innovative gateway technology to capture real-time production data from various manufacturing resources.

At present, how to use MAMS technology to achieve DAI in the virtual environment efficiently and how to design the interactive mechanism between agents to achieve self-organizing production are still open problems. ⁹ Usually, researchers focus on establishing a dynamic dispatching mechanism of tasks, but ignore the application environment. The dynamic dispatching approach of the workshop needs to be constructed after realizing the self-organizing control of production. Take the processing workshop as an example. First, it needs to ensure that MAMS can realize the self-organizing part-processing. Then, the dynamic task-dispatching mechanism designed on this basis can be used in practice.

This paper proposed a dynamic manufacturing control system for the processing workshop. In this system, the connection/mapping approach between agents and physical entities is constructed to form the distributed control of production. The main contributions of the paper are summarized as follows:

• A distributed control structure based on multi-agent for processing workshop is presented, which can realize the self-organizing processing of the parts.

• Based on this control structure, a new value-based task dispatching method is proposed. The MAs and PAs implement a matching mechanism through interaction.

The following research content is arranged as follows. Section 2 introduces a new production control structure based on multi-agent to realize the self-organizing part-processing. The dynamic dispatching mechanism and the interaction mechanism between agents in the processing are also studied in this section. Section 3 introduces an active application. The effectiveness of the control system proposed in this paper is proved by practical verification in this section. Section 4 summarizes the research work presented in this paper.

Dynamic manufacturing control system for processing workshop

Distributed control structure based on multi-agent

The processing workshop is a discrete and complex system. It mainly includes processing equipment (mainly machine tools), logistics handling equipment (such as manipulators and AGVs), warehouses, and workers. This paper does not discuss the integration of workers because an independent management system will run this function generally. As mentioned above, the behaviors of workers will cause the instability of the processing time, which is considered as a disturbance in this paper.

The processing workshop can be divided into different units with equipment. Taking a machine tool as an example (as shown in Figure 1), the unit divided around the machine tool includes its buffer and accessory equipment (such as sensors, RFID readers, and handheld devices). This paper refers to such units as processing cells. At the physical level, embedded technology is used to unify equipment interfaces. At the software level, each “processing cell” has its own corresponding MA. MAs can realize the control of the equipment.

Figure 1.

Processing cell and machine agent.

This paper divides the MA into two parts, as shown in Figure 1. The first part is the “adaptation layer.” It is used to implement the action-driven and information acquisition of the “processing cell.” The processing cell usually contains a variety of controllers. For example, in a processing cell for milling, the milling machine is controlled by a CNC, the buffer is controlled by a microcomputer, and the RFID reader also has its own controller. The “adaptation layer” will establish connections with these controllers, and it consists of two parts, including “link library” and “execution function API.” The “link library” is used to realize the information interaction with the controllers, and the Dynamic Link Library (DLL) corresponding to the underlying device controllers are integrated in it. The “execution function API” encapsulates function interfaces for implementing machine control.

The second part in MA is the “analysis layer.” Java Agent DEvelopment (JADE) framework is used to develop the agent program. ²¹ Communication-module, decision-module and controller-module are constructed to realize main logical functions of MAs. The communication-module encapsulates the interface of message exchange between two agents. Information from other agents and the “analysis layer” is processed in the decision-module. The content of the decisions from this module can be the active control of the “processing cell,” or replying to the information from other agents, or initiating a negotiation between multiple agents. The controller-module connects to the adaptation layer and realizes the agent’s control of the device. It realizes the autonomy of the “processing cell” together with the decision-module.

With the development of information technology, RFID and two-dimensional code technology have been used in workshops, which make it possible to locate the parts and trace the processing information. Take the RFID technology as an example. The pallets that carry the parts are usually equipped with RFID tags. When leaving the warehouse, the basic information of this batch of parts will be written to the RFID tag, including model, task start time, delivery time, task ID, etc. The MA of the warehouse notifies the MA of the AGV to pick up the parts. An RFID reader is installed on the AGV. After the MA corresponding to the AGV reads the part information through the “adaptation layer,” it confirms the receipt of the correct parts.

Different batches of parts will generate corresponding PA programs at the software level. PAs are the mapping of the parts in the software. They are automatically generated when the parts enter the workshop. Its structure is shown in Figure 2.

Figure 2.

Operation model for part agent.

The development of the PA program is also based on the JADE framework. The communication and decision modules in the PA are the same as the role in the MA. The data module is used to store part information, including task information, historical processing information, current processing state, etc. After processing is completed, this information will be synchronized into the enterprise database to perform task traceability.

Compared to the MA, the PA lacks the control module. The connections of actual control between PAs and the parts need to be executed through interaction with MAs. For example, the RFID reader in a processing cell is controlled by the corresponding MA. When a part pallet enters the buffer, it will be perceived by this MA. The MA will communicate with the PA. After that, the PA implements the part status update. Similarly, when the MA prepare to process this batch of parts, PA will send the necessary information to MA, including the NC code and preparations that need to be performed before processing. The MA will handle this information and perform part machining operations.

MAs and PAs realize distributed control of the workshop layer at the software level. MAs can control the “processing cells.” The negotiation between MAs and the negotiation between MAs and PAs can realize part transportation and part processing. The distributed control structure proposed in this paper can achieve dynamic handling of production events in cooperation between agents.

Before realizing dynamic task-dispatching, the control structure proposed in this paper first ensures the realization of self-organizing part-processing, so that MAMS can operate in the actual environment. This paper verifies this function in Section 3.

Value-based task dispatching mechanism

The control system can be achieved dynamic and real-time requirements for the production process through defining negotiation rules between agents and designing functional agents. It includes how to deal with various faults, disturbances, and the implementation of dynamic dispatching. This paper mainly discusses how to achieve dynamic dispatching. In order to make subsequent content easier to understand, the following definitions are advanced:

• Task: In this paper, a task refers to the one processing step of a batch of parts. Tasks can be done by different machine tools. Tasks are published by PAs, and implemented by MAs. This paper ignores the logistics here and only considers the processing task.

• Buffer: The workshop has a total buffer center. When a task has no equipment to undertake temporarily, the parts corresponding to the task will be sent to the buffer center. Simultaneously, each machine has its own corresponding buffer. When the machine is busy and the next task arrives, the task will enter the buffer first. The machine buffer capacity is “1,” which can only accommodate one task.

The contract network protocol (CNP) is a typical dynamic dispatching mechanism and is applied to various multi-agent manufacturing systems. For the control system proposed in this paper, if the CNP is used in the dynamic dispatching approach, PAs and MAs will form the following negotiation mechanism: (1) When a PA release a task, MAs that have the ability will respond. (2) These MAs will analyze the task and give offers according to the current machine status. (3) The PA will analyze the offers of the MAs and select one appropriate MA.

CNP is a hard real-time dynamic dispatching mechanism. However, the environmental knowledge mastered by a single agent is limited. Under this mechanism, the PA can only make decisions based on local information. When there are multiple tasks in a short period, the contract network mechanism follows the first-come-first-serve principle, and the decision made by the agent ignores the impact of other tasks.

This paper proposes a new task dispatching mechanism. MAs can obtain device information in real-time through the adapter layer. It is difficult to predict accurate processing time before task execution due to the pre-preparation work by humans and the disturbances in the processing, but the expedition of the MA on the remaining time will become more accurate with the processing, especially the final stage of the task. Based on this characteristic, this paper proposes a dynamic dispatching approach based on scrolling window mode, as shown in Figure 3.

Figure 3.

A dynamic dispatching approach based on scrolling window mode.

In this paper, time is discretized. A time interval t_w is set, and is called “window time.” New tasks triggered in one “time window” will be executed by unified dispatching. Therefore, t_w is usually small. Assuming that the regular processing time of each task is 5–25 min, t_w can be set to 0.5 min.

MAs corresponding to the idle machine tools and the machine tools that can complete tasks less than t_w will enter the pool to be dispatched. PAs that are waiting for processing will release tasks. The tasks will match the MAs in the dispatching pool. If the machine tool corresponding to the MA can process this task, the MA and the PA will confirm their connection value between them. The connection value between MA and PA can be affected by a variety of factors, and the total connection value can be expressed as:

$$V_{\mathrm{ij}}=\sum {\lambda }_k·V_{\mathrm{ij}}^k$$

(1)

In equation (1), $V_{\mathrm{ij}}$ represents the connection value between MA _i and PA _j . It can be composed of multiple sub-items, and each sub-item has its own weighting factor ${\lambda }_k$ . For example, the current workshop needs to guarantee the delivery date and make energy consumption as small as possible. The connection value between MAs and PAs can be expressed as:

$$V_{\mathrm{ij}}={\lambda }_1·V_{\mathrm{ij}}^1+{\lambda }_2·V_{\mathrm{ij}}^2$$

(2)

The value of the sub-item is negotiated by MAs and PAs. PAs will give information or reward of the sub-item. For example, $V_{\mathrm{ij}}^1$ represents the urgency of the delivery time. The PA will first give the reward of this sub-item. The more urgent the delivery time is, the higher the reward $V_{\mathrm{ij}}^1$ is, which represents that MAs execute it will get higher profit. For example, a three-axis machining center and a five-axis machining center can perform a task simultaneously, but they consume different time. The consumption time here is estimated by historical data, or the NC code execution time multiplies a safety factor. It should be noted that the expected time is usually in error with the real processing time. This paper doesn’t use it to generate scheduling sequences but used to make single-step decisions.

The MAs will discretize the expected consumption time and then obtain how many time windows are required to implement the task. The reward of the task will be averaged to these time windows. The connection value of sub-item between MAs and PAs will be decremented with the increase of time windows. It will be calculated as follows:

$$V_{\mathrm{ij}}^k=\sum _{t=0}^{T_{\mathrm{ij}}-1}{\delta }^t·\frac{V^k}{T_{\mathrm{ij}}}$$

(3)

In equation (3), $\delta$ is the discount factor. Its value is less than 1, and it is usually close to 1. $T_{\mathrm{ij}}$ represents the total number of time windows that MA _i needs to go through when performing tasks released by PA _j . Through equation (3), the timeliness of value is taken into account in the negotiation between MAs and PAs. Generally, the value of the positive sub-item is greater than zero. The faster it is completed, the higher the connection value of this sub-item. The value of the negative sub-item is less than zero. The shorter the time, the smaller the connection value of this sub-item.

Let us see an example to make the process clear. It assumes that the PA _j releases a task with a reward of 0.6 for the “urgency of delivery date” ( $V^1=0.6$ ). The MA _i , which has entered the dispatching pool, confirms that it can take this task. The MA _i is expected to be processed in three-time windows (usually not so few in practice). This paper uses a discount factor of $\delta =0.9$ , and $V_{\mathrm{ij}}^1=0.2+0.9\times 0.2+0.9^2\times 0.2=0.542$ .

$V_{\mathrm{ij}}^2$ represents the value of energy consumption. At this point, PA will give basic information about the task. MAs will expect the energy consumption of the task in t_w and the number of time windows that need to experience. The value of energy consumption needs to be normalized. Since the energy consumption should be as small as possible, it is necessary to reverse the value when determining its reward.

For example, the maximum energy consumption value of a “processing cell” in the t_w is 10, and the MA _i predicts energy consumption in the t_w is 6. Then, the reward of energy consumption obtained in the t_w is −0.6. The MA _i is expected to be processed in three-time windows. Therefore, the reward of energy consumption is −1.8 ( $V^2=-1.8$ ). This paper uses a discount factor of $\delta =0.9$ , and $V_{\mathrm{ij}}^2=-(0.6+0.9\times 0.6+{0.9}^2\times 0.6)=-1.626$ .

In the end, the total connection value between MA _j and PA _i is obtained from the equation (2). The weighting factors ( ${\lambda }_k$ ) need to be determined according to the order and importance of each sub-items. For the above example, it is assumed that the delivery time is more important, so this paper defines ${\lambda }_1=2$ , and ${\lambda }_2=0.1$ . Then, $V_{\mathrm{ij}}=1.247$ .

After confirming the connection value between MAs and PAs, the task dispatching of each round is transformed into a bipartite graph problem (as shown in Figure 4). This paper sets an arbitration agent to handle this problem. The connection information between MAs and PAs will be registered in the arbitration agent. It will find the best solution for this round of dispatching. The optimization goals are as follows:

$$\begin{array}{c}\max \sum _{i=0}^I\sum _{j=0}^J{V}_{\mathrm{ij}}·a_{\mathrm{ij}}\hfill \\ \begin{array}{cc}s.t.\sum _{i=0}^I{a}_{\mathrm{ij}}=1,\hfill & j=1,...,J\hfill \\ \sum _{j=0}^J{a}_{\mathrm{ij}}=1,\hfill & i=1,...,I\hfill \end{array}\hfill \end{array}$$

(4)

Figure 4.

Forming a dispatching plan.

Where:

$$a_{\mathrm{ij}}\{\begin{array}{cc}1\hfill & \mathrm{if}\mathrm{P}{\mathrm{A}}_j\mathrm{is}\mathrm{assigned}\mathrm{to}\mathrm{M}{\mathrm{A}}_i\hfill \\ \multicolumn{1}{c}{0}& \mathrm{if}\mathrm{P}{\mathrm{A}}_j\mathrm{is}\mathrm{not}\mathrm{assigned}\mathrm{to}\mathrm{M}{\mathrm{A}}_i\hfill \end{array}$$

Here, $a_{\mathrm{ij}}$ represents the connection between MA _i and PA _j . Finally, the result $\left[a_{\mathrm{ij}}\right]$ calculated by the arbitration agent is the current dispatching plan.

Implementation in the actual environment

A processing workshop in the laboratory environment is used to implement the system proposed in this paper. The workshop layout is shown in Figure 5. Four types of machine tools are used in it, and each type has two. Their functions are shown in Table 1. RFID technology is used in the system. The RFID reader will perceive the parts in the whole processing cycle, including leaving the material warehouse, transportation, processing, and entering the finished warehouse.

Figure 5.

A processing workshop case for verification.

Table 1.

The machine tools used for the experiment.

Name	Amount	Processing capacity	The number of buffer stations
CNC lathe	2	Turning	2
CNC milling machine	2	Milling/carving	2
CNC engraving machine	2	Carving	2
Machining center	2	Turning/milling/carving	2

This paper carries out the “processing cells” division in the workshop. Embedded technologies are used to integrate interfaces in each “processing cell” and deploy MA programs at the software level. The hall sensors are applied to monitor energy consumption for machines. MAs will record the historical energy consumption data of the corresponding “processing cells.” During the system’s operation, the energy consumption required in the next step is predicted based on the previous energy consumption value.

In this experiment, the number of each batch of parts is set to “1.” The processing feature of the part can be customized, and the customer can set size information in the feature, so the processing time of each part is not fixed. In the experimental system, the part orders reach the workshop layer through the network, and the corresponding PAs will be formed in the software layer.

By defining the negotiation mechanism between the agent during the processing, the system can achieve self-organizing production of tasks. One of the actual cases is shown in Figure 6. Parts, robots, AGVs, and machine tools achieve self-organized part-processing through the multi-agent architecture. The negotiation mechanism between agents is as follows:

• Step1. The PA notifies the MA of AGV transporting the part.

• Step2. After the part enters the station, the MA of AGV informs the MA of the manipulator to pick up this part. At the same time, the PA informs the MA of the CNC milling machine with the part information, including NC code and the preparation work.

• Step3. The MA of the CNC milling machine notifies staff to complete the preparation.

• Step4. After the preparation is confirmed, the MAs of the machine and manipulator will cooperate to complete the fitting of this part.

• Step5. The MA of the CNC milling machine starts processing.

Figure 6.

A negotiation case for processing in the experiment.

Simultaneously, the system can handle various processing events by defining the negotiation mechanisms. Here, this paper verifies the effectiveness of the dynamic dispatching mechanism proposed in this paper. The contract network mechanism and the dynamic dispatching mechanism studied here are used separately. Figure 7 shows the results of these two mechanisms in a same processing scene.

Figure 7.

Results of the two mechanisms in the same processing scene. The equipment represented by M1 to M6 in the figure are as follows: M1, a CNC lathe; M2, a CNC lathe; M3: a CNC milling machine; M4, a CNC milling machine; M5, a machining center; M6, a machining center. At “T₁,” the parts “P7” and “P8” are assigned to the workshop layer. “T₂” is the delivery time of the part “P1.” (a) A result of the contract network mechanism and (b) A result of the dynamic dispatching mechanism proposed in this paper.

In this processing scene, the workshop will get parts “P7” and “P8” at “T₁.” At this point, M2 (a CNC lathe) is about to complete the task. Under the contract network mechanism, the P7 and P8 will first consider their own interests and occupy the idle resources to start processing. After the task of P1 in M1 is completed, the machining of the following processing step for P1 cannot be implemented immediately due to resources have been occupied. Finally, the completion time of P1 exceeds the delivery date. Under the dynamic dispatching mechanism proposed in this paper, parts P7, P8, and P1 will be considered together and make resource value matching to get a reasonable dispatching plan. The results show that the latter mechanism is better than the previous one when more concurrent tasks arise.

Then, this paper extends the experiment time, and compares the performances in terms of delivery guarantee and energy-saving. The sum of the overdue time and the total energy consumption are counted and normalized. The final comparison result is shown in Figure 8.

Figure 8.

The performance of the delivery period guarantee and the performance of energy-saving.

The results show that the task dispatching mechanism proposed in this paper can effectively reduce the overdue risk of tasks. Simultaneously, this mechanism is also better than the CNP dispatching mechanism in terms of energy consumption. The results of the above two tests also confirm the effectiveness of the control system presented in this paper.

Conclusion

This paper introduces a dynamic dispatching control system for processing workshop to improve the ability of dealing with the dynamic task and brings the following main innovations:

• A connection/mapping approach between agents and physical entities is proposed. The structures of part agent (PA) and machine agent (MA) oriented to the processing are constructed.

• A value-based task dispatching method is studied. An arbitration agent is designed to implement the dispatching of each task through value matching.

• The timeliness of value is considered in the new matching calculation method. A task dispatching mechanism based on the scrolling time window is investigated.

An actual experiment is carried out to verify the advantages of this dynamic control system. However, it should be noted that this paper ignores the logistics system and the staff management system. The dynamic dispatching in this paper only occurs between processing equipment and parts. The authors will integrate these subsystems into a unified dynamic control framework in future works.