Abstract
With the continuous innovation of technology, automated guided vehicles are playing an increasingly important role on manufacturing systems. Both the scheduling of operations on machines as well as the scheduling of automated guided vehicles are essential factors contributing to the efficiency of the overall manufacturing systems. In this article, a hormone regulation–based approach for on-line scheduling of machines and automated guided vehicles within a distributed system is proposed. In a real-time environment, the proposed approach assigns emergent tasks and generates feasible schedules implementing a task allocation approach based on hormonal regulation mechanism. This approach is tested on two scheduling problems in literatures. The results from the evaluation show that the proposed approach improves the scheduling quality compared with state-of-the-art on-line and off-line approaches.
Keywords
Introduction
With the globalization of markets, the continuous innovation of technology, and the short life of products, the traditional manufacturing industry is facing tremendous challenges. This situation has promoted the development of manufacturing systems toward the direction of discretization, intelligentization, and autonomy. A manufacturing system typically consists of several production operation units, an automated storage/retrieval system (AS/RS) for raw material and finished products, and an automated material handling system for material transfer between units. In order to assure the high agility and flexibility, manufacturing systems require an intelligent and automated material handling system. 1 An automated guided vehicle system (AGVS) is one of the most efficient material handling systems due to high flexibility, safety, and small space utilization of automated guided vehicles (AGVs). In recent years, the research on AGV technology has strengthened their flexibility, intelligence, and autonomy.2–5
An AGVS should ensure normal and efficient material transfer, meanwhile, reducing the transportation cost, work-in-process (WIP) inventories, and total operation cost.5,6 To achieve these goals, many approaches and solutions are used in manufacturing systems. In these systems, scheduling problems are optimization processes of resource and task allocation. Both the scheduling of operations on machines as well as the scheduling of AGVs are essential factors contributing to the efficiency of the manufacturing systems. 7 AGV scheduling problems depend on the machine scheduling for task execution, specifically on the starting and ending time assigned by the operations’ scheduling. The objective of AGV scheduling is to optimize the material handling, executed by AGVs, among the machines and the AS/RS. AGV scheduling can be achieved by both off-line and on-line approaches. Off-line scheduling is aimed to manage all activities for the entire scheduling period, while on-line scheduling carries out the allocation and dispatching in real-time, during the execution of the plan.
Off-line AGV scheduling is a non-deterministic polynomial-time (NP)-hard problem that must take into account machine scheduling and AGV scheduling simultaneously. Numerous research results show that NP-hard scheduling problems can be solved by heuristic approaches (HAs)8–10 as well as off-line AGV scheduling problem. 11 In order to optimize AGV scheduling, Confessore et al. 12 propose a network-based HA for the vehicle dispatching. Bilge and Ulusoy 13 exploit the interactions between machine scheduling and AGV scheduling in a flexible manufacturing system (FMS) by addressing them at the same time. They define a nonlinear mixed integer-programming model to formulate the problem. They apply a heuristic-based iterative approach to generate machine schedules that are used to define time windows for AGVs; afterward a feasible solution for AGV schedules is searched. Besides, several more researches14–17 solve scheduling problems in the literature 13 using meta-heuristic techniques.
The goal of off-line scheduling is to generate an optimized schedule over a period of time, during which the schedule is not expected to require any change. Nevertheless, during production, disturbances are likely to cause the unavailability of the original schedule. Therefore, on-line scheduling or dispatching approaches can substantially improve the performance of AGVSs.18–20 There are two on-line approaches for AGVSs: centralized and decentralized (or distributed). A centralized approach has one controller for all AGVs in the system.21–23 Whereas, in a distributed approach there are several subsystems controlled by different independent controllers based on their local knowledge. Multi-agent systems (MASs) and holonic manufacturing system (HMS), with distributed characteristic, can be used to implement on-line AGV routing and dispatching. Lau and Woo 24 describe an agent-based dynamic routing strategy for generic automated material handling systems. Morten and Olivier 25 present a holonic-based approach for AGVS developed for the automated paint-shop. Babiceanu et al. 26 introduce a holonic control approach for scheduling material handling devices in the cellular manufacturing environment. There are also some researchers applying distributed approaches to deal with classic objectives existing for AGV routing and dispatching, such as deadlock and conflict. Singh and Tiwari 27 propose a MAS approach for the operational control on AGV conflict-free routing. Breton et al. 28 introduce a routing approach based on MAS for deadlock avoidance and conflict free. Wallace 29 presents an agent-based AGV controller to maintain deadlock-free flow of AGVs within a free space world model.
Although there have been many researches on on-line AGV routing and dispatching by applying distributed approaches, very few carry out the study on on-line scheduling for machines and AGVs simultaneously. To the best of the authors’ knowledge, only one work 30 proposes a MAS approach for simultaneous scheduling of machines and AGVs. The approach works in an on-line environment and generates feasible schedules using negotiation and bidding mechanisms between distributed agents. MAS negotiation and bidding mechanisms have the advantage of simplicity. Nevertheless, since they are a type of explicit coordination mechanism, when the complexity of the manufacturing environment increases (the number of machines and AGVs is large, together with a high number of new tasks appearing at run time), the number of messages passing is also large, and time consumption in some situations is not appropriate for given feasible solutions in due date.
Intelligent models inspired by human body information processing mechanism have become a new research focus in the Artificial Intelligence research field. For example, endocrine system is a core part of human body information processing mechanism. Its unique processing method, based on the principle of hormone diffusion and reaction, has given researchers a lot of inspiration.31,32 The hormone-regulation mechanism is one kind of implicit coordination approach that leads to quick coordination between the different components of the entire system. Compared with MAS negotiation and bidding mechanisms, this mechanism implements simpler coordination and requires less message passing among the system components. In this work, an approach for distributed and on-line scheduling of AGVs and machines that uses the hormone-regulation mechanism is proposed.
The remainder of the article is organized as follows. Section “On-line scheduling model” presents an on-line scheduling model inspired by the principle of hormone diffusion and reaction to agilely deal with emergent tasks. Section “Allocation mechanism based on hormone-regulation mechanism” introduces a transportation task allocation mechanism based on hormone-regulation mechanism. Section “Distributed cooperation mechanism for on-line scheduling” illustrates a hormone regulation–based cooperation mechanism between machines and AGVs for on-line scheduling. Section “Experimental study” presents an evaluation study in which the proposed approach is compared against other state-of-the-art approaches. Section “Conclusion” describes the conclusions and future works.
On-line scheduling model
On-line scheduling approach
An on-line problem for simultaneous scheduling of machines and AGVs can be modeled as an on-line job shop scheduling problem in which the materials are transported by AGVs. The process route plan generated from an on-line job shop scheduling determines the key time nodes of an on-line AGV scheduling. In recent years, on-line job shop scheduling problems for manufacturing systems have been studied by many researchers.33–35 In our previous work, 36 a shop floor re-scheduling approach was presented. This technique can deal with on-line scheduling for machines and operations proposed in this article. Thus, the focus of this article is the scheduling of transportation tasks between AGVs and machines, shown in Figure 1.

Scheduling of transportation tasks between AGVs and machines.
This scheduling approach is collaborative and distributed and can be described in an abstract way as follows. The collaborative scheduling process is divided into three stages. In stage I, the scheduling process is triggered when an operation is started on a machine; the machine must deal with the subsequent transportation throughout the shop floor before the operation is finished. The machine needs to find a suitable AGV that can execute the transportation. To this end, it sends the demand requirement to the AGVs in the system. In stage II, when a given AGV realizes the demands from machines (sensing the environment or receiving a message), it calculates its transportation efficiency for every demand and selects the demand with the highest transportation efficiency according to its own criteria, and then communicates its choice back to the relevant machine. In stage III, the machine receives/senses the feedback information, then evaluates the different alternatives and selects the most efficient one. Finally, the machine allocates the transportation task to the selected AGV. In the next subsections, an on-line scheduling model inspired by the principle of hormone diffusion and reaction is detailed.
Information processing mechanism in endocrine system
The endocrine system of the human body is a complex physiology network formed by a variety of endocrine glands. These endocrine glands can influence each other by secreting, transmitting, and responding to different hormones. 37 Endocrine glands with different functions have their own proprietary hormone receptors, which react to hormones in the humoral environment. However, endocrine glands diffuse hormones into the humoral environment with no constraints; the hormone receptors only respond to the specific hormones. The principle of hormone diffusion and reaction in the endocrine system is illustrated in Figure 2. The solid lines with an arrow represent the hormone secretion between endocrine glands or hormone autocrine in independent endocrine glands, whereas the dotted line with an arrow indicates the hormone feedback of endocrine glands from the participant to the initiator. For example, affected by a stimulus, endocrine gland “A” secretes hormone “A” and releases it into the humoral environment. Sensing the stimulus from hormone “A,” endocrine gland “B” secretes hormone “AB” and autocrine hormone “B” simultaneously. Then, endocrine gland “B” feeds back hormones “AB” and “B” to endocrine gland “A.”

Principle of hormone diffusion and reaction in endocrine system.
The principle of hormone diffusion and reaction in endocrine system is provided with the following characteristics:
Information transmission functions. There is no centralized control part but only one humoral environment for hormone transmission. Moreover, the behaviors of hormone secretion in endocrine glands are only related to the hormone information in the humoral environment. Thus, it is not required a point-to-point communication.
Specificity. A hormone is a kind of chemical component released by endocrine glands. The combination of the chemical component and hormone receptors is specific, that is, a hormone receptor only reacts to a specific chemical component (hormone). The specificity of hormone is determined by the genetic information.
Synergy and antagonism between hormones. In the endocrine system, endocrine glands release hormones through stimulus in the humoral environment, where synergy and antagonism between hormones emerge from the different effects of hormones on the same gland. These kinds of interactions have the effect to stabilize the system environment.
By employing the features of hormone transmission and interaction, the endocrine system of the human body can achieve rapid adjustment and maintain high adaptability.
On-line scheduling model inspired by the principle of hormone diffusion and reaction
To solve the on-line scheduling problem, an on-line scheduling model inspired by the principle of hormone diffusion and reaction is proposed (Figure 3). Acting as endocrine glands, the machines play the role of initiators providing optimization and coordination services to AGVs and AGVs play the role of participants providing response and optimization services. In our model, it is assumed that machines and AGVs have the capacity to handle calculation, comparison, and decision-making operations, since the computation power and the optimization capability are embedded in them. In the proposed model, the tasks act as the stimulus in the system; the shop floor environment acts as the humoral environment. In this way, the diffusion and reaction information are similar to the secretion and feedback of hormones.

On-line scheduling model inspired by the principle of hormone diffusion and reaction.
As illustrated in Figure 3, when operation task “a” is executed on machine “A,” the subsequent need for transportation task “a” emerges. Stimulated by that need, machine “A” evaluates the degree of demand for AGVs, then it packages the demand and transportation task information into a hormone, and finally diffuses the hormone into the shop floor environment. When the AGVs sense the hormone information, they are stimulated by the demand and the transportation task simultaneously. In response to the stimulus, each AGV evaluates how the demand influences itself and its performance to complete the transportation task; then it packages the evaluated value into relevant hormone information and releases this information to machine “A.” According to the feedback hormone information, machine “A” allocates the transportation task to the relevant AGV. In the process of on-line scheduling, the information evaluation of machines and AGVs is similar to the process of responding to stimulus and secreting hormone, which is a kind of self-adjustment and adaptation mechanism to maintain the balance of the system. By utilizing this model, the transportation tasks can be rapidly allocated without a supervisor. In the next section, the allocation process of transportation tasks is described in detail.
Allocation mechanism based on hormone-regulation mechanism
Hormone-regulation mechanism background
Farhy
38
defined the basic law for modeling the secretion of hormones by endocrine glands. In this model, the hormone regulation complies with Hill function which is composed of the rising function s and decreasing function
where C is variable of hormone concentration, T is a threshold of hormone concentration, and T > 0; n is the Hill coefficient and n > 1.
If a hormone x is controlled by a set of hormones
where Sx0 represents the initial secretion speed of hormone x, Ci is the concentration of hormone i, and coefficient ai is a positive constant associated with hormone i.
For example, the glucagon secreted by pancreatic islet α-cells has the function of raising blood sugar concentration; on the contrary, the insulin secreted by pancreatic islet β-cells has the function of reducing blood sugar concentration. When the blood sugar concentration is low, islet α-cells promote the glucagon secretion, and hormone regulation complies with up-Hill function for rapidly improving the blood sugar concentration; on the contrary, when the blood sugar concentration is high, islet β-cells promote the insulin secretion and hormone regulation complies with down-Hill function for agilely reducing the blood sugar concentration. Because hormone regulation has the characteristics of monotonicity and non-negativity, the affection of one hormone to another hormone can only be the stimulation or inhabitation. In biology, insulin and glucagon exist at the same time and have the function of mutual antagonism to regulate blood sugar concentration.
If
There are not only antagonistic effects but also synergistic effects between different hormones. For example, glucagon and epinephrine have the synergistic function of elevating blood sugar concentration. If
The next sections describe the design of AGV allocation approach inspired by hormone-regulation mechanism.
Time parameters in scheduling
In the on-line scheduling problem of machines and AGVs, a task has two main stages: the task being operated by a machine (TO) and the task being transported by an AGV (TT), shown in Figure 4. The notations are given below:
st is the start time of an operation task.
ct is the completion time of an operation task.
ot is the duration of an operation task.
lt is the loading time of a transportation task.
ult is the unloading time of a transportation task.
tt is the duration of a transportation task.
M is the set of machines.
K is the set of AGVs.
J is the set of jobs.
L is the set of operations of a job.
Z is the makespan.

Two main stages of a task.
The objective of on-line scheduling problem of machines and AGVs is to minimize Z. According to the interrelation between time parameters, constraints are extracted and formulized as follows
Formula (5) presents a time constraint between adjacent operations of a job and Formula (6) presents time constraints between AGV and adjacent operations of a job. In the on-line scheduling problem, only ot and tt are considered as given parameters and other parameters are variables to be determined. According to the constraints (5) and (6), interrelation between parameters is determined. Taking the operation l of job j as an example, calculation processes of the above parameters are illustrated as follows. When an operation task (TOjl) is executed on a machine, stjl and otjl are determined. And then ctjl can be obtained in accordance with equation (7)
According to the operation state of AGV (i.e. transporting a task; or being idle and waiting for a task to transport), the loading time of AGV is described by the following equations
where t is the current time;
where DLjl is the destination location of the operation l of job j. The unloading time of AGV k is computed according to equation (10)
Finally, the makespan is selected in accordance with equation (11)
where Lj is the number of the last operation of job j.
Allocation mechanism
In the case of normal operation, hormone concentrations of the multi-AGV system are in the equilibrium state. Once a transportation task emerges in the manufacturing system, the balance will be disturbed. It is assumed that the secretion of hormone is triggered in the manufacturing environment, stimulated by the transportation task. The level of hormone concentration is related to the effect on the system balance caused by the initiation of transportation task. If the effect on the system balance is small, the level of hormone concentration released by the system is low; otherwise, the level of hormone concentration is high. Taking this principle as a basis, a transportation task allocation mechanism is proposed. In the following subsections, the operation l of job j is taken as an example to describe the allocation mechanism in detail.
Hormone secretion of machines
When an operation task
where, c is a positive constant.
An example of hormone secretion curve of a machine is shown in Figure 5 according to equation (12). When

Hormone secretion curve of machine stimulated by transportation task.
Hormone secretion of AGVs
When an AGV with transport capability senses the stimulation of
where,
An example of hormone secretion speed curve of AGV stimulated by

Hormone secretion speed curve of AGV stimulated by
Besides
When
An example of hormone secretion speed curve of AGV stimulated by ult (Figure 7) is plotted according to equation (14). When

Hormone secretion speed curve of AGV stimulated by ult.
As mentioned above, the hormone secretion of AGV
k
is simultaneously influenced by two independent hormones (
Decision making
When a machine senses the set
A machine selects the right (i.e. optimal) AGV according to the secretion speed of hormone, in which a machine’s need for an AGV and the performance of an AGV to perform a transportation task are taken into account simultaneously. In the selection of a transportation task, a weighted loading time is proposed as follow
The weighted loading time reflects transportation efficiency between different tasks considering the influence of hormone
Therefore, stimulated by multiple tasks to transport, an AGV will choose the task with the highest transportation efficiency to secrete hormone. Taking transportation tasks as the communication media of mutual selection between AGVs and machines, the arrival of tasks on time and the transportation efficiency of the AGV system can be ensured simultaneously. In the processes of optimization, AGV can select the next task to transport ahead of schedule, so the coherence of processing between previous and the next operations can be guaranteed.
Distributed cooperation mechanism for on-line scheduling
In this section, a distributed cooperation mechanism for on-line scheduling is elaborated. This mechanism is mainly to solve mutual selection and optimization between machines and AGVs. The goal of the mechanism is that the distributed system can achieve an optimized production status and maintain a high production level. As shown in Figure 8, machines and AGVs are considered autonomous and cooperative cells 39 that follow the steps given below to conduct their inherent operations:
Step 1. When an operation task
Step 2. Machine cell m releases hormone information
Step 3. When AGV cell k senses the set
Step 4. AGV cell k extracts the required information from the set
Step 5. Stimulated by
Step 6. AGV cell k feeds back hormone information
Step 7. When machine cell m senses the set
Step 8. A loop between Steps (1)–(7) is repeated until all the transportation tasks are allocated, then the makespan is selected according to equation (11).

Distributed cooperation mechanism between machines and AGVs.
Using this allocation mechanism, the machines can allocate the transportation tasks according to secretion speed of hormone and the AGVs can select the transportation tasks with the premise of transportation efficiency. Finally, the objective of distributed and on-line scheduling of machines and AGVs can be achieved.
Experimental study
In order to simulate the proposed hormone regulation–based approach (HRA) for on-line scheduling, a series of instances were selected from the benchmarks of Bilge and Ulusoy. 13 These instances were used in both off-line and on-line scheduling by many researchers.11,13–17,30 In this article, the performance comparison between our proposal against off-line 16 and on-line 30 approaches of the specialized literature was carried out. In the experiments, the performance measure selected was makespan. The performance of the off-line approach is considered as the lower bound of on-line approaches. The constraints and assumptions of the experiments were in the following:
One operation cannot be started before its preceding operations are finished and each machine is provided with sufficient input/output buffer space.
The processing parameters of the operation tasks are deterministic such as operation time and setup time.
All AGVs work with the same speed and transport a single task at a time.
Preemption is not allowed in transportation, that is, once task transportation is started, it must be finished without interruption.
Disturbances such as conflict, deadlock, delay, and resource malfunction, are not considered.
Let’s assume an experimental scenario in which two AGVs travel between four machines and one load/unload (L/U) station. Figure 9 shows the layout schemes of the instances, Table 1 lists the transportation time matrix of AGV, and Table 2 gives the data sets of jobs. In the design of this test problem, the ratio between transportation time and operation time is taken into consideration as a significant characteristic. The ratio can be described as
Transportation time matrix of AGV. 13
L/U: load/unload.
Data sets of jobs. 13

Layout schemes. 13
All procedures were implemented in JAVA under JDK1.7.0 and the experimental tests were carried out on a 2.0 GHz Intel Core Duo computer with 2 GB RAM under Windows 7. Combining the 10 job sets and the 4 layout schemes from Tables 1 and 2, 82 test instances were created. These instances were tested using an off-line HA,
16
a MAS approach,
30
and the proposed HRA. The off-line HA determined a lower bound of the makespan, so the aim of experimental study is to compare our proposal (HRA) against MAS and determine the deviation of both MAS and HRA with respect to HA (Dev1, Dev2). Thus, we also show the deviation of HRA with respect to MAS (Dev3). The results for the case
Comparison results in the case of
MTD: mean total deviation. The bold values represent optimal makespan of the three approaches.
Comparison results in the case of
MTD: mean total deviation. In two tables, the bold values represent optimal makespan of the three approaches.
The results show that our approach HRA outperforms MAS in most instances of Tables 3 and 4. It can be observed that HA obtained the best results in almost all instances due to the fact that it is an off-line heuristic and all information of the scheduling problem is known in advance.
From the results in Table 3, the mean total deviation (MTD) of HRA to HA is 33.83%, which means that HRA does not go beyond HA in makespan. This result is in accordance with our expectations as HRA is not an off-line algorithm but on-line and distributed approach. Compared with an on-line approach (MAS), 55% of the results from HRA are better than MAS and the MTD of HRA to MAS is −1.65%. Thus, HRA shows better performance in makespan. From the results in Table 4, the performance of HRA is better. Compared with on-line approach, the performance of HRA is better than MAS in 69% test instances and the MTD of HRA to MAS is −5.28%. Because the MTD of HRA to HA is 5.23%, it shows that the optimization capability of our approach is closed to off-line method. Similar results are obtained for EX110, EX220, EX630, and so on. In other instance of EX820, is shows the same performance with HA. Similar results are obtained for EX 620, EX830, and EX 840. By comparing the results, it is clear that HRA exhibits better performance in the case of relatively low
The Gantt chart of instance EX23 is shown in Figure 10. The numbers in the Gantt chat represent the serial number of tasks. For example, the number “21” in M2 represents the first operation task of job2 and the number “21” in AGV1 indicates the first transportation of job2. As can be seen from the Gantt chart, at the time 48, a new round of mutual selection between machines and AGVs starts; two AGVs have the transportation capacity to accept more transportation tasks; and operations “61,”“52,” and “32” demand AGVs for transportation. In AGV selection stage, affected by the hormone H and loading time, the two AGVs select the operation “61” with the lowest weighted loading time; then in machine selection stage, AGV2 is selected as the winner. Because compared with AGV1, AGV2 can finish the task with lower deviation and the hormone secretion speed is lower than AGV1. The mean value of transportation task of AGV is 11.2 which can be computed using the data in Table 1. As shown in the Gantt chart, 21 transportation tasks are implemented by two AGVs and the distribution of time for AGVs is shown in Figure 11. Of the transportation tasks, 90.48% have time consumption less than 15. As shown in Figure 12, the utilization rate of two AGVs remains at a high and balanced level.

Gantt chart of EX23.

Distribution of time for AGVs.

Utilization rates of AGVs and machines.
Conclusion
In this article, HRA for distributed and on-line scheduling of machines and AGVs is proposed. The principle of hormone diffusion and reaction in endocrine system provides a new information processing method for the on-line scheduling model to agilely cope with new arrival tasks. According to hormone-regulation mechanism, the deviations between completion time and planned time in on-line allocation processes are used to optimize makespan. Thus, in the mutual selection between machines and AGVs, machines can allocate transportation tasks according to the minimum secretion speed of hormone and AGVs can select transportation tasks with the highest transportation efficiency. As the tasks are well allocated, good performances of the system can be obtained.
The proposed approach was tested and compared with classical instances in the literature. The performance of HRA was close to off-line approaches. Especially in the low ratio, the performance in some instances was the same with off-line approaches. Compared with the on-line approach, performance of HRA was better in more than half of the test instances, and the rest was the same or close to. The results show that HRA reflects a promising performance in terms of distributed and on-line scheduling of machines and AGVs. The application of hormone regulation in this area shows great potential. Especially, its reaction and diffusion mechanisms can quickly respond and satisfy scheduling requirements in the system. And it expands the view of research on dynamic scheduling problems. Future research can be directed toward the decision-making modules in different stages to improve the performances of the scheduling approach. Besides, the proposed approach can be enhanced taking into account conflict, deadlock, delay, recourse malfunction, rush orders, and so on.
Footnotes
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was sponsored by the National Natural Science Foundation of China (NSFC) under Grant Nos 51175262 and 51575264 and the Jiangsu Province Science Foundation for Excellent Youths under Grant No. BK2012032. This research was also sponsored by the CASES project which was supported by a Marie Curie International Research Staff Exchange Scheme Fellowship within the 7th European Community Framework Programme under grant agreement no. 294931.
