Towards a hybrid software architecture and multi-agent approach for autonomous robot software

Abstract

To support robust plan execution of autonomous robots in dynamic environments, autonomous robot software should include adaptive and reactive capabilities to cope with the dynamics and uncertainties of the evolving states of real-world environments. However, conventional software architectures such as sense-model-plan-act and behaviour-based paradigms are inadequate for meeting the requirements. A lack of sensing during acting in the sense-model-plan-act paradigm makes the software slow to react to run-time contingencies, whereas the behaviour-based architectures typically fall short in planning of long-range steps and making optimized plan adaptations. This article proposes a hybrid software architecture that maintains both adaptivity and reactivity of robot behaviours in dynamic environments. To implement this architecture, we further present the multi-agent development framework known as AutoRobot, which views the robot software as a multi-agent system in which diverse agent roles collaborate to achieve software functionalities. To demonstrate the applicability and validity of our concrete framework and software architecture, we conduct an experiment to implement a typical case, for example, a robot that autonomously picks up and drops off dishes for remote guests, which requires the robot to plan and navigate in a highly dynamic environment and can adapt its behaviours to unexpected situations.

Introduction

An autonomous robot is a goal-oriented system with various tasks that is expected to perform tasks independently. The environment in which autonomous robots reside is generally open and dynamic, and the evolution of states is uncontrollable and unpredictable by human efforts. Moreover, tasks for robots in real-world environments are usually strict and have real-time constraints that require the robot to act efficiently and respond quickly. Therefore, the robot is expected to execute the plans adaptively, robustly and efficiently. To support autonomous robot operation in dynamic environments, autonomous robot software has the following expectations:

• Adaptive: Due to inevitable changes in the tasks and environment, the software should continuously adapt the planned behaviours to various changes that occur in the environment such that behaviours are valid and effective in the dynamic environment.

• Reactive: The real-time constraints on tasks require the robot to rapidly respond to external events, and thus the software is expected to engage in reactive planning for emergent situations for which time-consuming adaptation process fails to meet with the time constraints.

The expectations of both adaptivity and reactivity for autonomous robot software are much more challenging than for software agents. Traditionally, software agents are acting based on the expectation that the software system will behave exactly as specified. However, this expectation is no longer valid if the physical world in which the robots are situated in is inside the loop, as robot beliefs usually not represent true facts about the world and robot software may not behave as expected. In this case, studies on architectural structure of autonomous robot software become important to tackle this issue. Moreover, the new challenges for autonomous robot software pose the following requirements for software engineering practices, such as concurrent and distributed in software structures, ¹ robust in overall performance of robot software, suitable for autonomous software abstraction and modular in software structure. ²

The existing studies on robot software architectures and implementation approaches have progressed and shown significant advances in modelling and development of a complex robot software system. However, these efforts appear inadequate to meet new requirements presented by autonomous robot software.

In terms of previous works on robot software architectures, the sense-model-plan-act (SMPA) architecture ³ first introduces autonomy in autonomous robot software by incorporating modelling and task planning capabilities. However, because the approach is established under assumptions related to static or predicable connections between sensing and acting, it appears rather dangerous for autonomous robots to act in a dynamic world because no sensing is involved while the robot acts to reach planned goals. Another behaviour-based architecture facilitates continual environmental awareness and strong reactivity, although it proves difficult to plan robot behaviours in an optimal manner to achieve long-range and complex goals.

For software implementation approach, both object-oriented and component-based approaches apply software engineering principles that demonstrate strong modularity, reusability and concurrency in robot software products. However, it has become apparent that these approaches are limited in robot programme abstractions because the programme models such as static objects usually feature in static structures and passive interaction behaviours. However, in agent-oriented techniques, an agent entity is commonly conceived of autonomous structures and proactive behaviours, which is able to choose appropriate actions according to its capabilities, skills and also the situations. Particularly, multi-agent systems have emerged as a powerful technique for decomposing, abstracting and organizing a complex robot software. The agent-oriented approach represents a means for introducing autonomy, distribution, collaboration and other advanced features in robotic programming. ⁴

This article attempts to maintain both adaptivity and reactivity in autonomous robot software by presenting a hybrid robot software architecture and adopting a multi-agent implementation approach. The proposed architecture integrates advantages of the SMPA paradigm and behaviour-based architectures to achieve both adaptive and reactive capabilities. Moreover, we present a multi-agent development framework known as AutoRobot (https://www.trustie.net/projects/969) that offers basic agent abstractions to implement robotic software applications.

The rest of this article is organized as follows. ‘Related work’ section discusses several related works. ‘Motivation example for the autonomous robot’ section illustrates a motivation example for autonomous robots. ‘Abstract model of hybrid architecture’ section presents the proposed architecture. ‘Formal framework of hybrid architecture’ section discusses how to implement the architecture based on the AutoRobot multi-agent system. In ‘AutoRobot: A multi-agent implementation framework’ section, a conceptual framework is established to compare AutoRobot framework with other robotic software frameworks. In ‘A conceptual framework for comparison’ section, an experiment is conducted to validate our concrete framework and software architecture. ‘Experiment’ section concludes this article with some discussions on some future directions.

Related work

Together with implementation approaches, robot software architecture is a fundamental research topic for modelling and programming of autonomous robot software with ever-increasing complexity. ⁵ A large body of literature is available on the main paradigms of the SMPA approach and behaviour-based architectures as well as various software implementation approaches.

Robot software architecture

SMPA architecture. Ruehl et al. ⁶ extended the classic sense-plan-act cycle with a discretization step before planning and a verification step for generating steps in support of planning executable manipulation actions. The additional steps work together to map the real-world environments to an approximation for the symbolic planner and to check the feasibility of the generated plans to ensure high feasibility and reliability of the robot plan execution.

Wotawa ⁷ presented a robot software architecture that extends the sense-plan-act paradigm with model-based reasoning ability to handle the cases of internal faults and external events. A model-based reasoner is used to identify the root causes of the detected deviations between the observed and expected behaviours and has access to both internal states of the controller and the sensor information.

Mansouri and Pecora ⁸ extended qualitative spatial knowledge in a sense-plan-act loop to specify sophisticated robot behaviours in purely metric terms, including matching of perceived context with world knowledges, instantiating plans into the metric space of the real world and reacting to possible contingencies.

Behaviour-based architecture. The subsumption architecture ⁹ is among the most influential work in behaviour-based robotic studies. Layers of a control system are built in correspondent to each levels of competence of robot behaviours, and a new layer can be simply added to the next higher level of overall competence.

The architecture proposed by Eckhardt ¹⁰ merges different capabilities such as automatic navigation and self-consciousness into a consistent framework, thus enabling the robot to navigate in a complex natural outdoor environment by relying on only a single on-board camera as its sensory input.

An integrated behaviour-based control architecture ¹¹ (iB2C) addresses issues of coordination of the competing parallel behaviours, identification of error sources in emergent system behaviours and structuring of the architecture development process. The iB2C integrates a wide variety of concepts, such as the behaviour fusion mechanism, behaviour interaction patterns and arbitrary behaviour transfer functions, thus allowing development of large-scale behaviour networks.

The ALLIANCE-robot operating system (ROS) architecture ¹² is a fully distributed, fault tolerant and behaviour-based framework that delivers abstract classes corresponding to ALLIANCE ¹³ model elements based on ROS. ¹⁴ The framework encapsulates the ROS mechanism and python libraries to construct basic function units to facilitate robot behaviour description and fault-tolerant multi-robot task allocation.

Implementation approach

Object-oriented approach. Kramer and Scheutz ¹⁵ presented advanced robotics interface for applications to facilitate a high-level and structured environment for writing robot controllers. The interfaces provide certain powerful and reusable modules for programming common robot tasks such as mapping and localization.

Yet another robot platform (YARP) ¹⁶ is an open-source project that supports infrastructure-level software development by facilitating code reuse and modularity. YARP proposes a key concept of the port class as an active object to manage multiple connections for a given unit of data either as input or output.

The object-oriented robotics Application Programming Interface (API) ¹⁷ is used to develop software for industrial robotic applications, which offers an abstract, extensible domain model and enables common functionality that can be easily used by application developers. By delivering basic concepts for world modelling, devices and action specifications, this approach promotes reuse of logic common to industrial robotic applications.

The ArmarX ¹⁸ is a robot software framework that eases the realization of higher level capabilities for complex robotic systems. ArmarX adopts well-defined interface definitions and flexible communication mechanisms, in contrast to traditional message protocols, to support the disclosure of states of a robotic system.

Component-based approach. The player/stage ¹⁹ system is a robot device interface platform that applies to programming and simulation of distributed multi-robot and sensor network systems. The player component serves as the device and server interface for users to access the robot hardware devices over the network, and the stage component refers specifically to a graphical, two-dimensional simulator for modelling robot devices in a self-configuration environment.

Webots ²⁰ has emerged as a mobile robotics simulation software tool that supplies programmers with a three-dimensional prototyping environment for modelling, programming and simulating mobile robots. A library of robot sensor and actuator components allows users to plug in a self-defined robot model and tune it individually.

Orca ²¹ is a component-based software framework that facilitates definitions and developments of building blocks and connects them to achieve complex robotic systems ranging from a single robot to distributed sensor networks. The goal of Orca is to introduce unified usage patterns to promote software reuse in robotics.

The mobile and autonomous robotics integration environment ²² (MARIE) approach supports rapid prototyping and integration of existing software components for robotic software systems. MARIE collects a wide range of black box and white box components to allow addition of new functionalities to the robot software.

The ROS ¹⁴ is a middleware-based component platform that manages modules using inter-process communications. Software built around ROS can use fine-grain licensing of their various components, and individual modules can incorporate software from the robotic community. The actionlib stack from ROS is a significant and convenient approach to achieving real-time information sharing between server and client nodes. The actionlib offers a standardized interface with an ActionServer that tracks the long-running goals pursued by one or several ActionClients.

Virtual robot experimentation platform ²³ (V-REP) is a versatile and scalable simulation framework that allows for direct incorporation of various control techniques. This framework adopts a component-based ideal of offering built-in and ready-to-use robot functionalities as well as various programming approaches.

Agent-based approach. The virtual operator multi-agent system (VOMAS) ²⁴ was developed to support spontaneous generation of a task without the need to replan. The system functionality is achieved by incorporation of three roles for agents: the VO for high-level planning, the robot agent for managing low-level physical effectors and the user agent for users to interact with the robot.

Vallejo et al. ²⁵ proposed a multi-agent architecture to support swarms of robots to handle surveillance tasks. The system is structured into three layers, namely, a perceptual layer for perception inputs, the conceptual layer for analysing the information from the sensors and the decision-making layer for high-level reasoning. The reasoning kernel deploys a set of reasoning agents for each mobile robot to share knowledge related to a certain problem. The architecture also establishes a communication model among agent entities, which improves scalability and flexibility of information exchanges.

The COROS developed by Koubâa et al. ²⁶ is a multi-agent software architecture for cooperative robots in which each of the agents represents a robot machine or a monitoring/control workstation. The COROS system consists of five conceptual subsystems, that is, communication, ROS interaction layer, robot control, application and knowledge bases. Each of the systems offers an abstraction interface to support design and implementation for developers. The COROS framework promotes software reuse and modularity by presenting generic classes and packages for low-level implementation details.

Motivation example for the autonomous robot

To better illustrate our ideas, we describe the example of an autonomous robot that serves dishes to guests located in different guest rooms. This example is used throughout this article to illustrate our proposed approach. Figure 1 shows the layout of a typical restaurant in which our robot operates. The environment is not assumed to be specifically structured in a manner that makes it easy for the robot to act. The restaurant environment is typified by dynamics and uncertainties because it contains moving obstacles, such as various people of guests and staffs who randomly move around in the restaurant without notifying the robot as well as static obstacles such as walls or furniture. The robot is expected to act autonomously and is equipped with functionalities that help it to accomplish tasks. To understand the challenges for autonomous robot software in real-world environments, we consider the following specific scenarios in the above example.

• Goal pursuing. The robot must deliberate over complex situations and adapt a predefined path routine to changed environments during execution of the navigation plan to pursue successful task goals.

Figure 1.

Motivation scenario of an autonomous robot serving in a restaurant environment. R1 denotes the robot that serves dishes to guests, whereas R2 and R3 represent the moving entities (other robots or guests).

In most robotic applications, the robustness of plan execution requires the robot to be able to adapt pre-made plans to unexpected situations that occur during operation. For example, certain guests might accidentally stand in the path of robot navigation, and the robot cannot expect the guests to act to suit it. In this case, the robot must make adaptive decisions on how to find another applicable path to the target room without colliding with guests.

Obstacle avoidance. If an obstacle suddenly moves towards the robot, in addition to computing an alternative path routine to the target position, a more important and urgent task for the robot is to enact instantaneous measures to avoid obstacles and ensure robot safety.

The reactivity of robot behaviours towards difficult real-time situations poses an essential challenge for software in autonomous robots. Situations that require high reactivity of the robot software usually address with the safety of robot itself or the situated environments, such as collapse with other agents and low-power emergencies, which require a rapid response to avoid damage to robot components or environmental entities.

Abstract model of hybrid architecture

Design considerations

To address the challenges discussed above, a robot software architecture must incorporate the ability to support plan adaptation and quick reaction to exceptional situations. To address the limitations of the existing architecture paradigms, the primary problem is to ensure that during the task-following process, additional processes of sensing and monitoring occur can run in parallel to update real-time environment states, monitor and detect run-time contingencies during plan execution and address exceptional situations in uncertain real-world environments. The problem can be categorized into issues of architectural structure and interaction mechanism in design of the software architecture.

Architectural structure: The single execution pipeline of the SMPA architecture makes it difficult to continuously sense the environmental states through sensor devices and increases time delays between sensing and acting. The ideal of incorporating additional interaction pipelines between sensing and other functional components attempts to enable direct access to sensor data to reduce the time delays between sensing and planning.

Interaction style: The styles of communication specify the types of messages transferred between two interacting functional components, which include data-based flows and event-based flows. Data-based communication delivers actual information such as sensor data and plans to other components for further analysis and synthesis, while the event-based communication delivers notification messages related to control commands that trigger or activate reactions from other components.

Overview

An overview of a hybrid software architecture is shown in Figure 2. Adaptive control involves the deliberation operations of modelling, planning and monitoring components responsible for problem solving, plan execution monitoring and plan adaptation. Reactive control is closely related to robot physical operations in which the sensing and acting components operate directly on the situated environments. Reactive control enables a rapid response to unexpected situations with reactive planning strategies and without the need for time-consuming high-level reasoning activities.

Figure 2.

High-level framework of hybrid software architecture.

The workflow of the proposed architecture can be summarized in the following steps:

1. In the initial stage, the sensing component gathers information on the situated environments and sends it to the modelling components.

2. The modelling component merges the sensor information and establishes the initial state (model) of the robot.

3. Based on the initial state, the planner computes and creates a set of ordered plans to perform the desired task. When a plan is instantiated, a list of corresponding expectations related to the execution of the plan is also generated. Once a plan has been generated, it is sent to both the acting component for execution and the monitoring component for plan expectation monitoring.

4. After receiving a plan, the acting component further refines each of the abstract plans into one or several primitive actions that can be directly handled and executed by the robot actuators. Throughout the process of plan execution, the states of the robot environment could change with the effects of the actuator physical operations.

5. During plan execution, the monitoring component accesses essential sensor information to assess whether the expectations are met during plan execution by comparing them with the obtained observations.

6. If any of the plan expectations are violated, the deviation region is sent back to the planner and triggers the process of remodelling and replanning. At the same time, the monitoring component notifies the acting component to block the plan execution and wait for updated plan solutions.

7. Moreover, this approach allows direct connections between the sensor and actuator components to achieve rapid responses towards unexpected situations with reactive strategies.

Formal framework of hybrid architecture

In this section, we establish a formal framework to specify the concepts and procedures in our proposed architecture.

Preliminary

Definition 1

(State) Suppose a robotic system has n state fields, then the n-dimension state space can be defined as $S=\{d_1,d_2,\dots ,d_n\}$ with each d_i representing a variable of ith state field.

Definition 2

(Model) A world model M is denoted by M = S_o, which describes the initial state before a robot plans to do a task.

Definition 3

(Event) Let φ be a predicate, then effects of an event e can be defined as:

• a state field addition +φ, whereby a state variable φ is added to the state space; and

• a state field deletion −φ, whereby a state variable φ is removed from the state space.

Definition 4

(Action) An action is a primitive process that can be directly executed by a robot platform, and the execution of an action involves no deliberation. An action basically consists of an action name and arguments and can be defined as a tuple $a=<\psi ,\varpi ,∊>$ where:

• ψ, the identifier, is a first-order predicate $p({\tau }_0,\dots ,{\tau }_k)$ where ${\tau }_0,\dots ,{\tau }_k$ are variables.

• ϖ, the precondition, is a first-order formula whose free variables also occur in ψ.

• ∊ is a set of first-order predicates representing effects of the action, which updates relevant state variables of the state space.

Definition 5

(Planning) A planning function is described as

$Plan(S_o,S_t):\{S_o,S_t\}\to P\cup \left\{failure\right\}$

where S_o is the initial state specification, S_t denotes the goal state specification and P denotes the set of generated plans.

Definition 6

(Plan) A plan is a long-range and high-level task step generated by a task planner. The refinement and decomposition of a symbolic plan generally require deliberation capability for plan achievement. We define a plan as $p=<\text{def,\hspace{0.17em}}{S}_o,S_t,S_m,bd,\prod >$ , where:

• def declares the parameters in the plan.

• S_o indicates the initial state of the robot before plan execution.

• S_m denotes the maintenance expectances that indicate certain state constraints needing to be satisfied during plan execution.

• S_t denotes the effect expectances (terminal state) after the successful execution of the plan.

• bd is the plan body that consists of a finite and possibly empty sequence of actions [ $a_0,\dots ,a_n$ ], where $S_o\supseteq \varpi (a_0)$ and $∊(a_n)\supseteq {\text{S}}_t$ .

• $\prod =\{{\theta }_1,{\theta }_2,\dots ,{\theta }_n\}$ contains names of associated sensors (such as camera, sonar and bumper) that are expected to monitor plan execution.

For a robot that executes the plan $p=<\text{def},S_o,S_t,S_m,bd,\prod >$ , the execution process is denoted by

$\text{ExecutePlan}(p):S_o\times bd\stackrel{S_m}{\to }{S}_t$

which indicates that a plan expects to execute the action sequence bd under maintenance expectations S_m from the initial state S_o and achieves the effect expectations S_t (the terminal state).

Definition 7

(Strategy) A strategy is a reactive and short-range response towards a timely emergent situation, which consists of instantiated actions in reacting to an external event. The difference between a plan and a strategy is that a plan consists of long-range action steps to achieve high-level goal, which requires synthesizing and reasoning by a planner, whereas a strategy contains only short-range actions to make local effects. We define a strategy as $\eta =<\text{e,\hspace{0.17em}}\Theta >$ , where e denotes an event that triggers the strategy, and $\Theta =\{a_0\circ \cdots \circ a_n\}$ is an ordered set of atomic actions that are retrieved from a library of actions, where ∘ represents certain temporal constraints as ∨ or →.

Definition 8

(Topic) The perception information obtained by sensors can be flexibly accessed through topic-based communications, and data messages are delivered to all components that have registered to the topics.

A topic is defined as t = (ρ, ν), where ρ represents the sender of the topic and ν denotes vector containers that stores sensor data messages transferred through the topic. Besides, each sensor can publish a single topic while a registrant can simultaneously register to several topics and receive different types of sensor data messages.

$\begin{array}{l}\text{PublishTopics}:\{{\theta }_1,{\theta }_2,\dots ,{\theta }_n\}\to \{t_1,t_2,\dots ,t_n\}\\ \text{RegisterTopic}s:\{t_1,t_2,\dots ,t_n\}\to \{{\nu }_1,{\nu }_2,\dots ,{\nu }_n\}\end{array}$

Adaptive control

In this section, we describe an adaptive control procedure in the hybrid architecture. The general procedure is shown in algorithm 1, and we describe certain of the key steps in additional detail below. The first step is the planner, which generates applicable plans to reach the goal state from the initial state (model). Both the initial and goal states in this work refer to the external states of the robot and situated environment, which relate to the task descriptions. The second step is to monitor plan execution as soon as the plan is executed (see algorithm 2). Monitoring of the plan execution consists of comparing both maintenance and effect expectations with the real-time perception information and subsequently sending the detected plan deviations back to the planner if anomalies are detected. The following step syncs robot perception model with the changed states of the external environments if plan anomalies are detected and replanning is required (see algorithm 3).

Monitoring of the plan expectations. Once a plan p has been generated, the planner passes a list of expectations s_m and s_t as well as types of related sensors $\prod =\{{\theta }_1,{\theta }_2,\dots ,{\theta }_n\}$ to the monitor as the plan execution begins.

As shown in algorithm 2, a first step in monitoring extracts the sensor types and activates the corresponding sensors to begin working. The sensors attempt to send the perception data messages via topic-based communications. Topic-based communication allows diverse components such as monitoring and modelling components to dynamically access the applicable sensor information through the broadcast topics at any time, without needs to notify the sensor providers. The sensors activate to work by event notifications of e_pub = ‘onStartPublishing’ from the monitor, and then corresponding sensors θ_i start to publish topics t_i to provide external access to data messages. The monitor registers to the topics { $t_1,t_2,\dots ,t_n$ } which are required by the plan and receives corresponding perception data { ${\nu }_1,{\nu }_2,\dots ,{\nu }_n$ } as soon as it receives event notifications e_reg = ‘onStartRegistering’ from actuators when plan executes.

The second step coordinates the paces of monitoring process and acting process. To monitor both the maintenance and effect expectations, the monitoring operations must be synchronized with the run-time state of the plan execution, namely, the in-execution phase and post-execution phase. As the plan is carried out, the maintenance expectations must be checked to guarantee the run-time safety, and after execution of the plan, the post-effects must be verified to decide whether the actual effects achieve the expectations. Synchronization of these operations is achieved by event notifications from the actuators. As the actuators begin to execute the plan, a start notification e_in = ‘onStartExecution’ is sent to the monitor to activate the monitoring process for the maintenance expectance. After the actuator executes the plan, it also sends notifications e_post = ‘onPostExecution’ to activate the monitoring process for the effect expectance.

If the monitor detects deviations that violate the maintenance or effect expectations, it first sends notifications to the actuator to stop the current operations in the plan execution or block the next plan execution, and the detected deviations Δ are passed back to the planner to update the planning model accordingly.

Updating of the plan model. If the monitor detects regions of expectation anomalies, this information is sent back to the planner to update the planning model accordingly. In algorithm 3, the planner extracts the state variables that cannot be satisfied by the situated environments and calls for the modelling component to update the model according to anomalies $\Delta =\{d_1,d_2,\dots ,d_j\}$ . As each state field d_i is set corresponding to a certain aspect of perception data ν_i, the modelling component registers to the corresponding topics to access the latest sensor data required for updating of the world models. After updating of the model, the planner begins to generate new plans based on the updated models.

Reactive control

The low-level reactive control flow allows rapid responses with effective measures to address the time-sensitive emergent situations in which strict time constraints are imposed and the time-consuming deliberation process struggles to meet the requirements. For example, if a robot is in danger of falling on the floor, the reactive strategy in which the robot protects its head with its arms can be effective in reducing the risks of serious damage, and this process might occur too late if recovery decisions are finally made by the adaptive procedure.

In the reactive flow (algorithm 4), the sensors offer data services for other components and are also capable of estimating and predicting possible emergencies, by analysing data flow related to some of safety-critical measurements of the robot’s well-being, such as the safety distance, motor temperature and posture balance. Sensors are enabled for the essential capability of analysing and handling raw data to detect possible run-time contingencies as quickly as possible, especially in highly dynamic environments so that reactive strategies can be applied to cope in a timely fashion.

When the sensor detects exceptional situations, event notifications are sent immediately to the planner component for reactive planning. The planner maintains a set of strategies that correspond to the events related to the internal states of the robot system, and the strategy has priority over all other task-oriented plans that are currently underway. Upon receiving notifications of emergencies, the planner first applies applicable short-range strategies to handle the situation as quickly as possible and deliberates over situations to check whether it is still necessary to plan adaptations of long-range steps to pursue the original goal.

AutoRobot: A multi-agent implementation framework

This section presents an implementation framework in AutoRobot to implement the architecture, which is based on the ideas and technology of multi-agent systems. A concrete implementation framework for the motivation example is showed in Figure 3. The implementation framework is built on the Java agent development framework ²⁷ (JADE) platform and ROS. ¹⁴

Figure 3.

The AutoRobot multi-agent implementation framework.

JADE acts as middleware for the development and execution of agent-based applications, and the reasons why JADE is a good choice are twofold. First, in terms of software functionality, JADE displays strengths in explicitly specifying various types of behaviours for an autonomous agent, delivering an agent interaction mechanism that can satisfactorily facilitate flexible and autonomous behaviour interaction among agents and can be extended to incorporate the planner capability from the AI community, in comparison with the actionlib facilitates in ROS. Second, JADE shows a great number of strengths in programming and developing agent-based software, such as free access, interoperability, powerful facilitates of communication protocols and so on. ROS is fundamental middleware that offers unified interfaces over heterogeneous robotic hardwares. The ideal of integrating the agent-oriented techniques with ROS middleware is intended to improve the reusability and portability of agent programmes among diverse robotic platforms and to reduce programming complexity and efforts.

The agent layer builds up the high-level programme abstraction over a target robot software application, which consists of the diverse roles of the interacting agents. For each agent role in AutoRobot framework, the autonomy of agent behaviours is achieved by a popular belief-desire-intention (BDI) deliberative structure for implementation of bounded rationality and autonomy in an agent, or a simple reactive structure for implementation of a stimulus-response behaviour, according to different role specifications. A deliberative process generally first calculates the desires according to the recent beliefs and then decides about the intentions. The so-called reactive agents are generally realized by a direct (complex or simple) mapping from inputs (sense) to outputs (act). The roles of the agents are briefly summarized as follows:

The PlannerAgent performs modelling and planning jobs, including activities of establishing world models and planning over a specific problem domain. In a BDI implementation, the belief stack stores the internal model of the recent state of the environment and the global task goals to which that the agent commits. Plans that are instantiated and adopted by the agent are known as intentions with the agent committing to the applicability of those plans.

The DispatcherAgent acts as a mediator that dispatches the generated plans to a specific actuator agent capable of the corresponding action. Implementation of this agent involves with a simple reactive structure and maps a specific plan (sense) to a capable agent (act) in an efficient manner.

The MonitorAgent links the monitoring process and is responsible for extracting expectations of the currently executed plan and distributing the observation plans to the relevant sensor agents. The MonitorAgent is also implemented as a BDI-style agent, and its belief stack stores the execution states of a currently executed plan with intentions composed of computed sensor topics from the plan expectations.

An ActuatorAgent is implemented as the abstraction over the robot physical actuator and maintains a simple reactive structure, effectively managing the individual robot actuator and translating the plan into primitive actions.

A SensorAgent controls an independent sensor device of the robot and is implemented to perform a stimulus-response behaviour aimed towards external state changes.

The autonomous properties of each agent role are achieved from two aspects. From the viewpoint of programming concepts, an agent embodies higher abstraction than a static object and manages its proactive behaviours according to external situations. From the viewpoint of the role in a multi-agent system, each agent with a specific role generally maintains either a complex deliberation process or a relatively simple reactive process and has full autonomy in determining its interaction with other agents.

To develop a specific application software, we need only to determine how many sensor and actuator agents (according to the types of robot hardware and application requirements) are required and implement agent classes as subclasses of SensorAgent and ActuatorAgent. Considering the aforementioned example, we need to implement several agent classes such as a WalkerAgent, a KinectAgent and a BumperAgent on behalf of an independent sensor device, besides the planner, dispatcher and monitor agents.

In the low-level ROS layer, each robot controller programme is implemented as an ROS node which offers basic robotic services. With well-designed interfaces for low-level controller nodes, the agent-level programmes only address interactions among diverse roles of agents, regardless of the redundant work of implementing elementary robotic functionalities.

A conceptual framework for comparison

To discuss both strengths and weaknesses of our approaches, we establish a conceptual framework to compare AutoRobot software framework to a subset of the popular robotic software frameworks aforementioned in the related work section. As a robotic software framework is an integral and concrete production of robot software architecture, software engineering methodologies, tools and libraries and so on, we propose three categories of criteria for comparison which cover architectural characteristics, software engineering characteristics and platform supports.

F1: Architectural characteristics

Architectural characteristics of a robotic software framework refer to fundamental robotic functional issues, such as control models, high-level structures of a robotic software and software capabilities.

F1.1: Adaptive capability. Adaptive functions of a robotic system closely relate to whether a framework offers explicit or implicit design models or structure prototypes in facilitation of planning, reasoning and synthesizing over deterministic or nondeterministic world models.

F1.2: Reactive capability. Robot software reactivity depends on whether the software architecture implements basic sensor-effector reacting loops and performs reactive strategies towards emergent situations.

F1.3: Multi-robot cooperation. Development of software for a multi-robot software application generally requires that the robotic software framework offers convenient facilities to solve coordination issues of robot teams such as control models, distributed communication protocols, knowledge bases and so on.

F2: Software engineering characteristics

Software engineering characteristics refer to good practices and principles for designing and implementing a robotic software system in high quality.

F2.1: Reusability. Reusability depends on whether the software architecture or middleware provides carefully designed programme interfaces and good portability of successful robotic software products among heterogeneous robot platforms.

F2.2: Maintenance. A robotic software in good maintenance allows flexible and easy modification of a certain robotic functionality, such as task planning algorithms and off-the-shelf knowledge reasoning engines.

F2.3: Modularity. A modular robotic software architecture consists of several components of high cohesion and low coupling, with the dependences among components being kept at minimum to obtain a maintainable and scalable software.

F3: Platform supports

Platform supports refer to framework facilities or tools that affect multiple components (or the system as a whole) and are not tailored to individual architectural components, application domains or particular stages of robotic application development.

F3.1: Operating system. A compatible robotic software framework among diverse operating systems allows a designer to freely choose programming environments, which can greatly promote the adoption and popularity among robotic programmers.

F3.2: Programming language. Programming languages such as object-oriented language hide irrelevant or cumbersome implementation details from end users, which reduce programming efforts and make it easy for beginners.

F3.3: Reusable packages. The framework must establish well-structured abstractions over basic robotic functionalities and implement them as reusable packages to ease development efforts for programmers.

For each robotic software framework, a value has been assigned for the criteria based on the publications and user experiences. There are two types of assignments made in the comparison: (1) ternary, signified by □ for not supported, ⊟ for partially supported and ⊞ for well supported; and (2) listings, signified by brief text descriptions. The following shorthand column headings are used to designate particular systems: P/S-player/stage, W-Webots, Or-Orca, MA-MARIE, VO-VOMAS, YA-YARP, R-ROS, V-V-REP, AX-ArmarX, CO-COROS, AL-ALLIANCE-ROS and AR-AutoRobot.

Table 1 has evaluated 12 robotic software frameworks with respect to a set of relatively common criteria supporting designs and implementations of robotic software applications. The comparison results suggest that AutoRobot outweighs most of other robotic software frameworks in the following aspects: (1) AutoRobot provides explicit control models and architectural structures to support adaptivity and reactivity of robot behaviours, (2) AutoRobot carries out good software engineering practices such as reusability and modularity and (3) AutoRobot supports interoperable and portable robot programmes on different operating system. However, the limitations are also obvious. First, our framework lacks essential facilitates or theoretical foundations for multi-robot software development; second, the lack of well-organized and systematic development packages to facilitate programming makes AutoRobot less popular with the robotic community.

Table 1.

General aspects of the compared robotic software frameworks.

Criteria	P/S	W	Or	MA	VO	YA	R	V	AX	CO	AL	AR
F1.1	□	□	□	⊟	⊞	⊟	⊟	□	⊞	⊟	⊟	⊞
F1.2	□	□	□	⊟	⊞	⊞	⊟	⊟	⊟	⊟	⊞	⊞
F1.3	⊞	⊞	□	⊟	⊞	⊟	□	□	□	⊞	⊞	□
F2.1	⊟	⊟	⊞	⊞	⊟	⊞	⊞	□	⊞	⊞	⊞	⊞
F2.2	⊟	⊟	⊞	⊞	⊞	⊞	⊟	⊟	⊟	⊟	⊟	⊞
F2.3	⊟	⊟	⊟	⊞	⊞	⊞	⊟	⊞	⊞	⊟	⊟	⊞
F3.1	U	U, W	U, W	U	U, W	U, W	U	U, W	U, W	U	U	U, W
F3.2	C++, P, J	C, J	C++	C++	J	C++	C++, P	Lua	P, C++, J	C++, P	C++, P	C++, J, P
F3.3	⊟	⊟	⊞	⊞	⊟	⊞	⊞	⊞	⊞	⊞	⊟	⊟

U: Unix; W: Windows; P: Python; J: Java; P/S: player/stage; W: Webots; Or: Orca; MA: MARIE; VO: VOMAS; YA: YARP; R: ROS; V: V-REP; AX: ArmarX; CO: COROS; AL: ALLIANCE-ROS; AR: AutoRobot.

Experiment

To validate the feasibility and effectiveness of our AutoRobot multi-agent development framework together with the underlying hybrid architecture, we implemented the aforementioned example under the AutoRobot development framework and ran the software on a real robot to test whether it can manage to cope with the challenges presented by real-world environments.

Problem formulation

The overall process of plan execution for the example is illustrated in Figure 4. To accomplish the delivery task, three high-level plans are synthesized to reach the goal, and we take the navigation plan as an example to demonstrate the adaptation and reaction processes. The navigation plan is refined into a set of atomic actions with computed position parameters (x, y and z). When a moving obstacle is detected that threatens the robot’s safety, an instantaneous short-range strategy (Δx, Δy and Δz) is applied to adjust the robot’s position to avoid the obstacle. After reactive adjustment, the following path routine is computed and adapted to pursue the original goal destination.

Figure 4.

Plan adaptation and reactive adjustment for navigation in the example.

Software implementation

Figure 5 demonstrates the software implementation details in a statechart diagram. The first step is to create and initialize all roles of agents by registering the agents with the agent management system for global agent identifiers. Additionally, a sensor agent also needs to register with the topic management service (TMS). The role of TMS is to support sending messages about a given topic. Such messages will be received by all subscriber agents that have previously registered to that topic. An actuator agent needs to register with the directory facilitator, the role of which is to provide the default yellow page service for managing all of the actuator agents. The only subscriber agent in our example is the WalkerAgent which manages robot movement system and registers to several sensor topics for data inputs. A publish–subscribe mechanism in our multi-agent framework is extremely convenient and useful for complex single robot applications that involve a great many actuator subscribers, with flexible and efficient message (sensor data) communications between an actuator subscriber and single or multiple sensor publishers.

Figure 5.

The software implementation details of the motivation example.

After initialization, the PlannerAgent waits for the user order to obtain the destination position, plans an optimal path based on prior knowledge and sends the generated plans to the DispatcherAgent and MonitorAgent for further processing. In the case of successful type matching between the plan and a capable actuator, the DispatcherAgent sends a plan message to the corresponding actuator. When receives the plan, the WalkerAgent begins to interpret the symbolic plan into executable actions, instantiate action parameters such as initial and target positions and drive the movement system. While the WalkerAgent executes the navigation plan, the MonitorAgent evaluates the plan expectations with real-time sensor data to check whether the expectations are satisfied. In the case in which certain expectations are violated, the MonitorAgent notifies the WalkerAgent to block the execution of the current plan and also triggers the PlannerAgent to update plans.

Separately, the software incorporates a continuous sensing process aimed towards the situated environments. The roles of sensor agents are twofold: one role passes the raw data to other agents for further analysis and synthesis as a source of perception information and the other role discovers obstacles via embedded recognition algorithms. For example, the KinectAgent incorporates essential pattern recognition algorithms to recognize possible obstacles in the captured photos, and the BumperAgent sets the threshold pressure values for awareness of the direction and positions of the contacting obstacles. In the case of obstacle recognition, event-based notifications are sent to the PlannerAgent for reactive response and adaptive planning. The PlannerAgent incorporates a set of reactive strategies that correspond to different events. Moreover, the plan adaptation dynamically adjusts the parameters of the initial and target positions with the aim of pursuing original target destination.

Evaluation

In this experiment, we compare three different software architectures that are introduced in related works: (1) SMPA paradigm, (2) behaviour-based reactive model and (3) our proposed hybrid approach. To perform the experiment in a lab environment (see Figure 6), we implement the software in three different architectures on a humanoid robot NAO to play a role of serving dishes to guests. Additionally, as NAO is approaching the target position, we drive two Turtlebot robots to move around as moving obstacles, to test if NAO can successfully avoid them and go on with its goal in a timely fashion. Each instance of this experiment has been performed twice with moving obstacles running at low and high speed, respectively. The performance metrics we try to measure are the times of successful accomplishment of navigation goals, the reaction time to unexpected events and the time cost of whole plan execution.

Figure 6.

Testing scenario in the experiment.

Implementation of other software architectures. To evaluate architectural performances, we implement the other two software architectures in our AutoRobot multi-agent development framework but with diverse organizational structures.

SMPA architecture: Four agent roles are designed to perform functionalities of each system component, with feedback from sensor agents being considered only at the beginning of motion planning in the planner agent. In the implementation, sensor agents provide a model of the environment, in which a planner agent plans an optimal and collision-free path for actuator agents to follow with.

Behaviour-based reactive architecture: Several agents in diverse levels of competence are created to decompose the navigation problem vertically, with each level of competence specifying a class of valid agent behaviours. ⁹ The lowest level to the highest level of competences in our implementation are defined as follows: avoiding obstacles, wandering around without colliding with others, building a map of situated environment and planning paths from one place to another.

Results. Figures 7 and 8 visualize the trajectories of NAO robot navigating in the presence of both low-speed and high-speed Turtlebot robots. The Turtlebot robots are programmed to run to the NAO robot at certain locations that indicated by the arrows. The faster the Turtlebot robots run, the higher is the difficulty for NAO to successfully avoid them and reach the goal. In the case of Turtlebot robots running slowly, three system architectures all successfully detect the presence of dynamic obstacles and manage to reach the target position. However, in the high-speed case, the SMPA approach fails to detect the presence of ‘turtlebot1’ and stops navigation due to an unexpected collision with ‘turtlebot1’, while the other two approaches perform very well under such a dynamic condition.

Figure 7.

Navigation trajectories of NAO robot in the presence of low-speed obstacles.

Figure 8.

Navigation trajectories of NAO robot in the presence of high-speed obstacles.

In Table 2, we report both reaction time and overall execution time for all the successful trials. The hybrid approach costs slightly less time than the behaviour-based approach, whereas the SMPA is significantly slowest. The difference in the time costs can be attributed to the facts that SMPA starts planning to the next position when having reached the old one and has to wait until a solution has been found, and behaviour-based reactive approach fails to optimize global path routes to the long-range goal.

Table 2.

Time costs of reaction and overall execution process.

Architecture	Occasion	Difficulty
		Low	High
	Reaction 1	5.78 ± 0.14	–
SMPA	Reaction 2	6.20 ± 0.22	–
	Overall	50.45 ± 0.04	–
	Reaction 1	3.89 ± 0.84	5.23 ± 0.13
B-reactive	Reaction 12	4.25 ± 0.30	7.42 ± 0.84
	Overall	40.24 ± 0.54	50.31 ± 0.04
	Reaction 1	4.03 ± 0.23	5.12 ± 0.91
Hybrid	Reaction 2	4.79 ± 0.27	6.90 ± 0.39
	Overall	37.23 ± 0.06	43.12 ± 0.93

SMPA: sense-model-plan-act.

Conclusion and future works

Autonomous robots are complex social-cyber-physical systems that typically operate in an open environment and use autonomous behaviours to accomplish the assigned tasks. Such requirements of autonomous robot software challenge the existing robot software architectures and resultant implementation approaches from software engineering perspectives. This article proposes a solution to the design and development of autonomous robot software. The contributions of the article are twofold and are described as follows:

Hybrid robot software architecture. The architecture integrates the advantages of both the SMPA paradigm and behaviour-based architectures to achieve both adaptive and reactive capabilities of robot behaviours.

The AutoRobot multi-agent development framework. Different functional components in the architecture are implemented as diverse agent roles. Autonomy of the agent behaviours and continuity of interactions are achieved via the autonomous individuals and their cooperation.

We have successfully developed several cases of autonomous robot software to validate the effectiveness of our proposed architecture and framework. In future work, we will further explore the following topics: (1) safety specifications of robot behaviours to avoid possible conflicts between the applied strategies and the currently executed plans, (2) the task allocation issue of how to conduct a reasonable plan decomposition to match the physical capabilities of robot actuators and (3) unified interfaces for sensor data transferring between the agent programming platform and the ROS system.