Abstract
Nowadays, a major challenge in the development of advanced robotic systems is the creation of complex missions for groups of robots, with two main restrictions: complex programming activities not needed and the mission configuration time should be short (e.g. Urban Search And Rescue). With these ideas in mind, we analysed several robotic development environments, such as Robot Operating System (ROS), Open Robot Control Software (OROCOS), MissionLab, Carnegie Mellon Robot Navigation Toolkit (CARMEN) and Player/Stage, which are helpful when creating autonomous robots. MissionLab provides high-level features (automatic mission creation, code generation) and a graphical mission editor that are unavailable in other significant robotic development environments. It has however some weaknesses regarding its map-based capabilities. Creating, managing and taking advantage of maps for localization and navigation tasks are among CARMEN’s most significant features. This fact makes the integration of MissionLab with CARMEN both possible and interesting. This article describes the resulting robotic development environment, which makes it possible to work with several robots, and makes use of their map-based navigation capabilities. It will be shown that the proposed platform solves the proposed goal, that is, it simplifies the programmer’s job when developing control software for robot teams, and it further facilitates multi-robot deployment task in mission-critical situations.
Introduction
Today’s state of development of autonomous robots tries to satisfy a problematic requirement: robots should be able to achieve ever more complex tasks, such as search and destroy explosives, locate catastrophe victims and various other tasks, that must be carried out in environments populated by human beings but by means of several robots. A sample mission could be a group of mobile robots, one leader and several slaves, which explore different rooms in a building trying to find a target (a wounded man, explosive material or some other objective). The leader has at its disposal any instruments needed to heal the wounded or to deactivate an explosive device. Slaves explore the rooms and notify the leader (and the other slaves) if any of them finds the target. Once notified, the leader proceeds to the indicated localization.
This task implies high levels of abstraction, with primitives of a high level such as ‘find victim/explosive’, ‘send FOUND message to leader robot’, ‘send location of this unit’, ‘identify room H4 in this building’, ‘enter room H4’ and many others. The human team in charge of deployment needs to be able to configure simply and quickly the whole mission. The work of robot developers should also be simplified in such a way that they have a module that incorporates functionalities like tracking, mapping, route planning, artificial intelligence algorithms and more. All these should be done without carrying out programming tasks.
In this sense, it would be convenient to have tools that help us manage growing complexity. This includes both robot development and deployment of robots for the mission. That should be the main goal of today’s robot development environments (RDEs). Several RDEs that help us to develop autonomous robots have been reviewed (ROS, OROCOS, MissionLab, Carnegie Mellon navigation (CARMEN), Player/Stage), as shown in ‘Evaluation of different alternatives’ section. Indeed, when using MissionLab, most of our computer engineering students are able to complete the sample mission in a single session without previous knowledge of the tool.
Users of MissionLab can use existing behaviours to build complex missions with several robots using a graphical editor (CfgEdit). This tool allows us to configure missions graphically with several robots or even groups of robots, where each robot is guided by a finite state machine. Robots can communicate with each other and can have joint behaviours based on the societal agent theory. 1 Users do not have to write a single line of code. Moreover, MissionLab has a case-based reasoning server, which can automatically generate mission plans or receive high-level orders at runtime using a language called command description language.
The development of the mission can be done in less than 1 h using CfgEdit. We only have to graphically build the state machine of each robot, start the message server (iptserver) and execute the processes that controls the hardware of each robot (HServer). Finally, we push the button to run the mission in CfgEdit. It is even easier to simulate the execution of the mission, we only need to launch the message server before running the mission.
As shown in ‘Evaluation of different alternatives’ section, none of the currently widespread RDEs can do the same so easily, so fast, with a graphical user interface and without writing any code. MissionLab provides very advanced features (automatic code generators, graphical mission definition, automatic creation of missions, etc.) not available on ROS, OROCOS, Player/Stage and so on. Further, MissionLab has demonstrated its strengths in publications related to several areas like learning, 2 hierarchical behaviour, 3 multi-robot formation, 4 multi-robot task allocation 5 and simultaneous localization and mapping (SLAM). 6 Its usability has been thoroughly tested, 7 and indeed the project is still alive and it is being used in projects like micro autonomous systems and technology 8 and also in robotic missions intended for counter-weapons of mass destruction. This has been done in defence threat reduction agency, 9 where the performance of MissionLab was verified using process algebra. According to a recent publication, 10 works are in progress for software verification purposes.
The two main inconveniences of MissionLab are that its last official version is targeted to a Linux distribution unsupported since 2006, and that it shows limitations creating maps or using them for localization and navigation.
We decided to improve map-based capabilities of MissionLab in order to solve its limitations (as shown in ‘Description of selected RDEs’ section). In particular, its usage of maps for navigation is very limited and indoor localization is not precise enough. To solve these deficiencies of MissionLab managing maps, we decided to integrate features from another open source RDEs. We chose CARMEN because of its better solutions for mapping, map-based localization and navigation. Besides, it is a very simple and portable RDE with very few dependencies, and the communication library that CARMEN uses (inter-process communication (IPC)) is similar to the library used by MissionLab interprocess communications toolkit (IPT) in many aspects. Both are different forks of a previous project called task control architecture (TCA).
We think that MissionLab has fallen into disuse for the broad audience because it is incompatible with the recent versions of operating systems. Hence, we updated those libraries, and we made available the services of modern operating systems (mainly a kernel-assisted thread library). Further, in MissionLab, we replaced its communication library (IPT) with the one used by CARMEN (IPC).
The new RDE, we created, integrates both MissionLab and Carmen, allowing MissionLab to take control of CARMEN robots, to get CARMEN sensor readings (odometry, sonar, laser) and to incorporate the best of CARMEN’s features (localization, navigation and mapping) in its missions. The design and implementation of the integrated architecture preserves backwards compatibility with both original RDEs. That means any development that makes use of the original version of MissionLab or CARMEN can use this new integrated RDE without any changes.
That is the main point of our work, providing an integrated RDE with the best capabilities of both. These capabilities are available again for new robotics developments, thus opening a way for even more interesting research like the integration of the resulting RDE into ROS.
Analysis of the considered RDEs
For this work, we have considered several RDEs (Table 1). In this section, we are going to review them, explaining why we have considered MissionLab and CARMEN over the alternatives and will describe them more in depth.
Comparison chart showing the characteristics of the considered RDEs.
RDEs: robotic development environments; IPC: inter-process communication; CORBA: common object request broker architecture.
Evaluation of different alternatives
Nowadays, there are very popular RDEs 11 like ROS, 12 OROCOS, 13 Player/Stage, 14 CARMEN, 15 MissionLab 16 and so on based on highly modular designs. Usually, these modules provide an input interface, an output interface and some configurable parameters. This provides scalability to add new algorithms and drivers.
Thanks to RDEs, we have a huge bunch of functionality that can help us in our robotic developments. However, although a robot is able to accomplish a lot of individual tasks (open doors, detect patterns, catch objects…), this does not guarantee that it is more autonomous or intelligent. A robot becomes more autonomous if it is able to appropriately combine these individual capacities and use them when it makes sense.
If we have a robot that can do tens or hundreds of little tasks and we want to do something smart with it taking advantage of all its skills, it seems obvious that we will need tools that help us to model the robot’s behaviour. These tools can be based on finite state machines, case-based reasoning or any other artificial intelligence technique. The same is applicable when building missions with tens or hundreds of robots involved. When robot teams are considered, instead of a single robot, we have the same challenge at a higher abstraction level. If we want a robot team to perform a mission in an efficient way, we need to synchronize and accommodate the behaviour of each robot in order to support the others to accomplish the mission objectives.
Using most RDEs (ROS, OROCOS, Player/Stage, etc.), the development of a mission similar to the one proposed in the introduction (several robots with two different roles) requires writing communication code, synchronization code or even the whole state machine, with or without supporting tools. Executing it may require starting many modules or creating a custom deployment file that help us starting the mission. This is a clear overhead because, instead of putting the focus on the design of the behavior of the robots and the logic of the mission, we have to spend a lot of time dealing with programming and intrinsics of the development environment.
It can be shown that the RDEs like ROS, OROCOS or Player do not have the main advantages of MissionLab in this sense. Looking at their official documentation, it is obvious that they are environments for developers with programming skills and knowledge of the underlying architecture (individual modules, messaging system, etc.). We think that, in order to allow the development of really complex robot behaviours and multi-robot missions, a higher abstraction level is needed. It is just not realistic to think about the development of intelligent robots able to perform hundreds of tasks and able to interact with their environment like humans, or complex missions with hundreds of robots, if we have to manually deal with low-level things like message passing or the management of the needed network of individual modules and behaviours.
Along with the graphical tools and automatic code generators to develop complex robot behaviours, the management of multi-robot missions is also a key feature of MissionLab. We can add more robots to a mission with a simple copy–paste operation in Cfgedit and setting the name of the hardware server controlling the robot when starting the mission. MissionLab also includes some behaviours to share information among robots. In other RDEs like ROS, OROCOS or Player, users have to deal with possible conflicts, have to define messages and have to implement the mission communication logic in the source code.
Player/Stage provides interfaces with robots and sensors systems and can simulate them, but it does not provide any component to manage or to synchronize robot behaviours or groups of robots out of the box. Although Player is mainly aimed to provide interfaces to a variety of robot and sensor hardware, it also provides some map-based features like localization (drivers amcl and ekfvmap) and navigation (wavefon driver). It provides a graphical tool to control those features (playernav), but it does not have tools to create maps based on sensor readings or to edit them.
The approach of OROCOS to handle these high-level features is closer to MissionLab. In OROCOS we can create XML files specifying relationships among several components that can be used by the deployer tool. It allows us to define finite state machines, to associate components hierarchically, these components can only run when needed and so on. It even provides a language called osd, allowing an easy definition of finite state machines. However, there is not any graphical tool that assist us in the creation of the necessary code. All needed XML, C++ and osd code must be written by hand.
ROS has a component called roslaunch that can read XML files to automate the process of launching ROS nodes, setting parameters and routing messages. In roslaunch files, we can define the static deployment of modules and the communications among them, but we cannot model dynamic changes that can occur during the course of a mission. We cannot specify finite state machines to lead the behaviour of the robots, the system does not start and stop individual behaviours irrespective of its output, and we have to explicitly define namespaces or remap messages in order to avoid conflicts to launch several instances of any module.
There are some graphical editors that manage roslaunch files like rxDeveloper or node_manager_fkie, but they suffer the limitations of roslaunch and do not avoid us having to directly deal with node connections, message names and so on. They simply make the edition of roslaunch files easier, in contrast with Cfgedit that automatically generates all necessary code and manages communications of all robots in a mission. ROS also contains packages to do mapping (like the slam_gmapping stack), navigation (like the navigation stack), visualization of missions (like rviz) or map editors (like Semantic Map Editor).
CARMEN provides intuitive graphical tools to create maps (vasco), edit them and its metadata (map_editor) and use them in navigation (navigatorgui). Its management of maps is remarkable. It contains an implementation of Hähnel’s map builder 17 to create maps with information about free and occupied zones (allowing intermediate probabilities), it supports off limits for navigation and can identify places using its names. Also, several maps can be associated using doors or elevators. CARMEN can localize the robots using laser range data and a particle filter. CARMEN navigation uses map information to calculate paths online and change them if they are obstructed, using the Konolige’s gradient descendant planner, 18 which has replaced the previous method (Thrun et al. 2001 19 in combination with Fox et al. 1997 20 ) due to reliability problems.
All mapping, localization and navigation information generated by CARMEN can be accessed from any other module connected to the system using the subscription mechanisms provided by IPC. The map-maker tool provided by CARMEN (vasco) uses logs with odometry and laser data to generate accurate maps, thanks to its scan matching algorithm. To correct the few fails the mapping algorithm could produce in the map, vasco allows user to do some changes like discard invalid data or rotate/translate the map. More advanced changes can be made with the map_editor tool. It allows users to change the probabilities, to specify place names, navigation off limits or even to create a map from the scratch.
Based on the descriptions above, ROS and CARMEN are the best candidates to provide the features required by MissionLab. We finally selected CARMEN because of its simple RDE focused on the needed features, less dependency, portability and modules for localization, navigation and mapping and very intuitive graphical tools for non-expert users. The similarity between the message servers of MissionLab and CARMEN is an added advantage because it allows the usage of a common messaging system for the integrated RDE.
Description of selected RDEs
In this section, we are going to explain the main characteristics and components, strengths and weaknesses of RDEs involved in our project, in order to help in understanding the integration we have carried out.
MissionLab is a set of software tools for developing and testing behaviours for single robots and groups of robots, with five main components. Mlab allows users to monitor missions, teleoperate robots and it is able to generate simulated data in order to test behaviours and missions. CfgEdit is a graphical tool to build complex missions with several robots for the users that only want to use existing behaviours. Robot executables are generated automatically by CfgEdit in a three-phase compilation using two intermediate languages called configuration description language (CDL) and configuration network language (CNL). CDL is used to recursively define abstract societal agents and CNL to model the distinct modules that compounds a mission and the data flow among them. However, they only have to be directly used by researchers who want to add new robot behaviours to MissionLab. HServer (Hardware server) directly controls all hardware of the robots and provides a standard interface for every robot and every sensor. It also estimates the robot position integrating information from different sources by means of several algorithms, like Kalman and particle filters. 21 Case base reasoning server generates mission plans based on specifications from users by retrieving and assembling components of previously stored successful mission plans using an extended 22 case-based reasoning method 23 that even allows to repair generated missions. 24
In the execution model of MissionLab (shown in Figure 1), each robot executable drives its own robot using an instance of Hserver. Robot executables (MISSION in Figure 1) can communicate with each other and with mlab, by means of IPT, to inform about its status or receive further orders.
Execution model of MissionLab. All communications are made using IPT.
The strengths of MissionLab are its high-level features (automatic mission creation, code generation), its CfgEdit tool and the integration of position information from different sources that HServer provides. But, on the other hand, its map-based capabilities are very limited. MissionLab does not have any map-based localization feature. Outdoors, we can use GPS to make global localization, but indoors MissionLab localization diverges, and this limits the precision, duration and complexity of the missions that we can implement. Map-based navigation capabilities are not the strengths of MissionLab either. It can calculate routes offline using the A* algorithm but requires a special map file. This feature is disabled by default in CfgEdit, and once the mission starts the path is fixed and is not recalculated under any circumstances. MissionLab has another feature for online navigation using the D* Lite algorithm. It is present only in one MissionLab behaviour (GoTo_DStar) and its data are not published through the MissionLab message server (iptserver); therefore, other MissionLab behaviours cannot take advantage of the navigation information. Its usefulness when operating in well-known environments is limited, especially indoors, as it is not backed by map-based precise localization. There are no tools to create, edit or visualize the maps used by this feature (nor A* feature) or the generated navigation plans. Also, as its own authors claim, this D* navigation is intended to be used in unknown environments. 25
CARMEN is a modular software designed to provide basic navigation primitives including base and sensor control, logging, obstacle avoidance, localization, path planning and mapping. CARMEN modules follow a three-layer architecture. 15 The modules of the bottom layer directly interact with the hardware of the robot, provide abstract base and sensor interfaces, calculate odometry and deal with simple rotations and straight-line motions. This layer includes drivers for a wide range of commercial robot bases and a simulation platform for all of them. The modules of the second layer implement the navigation and localization primitives of the robot. The third layer is reserved for user-level tasks using primitives from the second layer.
Communications among CARMEN modules are handled using a separate package called IPC system. Even though the IPC is distributed along with CARMEN, it is indeed a separate software development. The maturity and stability of IPC make CARMEN a very reliable system. The IPC supports multithreaded environments and connections with several IPC servers, however, it does not support both things at once. This has been one of the problems that we have had to solve in order to fully integrate MissionLab and CARMEN.
Figure 2 shows the execution model of CARMEN in which the different CARMEN modules (base, robot, localize and navigate) cooperate using IPC messages. The base module directly accesses the robot hardware, sends IPC messages with information about sensors and receives messages to control the actuators. The robot module sends control messages, receives messages from base and provides a common interface for all types of robots. The robot module receives instructions from other modules (robotgui when teleoperating or navigator when moving autonomously) and forwards them to the base module using the IPC message CARMEN_BASE_VELOCITY. It provides odometry data to other modules using the message CARMEN_BASE_ODOMETRY and also provides an elementary collision detection that is able to stop the robot in front of obstacles. The localize module receives odometry and laser data from robot module and sends the estimated global position of the robot using the IPC message CARMEN_LOCALIZE_GLOBALPOS. This localization is considered the most reliable in CARMEN and is used in all other modules to make any decision. For example, the navigate module, receives this position, calculates paths and sends instructions to the robot module.
Execution model of CARMEN. Using a modular design, it provides the basic navigation primitives related to maps.
The main strengths of CARMEN are its features related to maps (localization, navigation and mapping) and its easy usage and installation, with very few dependencies. However, in order to use CARMEN in complex robotics projects, we miss a better multi-robot support, the possibility of using a Kalman filter to estimate the position of the robot, the ability of combining several robot behaviours and some graphical user interfaces to assist developers to create complex robot software without having to implement the complete robot logic programmatically.
Specifications of the integrated architecture
Based on the similarities, advantages and disadvantages of both RDEs, we decided to address the integration of MissionLab and CARMEN following these specifications: Both systems must preserve a total backwards compatibility with third-party developments related to MissionLab or CARMEN. Thus, both systems must be able to run separately as usual. The resulting system must be multi-robot, allowing the usage of several CARMEN robots in a single mission. The resulting system must be able to use either MissionLab or CARMEN robot drivers. MissionLab must retain total control of the robots and missions and the final say about the estimated robot position, because it has more advanced control features and is able to fuse the output of several localization algorithms. Localization information generated by CARMEN must be available in MissionLab to improve the estimated position. Sensor readings from CARMEN must be available in MissionLab as if they were provided by a MissionLab driver. Map-based navigation from CARMEN must be available in MissionLab in order to combine them with other MissionLab behaviours and use them in CfgEdit. The resulting system must be able to run natively in recent versions of Linux.
Conceptual design of the integrated architecture
Based on the specifications in ‘Specifications of the integrated architecture’ section, we designed and implemented the integrated platform. We have taken into account that there are two key points in the execution model of CARMEN (Figure 2): the message CARMEN_LOCALIZE_GLOBALPOS with the position of the robot, and communications (CARMEN_BASE_VELOCITY, CARMEN_BASE_ODOMETRY) between robot and base modules. Intercepting these messages, it is possible to take control of the robot, because we control the estimated position of the robot, we receive all information from sensors and the final movement decision from CARMEN, and we can send our own movement orders regardless of the CARMEN decision. This is the design we have followed to allow MissionLab to take control of CARMEN robots.
The execution model of the integrated RDE follows the design depicted in Figure 3, that represents a mission with a pure CARMEN robot (controlled by the modules on the left of the image and HServer A), a pure MissionLab robot (controlled by HServer B) and an hybrid MissionLab–CARMEN robot (using CARMEN modules on the right of the image, but directly controlled by HServer C so it does not need the base module of CARMEN). All the communications among CARMEN modules for each CARMEN robot are performed through its own IPC server as usual in CARMEN. Thus, if a mission contains two CARMEN robots, there must be two distinct IPC servers. This can take place in the same machine but, in this case, they must run at different ports. Meanwhile, MissionLab communications can be done using the default IPT server as usual in a stand-alone execution, but it is recommended to use the new IPC-Adapter library that we have developed instead, because it provides a full compatibility with latest Linux distributions. The details about this library are explained in ‘IPC-Adapter’ section.
Overall design of the proposed architecture showing the interaction between MissionLab and CARMEN components.
Each pure MissionLab robot that only uses MissionLab drivers has its own HServer process (HSERVER B in Figure 3). There are no changes in this regard in comparison with the official MissionLab.
Each CARMEN robot integrated in a MissionLab mission must have its own associated HServer process (HSERVER A and C). In this association, the control of the robot hardware may be done either by a CARMEN base driver (HSERVER A) or by a MissionLab driver (HSERVER C). When a CARMEN base driver is used, HServer controls the robot and gets the odometry and sonar readings from it through a new HServer driver that intercepts key messages in order to take control of the robot. In either case, other new drivers allow HServer to get the laser readings and the estimated robot position from the laser and localize CARMEN modules, respectively.
When any of these new CARMEN-related HServer drivers starts, MissionLab intercepts the CARMEN internal robot communications using a new message hooking feature that we have implemented for CARMEN (‘Interception of CARMEN messages’ section). Using this feature, HServer takes control of the CARMEN robot and can send odometry and sonar messages to the corresponding IPC server in order to make possible that MissionLab robots use navigation and localization features from CARMEN. This is explained in detail in ‘Low-level architecture operation: integrating drivers and localization features’ section.
CARMEN navigation features are integrated in MissionLab in a higher abstraction level. New MissionLab CDL behaviours are able to send navigation commands to CARMEN and receive CARMEN movement information in order to be fused with the output of other MissionLab behaviours if desired. This is explained in ‘High-level operation: integrating CARMEN navigation’ section.
To support this design and to make some improvements on IPC, a new mechanism was implemented to allow the interception of CARMEN messages, new HServer drivers were created to interface with CARMEN base, laser and localize modules and two new CDL behaviours integrated CARMEN navigation features with the MissionLab graphical editor (CfgEdit).
We distinguish between what we have called low-level integration that includes integration of sensor readings, robot motion and localization; and high-level integration that includes navigation and the combination of CARMEN movement decisions with other MissionLab behaviours.
Tasks and features needed for the integration
In order to achieve our goals, prior to the integration, we had to prepare both RDEs. In this section, we will explain the four major tasks and features we have carried out to support our design.
Migration of MissionLab to recent Linux distributions
To meet our objectives, we had to port MissionLab to recent Linux distributions. This requires to solve a lot of small problems caused by the evolution of third-party libraries, fix bugs, memory leaks and also some non-trivial problems. At first, it was necessary to replace the thread library used by MissionLab (cthreads) because it is an unsupported user-level thread library that does not work in recent Linux distributions. We chose a kernel-assisted library like pthread for this replacement because it is nowadays a widespread standard.
After that, we had to replace the communication library used by MissionLab (IPT) because it is not completely re-entrant. It works well with the threading library that the original version of MissionLab uses (cthreads) because it is an user-level library and changes between threads only happen in calls to this library, all of them out of the IPT code. However, that library is neither available nor compatible with recent Linux distributions and the migration to a library like pthread, in which changes between threads may occur at any time and threads can run concurrently on different processors, exposes synchronization problems in IPT that make it unusable.
We chose the communication library used by CARMEN (IPC) for this replacement due to several reasons: it supports multithreaded applications, it allows modules to connect with several IPC servers at once, and it has similarities to IPT because both are forks of the same project (TCA). These similarities include that both use the same format (eXternal Data Representation (XDR)) 26 to define messages. Moreover, it has been used and tested in other important projects (at National Aeronautics and Space Administration of the United States (NASA), Defense Advanced Research Projects Agency of the United States (DARPA), Carnegie Mellon University…), and it is distributed under the simplified Berkeley Software Distribution license, which allows us to modify and redistribute the code within projects like this one.
IPC adapter
IPC adapter is a new component developed for MissionLab, which implements the IPT interface that MissionLab uses, relying in the IPC library from Carnegie Mellon University (Figure 4).
Replacement of IPT by IPC, thanks to a bridge module (IPC adapter) that provides the same interface. IPC: inter-process communication.
IPT provides more features than IPC but not all of them are used by MissionLab. Before the replacement, we stripped down the IPT library to the minimal set that allows MissionLab to work, discarding any additional feature, as our goal developing IPC adapter was to replace IPT in MissionLab (not to fully re-implement IPT). Afterwards, we implemented this resulting interface using IPC.
Registering and sending broadcast messages were implemented in a straightforward way, since both libraries use the same language (XDR) to define messages and both provide the message broadcasting feature. However, MissionLab does not use so much broadcasting messages. It uses mostly direct messages from one module to another. This way, MissionLab can manage multi-robot missions without any robot being disturbed with messages addressed to others. This behaviour was a drawback for our new IPC adapter component, because IPC does not have the ability to send direct messages.
To solve this problem, we had a policy that allows IPC adapter to send messages only to desired recipients. Each time a MissionLab module registers a message using the IPT interface, IPC adapter registers two messages: one with the same name to handle the broadcasts messages and query replies (messagename) and another one that concatenates the module name and message name to handle direct messages (modulename_messagename). Thus, when MissionLab wants to send a direct message, IPC adapter sends a broadcast message without using the original name, but the module name concatenated with the message name. This ensures that only the correct modules receive these messages, because only they have registered them with this name. This solution has an additional advantage. It allows an easier debugging and information sharing because all messages are accessible by other processes. The resulting IPC adapter component generates a library that MissionLab can link and use without changing a single line of code.
IPC enhancements
The main challenge related to IPC in this project was to make it connect to several IPC servers in a multi-threaded environment like MissionLab. IPC allows multi-threaded usage and connections to several servers but does not allow both things at the same time. Due to some implementation details, a thread may want to send a message or a response to one server, but finally it could be delivered to the wrong one. So we had to re-implement some internals of IPC to allow that usage.
The last problem we had to deal with in this project related to IPC library was the simultaneous reception of messages from different servers by different threads. That usage caused an important performance loss in the MissionLab–CARMEN integration due to deficiencies in IPC. We fixed them resulting in an improvement of the performance compared to the original version of MissionLab without adding any unwanted side effect. Since our source code is publicly available, anyone can take advantage of those improvements and fixes for his own developments.
Interception of CARMEN messages
The most elegant way we found to allow MissionLab to take control of CARMEN robots at runtime is to be able to intercept messages among their modules. This way, if developers want to use only CARMEN, they can do it as usual and, if they want to take advantage of MissionLab features, they only have to start it. Since all the information in CARMEN is sent among modules using IPC messages, a good way to take control of CARMEN robots is to have the ability of intercepting and rerouting these messages.
Although CARMEN is supposed to use abstract interfaces for communications to allow an easy transition to other communication library if necessary, not all communication code is hidden by those abstract interfaces. Most modules send their messages directly using IPC functions. Because of that issue, we cannot implement our rerouting mechanism without either changing all these calls or making changes in IPC. The last option was chosen because the first one would force us to change every CARMEN module and would break the compatibility with other development using CARMEN. To do so generically, we implemented a new function called IPC_hook in the IPC library that takes the name of the message we want to reroute as the first parameter, and the new name for the message as the second parameter. This function maintains a hash table that stores message–destination pairs, which is checked each time a message is to be sent.
Once we implemented the new IPC_hook feature, the ability of rerouting any CARMEN message was implemented in a very straightforward manner. Since every CARMEN module connect to IPC servers using the same function (carmen_ipc_initialize), we modified it to register a new message called CARMEN_GLOBAL_HOOK_MSG. In the message handler, we take the name of the message which is going to be redirected and the new desired name for the message. We use both to call the new IPC_hook function. As result, MissionLab is able to send CARMEN_GLOBAL_HOOK_MSG messages to any CARMEN module and redirect all the messages that it needs in order to take the control of CARMEN robots.
Low-level architecture operation: Integrating drivers and localization features
Two of our project goals were to allow to the use of device drivers existing either in CARMEN or in MissionLab and take advantage of the best features of both RDEs. For that, it is necessary to publish the same information (odometry, laser, sonar) in both systems. Therefore, if we use CARMEN drivers for our robot, HServer may read this information and send it to the mission and Mlab console as usual. Otherwise, if we use MissionLab drivers for our robot, this information may be published through IPC to take advantage of CARMEN features.
In Figure 5 an overview of this integration is shown. On the left, it shows the different parts of HServer: the Robot class, which is the base of all robot drivers in MissionLab, the Pose Calculator module, which integrates different sources of information related to the robot position, and the new modules CARMEN BASE DRIVER, CARMEN GPS DRIVER and CARMEN LASER DRIVER for the integration with CARMEN. On the right, it shows the CARMEN modules that control the movement and the position of robots (robot, localize, laser and base). Communications between different modules are represented using arrows.
Integration of CARMEN localization and drivers showing the involved modules and messages.
To be able to use CARMEN drivers on MissionLab, we have developed a new robot driver called CARMEN BASE DRIVER in HServer to get odometry and sonar data from the CARMEN base module; a new laser driver (CARMEN LASER DRIVER) that gets data from the CARMEN laser module; and a new GPS driver (CARMEN GPS DRIVER) that gets data from the CARMEN localize module. These drivers must know the host and the port that the CARMEN robot uses in order to communicate with their associated CARMEN modules. Users may provide this information interactively through the HServer console, but it can be configured in the HServer configuration file for unattended starts as well.
To be able to use MissionLab drivers on CARMEN, we have modified the Robot class in order to publish the odometry and sonar data through IPC when the robot is directly controlled by HServer drivers. To do so, it is only needed to start HServer with the -s modifier and to specify the host and the port of the IPC server where these messages must be sent.
Other key point is that we need to keep a unique final decision for the position of the robot in MissionLab and CARMEN in order to preserve the coordination between them. When working with both systems together, MissionLab always has the final say about the robot position. We took this design decision because HServer allows to fuse position information from multiple sources using its PoseCalculator fuser based on Kalman and particle filters. In some scenarios, CARMEN localization loses accuracy and in these cases it is interesting to give more importance to other sources such as odometry and sensors that the robot may incorporate, like a GPS, a compass or an accelerometer. In our implementation, CARMEN_LOCALIZE_GLOBALPOS messages (that provide the best estimation of the robot pose in CARMEN) are used as a GPS input to the PoseCalculator fuser of HServer through the new CARMEN GPS Driver, and the accuracy of each CARMEN estimation is taken into account in HServer to fuse it with other sources of position data as best as possible.
To fully control the CARMEN robot, HServer mainly manages two key CARMEN messages: CARMEN_LOCALIZE_GLOBALPOS and CARMEN_BASE_VELOCITY generated by the CARMEN localize and robot modules, respectively. The first one is intercepted by the new CARMEN GPS DRIVER in HServer (represented by a red X in Figure 5), and it is then forwarded with the fused pose calculated by HServer to be used by any other module. The second one is redirected to be used by behaviours in the MissionLab mission as the CARMEN movement decision (see ‘Low-level architecture operation: integrating drivers and localization features’ section), and it is replaced by HServer with the final movement decision taken by MissionLab. HServer sends these CARMEN messages when any CARMEN driver is loaded.
High-level operation: integrating CARMEN navigation
The CARMEN navigation capabilities have been integrated in MissionLab behaviours. We have implemented a new MissionLab CDL behaviour, called CARMEN_NAVIGATE, that uses the CARMEN_NAVIGATOR messages to control the CARMEN navigator module. For that, a new function allows to get the CARMEN movement instructions from CARMEN_BASE_VELOCITY messages. This new simple behaviour has been finally included in two new tasks that can be used in CfgEdit: CARMEN_GoTo and CARMEN_Navigate, when the robot is being teleoperated or it is autonomously moving. Figure 6 shows a robot executable generated by MissionLab that uses our new CDL behaviour and its interaction with HServer and CARMEN modules (on the right of the image).
Integration of CARMEN navigation features with MissionLab redirecting the main messages used by CARMEN.
CARMEN_Navigate is a simple task that receives a goal position as a parameter and just follows the CARMEN movement command. CARMEN_GoTo is a compound task defined as a cooperation of the CARMEN_NAVIGATE behaviour with the predefined behaviours in MissionLab to avoid obstacles and to allow the teleoperation of the robot. CARMEN_GoTo receives as parameters the goal location, the weight that CARMEN commands have in the final movement decision, the weight that the avoid obstacles behaviour have in the final movement decision, and two more parameters to configure the avoid obstacles behaviour: the avoid obstacles sphere and the safety margin.
As usual, we can use CfgEdit to generate complex missions with many states and trigger using these new behaviours. This allows us, for example, to easily create a mission that moves a robot among several places in a map, and many other operations like picking up objects in these places, taking care of its battery level to return to the charging point if necessary.
Validation of the proposed RDE
We have used four test scenarios to ensure that our multi-robot software control architecture meets the specifications defined in ‘Specifications of the integrated architecture’ section and works as expected. So, we have checked its multi-robot feature that is naturally found in MissionLab, and the capabilities of map-based localization and navigation provided by CARMEN. With the two first examples, we have examined their compatibility and compared the results with the obtained using the official virtualized version of MissionLab and the last official CARMEN release. The rest of the examples could not be done with the original RDEs because they use features of both platforms and the integration that we have made. All tests have been executed in the most recent versions of Ubuntu, Fedora, Debian, OpenSUSE and CentOS to prove the last point of our specifications.
Simple MissionLab multi-robot mission
This test scenario demonstrates the compatibility with the official version of MissionLab (http://arce2.fis.usal.es/MissionLab.mp4; first point of our specifications; first point of our specifications) and the execution of a multi-robot mission (second point). Additionally, we compare the results with the same mission executed using the original version of MissionLab. The mission consists of three robots moving around a square using the MissionLab GoTo behaviour. These robots are simulated by the default HServer simulator. Due to the improvements that we have made on MissionLab (commented on ‘Migration of MissionLab to recent Linux distributions’ and ‘IPC adapter’ sections) the CPU usage has been reduced 40% and the memory usage is stable (no memory leaks), giving developers the opportunity of developing complex missions with a very long duration. The course of the mission is the same in both systems, there is not any noticeable difference.
Simple CARMEN mission
This test scenario proves compatibility with the official version of CARMEN (http://arce2.fis.usal.es/CARMEN.mp4). We start all the necessary modules and then, using the navigatorgui tool, we place a simulated robot in the map, we select a goal location and let the robot reach it. The behaviour of the robot, the memory consumption, the CPU usage and the mission performance are almost the same as observed with the official version. This is because IPC changes are only related to connections from several threads to several servers, and the changes made in CARMEN to intercept messages are never used in pure CARMEN missions.
Multi-robot MissionLab–CARMEN mission
This test scenario makes use of MissionLab and CARMEN integration (http://arce2.fis.usal.es/MissionLabCARMEN.mp4). The mission uses three simulated robots by both RDEs. The first one is simulated by the default HServer simulator, the second one is a simulated Pioneer-I provided by CARMEN and the third one is simulated by the default HServer simulator but also takes advantage of simulated laser readings by CARMEN and of their localization and navigation capabilities. The first robot uses the default MissionLab GoTo behaviour to move between two locations and the other two robots navigate between another two locations using the new CARMEN_GoTo behaviour. The behaviour of the robots is the expected one. This test validates our specifications 2–8. Localization information generated by CARMEN for the second robot is integrated in MissionLab through HServer, MissionLab has the control of the mission, sensor readings generated by CARMEN are propagated to MissionLab, and the mission uses our new CARMEN_GoTo behaviour that integrates CARMEN navigation with MissionLab. Figure 7 shows the HServer console (in the bottom-left corner) and the graphical tools of MissionLab and CARMEN during the mission (navigatorgui in the top-left corner, mlab in the top-right corner and robotgui in the bottom-right corner).
Screenshot taken during the course of our test with simulated CARMEN and MissionLab robots.
Multi-robot MissionLab–CARMEN mission with real robots
This test scenario introduces real robots in a MissionLab–CARMEN mission that we have created with Cfgedit (http://arce2.fis.usal.es/CleaningSystem.mp4). One custom robot equipped with a 2D laser range finder laser that acts as the leader of the mission and is managed by a CARMEN driver and one Roomba controlled by a MissionLab driver. The Roomba robot must follow the leader until it detects an open door with its laser. Once an open door is detected, the leader sends the position to the Roomba, it enters the room and then returns to follow the leader robot again looking for the next door. Since the Roomba robot does not have advanced sensors to accurately estimate its position, it hits the corridor with his bumpers looking for the open door. In this test scenario, as in the previous one, we cannot compare the performance results with the official versions of CARMEN and MissionLab because the success of these missions is only possible thanks to the integration we have made. With this test we validate the integration of a real robot driven by CARMEN with another one controlled by MissionLab in a collaborative mission.
Mission with an industrial forklift
Once we ensured that our RDE worked as expected, we tested its reliability in a more complicated environment. We created a mission that uses CARMEN localization, CARMEN navigation, our new MissionLab behaviour CARMEN_Navigate and HServer device drivers for the autonomous navigation of an industrial forklift among several places of an outdoor parking.
The forklift has an electric engine powered by a set of acid batteries that we also use to feed two computers with our integrated platform installed and one router to provide a wired network for the two computers and remote access. The forklift has sonars on the back and on both sides, a laser range finder on the front and encoders in the steering wheel and front wheels. Several motors managed by proportional–integral–derivative (PID) controllers (based on encoders) move the accelerator, the break the steering wheel and the lever to move the forklift forwards or backwards. One of the computers has HServer installed and uses it to read sensors and manage the set point for actuators through a controller area network (CAN) bus. Additionally, it runs several CARMEN modules: ipc for communications, laser to read the laser on the front, and param_daemon to serve the maps, and the parameters of the different modules, robot to communicate CARMEN modules with HServer and avoid obstacles, localize to estimate the position of the forklift based on the map and laser readings and navigate to calculate routes. The other computer runs CfgEdit to automatically generate the mission executable based on a state machine that cyclically moves the forklift among several places sending waypoints to the navigator module of CARMEN (thanks to our high-level integration) and avoiding obstacles, or temporarily stopping the forklift when it is not possible to avoid them. Obstacle avoidance is controlled by the robot module of CARMEN that is able to access all sensors, thanks to our low-level integration. Using an external laptop, we are able to run visualization tools like robotgui or navigatorgui to monitorize the mission that is totally autonomous.
The forklift was moving autonomously for more than 20 min (http://arce2.fis.usal.es/AutonomousForklift.mp4), 5 until we stopped it. It did not need GPS, gyroscope or a compass because CARMEN localization was enough. This test not only validates the correct integration between MissionLab and CARMEN but also demonstrates that the resulting RDE is reliable enough to be used in a complex environment in which a failure could cause significant damage. Figure 8 shows the industrial forklift during the test.
Industrial forklift autonomously driven by our integrated platform.
Conclusions
We have successfully integrated two of the best publicly available open source robot development environments: MissionLab and CARMEN. After all the changes we have made, both frameworks maintain a total backwards compatibility and fulfil the specifications we defined. The resulting system allows to develop multi-robot missions in which all robots can take advantage of the best features of both frameworks. The performance in terms of memory consumption and the CPU usage of MissionLab has been improved, allowing the development of more complex missions. The resulting platform has been acknowledged by the creators of MissionLab and a link to this work is available in their official homepage (http://www.cc.gatech.edu/ai/robot-lab/research/MissionLab/). 6 Along with the complete source code, we provide rpm and deb packages for the most used and recent Linux distributions (Fedora, Ubuntu, OpenSUSE, Debian, CentOS) and architectures (×86, ×86_64), to give other researchers, or even casual users, the opportunity of using this software easily in their developments. We also give instructions to reproduce our test cases that do not depend on custom hardware (http://gro.usal.es/web/research.php?name=missionlab§ion=index&) to ease the installation of this software and the reproduction of our results. We hope that this RDE is especially useful on educational environments and encourage the integration or implementation of its outstanding features inherited from MissionLab like graphical mission creation, automatic code generation, compound behaviours and so on in the most widespread robot development environments.
Footnotes
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Serrano F. J. has worked under the support of a University of Salamanca fellowship. Rodríguez-Aragon J. F. has worked under the support of a Junta de Castilla y León and European Social Fund fellowship.