Sage Journals: Discover world-class research

Abstract

The article presents a software architecture to optimize the process of prototyping and deploying robot controllers that are synthesized using model-based design methodologies. The architecture is composed of a framework and a pipeline. Therefore, the contribution of the article is twofold. First, we introduce an open-source actor-oriented framework that abstracts the common robotic uses of middlewares, optimizers, and simulators. Using this framework, we then present a pipeline that implements the model-based design methodology. The components of the proposed framework are generic, and they can be interfaced with any tool supporting model-based design. We demonstrate the effectiveness of the approach describing the application of the resulting synchronous dataflow architecture to the design of a balancing controller for the YARP-based humanoid robot iCub. This example exploits the interfacing with Simulink® and Simulink® Coder™.

Keywords

Model-based design robot controllers rapid prototyping and deployment robotic middlewares code generation actor-oriented programming

Introduction

In the past few decades, robotics has experienced a continuous shift from applications in constrained industrial environments to those involving autonomy, interaction, and collaboration with external agents. The adaptation of the robotic devices to new tasks often presents big challenges in both cost and time. Thus, the capability of prototyping a new controller and rapidly deploying it to the target robotic device is becoming more and more paramount. The canonical approach to develop a robotic controller can be summarized in two distinct phases.¹ In the first phase, the robotic controller is synthesized, tuned, analyzed, and possibly tested in a simulated environment. Arbitrarily complex models of the controlled system are typically exploited. In the second phase, the controller is ported to the real device, tuned again and executed. Each minor change to the controller requires iterating this entire process from start, and a lot of effort is spent to minimize manual operations.

Model-based design² (MBD) is a methodology that emerged to deal with the challenges introduced by the need to continuously improve complex systems. MBD aims to simplify the development by providing a common environment shared by people of different disciplines involved in the different design phases.³ Later changes of the original design either due to early mistakes or requirements modifications are easier to propagate, therefore time and cost of the development can be reduced.⁴ A characteristic of the MBD is that the iterative process of the continuous improvement is performed with unified visual tools, typically based on dataflow programming languages and frameworks. The dataflow naming originates from the view of programs as directed graphs of computations, where the data flows between their components.⁵ Controllers are usually described and analyzed using block diagrams, and the view offered by dataflow programming, composed of nodes and edges, naturally translates to blocks and signals. The analogy between graphs and block diagrams makes MBD particularly appropriate for controller design. A typical implementation of MBD consists of the following stages:

Plant modeling: creation of a mathematical description of both the dynamics of the controlled system and its environment.

Controller prototyping: implementation of the operational aim of the controller acting on the plant model.

System simulation: assessment of the controller performance in a simulated environment containing the plant model.

Controller deployment: adaptation of the resulting controller to run online on the controlled system operating in the real environment.

In the market, there are many available open-source and commercial software that implement the specifications of MBD. Most of them belong to a broader category of tools that fulfill the paradigm of actor-oriented programming⁶ (AOP). Contrarily to object-oriented programming (OOP), where data structures interact via procedure calls, in actor-oriented languages concurrent objects are the first-class citizens. These objects are also called actors and they communicate with each other via predefined channels. Actors have well-defined interfaces that abstract their internal state and define constraints on how they can interact with the outside. The interaction between actors is never direct as the channel mediates it. In this way, the actors are independent entities that are not directly connected with other actors. The definition of AOP is very broad and many models of computation ⁷ can be identified to categorize the nature of the interaction between actors and channels. Each of these models is characterized by constraints in their execution, typically in the form of internal computation, internal state update, external computation, and type of communication between actors. Among all the available models, synchronous dataflow ⁸ is particularly suited for the design of robotic controllers. The computations performed by this model are triggered by the availability of new input data and the connections between actors are buffered. Examples of tools belonging to this domain are Simulink®,⁹ Xcos,¹⁰ Modelica®,¹¹ and LabVIEW™.¹²

The application of MBD to robotic controllers development can narrow the gap between control engineers, used to approach systems with block diagrams, and software engineers, used to procedural and OOP. However, this approach is not exempt from the complications introduced by system integration, which often introduces time-consuming obstacles. Particularly for what concern robotics, AOP languages by themselves are not the final solution. In fact, OOP still has a central role in the development of low-level algorithms. The aim of these actor-oriented languages is not substituting OOP, but complementing it. In fact AOP is more suitable to target the creation of applications that belong to higher abstraction layers, implementing a design principle compatible with the separation of concerns. ^13,14 It represents a valid choice to ease the interconnection of self-contained black-box functionalities, which represent the building blocks of any robotic controller.

In this work, we propose a software architecture composed of a framework inspired by AOP and a pipeline for its application to robot controllers design. The framework intends to reduce the effort spent on system integration while minimizing both code and functionality duplication. The pipeline implements all the MBD stages, and it aims to minimize the controllers lead time while automatizing as much as possible the prototyping and deployment processes. Rapid prototyping and continuous deployment are achieved interfacing the framework respectively with Simulink and Simulink® Coder™.¹⁵ Simulink provides out-of-the-box a wide library of black-box functionality exposed as blocks and also allows to be extended and integrated with external algorithms. Its status of visual programming and debugging is very mature and well-documented. Simulink Coder provides the automatic code generation capability that aids the implementation of the deployment stage of MBD, removing the need to port or adapt the controller to another domain before being executed in the target platform.¹⁶

Despite our tools selection, the framework has been designed in such a way to simplify the integration with other existing actor-oriented frameworks. The logic of the presented black-box functions (developed in OOP) is independent of them. This design allows to effectively separate the two programming domains while exploiting the best features from both.

More specifically, the work presented in this article is based on a previously introduced framework.¹⁷ From the status described in that work, the underlying software architecture considerably changed, but a big effort was spent to maintain as much as possible the same user experience. The whole-body interface layer proposed in the original work has been entirely removed, moving the responsibility of the robot abstraction to the middleware layer. Moreover, most of the improvements detailed in the same study have been adapted to the new architecture and implemented. Beyond a radical architectural advancement, the main extension presented in this work is the fulfillment of the automatic code generation support, fundamental to complete the implementation of MBD.

The article is structured as follows. First, we list tools and frameworks belonging to AOP and implementing the MBD pattern, and define a common terminology used throughout the article. Then, we present the architecture of the proposed software framework and outline how we implemented AOP for robotic controllers design. Successively, we describe how the proposed framework can be exploited to obtain a pipeline that implements the typical stages of MBD. We present a development cycle example for a balancing controller that targets a humanoid robot. We proceed by discussing the current limitations of this workflow and future improvements. Finally, we draw conclusions.

Background

Related software

MBD is a methodology that covers many software layers. Following a top-down view, the conventional unified tools typical of MBD usually share the following features:

support to automatically generate real-time code from a model,

present a graphical interface to visualize a model,

serve an engine to initialize the directed graph of computation,

provide a set of solvers to compute the output of each model element,

include a library of default black-box functions, and

offer a set of application programming interface (API) for interfacing with its engine.

Providing a complete taxonomy of the existing tools and frameworks is not a trivial task. To simplify the analysis, we limit the overview to frameworks that use the synchronous dataflow model of computation, ignoring those that also support concurrent actors. Considering the scope of the current work, we separate the existing solutions in two categories: hybrid and discrete-only.

Hybrid tools are the most generic and complete, they typically allow performing both continuous and discrete simulations. Since they provide solvers for each of the two domains, hybrid tools can execute both offline simulations of continuous ODE systems and their discrete equivalent which is compatible with real-time usage.

Discrete-only tools, instead, target only discrete-time systems, and their execution is limited to call an equivalent step function. Given their discrete nature, this second category is compatible by design with real-time usage.

Given these definitions, engines that belong to the hybrid group are Drake,¹⁸ OpenModelica,¹⁹ and the commercial software Simulink, Dymola®,²⁰ and LabVIEW.¹² Excluding Drake, all the others are unified visual tools which fully enter into the MBD framework. Simulink, in particular, is the engine that became the de facto standard for MBD. It implements all the MBD stages providing great flexibility and very simple user experience.

Other available engines are represented by those that emerged in the context of software engineering for robotics, all belonging to the category of the discrete-only tools. These types of engines are typically designed to support the development of software that runs in real time on a robotic platform, and so they do not support simulation-specific features such as continuous-time system modeling. Examples of this software are Stack of Task’s Dynamic Graph,²¹ Genom3,²² OpenRTM,²³ and Orocos.²⁴ A features comparison of the tools listed in this section is presented in Table 1.

For what concerns the deployment stage of MBD, we can identify few suitable frameworks that provide support of automatically generate code. The scope of this process is to convert a model prototyped as a directed graph to a low-level procedural representation. Nowadays, automatic code generation is a standard feature of the MATLAB system. The Simulink Coder toolbox allows generating optimized C and C++ code from a Simulink model, and it provides support to customize the sources injecting custom code during the generation process. Other frameworks that are worth mentioning are most of the software suites based on the Modelica¹¹ language, which typically support generating low-level code from their models.

Table 1.

Comparison of available synchronous dataflow frameworks.^a

Software	Hybrid	Visual tool	Code generation	Real-time native	Open source
Drake	×			×	×
OpenModelica	×	×	×		×
Simulink®	×	×	×
Dymola	×	×	×
LabVIEW	×	×	×
Xcos	×	×	×		×
Dynamic Graph				×	×
Genom3				×	×
OpenRTM				×	×
Orocos				×	×

^a Hybrid frameworks support simulating both continuous ordinary differential equation (ODE) and discrete systems. Visual tools offer a graphical interface to create the computational graph with the actors. Executing stand-alone code typically involves the generation of a low-level description of the graph; visual frameworks might or might not support exporting the graph through code generation. Note that the listed nonvisual tools are all C and C++ based, and their graph is already described in low-level languages. Furthermore, nonvisual tools might or might not support real-time execution.

The functional mock-up interface²⁵ (FMI), despite being outside the categorization described above, is still relevant to this overview. FMI is a standardized interface widely used in industry for model-based development. It is a feature-rich and production-grade tool with a clear standard, constantly improving at each release but, as most of the tools listed in this section, it was not available when we started the development of our software stack. In any case, its adoption in its current form would not be possible due to the lack of the support of vector messages between actors, shortcoming that will be removed in the upcoming version of the standard.

Other interesting frameworks for controllers design which are related to the cited engines are the Robotic Toolbox²⁶ and the Robotic System Toolbox.²⁷ The latter, particularly, is one of the few unified framework that fully implements MBD specifically for robotic controllers. It is based on the ROS²⁸ middleware, and it implements many of its features. However, the support of kinematics and dynamics has been added only recently, and it lacks the possibility to be extended to interfacing with third-party robotic libraries.

Terminology

The majority of the hybrid and discrete-only software listed in the “Related software” section share a software architecture composed of similar components. In view of the AOP architecture used in this work, we will make use the following terminology, illustrated in Figure 1:

Blocks are elements that provide self-contained functionality. They wrap algorithms exposing a black-box interface composed of inputs, outputs, and parameters.

Ports are virtual elements associated to block inputs and outputs. They store information to identify which kind of data is supported by the block (typically size and type).

Signals are the elements that connect ports of different blocks. When two ports are connected, they share their data.

Engines control the channel through which the blocks communicate. Engines typically create the computational graph and assign the blocks execution order. They also collect the block outputs and propagate them to the handled channel. They usually provide graphical tools to visualize blocks and help their interconnection by creating signals between them.

Figure 1.

Visualization of the implementation of an actor: the block.

These terms naturally translate to the definitions of the AOP framework: blocks map to actors, signals map to channels, and ports represent the interface between actors and channels.

Framework software architecture

This section describes the software architecture of the proposed framework. Firstly, the factory pattern and the plugin concepts are introduced. Their combined usage has direct applicability within an AOP context. Secondly, we describe in detail the two components that form the framework: BlockInformation ²⁹ and WHOLE-BODY TOOLBOX.³⁰ BlockInformation provides the support to AOP and the interfacing with third-party frameworks. WHOLE-BODY TOOLBOX provides a plugin library containing the actors that expose the robotic stack used for controllers design: robotic middlewares, rigid-body dynamics libraries, and robotic simulators. An overview of the main classes of these two projects is shown in Figure 2. In other terms, WHOLE-BODY TOOLBOX provides the algorithms, BlockInformation provides the back end of the software infrastructure that abstracts blocks and engines. The solvers and the front end are instead provided by the selected engine.

Figure 2.

Unified modeling language (UML) diagram of the main classes part of the proposed architecture. The BlockInformation package is composed of three components: core contains the common classes and interfaces, simulink and simulinkcoder contain, respectively, the abstraction of the Simulink® and Simulink® Coder™ engines.

Factory pattern and plugin libraries

Third-party engines typically offer a set of APIs that can be used to integrate external software inside their framework. To detach effectively the block implementations from the third-party engine, the combination of the factory pattern and dynamically loaded plugins represents one of the canonical solutions.³¹ With the factory pattern, objects are created from a factory function without the need to specify their class. Typically, a label or identifier is associated with this kind of objects, and only this information is required during their instantiation. Unfortunately, this is not enough to achieve the separation between engines and blocks, because the factory function only hides their allocation and the engine still needs to link against their implementation. This shortcoming can be overcome with plugin libraries dynamically loaded during runtime. In this case, the engine needs to have two information: the label associated with the implementation of the block and the name of the shared library that contains it. Once the plugin is dynamically loaded, the engine can instantiate block objects using a factory function without knowing anything about the class that implements them. Then, it can call their functionality through the common interface. The implementation of block classes is not constrained to any model of computation of AOP. In the most general form they can be asynchronous and concurrent.

The combined architecture of factory and plugins represents a natural implementation of AOP. In fact, the limitation of the engine to access the functionality of the blocks through their exposed abstraction layer enforces one of the key characteristics of actors: the exposure of a well-defined interface.

For what concerns robotic controllers, the separation layer introduced by the plugin-based factory pattern provides a great help in system integration. In fact, since the plugin libraries containing the blocks are engine-agnostic, they can be loaded from each engine without the need to recompile them. This means that a controller prototyped with one engine can load the same library of the deployed controller. The code duplication is hence minimized and the robustness of the system is improved because the logic of the blocks is shared. Another benefit of this architecture to the system integration is about dependencies. The stand-alone plugins can link against any third-party library without the need to operate on the layer specific to the engine.

BlockFactory

The concepts defined by AOP are implemented in a tool called BlockInformation. It allows creating blocks (the actors) that exchange data between each other through the signals connected to their exposed ports, as illustrated in Figure 1. The entities of AOP are mapped to C++ classes and interfaces, illustrated in Figure 2. BlockInformation also implements the factory pattern and provides support to dynamically load during runtime plugins that contain block objects.

To obtain engine-agnostic blocks, the 1 to be abstracted. For this scope, BlockInformation provides an abstraction layer called BlockInformation placed between blocks and engines. As shown in Figure 3, blocks can query information from the engine only through the BlockInformation interface, and engines can only call block functionalities through the Block interface.

Figure 3.

Software architecture outline. BlockInformation provides the Block and BlockInformation API along with the necessary tools to interface with the supported engines. The BlockInformation plugins contain the logic of the blocks, and they are loaded during runtime from the implementation of the engine API. This figure illustrates well the abstraction of the components of the architecture. The engine is only aware of its own API, which represents the entry-point that allow interfacing with it. The implementation of the engine API, once blocks are loaded, can call their functionality only through the Block API. Finally, the blocks contained in the plugins can communicate with the engine only through the abstraction provided by the BlockInformation API.

The interfacing with third-party engines can be achieved in two steps. Firstly, their own API or callbacks need to be implemented for loading during runtime the plugins containing the block’s logic. Secondly, to provide blocks the information from the engine they need, the BlockInformation interface needs to be implemented for the selected engine. In the current version of BlockInformation, we provide support of the Simulink and Simulink Coder engines. In this case, the implementation of their API corresponds in developing respectively a C MEX S-function and a target language compiler (TLC). BlockInformation provides these two files that are independent from the block implementation and can load generic objects implementing the Block interface.

The actor-oriented applications that can be built with BlockInformation are universal and not related by any means to robotic controllers. BlockInformation is engine-agnostic and can be interfaced with engines specific to the target application. System integration is simplified since it contains only a small number of classes and it has no dependencies. Beyond the scope of the presented work, BlockInformation can find applicability in fields such as electrical drives, communication systems, and power converters. Generally, it can cover all use cases that need exposing to the engines custom logic (either inlined in the block or wrapping external libraries) or interfacing with external devices while exposing only a simple and unified interface.

Whole-body toolbox

WHOLE-BODY TOOLBOX is a C++ plugin library that exposes canonical algorithms and utilities commonly used to develop robotic controllers, such as rigid-body dynamics algorithms and communication capabilities with robotic devices mediated by middlewares. These functionalities are wrapped as block entities, and they can be loaded independently by all the third-party engines supported by BlockInformation. To use the blocks in a Simulink model, the toolbox also provides a Simulink library that exposes the C++ classes as visual blocks, which can be imported by drag-and-drop and configured through text boxes and drop-down menus.

For historical reasons, the middleware we actively support is YARP.³² Our main target platform is the iCub humanoid robot,³³ even though all YARP-compatible real and simulated robots are supported out-of-the-box. As an example, a previous work¹⁷ showed a simulated whole-body controller running on both iCub and Walkman³⁴ robots.

Historically, WHOLE-BODY TOOLBOX was developed for whole-body control,³⁵ hence the name. In its last revisions, it became a generic robotic toolbox that can be used for any type of controller. The blocks implementing dynamics and kinematics algorithms are mainly based on iDynTree³⁶ and do not depend on any middleware. They can be used also with robots which are not YARP-based, outsourcing, in this case, the interfacing with the target platform to third-party plugin libraries. The only requirement for using the provided algorithms is the availability of an URDF description of the robot to control.³⁷

A complete software stack for robotic controllers typically involves the interaction with a physic simulator. The robotic simulator we chose to support is Gazebo.³⁸ The interaction between Simulink and Gazebo follows a co-simulation pattern, where the former is the master that issues forward step commands to the physic engine at each simulation step. The controller transparency between the real and the simulated robot is achieved by exposing the same network interface exploiting the abstraction layers provided by the YARP middleware. In the case of the simulated robot, the implementation of these interfaces are provided by Gazebo YARP Plugins.³⁹

The toolbox also provides generic utilities for robotic applications, such as discrete filters, Cartesian trajectory controllers,⁴⁰ and quadratic programming solvers based on qpOASES.⁴¹

The pipeline

In the “Framework software architecture” section, we introduced the proposed framework, described its architecture, and discussed how its components interact with each other. In this section, we will describe the pipeline that implements MBD from the point of view of the control engineer, detailing how it is practically used and how the components of the framework relate to each step of the development.

The proposed pipeline implements all the four stages on which the MBD pattern is derived. We will demonstrate a practical usage showing the steps to rapidly prototype and deploy a balancing controller,^42,43 executed on the humanoid robot iCub.³³ A simplified overview of the theory behind the controller is reported in the preceding work.¹⁷ Since we managed to maintain the compatibility of the controllers designed with the previous architecture, the experimental results of that study that use Simulink correspond to the prototyping phase of this pipeline. As explained more in detail below, thanks to the abstraction between the controller and the robot provided by the YARP interfaces, the pipeline includes few intermediate steps in addition to the stages defined by MBD.

The first stage of MBD is plant modeling. For controllers applications, the plant is typically composed of the robot and the environment where it operates. In our case, the model of iCub is represented by an URDF file, which stores its kinematic and dynamic properties. The model of the robot is generated semi-automatically from its computer-aided design, solution that allows obtaining a very detailed description of the robot. For what concern the environment, we use the default empty world provided by the physic engine running inside the simulator. The same applies to the interaction between the robot and the environment.

The implementation of the remaining stages of MBD is illustrated in Figure 4. The first row shows the prototyping phase of the pipeline and the second one shows the deploying phase. Referring to the figure, the depicted steps serve as follows:

This first step implements the controller prototyping stage of MBD. The controller is designed in Simulink using the default system blocks and the blocks provided by WHOLE-BODY TOOLBOX. In this case, when the user drops a block in the model, the S-function contained in BlockInformation loads the plugin library, and, using the factory method, it allocates the object that implements its logic.

When the controller is ready, it can be executed on the simulated robot model. WHOLE-BODY TOOLBOX provides a block for interfacing with Gazebo, synchronizing it with the simulation loop running in Simulink. This step provides the means of the system simulation stage.

In this additional third step, the control designer has the possibility to connect the controller, still running in Simulink from an external machine, to the real robot. Since the controller now needs to run in a real-time setting, the block used to interface with the simulator is substituted with a block that enforces the simulation loop to be synchronized with the real clock. Measurements and reference signals are gathered and streamed in real time.

Reached this point, the controller is already functional on both the simulated and the real robot. The last controller deployment stage starts with step 4. Exploiting the capabilities of Simulink Coder, the oriented graph visually created in Simulink is translated to an automatically generated C++ class. In our software architecture, Simulink Coder is handled as another engine (as illustrated in Figure 2), and a different implementation of the BlockInformation interface that abstracts the engine is used. A very important detail of this process is that the logic implemented by the WHOLE-BODY TOOLBOX blocks is not inlined in the autogenerated class. In fact, analogously to the behavior of any engine supported by BlockInformation, the plugin-based factory pattern is used. This means that the autogenerated C++ class loads the same plugin containing the logic of the robotic blocks that was used in the Simulink engine. Firstly, this helps to keep the behavior of the controllers running in different engines aligned. Secondly, assuming a constant controller graph, it simplifies the delivery of updates and fixes of the logic of the WHOLE-BODY TOOLBOX blocks. In fact, updated blocks can be deployed to the target platform by only distributing an updated plugin library, removing the need to regenerate the sources and rebuild the application. As last comment, it is worth noting that once the class has been generated and compiled, the presence of Simulink is no longer necessary.

This step in equivalent to step 2. In this case, though, the automatically generated controller is executed on the simulated robot.

Similarly, this step is equivalent to step 3 with the automatically generated controller.

The real deployment to the target platform is represented by this last step. Until now, the controller always ran from the external machine, communicating to the real robot through the network, exploiting the YARP observer pattern.³¹ The automatically generated class of the controller and the BlockInformation plugins are now compiled (or cross-compiled) for the onboard machine of the robot and, lastly, deployed. The comments about the choice of the plugin-based factory pattern of step 4 are even more central reached this last stage.

Figure 4.

Overview of the pipeline implementing model-based design. The prototyping phase, in the first row, assumes the availability of a model of the robot. In step 1, a Simulink® model of the controller is created. In step 2, the controller is tested in the Gazebo simulator using the robot model. In step 3, the same controller used in simulation is tested on the real robot, leveraging the robot transparency provided by exploiting the same YARP interfaces. All the computations of this phase are executed from an external machine running Simulink. The communication with the real robot is achieved through the YARP middleware. The second row illustrates the deploying phase. Exploiting Simulink® Coder™, in step 4, C++ code is automatically generated from the Simulink controller. Steps 5 and 6 perform the same tests of the previous phase from the same external machine, respectively, on the simulated and the real robot. This time, though, the autogenerated controller is executed. Eventually, in step 7, the controller is deployed to the computer in the robot head and runs stand-alone.

In this example, the controlled robots—simulated and real—refer to the same kinematic structure. One may wonder which modifications are necessary in this new architecture to run the controller on a robot endowed with a different number of degrees of freedom. One of the new features of WHOLE-BODY TOOLBOX is the presence of a configuration block, where it is possible to specify runtime information such as the name of the URDF model, the names of the controlled joints, and the name of the robot used to set up the YARP context. Excluding edge cases, this is enough to make controllers independent from the robot.

Limitations and future work

BlockFactory

The dataflow framework BlockInformation represents, as described in the previous sections, the abstraction layers between engines and black-box functions, supplied, for example, by plugins such as WHOLE-BODY TOOLBOX. This means that BlockInformation is responsible for exposing blocks in such a way that they can be properly configured by the solvers included in the engines. Currently, it only supports engines that provide discrete solvers with fixed step. This is the only requirement for models that have to be executed on a real-time system. However, BlockInformation was born as a generic dataflow framework and, when the deployment is not the final target, it should provide compatibility with continuous solvers which typically need to operate on the derivatives of the block state.

In its current state, the block interface is modeled to be a stateless system. The engine can only trigger the evolution of a block state by calling its output method since they are akin to instantaneous functions. However, stateful blocks can be extremely convenient in some use cases. Indeed, WHOLE-BODY TOOLBOX already contains blocks that hold an internal state, but it is hidden inside the implementation. One of the consequences of the presence of this hidden state is that blocks that need to know the step size cannot gather it directly from the engine, and this information must be passed as a parameter. This behavior cannot be very intuitive for the end user. Furthermore, is it more error prone since every time the user changes the step size, also the parameters of all blocks requiring it must be updated accordingly. This would not be necessary if the blocks would be modeled in such a way to expose their hidden state and rely on the engine features to address this shortcoming.

The FMI²⁵ represents a common standard as an alternative to the provided interfaces. Instead of a complete substitution, though, being able to expose blocks in their counterparts called functional mock-up units can be a valuable addition. This would open the interoperability with a plethora of tools that already support FMI, improving the integration of the models designed with BlockInformation in complex co-simulation environments.

Whole-body toolbox

WHOLE-BODY TOOLBOX currently grounds the interfacing with robots on the YARP middleware, and we are aware that there are not many existing YARP-based robots. Despite implementing the YARP interfaces for a new platform is not an insuperable task, it might limit the applicability of this pipeline. Going in this direction, a native implementation of the more common ROS middleware would enlarge the adoption of the proposed tools. A proof-of-concept of an ROS plugin implementing its publisher–subscriber pattern is already available.⁴⁴ On the same line, allowing to install the WHOLE-BODY TOOLBOX without its YARP component would be another possible improvement. In fact, the majority of the blocks are middleware-agnostic, and they could be already used in systems without any middleware installed. For instance, many use cases might benefit from the included algorithms for rigid-body dynamics.

The current support of simulating a kinematic structure consists of a co-simulation setup between Simulink and Gazebo that communicate through YARP messages, thanks to the Gazebo YARP Plugins. This entire system worked well for us in the past; however, its use is not as straightforward as it could be. In fact, to obtain a correct synchronization between the two simulators, all the components of this system should be started passing extra options. To simplify this process, it would be beneficial embedding the physic simulator inside a new block, treating it as a regular node of the graph. In this way, the synchronization could be greatly simplified taking advantage of the information available from being executed as part of the computational graph. This would also enable to execute headless simulations and allow us to open the graphical user interface only if visual feedback is required.

A limitation of WHOLE-BODY TOOLBOX that might restrict its applicability to generic tasks is the lack of maturity of the robotic perception stack. The main scope of our applications is balancing and locomotion; therefore, we always ignored perception and focused mainly on dynamics. Our controllers currently operate only on flat terrain, where perception is not required. However, creating new specific blocks to retrieve sensory data would be straightforward. An improved perception can then allow controllers to handle more structured scenarios that can be already simulated in Gazebo inserting the robot model into a structured world.

In the long run, we would like to add the support of existing machine-learning frameworks to embed networks and function approximators into our robotic controllers. Furthermore, we are planning to introduce the possibility to export controllers with an interface that exposes a set of parameters which would allow applying reinforcement learning algorithms.

Pipeline

The description of the pipeline reported in the previous section offers a general overview of its functionalities. However, it hides few caveats which might not be straightforward. In step 2, obtaining a model that can be effectively actuated in Gazebo requires tuning its PID gains. Finding a proper configuration is not straightforward and many iterations are necessary. Furthermore, this process has to be repeated again in step 3, when the controller is executed on the real robot. Once the right gains have been properly found, they can be reused in steps 5 and 6. However, these low-level configurations are not strictly specific to this pipeline. In fact, they are related to the YARP implementation of the robot, and these parameters are meant to be abstracted by YARP interfaces.

Similarly, it is interesting to analyze the factors that might differ between running the autogenerated controller from the external machine and from the onboard device of the robot. The communication between the controller and the robot—typically consisting of sensor measurements and references—are mediated in both cases by the transport layer handled by YARP. In the first case, since the controller is running in an external machine, the exchange of data occurs through the network transport layer. This type of data transfer introduces overhead and delays that might affect the performance of the controller. Deploying the controller to the onboard machine provides a great opportunity to mitigate this problem. However, this is not exempt to side effects. In fact, controllers are very sensitive to time delays and dealing with them is yet an open problem in many applications. Assuming that the same gains can be applied might hold surprises. In our experience though, controllers did not need any tuning. In any case, moving the computation of fundamental tasks such as motion control as close as possible to the actuators offers a tremendous possibility to enhance system robustness. Furthermore, the deployed controller is an optimized version of the one executed in Simulink. If the rate of the controller is slower than the rate of the robot measurements and the actuation bandwidth, the gain of speed might allow increasing the controller frequency, which is typically related to better performances.

A current limitation of the autogeneration process is how controller parameters stored in Simulink are handled. With the current BlockInformation version, due to how the code is generated, accessing them from the code is not very intuitive. As a consequence, it is not yet possible to obtain an autogenerated controller that, without the need of regenerating the sources, can be used on different YARP-based robots.

Conclusions

In this article, we presented a rapid prototyping and deployment architecture for robotic controllers based on the principles of MBD. The architecture is composed of a framework and a pipeline.

Developing and maintaining a controller in pure C++ is typically extremely demanding, and even minor architectural changes might require a considerable effort. In light of the fast prototyping aims, developing controllers using visual tools and then automatically generate optimized C++ code represents a great speedup.

As first component of the framework, we presented the actor-oriented tool BlockInformation. It abstracts generic algorithms and allows embedding them in generic applications modeled as directed graphs. The black-box functions that BlockInformation exposes are modeled as blocks with a predefined interface and are stored in collections as shared libraries. These libraries can then be loaded from third-party software that implement the MBD pattern. Among all the available possibilities, BlockInformation allows interfacing with Simulink and Simulink Coder. However, it streamlines the extension to other frameworks by providing a second interface that abstracts the engines from the block implementations.

Different kinds of robotic controllers are typically based on a limited set of elemental functionalities, and complex logic can be achieved by their composition. In this work, as second component of the framework, we presented WHOLE-BODY TOOLBOX, a collection of black-box functions for robotics representing the building elements of generic robotic controllers. This toolbox wraps a number of existing open-source projects belonging to the categories of robotic middlewares, rigid-body dynamic libraries, and mathematical optimization tools.

These two projects serve as the primary components of the proposed pipeline to rapidly prototype and deploy robotic controllers. In particular, the presented pipeline implements the rapid prototyping capability—idiomatic feature of MBD—by interfacing with the Simulink engine. The rapid deployment, instead, is achieved exploiting the automatic code generation support provided by Simulink Coder. We explained step by step how the entire process works, detailing how the stages of MBD have been implemented.

Ultimately, we explained the shortcomings of the current status of both the components of the presented framework and the resulting pipeline, and our plans to address them. The present condition of these projects is the outcome of many years of development, during which the architecture often changed and gave us the possibility to learn from our mistakes. Despite these continuous changes, a big effort has been spent to keep the experience of the control engineers that use this framework as consistent as possible. As attempted in the previous papers, we tried to be as critic as possible to our choices, being aware that the presented pipeline still has a big room of improvements.

To conclude, we would like to remark that the development of all the presented tools followed from their beginning an open-source and community-driven approach. From one hand, we could have never achieved the current development status and our results if we could not interface with existing open-source software such as middlewares, simulators, and libraries. We are grateful to the entire robotic community to provide and maintain them over time. From the other hand, collaborations with other research institutes—mainly belonging to the community built around the iCub humanoid robot—helped us to improve the robustness of the entire framework using it within different contexts. A great contribution, as an example, regards the interfacing with MATLAB. In fact, due to licensing limitation, we cannot test the pipeline thoroughly with many versions, and also the application of continuous integration pipelines presents many restrictions. A wider user base with diverse setup helped us debugging problems we would probably have never encountered.

Footnotes

Authors’ note

The content of this publication is the sole responsibility of the authors. The European Commission or its services cannot be held responsible for any use that may be made of the information it contains.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This was financially supported by the European Union’s Horizon 2020 research and innovation program under grant agreement no. 731540 (An.Dy).

ORCID iD

Diego Ferigo

References

Schlegel

Steck

Brugali

, et al. Design abstraction and processes in robotics: from code-driven to model-driven engineering. In Ando

Balakirsky

Hemker

, et al. (eds) Simulation, modeling, and programming for autonomous robots, vol. 6472. Berlin, Heidelberg: Springer, 2010. pp. 324–335.

Smith

Prabhu

Friedman

. Best practices for establishing a model-based design culture. SAE Technical Paper 2007-01-0777. DOI: 10.4271/2007-01-0777.

Lennon

. Model-based design for mechatronic systems. Electron World 2008; 114(1865): 23–26.

Brugali

. Model-driven software engineering in robotics: models are designed to use the relevant things, thereby reducing the complexity and cost in the field of robotics. IEEE Robot Autom Mag 2015; 22(3): 155–166.

Johnston

Hanna

JRP

Millar

. Advances in dataflow programming languages. ACM Comput Surv 2004; 36(1): 1–34.

Agha

Mason

Smith

, et al. A foundation for actor computation. J Funct Program 1997; 7(1): 1–72.

Lee

Neuendorffer

Wirthlin

. Actor-oriented design of embedded hardware and software systems. J Circuit Syst Comput 2003; 12(03): 231–260.

Lee

Messerschmitt

. Synchronous data flow. Proc IEEE 1987; 75(9): 1235–1245.

Simulink—simulation and model-based design. Available at: https://www.mathworks.com/products/simulink.html.

10.

Xcos. Available at: https://www.scilab.org/software/xcos.

11.

Tiller

. Introduction to physical modeling with Modelica. Berlin: Springer Science & Business Media, 2012.

12.

LabVIEW 2018 —National instruments. Available at: http://www.ni.com/en-us/shop/labview/labview-details.html.

13.

Mens

Wermelinger

. Separation of concerns for software evolution. J Softw Maint Evol Res Pract 2002; 14(5): 311–315.

14.

Vanthienen

Cher

Bruyninckx

. The 5c-based architectural composition pattern: lessons learned from re-developing the iTaSC framework for constraint-based robot programming. J Softw Eng Robot 2014; 5(1): 17–35.

15.

Simulink Coder. Available at: https://www.mathworks.com/products/simulink-coder.html.

16.

Bruyninckx

Klotzbücher

Hochgeschwender

, et al. The BRICS component model: a model-based development paradigm for complex robotics software systems. In: Proceedings of the 28th annual ACM symposium on applied computing—SAC ‘13. Coimbra, Portugal, March 2013, p. 1758. New York: ACM Press.

17.

Romano

Traversaro

Pucci

, et al. A whole-body software abstraction layer for control design of free-floating mechanical systems. J Softw Eng Robot 2017; 8(1): 89–103.

18.

Tedrake

Team

. Drake: a planning, control, and analysis toolbox for nonlinear dynamical systems. Available at: https://drake.mit.edu/ (2016).

19.

OpenModelica. Available at: https://openmodelica.org/.

20.

Systemes®

. Dymola. Available at: https://www.3ds.com/products-services/catia/products/dymola/.

21.

Mansard

Stasse

Evrard

, et al. A versatile generalized inverted kinematics implementation for collaborative working humanoid robots: the stack of tasks. In: 2009 International conference on advanced robotics. Munich, Germany, 22–26 June 2009.

22.

Mallet

Pasteur

Herrb

, et al. GenoM3: building middleware-independent robotic components. In: 2010 IEEE international conference on robotics and automation. Anchorage, AK, USA, 3–7 May 2010, pp. 4627–4632. IEEE.

23.

Ando

Suehiro

Kotoku

. A software platform for component based RT-system development: OpenRTM-Aist. In Carpin

Noda

Pagello

, et al. (eds) Simulation, modeling, and programming for autonomous robots. Lecture notes in computer science, pp. 87–98. Berlin, Heidelberg: Springer.

24.

Bruyninckx

Soetens

Koninckx

. The real-time motion control core of the Orocos project. In: 2003 IEEE international conference on robotics and automation (Cat. No.03CH37422), vol. 2, Taipei, 14–19 September 2003. pp. 2766–2771. IEEE.

25.

Functional Mock-up Interface. Available at: https://fmi-standard.org/.

26.

Corke

. Robotics, vision and control. Berlin: Springer, 2017.

27.

Robotics system toolbox. Available at: https://it.mathworks.com/products/robotics.html.

28.

Quigley

Gerkey

Conley

, et al. ROS: an open-source robot operating system. In: ICRA workshop on open source software, Kobe, Japan, May 2009.

29.

Blockfactory: a tiny framework for wrapping algorithms for dataflow programming. Available at: https://github.com/robotology/blockfactory (2019).

30.

Whole-body toolbox: a Simulink toolbox for rapid prototyping of whole body robot Controllers. Available at: https://github.com/robotology/wb-toolbox (2019).

31.

Gamma

Helm

Johnson

, et al. Design patterns: elements of reusable object-oriented software. Addison-Wesley professional computing series. Reading, MA: Addison-Wesley, 1995.

32.

Metta

Fitzpatrick

Natale

. YARP: yet another robot platform. Int J Adv Robot Syst 2006; 3(1): 8.

33.

Metta

Natale

Nori

, et al. The iCub humanoid robot: an open-systems platform for research in cognitive development. Neural Netw 2010; 23(8–9): 1125–1134.

34.

Ferrati

Settimi

Muratore

, et al. The walk-man robot software architecture. Front Robot AI 2016; 3: 25.

35.

Whole-Body Control—IEEE Robotics and Automation Society. Available at: https://www.ieee-ras.org/whole-body-control.

36.

Nori

Traversaro

Eljaik

, et al. iCub whole-body control through force regulation on rigid non-coplanar contacts. Front Robot AI 2015; 2: 6.

37.

URDF. Available at: http://www.ros.org/wiki/urdf.

38.

Koenig

Howard

. Design and use paradigms for gazebo, an open-source multi-robot simulator. In: 2004 IEEE/RSJ international conference on intelligent robots and systems (IROS) (IEEE Cat. No.04CH37566), vol. 3. Sendai, Japan, 28 September–2 October 2004, pp. 2149–2154. IEEE.

39.

Mingo Hoffman

Traversaro

Rocchi

, et al. Yarp based plugins for gazebo simulator. In Hodicky

(ed) Modelling and simulation for autonomous systems. Cham: Springer International Publishing, vol. 8906, 2014, pp. 333–346.

40.

Pattacini

Nori

Natale

, et al. An experimental evaluation of a novel minimum-jerk Cartesian controller for humanoid robots. In: 2010 IEEE/RSJ international conference on intelligent robots and systems. Taipei, 18–22 October 2010, pp. 1668–1674. IEEE.

41.

Ferreau

Kirches

Potschka

, et al. qpOASES: a parametric active-set algorithm for quadratic programming. Math Program Comput 2014; 6(4): 327–363.

42.

Nava

Ferigo

Pucci

. Exploiting friction in torque controlled humanoid robots. In: 2018 IEEE/RSJ international conference on intelligent robots and systems (IROS), Madrid, Spain, 1–5 October 2018.

43.

Pucci

Romano

Traversaro

, et al. Highly dynamic balancing via force control. In: 2016 IEEE-RAS 16th international conference on humanoid robots (humanoids). Cancun, Mexico, 15–17 November 2016, pp. 141–141. IEEE.

44.

blockfactory-demo-ros: A proof-of-concept plugin of a ROS toolbox for dataflow programming based on blockfactory. Available at: https://github.com/diegoferigo/blockfactory-demo-ros (2019).

A generic synchronous dataflow architecture to rapidly prototype and deploy robot controllers

Abstract

Keywords

Introduction

Background

Related software

Terminology

Framework software architecture

Factory pattern and plugin libraries

BlockFactory

Whole-body toolbox

The pipeline

Limitations and future work

BlockFactory

Whole-body toolbox

Pipeline

Conclusions

Footnotes

Authors’ note

Declaration of conflicting interests

Funding

ORCID iD

References