Sage Journals: Discover world-class research

Abstract

Autonomous navigation of cyborg insects in complex environments remains a challenging issue. Cyborg insects, which combine biological organisms with electronic components, offer a unique approach to tackle such challenges. This study presents a biohybrid behavior-based navigation (BIOBBN) system that enables cyborg cockroaches to navigate complex environments autonomously. Two navigation algorithms were developed: reach-avoid navigation for less complex environments and adaptive reach-avoid navigation for more challenging scenarios. This algorithm, especially the second one, leveraged the cockroaches’ natural behaviors, such as wall-following and climbing, to navigate around and over obstacles. Experiments in simulated environments, including sand and rock-covered surfaces, demonstrate the effectiveness of the BIOBBN system in enabling cyborg cockroaches to navigate and reach target locations. The denser second scenario required more time due to increased obstacle avoidance and natural climbing behavior. Overall performance was promising, highlighting the potential of biohybrid navigation for autonomous cyborg insects in navigating complex environments.

Introduction

The ability of mobile robots to navigate and avoid obstacles is crucial, especially in environments that are unknown and unstructured. Integrating and processing commonly used obstacle avoidance sensors such as ultrasonic, infrared, camera, and Lidar into mobile robots is crucial for safe, collision-free, and efficient navigation.^1–6 Fortunately, some sensors are easily integrated and attached to mobile robots without worrying about size, weight, complex processing algorithms, and power requirements. To address the challenges of autonomous navigation in unknown environments, researchers have developed mobile robots that leverage their size and payload capacity to accommodate various sensors and advanced navigation algorithms which enable the robots to navigate their surroundings autonomously.^7–11 Common navigation controls applied in mobile robots include reinforcement learning/neural network/deep learning, fuzzy logic, potential field method, and behavior-based navigation.^10,12–15

Behavior-based navigation (BBN) offers a powerful method for autonomous navigation in mobile robots, particularly in unknown environments.^11,16,17 BBN excelled in multi-agent scenarios where robots needed to reach goals, avoid obstacles, and maintain formations.¹⁷ BBN was inspired by animal behavior that decomposes complex tasks into simpler behaviors.^18,19 These behaviors are then integrated and coordinated to accomplish a desired overall objective, such as obstacle avoidance and goal attainment. The selection or combination of active behavior can be enhanced through additional methods like machine learning/deep learning or fuzzy logic.^11,20–24 Rather than following a rigid and intricate navigation/control strategy, the robot adapts to its environment in real time by utilizing sensory information and its internal state. BBN can be constrained by imprecise or incomplete knowledge of system and environmental factors, as well as difficulties in accurately representing the environment and robot location due to sensor data errors.^18,25 However, the approach prioritizes pre-defined reactive behaviors, making it computationally less demanding than methods like vision-based navigation or simultaneous localization and mapping (SLAM), which can be a constraint for resource-constrained robots. Consequently, this navigation method is well-suited for centimeter-scale robots that face constraints in size, computational resources, and power consumption.

Miniaturizing mobile robots to insect scale (centimeter scale) and integrating them with various sensors and complex algorithms poses a significant challenge. Insects innately possess capabilities for navigating through complex environments, including mechanisms for natural obstacle detection and avoidance. Biohybrid or cyborg insect robots leverage these innate abilities by combining living insects with wireless backpack stimulators or biomechanical interface technologies to control movement.^26–32 These cyborg systems employ the insect’s entire body for locomotion rather than requiring the development of complex mechatronic components, thus eliminating the need to completely artificially construct the robotic system.^33–36 The lightweight wireless backpacks used to direct cyborg insect navigation consume minimal power compared to artificial robots, which demand significantly higher power outputs to operate actuators such as hydraulic, pneumatic, or motors.^33,35,36

Cockroaches and beetles are the most commonly utilized as ground terrestrial cyborg insects due to their size and terrestrial locomotion.^{26,30,32,37–44} Cyborg insects could be steered manually along a predetermined path, enabling them to avoid simple obstacles.^27,45–47 This was achieved by stimulating their antennae and cerci to control the steering and forward motion.²⁷ Autonomous navigation incorporating feedback from a Kinect camera had been applied to control the cyborg insect to follow a predetermined path using only steering control in a free obstacle environment.⁴⁸ Researchers successfully developed a navigation system using proportional control to steer the cyborg insects autonomously along a predetermined path.^49,50

Most previous studies in autonomous navigation for cyborg insects were implemented on flat surfaces without obstacles.^48–50 Autonomous navigation in an unknown environment has been demonstrated by incorporating obstacle negotiation and onboard human detection.²⁹ The navigation performance was evaluated against obstacles without sharp corners like low barriers and tall walls of known height. The obstacle negotiation was based on its innate locomotion and natural behaviors. The navigation attempted to retain and cooperate with these abilities measured using an inertial measurement unit (IMU) and motion capture. Researchers have developed a cyborg cockroach for autonomous pipeline navigation.²⁸ Onboard sensors including a camera and IMU were applied to enable autonomous navigation control in confined dark pipelines via image processing and orientation sensing. Straight pipeline navigation was demonstrated successfully, incorporating wall-following behavior. However, more complex environments including obstacles remained a challenge for autonomous navigation. Commonly, navigation relied on external motion tracking to navigate the cyborg autonomously.

Cockroaches possess the innate ability to avoid obstacles through tactile sensing via their sensitive antennae making them suitable utilization in search and rescue missions. They can follow obstacle contours in their environment naturally (wall-following behavior). However, they often approach and remain stationary for indefinite periods, especially in the cornered area.^51–54 Therefore, to adequately function as cyborg insects in complicated environments, their natural behavior to stop for indefinite periods in such areas needs to be addressed. Moreover, integrating it with the commonly used obstacle-detection sensors such as ultrasonic, infrared, and lidar sensors poses a great challenge due to size, weight, and power consumption. Therefore, implementing an onboard obstacle avoidance system would enable it to navigate around corners and constrained spaces, avoiding prolonged halts.

Developing autonomous navigation for cyborg insects including obstacle detection and avoidance has not been explored by researchers extensively, especially in unknown and unstructured environments. Applying obstacle detection with low power consumption, lightweight, and small size is challenging. Our previous study selected a Time-of-Flight (ToF) laser-ranging module for obstacle distance measurement due to its lightweight and low power consumption.⁵² Cockroaches tend to approach obstacle areas with acute angles and stay in that area for a certain amount of time. By implementing the simple feedback control based on the onboard sensors, that is, IMU and ToF sensors, autonomous navigation in the presence of sharp-angled obstacles has been demonstrated in an unstructured environment with a flat surface. The cyborg insect avoided the obstacles and escaped from the sharp corner areas without stopping or becoming trapped. Although the previous work could navigate the cockroach to avoid the obstacles and escape the unstructured area, the goal-based navigation could not be attained because there was no measurement and feedback on the cockroach’s position.⁵² This article overcomes this challenge through the implementation of a 3D motion capture system.

In this study, onboard sensors (IMU and ToF sensors) were implemented to measure the cyborg cockroach motion and obstacle distance. The cyborg position was measured using an offboard sensor from motion capture. Previous BBN methods in mobile robots relied on pre-programmed responses triggered by sensor data. Cyborg insects possess inherent biological advantages like agile locomotion and obstacle avoidance. To leverage these capabilities, BBN was augmented with the free-walking motion (FWM) of a cyborg insect called biohybrid behavior-based navigation (BIOBBN). It allowed the navigation system to work alongside the insect’s natural behaviors, benefiting from both the pre-programmed responses and the insect’s inherent locomotion. While BBN relies on a continuously active behavior to constantly issue commands guiding the robot away from obstacles and toward its goal, the proposed BIOBBN approach eliminates this need for persistent behavioral commands driving movement. This is achieved through BIOBBN’s integration of FWM which reduces the number of stimulations and harnesses the cyborg insect’s innate locomotion, allowing it to navigate autonomously between the intervals of stimulation commands.

Two sets of BIOBBN navigation algorithms were designed to address varying obstacle densities. Reach-avoid navigation was tailored for low-density environments, prioritizing goal-reaching with minimal obstacle avoidance. In contrast, adaptive reach-avoid navigation was designed for high-density environments, incorporating obstacle avoidance and extended obstacle avoidance/wall-following techniques. Both navigation systems were tested in complex environments with challenging obstacles like small rocks and soil grains, deviating from flat surfaces. Experiments were conducted to measure the impact of this rough terrain on electrical stimulation input, assessing turning and forward responses on both flat and sandy/rocky surfaces. The first navigation system could utilize a bulkier and heavier electronic backpack, while the second needed a more compact, lighter one to accommodate its complex terrain navigation. The results demonstrated successful autonomous navigation, guiding cockroaches to target areas while avoiding obstacles and preventing entrapment in complex terrain features.

Material and Methods

Cyborg insect

Madagascar hissing cockroaches measuring from 6 to 7 cm in length were selected for the cyborg insect platform. To anesthetize them, carbon dioxide was used to induce anesthesia in an airtight container for 3 min. This method could put the cockroach to sleep for around 10 to 15 min, allowing for surgical implantation of electrodes into sensory organs and thorax. Silver wires with a diameter of 0.127 mm (A-M systems) were implanted on both antennae, both cerci, and the thorax using a methodology as presented in previous studies.^27,52 Following the surgical electrode implantation, the cockroaches were put back into containers for a minimum 24-hour recovery period before experiments. The utilization of cyborg insects in this study was approved by the animal experiment committee at Osaka University (approval number, 2023–5-0).

Wireless backpack stimulator

To address the challenge of cyborg insect autonomous navigation in unknown and unstructured environments containing complex obstacles and granular small debris/rocks, two miniature wireless backpacks were developed as summarized in Supplementary Table S1. They incorporated various onboard sensors but still maintained a small form factor. Both backpacks have dimensions of 3 cm in length and 2 cm in width. A 40 mAh lithium polymer battery was selected to power the backpack. The battery could power all onboard sensors and microcontrollers for approximately 19 min (the thermal image sensor was turned off), providing a suitable operational duration for the navigation experiments. By maintaining a small size yet integrating multiple sensing modalities, the developed backpacks potentially enable the autonomous navigation capabilities needed in complex environments as shown in Figure 1.

FIG. 1.

Proposed two wireless backpacks for autonomous navigation of cyborg insects in two different complex environments. (A) First wireless backpack stimulator with three ToF, and 9 Degrees of Freedom (DOF) IMU sensors. (B) First wireless backpack mounted on the cockroach (C) Cyborg cockroach with 4-mm reflective markers for external localization with motion capture system. (D) Second wireless backpack stimulator with three ToF, and 9-DOF IMU sensors. (E) The second wireless backpack is mounted on the cockroach (F) Cyborg cockroach with 4-mm reflective markers attached to the second backpack. The second backpack, weighing only 2.9 g including battery, is significantly lighter and more compact than the first backpack (5.66 g). Its reduced size and weight make it better suited for navigating more complex environments with denser obstacles and more challenging terrain. IMU, inertial measurement unit; ToF, Time-of-Flight.

Both backpacks utilized an ultra-low-power microcontroller from Insight SiP to support the computational and wireless communication requirements within the limited size. ISP1807 32-bit ARM Cortex-M0+ MCU (Insight SiP, France) was selected as the main computing unit. Communication with the PC was enabled via an integrated BLE 5.0 module. Environmental monitoring was conducted using the SHT40-AD1B-R2 relative humidity and temperature sensor from Sensirion AG. ToF laser-ranging module (VL53L0X from STMicroelectronics) was selected due to its compact size and low-power power consumption (20 mW maximum) to facilitate onboard obstacle detection. It is capable of measuring distances from 2 cm to 120 cm. Three VL53L0X ToF sensors were placed on the front, right, and left sides of the wireless backpack to create an obstacle-detection system for the cyborg cockroach.

Thermal imaging was provided by the HTPA32x32d infrared thermopile from Heimann Sensor. It is attached solely to the first backpack. It can generate a 32 × 32 pixel temperature gradient image. Previous studies have demonstrated the implementation of thermal image sensors for onboard human detection augmented with machine learning.^29,52

ICM-20948 9-axis IMU from InvenSense was utilized to measure the motion of the cyborg insect. It consisted of accelerometer, gyroscope, and magnetometer, supplying kinematic data to enable feedback control and detect the FWM of cyborg insect. All sensors were integrated into the backpack assembly using surface-mount printed circuit boards. The wireless backpack generated a biphasic ±3.3V 50 Hz square wave for electrical stimulations. Three electrodes were implanted in the left and right antennae and cerci to conduct the biphasic signal. A single common ground electrode was inserted approximately 3 mm deep near the midline of the thorax. To measure the cockroach position with offboard motion capture, four 4-mm retroreflective markers mounted on a 3D-printed linkage were rigidly attached to the backpack as shown in Figure 2C and F. Combining onboard sensing with offboard position tracking enabled autonomous navigation capabilities in complex simulated environments as depicted in Figure 2A and B.

FIG. 2.

Two experimental test areas were designed to test the autonomous navigation capabilities of cyborg insects. In the first scenario, testbed (A) featured a granular terrain with coarse and small rocks, simulating a challenging environment for small-scale, fully artificial mobile robots. Three V-shaped obstacles with small corner angles (60° and 70°) were placed on the testbed, along with a rectangular obstacle near the goal area. In the second scenario, testbed (B) presented a more complex and denser obstacle having four V-shaped, and long obstacles, designed to be more difficult and challenging for the cyborg insects to navigate. Irregularly shaped and sized stones were added to the terrain, and the goal area was enclosed by obstacles with a 7.5 cm height, except for an entrance gate on the back. The objective for the cyborg insects was to navigate autonomously through these complex, unknown, and unstructured environments. This involved avoiding obstacles, escaping obstacle corner areas, and successfully reaching the goal area (square area with a green circle) from the starting point. Small rocks and soil grains ranging from 1 to 19 mm were scattered throughout both scenarios.

The initial electronic backpack, incorporating a thermal image sensor and flexible flat cables, was heavier and bulkier, limiting its locomotion performance in complex environments. Designed for less complex environments, the first backpack, as shown in Figure 2A, was utilized in an initial experiment. This experiment highlighted the need for improvements in both hardware and software components to enable navigation in more complex scenarios. To address these limitations, the backpack design was refined, resulting in a significantly lighter and more compact second iteration, weighing only 2.9 g compared to the initial 5.66 g. This reduced weight and size enabled the cyborg insect to navigate more effectively through complex terrain with higher-density obstacles, as depicted in Figure 2B.

Experimental setup

An offboard motion capture system was employed to provide accurate position measurement and feedback on the cyborg insect. Vicon Vantage motion tracking system, consisting of four Vantage V8 and four Vantage V16 cameras, was used to measure the cyborg insect’s two-dimensional movements. It operated continuously during autonomous navigation at a frame rate of 100 Hz. Position tracking was measured through four 4-mm retroreflective hemispherical markers attached to the cyborg’s wireless backpack. Due to physical constraints of small size and lightweight, the motion capture system was not suitable for onboard use on the cyborg insect platform. However, it provided real-time position and heading feedback for reaching goal behavior navigation and served as ground truth for evaluating autonomous navigation performance.

Experiments were conducted in two different simulated scenarios to test and validate the proposed autonomous navigation as seen in Figure 2A and B. Both scenarios employed complex obstacle courses to test the navigation capabilities of the insects. The first testbed included three V-shaped obstacles and a tall vertical obstacle near the goal. The second scenario featured a more intricate course with four V-shaped obstacles, longer and taller walls, increased amounts of rocks and sand, and randomly placed obstacles of varying shapes and sizes. Navigating this more challenging environment would demand exceptional agility from the cyborg insects. Based on the previous study, cockroaches tend to approach and remain stationary at corner junctions within such V-shaped obstacles, making it difficult enough for obstacle negotiation and autonomous navigation.⁵² Both scenarios utilized a testbed with a surface composed of granular soil and small rocks to simulate a more challenging environment.

A green circle area with a 7 cm radius represented the goal/target that must be reached by the cyborg insect. In the first scenario (Fig. 2A), cockroaches were positioned facing inward at the corner of the first obstacle, toward the arena’s goal; in the second scenario (Fig. 2B), cyborg insects were oriented in the opposite direction of the first scenario, facing away from their designated destination. Navigation performance was evaluated based on each cyborg insect’s ability to autonomously navigate the test arena, avoiding obstacles while reaching the goal without becoming trapped or stopped. This experimental testbed setup can be used to test and validate the proposed BIOBBN to resolve complex navigation challenges in unknown and unstructured environments.

To evaluate the performance and repeatability of the proposed navigation system, seven cyborg cockroaches were tested in 26 trials (N = 7, n = 26) within the first scenario (Fig. 2A) utilizing the first backpack. Between trials, each cockroach was allowed a rest period of 3 to 5 min. The second backpack, being lighter and more compact, was utilized in the second complicated scenario, employing the second algorithm, which incorporated three behaviors. The navigation system was evaluated and tested using eight cyborg insects (N = 8, n = 26). Considering the complexity of the second scenario, reaching the goal area will be deemed successful if the cyborg insect can climb to the top of the obstacle and maintain its position on the top of the goal wall without descending. The autonomous navigation performance will be evaluated using four parameters: total stimulation time applied to the sensory organs, navigation time, distance traveled, and final position deviation from the goal circular area (diameter: 14 cm). Navigation time refers to the time required for the proposed feedback navigation controller to guide the cyborg cockroaches around obstacles and to the designated goal area (xg, yg)/final position from their starting point. The distance traveled (Dt) by cyborg cockroaches under autonomous navigation can be calculated using Equation (1) based on the trajectory obtained from the motion capture system (xc, yc). The final position deviation is the distance between the cyborg’s final position and the goal area. $D t = \sum_{i = 0}^{n - 1} \sqrt{{(x c_{n + 1} - x c_{n})}^{2} + {(y c_{n + 1} - y c_{n})}^{2}}$ (1)

Biohybrid behavior-based navigation

Both scenarios (reach-avoid navigation and adaptive reach-avoid navigation) employed two primary behaviors: reaching the goal and avoiding obstacles. In the first scenario, with its less complex terrain, these behaviors were sufficient to guide the cockroach. However, the second scenario’s increased obstacle density and complexity necessitated modifications. The obstacle avoidance behavior was adapted into two distinct behaviors: avoiding obstacles and extended obstacle avoidance/wall-following.

Reaching goal behavior steered the cyborg insect to move toward the target/goal location where there was no obstacle near the insect. The feedback control stimulated the antennae and cerci of the cyborg insect based on the feedback signals from IMU and motion capture. Avoiding obstacle behavior and extended avoiding obstacle/wall-following behaviors allow the cyborg insect to detect and maneuver around any obstacles when an obstacle is detected in proximity based on the obstacle distance thresholds. When the obstacle has been avoided, it will resume the reaching goal behavior. Both scenarios utilized steering and forward motion control integrated with FWM.

Reaching goal behavior

Onboard sensing was provided by a 9-axis IMU to measure longitudinal acceleration for motion detection/FWM. Localization feedback was obtained from the Vicon motion tracking system, which measured the position (xc, yc) and heading $(ψ)$ of the cyborg insect using retroreflective markers affixed to its backpack (Supplementary Fig. S1). To navigate the cyborg insect toward the goal, the shortest distance (r) was calculated using its current position data (xc, yc) along with the fixed target/goal coordinate (xg, yg) as written in Equation (2). The desired heading angle $(ψ_{d})$ is calculated using the Cosine rule as expressed in Equation (3). $[\begin{matrix} δ x \\ δ y \end{matrix}] = [\begin{matrix} x g - x c \\ y g - y c \end{matrix}], r = \sqrt{δ x^{2} + δ y^{2}}$ (2) $ψ_{d} = \cos^{- 1} (\frac{δ y^{2} + r^{2} - δ x^{2}}{2 δ y r})$ (3)

The reaching goal behavior was designed to generate appropriate steering commands to guide the cyborg insect toward the goal until its orientation and trajectory gradually aligned toward the goal coordinates as shown in Figure 3A, B, and C. To determine the required turning angle, an angular difference $(δ ψ)$ between the real-time measured heading $(ψ)$ and the desired heading angle $(ψ_{d})$ toward the goal can be calculated using equation (4). $δ ψ = ψ - ψ_{d}$ (4)

FIG. 3.

Biohybrid behavior design for a cyborg insect incorporating FWM. (A) Steering stimulation strategy for reaching the goal. Forward motion strategy for reaching the goal (B) with cerci stimulation, (C) without cerci stimulation. (D) Steering stimulation strategy for avoiding obstacle. Forward motion strategy for avoiding obstacle (E) with cerci stimulation. (F) without cerci stimulation. For complex and dense environments in the second scenario (Fig. 2B), modified steering control was implemented using either right (G) or left antenna stimulation (H). An extended avoiding obstacle/wall-following behavior i.e., (I) and (J) was incorporated to navigate areas with long walls or obstacles. This behavior prioritized wall-following when long obstacles were encountered but transitioned to goal-seeking when clear paths became available. FWM was incorporated without providing electrical stimulation to reduce the number of stimuli required and leverage the insect’s natural and agile locomotion during autonomous navigation. FWM, free-walking motion.

Cockroaches primarily move in a longitudinal direction (forward motion). Consequently, only longitudinal acceleration $(a_{x})$ was measured to detect the cyborg insect’s movement. However, this measured acceleration is subject to noise, making it challenging to accurately calculate longitudinal velocity $(v_{x})$ . The noisy signal hindered the direct implementation of a method to distinguish between the cockroach’s stationary and walking states.⁴¹ To address this issue, a simple parameter $(a_{diffsum})$ was introduced to detect FWM, as defined in Equation (5).⁵² The threshold value $(a_{diffsum_t})$ for this parameter was determined experimentally, coinciding with the moment the cockroach began to move slowly. A sliding window of 10 data points was used, where $(a_{x (i + 1)})$ represents the current longitudinal acceleration and $(a_{x (i)})$ represents the previous longitudinal acceleration measured from the IMU. The selected values for the first and second backpacks are summarized in Table 1 and Table 2, respectively. When the FWM was detected and the cockroach heading angle was in the yellow area/cerci zone $(- δ ψ_{t} < δ ψ < δ ψ_{t})$ , stimulation was not provided to the cockroach’s antennae. FWM was incorporated into forward motion control under reaching goal behavior to reduce the number of simulations given to the antennae. If the cockroach’s heading angle was within the yellow zone and FWM was detected, no stimulation was applied to the left or right antennae. This behavior was implemented in both scenarios. An angular heading difference threshold $(δ ψ_{t})$ of ±12 degrees was chosen for the yellow area. This threshold could minimize heading errors toward the goal and reduce unnecessary stimulation to the antennae, preventing repeated unnecessary stimuli.

Table 1.

Control Threshold Parameters for Behavior-Based Navigation in the First Scenario (Fig. 2A)

Symbols	Definitions	Values	Units
$δ ψ_{t}$	Angular heading difference threshold between the measured heading angle (ψ) and the calculated desired angle (ψ_d)	±12	degree
$a_{diffsum_t}$	Difference summation of longitudinal acceleration threshold	0.7	cm/s²
$d_{F t}$	Obstacle front side distance threshold	19	cm
$d_{R t 1}$	First right-side obstacle distance threshold	5.5	cm
$d_{R t 2}$	Second right-side obstacle distance threshold	3	cm

Table 2.

Control Threshold Parameters for Behavior-Based Navigation in the Second Scenario (Fig. 2B)

Symbols	Definitions	Values	Units
$δ ψ_{t}$	Angular heading difference threshold between the measured heading angle (ψ) and the calculated desired angle (ψ_d)	±12	degree
$a_{diffsum_t}$	Difference summation of longitudinal acceleration threshold	0.5	cm/s²
$d_{F t 1}$	First obstacle front side distance threshold	7	cm
$d_{F t 2}$	Second obstacle front side distance threshold	10	cm
$d_{R t 1}$	First right-side obstacle distance threshold	7	cm
$d_{R t 2}$	Second right-side obstacle distance threshold	7.5	cm
$d_{L t 1}$	First left-side obstacle distance threshold	7	cm
$d_{L t 2}$	Second left-side obstacle distance threshold	7.5	cm

For steering control, the stimulations provided to the left antenna $(u_{L G})$ and right antenna $(u_{R G})$ under reaching goal behavior are written in Equations (6) and (7). Stimuli were given when the cockroach faced in the antennae zone as shown in Figure 3A. Cerci stimulation under reaching goal behavior $(u_{C G})$ was given when the angular difference $(δ ψ)$ was inside the cerci zone and the cockroach stopped/FWM was not detected $(α_{diffsum} < a_{diffsum_t})$ as written in Equation (8). The stimulation output commands were converted into a simple binary format to interface with the cyborg insect’s electrical stimulation control. An output value of 1 indicated that a stimulation signal was given to the specified sensory organ (left antenna, right antenna, or cerci). In contrast, an output of 0 meant that no stimulation signal from the navigation output was given. The inclusion of FWM in the proposed navigation allowed the cyborg’s natural behaviors to traverse complex terrain without interference from the given electrical stimulation. $a_{diffsum} = \sum_{i = 0}^{9} | a_{x (i + 1)} - a_{x (i)} |$ (5) $u_{L G} = {\begin{matrix} 1 \\ 0 \end{matrix} \begin{matrix} (δ ψ < - δ ψ_{threshold}) \\ otherwise \end{matrix}$ (6) $u_{R G} = {\begin{matrix} 1 \\ 0 \end{matrix} \begin{matrix} (δ ψ > δ ψ_{threshold}) \\ otherwise \end{matrix}$ (7) $u_{C G} = {\begin{matrix} 1 \\ 0 \end{matrix} \begin{matrix} (- δ ψ_{t} \leq δ ψ \leq δ ψ_{t}) \land (a_{diffsum} \leq a_{diffsum_t}) \\ otherwise \end{matrix}$ (8)

Avoiding obstacle behavior

The onboard sensors from the IMU and ToF distance sensors provided feedback to enable obstacle avoidance behavior. For the first scenario, obstacle distance measurements from the front $(d_{F})$ and right $(d_{R})$ of ToF sensors were applied to detect obstacles and initiate turning motions for steering control as shown in Figure 3D. The commanded stimulation to the right antenna $(u_{R O})$ under avoiding obstacle behavior for turning left to avoid obstacles based on the front and right distance readings is expressed in Equation (9). FWM was included in the behavior of avoiding obstacles to reduce the stimulation given to the cerci (Fig. 3D and E). For forward motion control, stimulation to the cerci ( $u_{C O}$ ) was provided when there was no obstacle in front of the cyborg, and the obstacle was on the right side of the cyborg (Fig. 3F). Additionally, the measured $a_{diffsum}$ is less than $a_{diffsum_t}$ (no FWM detected) as expressed in Equation (10). This strategy allowed the cockroach to avoid the obstacle utilizing FWM with minimum stimulation to the cerci. To avoid the V-shaped obstacle, which had a length of 15 to 21.5 cm, the first front distance threshold $(d_{F t})$ was set at 19 cm. The second distance thresholds were set at lower values as written in Table 1 to prevent collisions with obstacles from the side while maintaining forward motion. $u_{R O} = {\begin{matrix} 1 \\ 0 \end{matrix} \begin{matrix} (d_{F} < d_{F t}) \land (d_{R} < d_{R t 1}) \\ otherwise \end{matrix}$ (9) $u_{C O} = {\begin{matrix} 1 \\ 0 \end{matrix} \begin{matrix} (d_{F} > d_{F t}) \land (d_{R} < d_{R t 2}) \land (a_{diffsum} \leq a_{diffsum_t}) \\ otherwise \end{matrix}$ (10)

In the second scenario, due to the highly dense obstacle environment, the obstacle avoidance behavior was divided into two modified strategies: avoiding obstacles and extended obstacle avoidance/wall-following. The modified obstacle avoidance focused on avoiding all obstacles through turning motions, as illustrated in Figure 3G and H. The stimulations applied to the right and left antennae are defined in Equations (11) and (12). To navigate the dense obstacle environment in the second scenario, the distance threshold parameters were lowered. The specific values for the first distance thresholds are provided in Table 2, enabling the cockroach to maneuver around obstacles and enter the designated goal entrance from either the left or right side. $u_{R O} = {\begin{matrix} 1 \\ 0 \end{matrix} \begin{matrix} (d_{F} < d_{F t 1}) \land (d_{R} < d_{R t 1}) \\ otherwise \end{matrix}$ (11) $u_{L O} = {\begin{matrix} 1 \\ 0 \end{matrix} \begin{matrix} (d_{F} < d_{F t 1}) \land (d_{L} < d_{L t 1}) \\ otherwise \end{matrix}$ (12)

Extended avoiding obstacle behavior/wall-following behavior

Previous research studies have demonstrated that cockroaches exhibit a natural wall-following behavior, particularly in confined spaces. This instinct allows them to navigate complex environments by following walls or the edges of long and tall obstacles.^{41,42,55–57} The onboard IMU and ToF distance sensors provided feedback to enable wall-following behavior. Obstacle distance measurements from the front $(d_{F})$ , left $(d_{L})$ , and right $(d_{R})$ of the ToF sensors were applied to detect surrounding walls and initiate forward motion when the cyborg stopped as expressed in Equation (13). The second distance for the left $(d_{L t 2})$ and right $(d_{R t 2})$ sides was set at 7 cm, taking into account the spacing between long obstacles and the increased obstacle density in the second scenario as summarized in Table 2. $u_{C W} = {\begin{matrix} 1 \\ 0 \end{matrix} \begin{matrix} (d_{F} > d_{F t 2}) \land (d_{R} < d_{R t 2}) \land (d_{L} < d_{L t 2}) \land (a_{diffsum} \leq a_{diffsum_t}) \\ otherwise \end{matrix}$ (13)

The block diagrams illustrating the navigation methods for the first and second scenarios are presented in Supplementary Figure S2, and Figure 4, respectively. The first scenario focuses on reaching the goal and avoiding obstacles in a sparse obstacle environment, while the second scenario incorporates three primary behaviors in dense obstacles and challenging terrain. The final stimulation decisions during autonomous navigation to stimulate the left antenna $(u_{L})$ , right antenna $(u_{R})$ , and cerci $(u_{C})$ are written in Equations (14) to (15). A signum function was applied to process the left antenna, right antenna, and cerci stimulations, limiting the output values to binary 0 and 1. The detailed 0 and 1 stimulation signals transmitted from the navigation algorithm and the wireless backpack are depicted in Supplementary Figure S3). The pseudocode for the first (reach-avoid navigation) and second (adaptive reach-avoid navigation) algorithms is outlined in Supplementary Figures S4 and Figure S5, respectively. Additionally, a detailed explanation of parameter tuning to adjust threshold parameters for diverse environmental conditions is provided in Supplementary Data. $[\begin{matrix} u_{L} \\ u_{R} \\ u_{C} \end{matrix}] = sign ([\begin{matrix} 0 & 0 & 1 & 0 & 0 \\ 1 & 0 & 0 & 1 & 0 \\ 0 & 1 & 0 & 0 & 1 \end{matrix}] [\begin{matrix} u_{R O} \\ u_{C O} \\ u_{L G} \\ u_{R G} \\ u_{C G} \end{matrix}])$ (14) $[\begin{matrix} u_{L} \\ u_{R} \\ u_{C} \end{matrix}] = sign ([\begin{matrix} 0 & 1 & 0 & 1 & 0 & 0 \\ 1 & 0 & 0 & 0 & 1 & 0 \\ 0 & 0 & 1 & 0 & 0 & 1 \end{matrix}] [\begin{matrix} u_{R O} \\ u_{L O} \\ u_{C W} \\ u_{L G} \\ u_{R G} \\ u_{C G} \end{matrix}])$ (15)

FIG. 4.

Block diagram of the BIOBBN control scheme implementing three behaviors for navigating complex terrain. The system incorporates feedback loops from onboard sensors (IMU and ToF sensors) and offboard sensor data from the motion capture system. ToF sensors were chosen for their compact size and precise short-range obstacle measurement. The cyborg cockroach’s position was tracked using an external 3D motion capture system. Obstacle distances from the ToF sensors and longitudinal acceleration from the IMU were input into the obstacle avoidance and extended obstacle avoidance/wall-following behaviors. Meanwhile, the cyborg insect’s measured position and heading were fed to the goal-reaching behavior. A simple parameter ( $a_{diffsum}$ ) was used to process acceleration data and detect FWM in real-time. In this second algorithm, three behaviors were implemented as described in Figure 3A, B, C, G, H, I, and J. BIOBBN, biohybrid behavior-based navigation; IMU, inertial measurement unit.

Results

Stimulation comparison between flat and sandy surfaces

Both navigation methods were tested in an experimental area containing sand and small rocks, a surface that has not been extensively explored or studied. To assess the impact of this rough terrain, experiments were conducted to measure the turning and forward responses to electrical stimulation on both flat and sandy/rocky surfaces. The measured responses are presented in Figure 5A and B. As illustrated in the results, cockroaches demonstrated significantly reduced responses to antenna and cerci stimulation on rocky and sandy surfaces compared to flat surfaces. Madagascar Hissing cockroaches utilize a combination of adhesive pads (arolium) and pretarsal claws to optimize adhesion to various substrates. While the claws enhance gripping on rough surfaces, the arolium activates during leg movement to facilitate adhesion on smooth surfaces.^58,59 However, increased surface roughness, such as sand or small rocks, can hinder the cockroach’s ability to grip, potentially reducing adhesion and friction. Furthermore, habituation to repeated stimuli could diminish the magnitude of the turning response until the cockroach ignores the given stimulation. In this study, we investigated the reduction in the angular response to repeated stimuli applied to the left antenna for one second with a rest/off stimulation period of 3–5 s per trial (n = 4 cockroaches). The results are shown in Figure 5C.

FIG. 5.

Turning and forward motion of cyborg Insects induced by electrical stimuli. (A) Turning response of cockroaches from left antenna stimulation on two different surfaces. (N = 4 cockroaches, n = 100 trials). (B) Forward motion response of cockroaches from cerci stimulation on two different surfaces. (N = 4 cockroaches, n = 100 trials). (C) Angular response from repeated left antenna stimuli (N = 4 cockroaches, n = 275 trials). The notation of **** indicates that p-value is less than 1 × 10⁻⁴ based on the t-test. Cockroaches exhibited significantly reduced responses to antenna and cerci stimulation on rocky and sandy surfaces compared to the flat surface. The reduced responses were observed due to reduced grip force on rocky and sandy surfaces. Repeated stimulation to the same left antenna demonstrated a decrease in turning angle magnitude over time/trials, eventually leading to stimulus disregard. The diminished turning and forward motion on sandy and rocky surfaces, along with the reduced response to the repeated stimulation, highlight the challenges inherent in autonomously navigating cockroaches in such environment.

The diminished turning and forward motion on sandy and rocky surfaces underscore the inherent challenges of autonomous cockroach navigation in such environments compared to flat surfaces. Additionally, habituation, or the reduction of angular response to repeated stimuli, poses a challenge for turning motion in autonomous navigation. Therefore, incorporating FWM into the navigation system could help reduce unnecessary repeated stimuli.

Reach-avoid navigation experiment

In the first scenario, a cyborg insect mounted with the first backpack (Fig. 2C) was placed facing the corner of the first obstacle. The cyborg could autonomously navigate around all obstacles and reach the goal area in unknown and unstructured environments (Supplementary Video S1). The cyborg’s velocity was computed through numerical differentiation from its position (xc, yc) measured by a motion capture system. Figure 6 presents the overall stimulations received by the sensory organs during autonomous navigation, including measured angles, obstacle distances, velocity, acceleration, and trajectory measured by onboard IMU, ToF distance sensors, and offboard motion capture. In this trial, the cockroach always responded to the given stimulation, and no electrode-tissue degradation was found.

FIG. 6.

Measured angles, obstacle distance, velocity, and acceleration during autonomous navigation in a complex environment (N = 1 cyborg cockroach, n = 1 trial). (A) Measured angular response, angular rate, angular difference $(ψ, \dot{ψ}, δ ψ)$ , and obstacle distance $(d_{F}, d_{R}, d_{L})$ vs. provided stimulation. (B) Measured velocity (v) and difference summation acceleration $(a_{diffsum})$ vs. provided stimulation. (C) Trajectory result from the initial position to the goal area measured using motion capture system (Supplementary Video S1). The navigation control activated its obstacle avoidance behavior when an obstacle came within range. This behavior was shown when the measured angular difference ( $δ ψ$ ) exceeded a threshold of ±12 degrees. Reaching goal behavior steered the cyborg toward the goal when an obstacle was outside the determined range. The velocity of the cyborg insect was calculated based on the measured position from motion capture, while the $a_{diffsum}$ was measured from the IMU sensor. Instead of measured velocity (v) from the motion capture, measurement from IMU was given to the feedback navigation. IMU, inertial measurement unit.

The measured cockroach’s heading response $(ψ)$ , heading rate $(\dot{ψ})$ , and heading angular difference $(δ ψ)$ during autonomous navigation are shown in Figure 6A. The right antenna was stimulated due to two conditions: reaching the goal and avoiding obstacle behaviors. The left antenna was stimulated under the reaching goal behavior. When the cockroach was put in front of the first obstacle, the cockroach was navigated to turn left by providing electrical stimulation to the right antenna, as shown with a negative angle of the cockroach’s heading $(ψ)$ . When there was an obstacle in the proximity of the cyborg cockroach under the threshold value, obstacle avoidance took over to steer the cockroach to avoid the obstacles as shown with a higher value of angular difference angle $(δ ψ)$ . Figure 6A shows the front, left, and right distances to obstacles as measured by onboard ToF distance sensors over time during autonomous navigation. It confirmed that the autonomous navigation system provided stimulation to the right antenna as the default steering command under obstacle avoidance behavior.

Figure 6B shows the velocity (v) and the difference summation of longitudinal acceleration ( $a_{diffsum}$ ) over time as measured by an offboard motion capture system and onboard IMU. When the velocity was approximately zero (i.e., when the cockroach had stopped or was not in free-walking motion), the $a_{diffsum}$ parameter value was small ( $\approx$ < 0.7 cm/s²). This confirms that the $a_{diffsum}$ parameter could be used to detect FWM despite exhibiting error detection. As shown in the figure, electrical stimulation was only provided to the cerci when FWM was not detected, for forward motion control in both obstacle avoidance and reaching goal behaviors. This control strategy minimized the number of stimulations applied to the cerci, as seen in Figure 6B. The cockroach was initially placed in the starting area and began navigating using obstacle avoidance. If the cockroach encountered the obstacle again, it was steered to avoid it. When in an unobstructed area, the reaching goal behavior guided the cockroach toward the goal area. This trajectory confirmed the system’s ability to navigate the cyborg insect around obstacles and toward the goal in a complex environment with a soil- and rock-covered surface, as shown in Figure 6C.

The overall results of the autonomous navigation tests in the first scenario, including highlighted stimulations on the trajectories, are presented in Figure 7. The results indicated that most cockroaches successfully navigated to the goal area in the first scenario, exhibiting minimal distance traveled and small final position deviations (Supplementary Video S3). The cockroach’s default behavior was to turn left when encountering obstacles (Fig. 3D and E). This strategy proved sufficient for the simpler obstacle course in the first experiment setup. However, a few trials deviated from the target area. Upon closer examination of cyborg cockroach with degraded navigation performance, it was observed that they were ignoring the repeated electrical stimulation provided to the same left antenna by the feedback navigation system (Supplementary Video S2). This ignored stimuli was due to tissue-electrode interface degradation and excessive repeated stimulation during navigation to reach the goal, leading to habituation to repeated stimuli and a reduced turning response. Moreover, the cockroaches had the electrodes implanted for over five days and attempted to dislodge or break them. Tissue-electrode interface degradation can be attributed to factors such as electrode displacement, tissue reactions to implants, biocompatibility issues, fibrous encapsulation, and electrode corrosion, as documented in previous studies.^30,60–65 This degradation could reduce the navigation performance in the turning motion. While habituation to repeated stimuli was observed in some trials, all cyborg cockroaches were able to autonomously navigate around obstacles and reach the target or near-target area without becoming trapped or halted in the complex environment. Due to the bulky design of the first backpack, two of 26 trials resulted in entanglement at the obstacle’s tip. However, the cockroaches were able to free themselves using their natural agility and crawling ability, demonstrating their resilience in escaping such a situation.

FIG. 7.

Overall trajectory results for autonomous navigation for cyborg insects in the first experimental area (Fig. 2A) utilizing the first backpack (N = 7 cockroaches, n = 26 trials). The black shaded area represents obstacles. Initially, some cyborg cockroaches exhibited repeated switching behaviors between obstacle avoidance and reaching the goal behaviors. Additionally, backpacks became entangled on the first obstacle’s tip in two trials out of 26 trials. However, the BIOBBN incorporating FWM enabled them to escape these situations. Overall, all cyborg cockroaches navigated all obstacles and escaped the cornered area, avoiding being trapped or stopped (Supplementary Video S3). Although a few trials showed that the cockroach ignored repeated electrical stimulation on the antenna for steering motion due to habituation/electrode-tissue interface degradation, overall, the cyborg insects could be autonomously steered to avoid/escape all obstacles successfully and reach the target or near the goal area, with the final position deviations observed in some cases. BIOBBN, biohybrid behavior-based navigation.

Adaptive reach-avoid navigation experiment

In one trial case using the second algorithm where the cyborg could reach the goal area by climbing the obstacle surrounding it, the cockroach was placed at a starting position facing 180 degrees opposite the goal. The cyborg initiated goal-seeking behavior, steering the cockroach toward the center of the goal area when no obstacles were detected and the angular difference $(δ ψ)$ exceeded a ±12 degree threshold. Obstacle avoidance was activated when obstacles were detected within specified distance thresholds. When no obstacles were within these thresholds and the angular difference $(δ ψ)$ was within the ±12 degree range, the cockroach navigated freely, as seen from 114 s to 121 s in Supplementary Figure S6. Upon encountering a tall obstacle wall, the cyborg executed a left turn to circumvent it. When only the front sensor detected an obstacle, the cyborg attempted to climb the wall. After returning to the ground, it reoriented itself towards the goal. Following the successful navigation of the fourth layer obstacle by climbing over it instead of avoiding it, the cyborg employed a similar strategy to circumvent the wall surrounding the goal area. The second algorithm was also implemented to allow the cyborg to approach a wall when only the front sensor detected an obstacle. In this trial, the left or right antennae were stimulated to avoid the obstacle and reach the goal area. The stimulation duration was increased during the climbing maneuver, as illustrated in Supplementary Figure S6. This suggests the second navigation facilitated the cyborg’s ability to circumvent the obstacle by climbing, rather than simply avoiding it. By modifying the obstacle avoidance rule in the second algorithm and applying a lighter, more compact electronic backpack, the natural agile locomotion of the cockroach could be augmented to navigate complex environments. The agile locomotions observed in 26 autonomous navigation trials are summarized in Supplementary Table S2.

In the second scenario, initially, the cyborg insects were positioned facing away from the goal area. All cyborg cockroaches successfully avoided obstacles and corners without becoming trapped. However, two out of 26 trials resulted in rollovers that could not be recovered from, preventing further navigation. Overall, the cockroaches failed to reach the goal area in 15.39% of trials due to rollovers or stopping at obstacles (Supplementary Video S4). Additionally, some cockroaches exhibited innate behaviors, such as climbing obstacles instead of avoiding them or stopping near the goal due to disregarding stimuli, especially in antennal stimuli. In two out of 26 trials, the cockroaches climbed obstacles and stopped walking on top of them, ignoring given stimuli.

Despite these challenges, most trials demonstrated successful goal-reaching, either by climbing onto the goal area’s wall, reaching the designated entrance, stopping on the obstacle goal’s top surface, or stopping with a position deviation. In 18 out of 26 trials (69.23%), cyborg insects successfully reached the goal area by climbing onto the wall, reaching the designated entrance, or stopping on the obstacle goal’s top surface (Supplementary Videos S5, S6 and S7). The cockroaches reached the goal area with a final position deviation of 15.4% (4 out of 26 trials). Throughout the trials, the cyborg cockroaches utilized their natural agility and diverse locomotion skills, including climbing, walking on top of obstacles, traversing irregular terrain, wall-following, and recovering from rollovers. The results of the autonomous navigation tests conducted in the complex experimental scenario (Fig. 2B) are presented in Figure 8, which features the cyborg insect trajectories with highlighted markers indicating instances of given electrical stimulation applied during navigation.

FIG. 8.

Overall trajectory results for the autonomous navigation for cyborg insects in the first experimental area (Fig. 2B) utilizing the second backpack (N = 8 cockroaches, n = 26 trials). The black line indicates the trajectory path for each trial starting from the initial position to the goal area. In the starting position, the cyborg insects were placed facing away from the goal area. All cyborg cockroaches avoided obstacles and a cornered area, without getting trapped or stopped. However, two trials out of 26 trials resulted in rollovers that could not be recovered from. Additionally, some cockroaches exhibited innate behaviors, such as climbing obstacles instead of avoiding them or stopping on the top surface of the obstacle due to disregarded stimuli. Despite these challenges, several trials demonstrated successful goal-reaching, either by climbing the goal area’s wall or using the designated entrance. Throughout the trials, the cyborg cockroaches utilized their natural agility and diverse locomotion skills, including climbing, walking on the top of obstacles, traversing irregular terrain, wall-following, and recovering from rollovers.

Discussion

Navigation performance

The obtained performance of both scenarios is depicted in Figure 9, with the first row representing the first scenario and the second row representing the second scenario. The stimulation time applied to the right antenna, left antenna, and cerci is shown in Figure 9A and E. These figures demonstrate that both proposed algorithms effectively reduce the number of stimulations time delivered to the cerci, with an average value of 1.7 s and 2.8 s, respectively. In the second scenario, the average stimulation time provided to the left and right antennae (47.5 and 40.9 s) was notably longer than in the first scenario (19.5 and 15.2 s for the right and left antennae). This increase in antenna stimulation in the second scenario can be attributed to the higher obstacle density, requiring more frequent stimulation to the antennae to avoid higher obstacle density. The low stimulation time for the cerci indicates that both feedback systems optimized the cockroach’s FWM, reducing and minimizing unnecessary repeated stimulation.

FIG. 9.

Navigation performance for the behavior-based feedback in two different terrains. Stimulation time given to the antennae and cerci (A) in sparse obstacle terrain and (B) in dense obstacle terrain. Required time for autonomous navigation, stimulation, and free stimulation time (C) in sparse obstacle terrain and (D) in dense obstacle terrain. Distance traveled during autonomous navigation (E) in sparse obstacle terrain and (F) in dense obstacle terrain. Final position deviation from the goal/target area (G) in sparse obstacle terrain and (H) in dense obstacle terrain. The notation of “ns” indicates that the p-value is not significant based on the t-test. Cyborg insects in dense obstacle terrain (Fig. 8) received more frequent stimulation, particularly to the antennae, to facilitate obstacle avoidance. The navigation system minimized unnecessary stimulation to both antennae and cerci during navigation. By reducing the number of stimulations and incorporating FWM, the early habituation response to the repeated electrical stimulation was mitigated, extending the operational time of the cyborg insects and innate locomotion.

The average navigation time in the first scenario was 77.1 s, with one outlier taking 203.2 s due to tissue-electrode degradation. In contrast, the average navigation time in the second scenario was 183 s. This longer duration can be attributed to the complicated terrain to navigate and cockroaches’ tendency to climb obstacles and ignore stimulation while navigating on obstacle surfaces. When encountering and touching obstacles, they frequently attempted to climb them rather than avoid them.

The distance traveled from the starting area to the goal area by cyborg cockroaches in the first scenario (180 cm × 180 cm) is shown in Figure 9C. The average distance traveled by the cockroaches was 404.7 cm. Among the observed trials, the cockroach that ignored the electrical stimulation commands traveled the farthest. The cyborg cockroaches successfully navigated around obstacles and toward the destination area. However, there were some final position deviations from the goal, as shown in Figure 9D. Out of 26 trials, 61.54% of the cyborg cockroaches successfully navigated to the designated goal area (radius: 7 cm) without deviating from their final position. The average final position deviation for the cockroaches that deviated from the goal area was 3.89 cm or 0.6 the body length. The average distance traveled in the second scenario, which was conducted in a complicated scenario (130 cm × 110 cm), was 302.6 cm, as depicted in Figure 9H. In 15.39% of total trials, the cockroaches failed to reach the goal area due to rollovers or stopping at the top of the obstacle surface. However, most trials demonstrated successful goal-reaching, either by climbing onto the goal area’s wall, reaching the designated entrance, stopping on the obstacle goal’s top surface, or stopping with a position deviation. In 18 out of 26 trials (69.23%), cyborg insects successfully reached the goal area by climbing onto the wall of the goal area, reaching the designated entrance, or reaching the obstacle goal’s top surface (small final position deviation). On average, the final position deviated 5.0 cm (0.77 the body length) from the target location in the second scenario.

The first navigation algorithm, paired with the bulkier initial electronic backpack, was designed for less complex environments with lower obstacle density. This configuration allowed for effective navigation using basic obstacle avoidance and goal-reaching behaviors. In contrast, the second navigation algorithm, equipped with a significantly lighter and more compact backpack, was optimized for navigating complex terrain with higher obstacle density. The reduced weight and size enabled the cyborg insect to employ more agile locomotion strategies, such as climbing and self-righting, to successfully traverse these challenging environments.

FWM has been incorporated into the navigation algorithm as a valuable feature for cyborg insects. A previous study utilized an IMU-based navigation algorithm that heuristically predicted insect situations to guide obstacle avoidance without employing a dedicated obstacle detection sensor.²⁹ The navigation was applied on flat terrain with low and tall wall-shaped obstacles, successfully demonstrating autonomous navigation around large, irregularly shaped concrete blocks and small hollow bricks. The cyborg’s natural locomotion enabled it to climb a 1.5-cm obstacle and navigate around irregularly shaped obstacles. However, this algorithm has not been evaluated in more complex environments with obstacles featuring sharp corners. A monocular camera’s depth map was utilized to anticipate obstacles and guide cyborg insects around them, preventing entrapment in obstacles with corners.⁶⁶ A cyborg’s navigation system necessitates an obstacle avoidance module to detect obstacles and steer clear. Without the obstacle avoidance system, the cyborg was unaware of obstacles and attempted to move directly toward its destination, leading to potential entrapment, particularly in areas with corners. By incorporating an obstacle avoidance system, the success rate of the navigation experiment increased to 73.3% (11 trials out of 15), while some cockroaches (26.7%) still became trapped in obstacles with corners. Further comparison of previous studies on cyborg insect autonomous navigation incorporating obstacle avoidance is provided in Supplementary Table S3). While onboard Wi-Fi camera offers the potential for obstacle detection, they often require complex image processing algorithms and consume more power (80–260 mA).^47,66 The ToF distance sensor provides accurate distance measurements with lower power consumption (20 mW), making it a more suitable option for onboard obstacle avoidance applications.⁵² By strategically positioning three ToF distance sensors on the front, right, and left sides of the cyborg insect, precise turning directions could be achieved. This onboard obstacle avoidance sensor configuration enabled the cyborg insect to successfully navigate around obstacles with sharp corners (no entrapment in the cornered areas/V-shaped obstacles), as demonstrated in this study.

Agile locomotion during autonomous navigation

While previous studies have not extensively explored and utilized the cockroach’s natural agile locomotion for autonomous navigation in complex environments, this study demonstrates its effectiveness. In the first scenario, two out of 26 trials involved brief entanglement of the backpacks on the first obstacle tip, but the cockroaches successfully avoided becoming stuck and escaped using their natural locomotion and crawling ability. In the more challenging second scenario, the proposed navigation system effectively leveraged the cyborg insects’ natural agility and adaptability to navigate the complex terrain, utilizing a lightweight and compact backpack stimulator that the insects could easily carry. The insects exhibited various innate behaviors while navigating the complex terrain, as illustrated in Figure 10 and Supplementary Video S8. In 9 out of 26 trials, the cyborg became entangled by the marker frame at the obstacle tips, but they were able to avoid getting stuck by utilizing their crawling locomotion ability. The cyborgs also demonstrated their ability to self-right after rollovers (3 successful self-righting out of 5 rollovers). Furthermore, they exhibited climbing behavior, successfully scaling the 7.5 cm goal area wall and entering the goal area. Additionally, they were capable of traversing from the ground to the top of obstacles and then returning to the ground. When encountering long wall obstacles, the cockroaches employed wall-following behavior. These innate agile locomotion abilities are crucial for navigating complex terrain and could be beneficial for future search and rescue missions. Given the natural complexity of post-disaster areas, augmenting the cyborg insect’s agile natural locomotion with autonomous navigation is essential for effective navigation in such environments.

FIG. 10.

Innate agile locomotion of cyborg insects during autonomous navigation (Supplementary Video S8). (A) Wall-following behavior in the long obstacle path. (B) Escaped being stuck at the tip of the obstacle. (C) Climbed obstacle wall, walk upon the top surface, and then return to the ground level. (D) Climbed the V-shaped obstacle using its natural climbing abilities instead of avoiding it. (E) Bypassed the designated entrance and climbed over the 7.5 cm obstacle to reach the goal area. (F) Reoriented itself after experiencing a rollover incident. The proposed navigation system successfully utilized the cyborg insects’ natural agility and adaptability to navigate complex environments, employing a lightweight backpack stimulator that the insects could easily carry. The insects exhibited various innate behaviors, including wall-following, climbing behavior, obstacle avoidance, and obstacle surmounting. Additionally, the cyborg insects demonstrated resilience by self-righting after rollover, demonstrating their excellent innate capabilities for navigating challenging terrain.

Future study

The cyborg insects still could be navigated to avoid all obstacles and reach the goal area with final position deviation without being trapped, although existed ignored stimuli during navigation. A previous study developed an algorithm that extends the control time of insect-computer biohybrid robots from minutes to hours.³¹ It tuned and adjusted the stimulus voltage based on the insect’s motion response, to reduce habituation onset. To prolong the insect’s turning response, researchers have explored alternating stimulation between the antennae and cerci.²⁸ Additionally, randomizing the timing of stimuli can help maintain angular responsiveness. Continuously adjusting stimulus intensity can further prevent habituation. Future research will investigate the use of biphasic signals with varying voltage and duration to extend operational time and minimize habituation. In two out of 26 trials, the cyborg insect experienced unrecoverable rollovers, hindering further navigation. To address this limitation, a bio-inspired 3D-printed artificial limb, as proposed in previous research, could potentially assist the insect with self-righting and locomotion.⁶⁷

As the research progresses towards more practical scenarios, the transition to using an offboard sensor like a 3D motion capture system will introduce new challenges. Limitations in size and power requirements will need to be carefully considered. To address these challenges, integrating Ultra-Wideband (UWB) technology into the electronic backpack offers a promising onboard sensor solution for the cyborg insect. UWB technology provides localization capabilities, making it well-suited for estimating the cyborg’s position relative to the environment.²⁹ By combining UWB with advanced sensor fusion techniques, a feasible solution can be developed to implement the proposed BIOBBN algorithm in real-world scenarios. This onboard sensor approach would overcome the constraints associated with an offboard motion capture system, allowing the cyborg to navigate autonomously without the need for an external infrastructure. Addressing these practical considerations and developing a robust onboard sensing system will be crucial next steps to translate the laboratory findings into deployable cyborg insect platforms capable of navigating complex environments.

Conclusions

This study introduces BIOBBN techniques to enable cyborg cockroaches to autonomously navigate complex environments. Two navigation algorithms equipped with different electronic backpacks were developed and tested in both sparse and dense obstacle scenarios. By integrating free-walking motion with programmed responses, these algorithms aimed to minimize external stimuli while leveraging the insects’ innate agility.

Onboard sensors and offboard motion capture systems provided crucial feedback for obstacle avoidance, free-walking detection, and precise position tracking. Experimental results demonstrated the effectiveness of the proposed BIOBBN approach. Cyborg cockroaches successfully navigated around obstacles, climbed over walls, and reached target areas in both simple and complex environments.

The adaptive navigation algorithm, in particular, showcased the ability to harness the insects’ natural agility to overcome obstacles rather than simply avoiding them. While the denser obstacle scenario led to longer navigation times due to increased obstacle avoidance and climbing behaviors, the overall performance highlights the potential of BIOBBN for enabling the autonomous navigation of cyborg insects in unknown, unstructured environments.

Footnotes

Authors’ Contributions

M.A.: Writing—review and editing, writing—original draft, visualization, methodology, analysis, data curation. X.Z.: Experiment and data curation. R.T.: Data curation, review. C.M.M.R.: Motion capture preparation, data curation. N.H.: Backpack preparation, cyborg insect implantation. K.Y.: Cyborg insect implantation and backpack set up. K.M.: Writing—review and editing, writing—original draft, supervision, project administration, methodology, funding acquisition, conceptualization.

Data Availability Statement

Data will be made available on request.

Author Disclosure Statement

The authors declare no conflict of interest.

Funding Information

This work was partly supported by the Japan Science and Technology (Moonshot R&D) Grant Number JPMJMS223A and JSPS KAKENHI Grant Number 22H04951.

Supplementary Material

References

Iwendi

, Alqarni

, Anajemba

, et al. Robust navigational control of a two-wheeled self-balancing robot in a sensed environment. IEEE Access, 2019; 7:82337–82348; doi: 10.1109/ACCESS.2019.2923916

Gopalakrishnan

, Tirunellayi

, Todkar

. Design and development of an autonomous mobile smart vehicle: A mechatronics application. Mechatronics, 2004; 14(5):491–514; doi: 10.1016/j.mechatronics.2003.10.003

Almasri

, Alajlan

, Elleithy

. Trajectory planning and collision avoidance algorithm for mobile robotics system. IEEE Sensors J, 2016; 16(12):5021–5028; doi: 10.1109/JSEN.2016.2553126

Zhao

, Fang

, Wang

, et al. Obstacle avoidance of multi-sensor intelligent robot based on road sign detection. Sensors (Basel), 2021; 21(20):6777; doi: 10.3390/s21206777

Adiuku

, Avdelidis

, Tang

, et al. Improved hybrid model for obstacle detection and avoidance in robot operating system framework (Rapidly Exploring Random Tree and Dynamic Windows Approach). Sensors (Basel), 2024; 24(7):2262; doi: 10.3390/s24072262

Yin

, Chen

, Liu

, et al. Autonomous navigation of mobile robots in unknown environments using off-policy reinforcement learning with curriculum learning. Expert Syst Appl, 2024; 247:123202; doi: 10.1016/j.eswa.2024.123202

Pandey

, Kumar

, Pandey

, et al. Mobile robot navigation in unknown static environments using ANFIS controller. Perspect Sci, 2016; 8:421–423; doi: 10.1016/j.pisc.2016.04.094

Munadi

, Radityo

, Ariyanto

, et al. Automated Guided Vehicle (AGV) lane-keeping assist based on computer vision, and fuzzy logic control under varying light intensity. Results Eng, 2024; 21:101678; doi: 10.1016/j.rineng.2023.101678

Wang

, Li

, Zhang

, et al. Review of wheeled mobile robot collision avoidance under unknown environment. Sci Prog, 2021; 104(3):368504211037771; doi: 10.1177/00368504211037771

10.

Cebollada

, Payá

, Flores

, et al. A state-of-the-art review on mobile robotics tasks using artificial intelligence and visual data. Expert Syst Appl, 2021; 167:114195; doi: 10.1016/j.eswa.2020.114195

11.

, Ran

, Wang

, et al. A behavior-Based mobile robot navigation method with deep reinforcement learning. Un Sys, 2021; 09(03):201–209; doi: 10.1142/S2301385021410041

12.

Al Mahmud

, Kamarulariffin

, Ibrahim

, et al. Advancements and challenges in mobile robot navigation: A comprehensive review of algorithms and potential for self-learning approaches. J Intell Robot Syst, 2024; 110(3):120; doi: 10.1007/s10846-024-02149-5

13.

Loganathan

, Ahmad

. A systematic review on recent advances in autonomous mobile robot navigation. Eng Sci Technol Int J, 2023; 40:101343; doi: 10.1016/j.jestch.2023.101343

14.

Tzafestas

. Mobile robot control and navigation: A global overview. J Intell Robot Syst, 2018; 91(1):35–58; doi: 10.1007/s10846-018-0805-9

15.

Zhu

, Zhang

. Deep reinforcement learning based mobile robot navigation: A review. Tsinghua Sci Technol, 2021; 26(5):674–691; doi: 10.26599/TST.2021.9010012

16.

Hacene

, Mendil

. Behavior-based autonomous navigation and formation control of mobile robots in unknown cluttered dynamic environments with dynamic target tracking. Int J Autom Comput, 2021; 18(5):766–786; doi: 10.1007/s11633-020-1264-x

17.

Balch

, Arkin

. Behavior-based formation control for multirobot teams. IEEE Trans Robot Automat, 1998; 14(6):926–939; doi: 10.1109/70.736776

18.

Marques

, Nunes

, de Almeida

. Olfaction-based mobile robot navigation. Thin Solid Films, 2002; 418(1):51–58; doi: 10.1016/S0040-6090(02)00593-X

19.

Anderson

, Donath

. Animal behavior as a paradigm for developing robot autonomy. Robot Auton Syst, 1990; 6(1–2):145–168; doi: 10.1016/S0921-8890(05)80033-8

20.

, Tang

, Meng

. Behavior-Based fuzzy control for mobile robot navigation. Math Probl Eng, 2013; 2013:1–10; doi: 10.1155/2013/561451

21.

Beom

, Cho

. A sensor-based navigation for a mobile robot using fuzzy logic and reinforcement learning. IEEE Trans Syst Man Cybern, 1995; 25(3):464–477; doi: 10.1109/21.364859

22.

Rusu

, Petriu

, Whalen

, et al. Behavior-based neuro-fuzzy controller for mobile robot navigation. IEEE Trans Instrum Meas, 2003; 52(4):1335–1340; doi: 10.1109/TIM.2003.816846

23.

. Fuzzy-logic-based reactive behavior control of an autonomous mobile system in unknown environments. Eng Appl Artif Intell, 1994; 7(5):521–531; doi: 10.1016/0952-1976(94)90031-0

24.

Aguirre

, González

. Fuzzy behaviors for mobile robot navigation: Design, coordination and fusion. Int J Approx Reason, 2000; 25(3):255–289; doi: 10.1016/S0888-613X(00)00056-6

25.

Maaref

, Barret

. Sensor-based navigation of a mobile robot in an indoor environment. Robot Auton Syst, 2002; 38(1):1–18; doi: 10.1016/S0921-8890(01)00165-8

26.

Kakei

, Katayama

, Lee

, et al. Integration of body-mounted ultrasoft organic solar cell on cyborg insects with intact mobility. Npj Flex Electron, 2022; 6(1):1–9; doi: 10.1038/s41528-022-00207-2

27.

Ariyanto

, Refat

CMM

, Zheng

, et al. Teleoperated locomotion for biobot between Japan and Bangladesh. Computation, 2022; 10(10):179; doi: 10.3390/computation10100179

28.

, Chen

, Yang

, et al. The autonomous pipeline navigation of a cockroach Bio-Robot with enhanced walking stimuli. Cyborg Bionic Syst, 2023; 4:67.

29.

Tran-Ngoc

, Le

, Chong

, et al. Intelligent insect–computer hybrid robot: Installing innate obstacle negotiation and onboard human detection onto cyborg insect. Adv Intell Syst, 2023; 5(5):2200319; doi: 10.1002/aisy.202200319

30.

Sanchez

, Chiu

C-W

, Zhou

, et al. Locomotion control of hybrid cockroach robots. J R Soc Interface, 2015; 12(105):20141363; doi: 10.1098/rsif.2014.1363

31.

, Lin

, Kai

, et al. A navigation algorithm to enable sustainable control of insect-computer hybrid robot with stimulus signal regulator and habituation-breaking function. Soft Robot, 2024; 11(3):473–483; doi: 10.1089/soro.2023.0064

32.

Dutta

. Cyborg insects could someday save your life. IEEE Pulse, 2019; 10(3):24–25; doi: 10.1109/MPULS.2019.2911818

33.

, Zhao

, Yu

, et al. A review of energy supply for biomachine hybrid robots. Cyborg Bionic Syst, 2023; 4:53; doi: 10.34133/cbsystems.0053

34.

Webster-Wood

, Guix

, Xu

, et al. Biohybrid robots: Recent progress, challenges, and perspectives. Bioinspir Biomim, 2022; 18(1):15001; doi: 10.1088/1748-3190/ac9c3b

35.

Zhou

, Mei

, Li

, et al. Progresses of animal robots: A historical review and perspectiveness. Heliyon, 2022; 8(11):e11499; doi: 10.1016/j.heliyon.2022.e11499

36.

Siljak

, Nardelli

PHJ

, Moioli

. Cyborg insects: Bug or a feature? IEEE Access, 2022; 10:49398–49411; doi: 10.1109/ACCESS.2022.3172980

37.

Caforio

, Punta

, Morishima

. Experimental modeling and variable structure control for cyborg cockroaches. IEEE Control Syst Lett, 2024; 8:67–72; doi: 10.1109/LCSYS.2023.3343595

38.

Poon

, Tan

DCL

, Li

, et al. (Invited) cyborg insect: Insect computer hybrid robot. Meet Abstr, 2016;MA2016-02(44):3221–3221; doi: 10.1149/MA2016-02/44/3221

39.

Nguyen

, Tan

, Sato

, et al. Sideways walking control of a cyborg beetle. IEEE Trans Med Robot Bionics, 2020; 2(3):331–337; doi: 10.1109/TMRB.2020.3004632

40.

Lin

, Li

, Zhang

, et al. Resilient conductive membrane synthesized by in-situ polymerisation for wearable non-invasive electronics on moving appendages of cyborg insect. Npj Flex Electron, 2023; 7(1):1–10; doi: 10.1038/s41528-023-00274-z

41.

Ariyanto

, Refat

CMM

, Hirao

, et al. Movement optimization for a cyborg cockroach in a bounded space incorporating machine learning. Cyborg Bionic Syst, 2023; 4:12; doi: 10.34133/cbsystems.0012

42.

Lin

, Kai

, Nguyen

, et al. A newly developed chemical locomotory booster for cyborg insect to sustain its activity and to enhance covering performance. Sens Actuators B Chem, 2024; 399:134774; doi: 10.1016/j.snb.2023.134774

43.

, Zhang

. Brain-Computer interface controlled cyborg: Establishing a functional information transfer pathway from human brain to cockroach brain. PLoS One, 2016; 11(3):e0150667; doi: 10.1371/journal.pone.0150667

44.

Erickson

, Herrera

, Bustamante

, et al. Effective stimulus parameters for directed locomotion in Madagascar hissing cockroach Biobot. PLoS One, 2015; 10(8):e0134348; doi: 10.1371/journal.pone.0134348

45.

Latif

, Bozkurt

. Line following terrestrial insect biobots. In: 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, 2012; pp. 972–975; doi: 10.1109/EMBC.2012.6346095

46.

Rasakatla

, Suzuki

, Tenma

, et al. CameraRoach: Various electronic backs packs for search and rescue. In: 2021 IEEE International Conference on Robotics and Biomimetics (ROBIO), 2021; pp. 1300–1303; doi: 10.1109/ROBIO54168.2021.9739264

47.

Rasakatla

, Tenma

, Suzuki

, et al.; Tokyo University of Agriculture and Technology 2-24-16 Nakacho, Koganei, Tokyo 184-0012, Japan. CameraRoach: A WiFi- and Camera-Enabled cyborg cockroach for search and rescue. JRM, 2022; 34(1):149–158; doi: 10.20965/jrm.2022.p0149

48.

Whitmire

, Latif

, Bozkurt

. Kinect-Based system for automated control of terrestrial insect Biobots. In: 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), 2013; pp. 1470–1473; doi: 10.1109/ICRA48891.2023.10160443

49.

Nguyen

, Dung

, Sato

, et al. Efficient autonomous navigation for terrestrial insect-machine hybrid systems. Sens Actuators B Chem, 2023; 376:132988; doi: 10.1016/j.snb.2022.132988

50.

Nguyen

, Sato

, Vo-Doan

. Burst Stimulation for Enhanced Locomotion Control of Terrestrial Cyborg Insects. In: IEEE International Conference on Robotics and Automation (ICRA) IEEE: London, United Kingdom; 2023; pp. 1170–1176.

51.

Wilson

, Nalepa

, Roth

, et al. Cockroaches: Ecology, Behavior, and Natural History. Johns Hopkins University Press: Baltimore; 2007.

52.

Ariyanto

, Refat

CMM

, Yamamoto

, et al. Feedback control of automatic navigation for cyborg cockroach without external motion capture system. Heliyon, 2024; 10(5):e26987; doi: 10.1016/j.heliyon.2024.e26987

53.

Crall

, Souffrant

, Akandwanaho

, et al. Social context modulates idiosyncrasy of behaviour in the gregarious cockroach Blaberus discoidalis. Anim Behav, 2016; 111:297–305; doi: 10.1016/j.anbehav.2015.10.032

54.

Clark

, Triblehorn

. Mechanical properties of the cuticles of three cockroach species that differ in their wind-evoked escape behavior. PeerJ, 2014; 2:e501; doi: 10.7717/peerj.501

55.

Jeanson

, Rivault

, Deneubourg

J-L

, et al. Self-organized aggregation in cockroaches. Anim Behav, 2005; 69(1):169–180; doi: 10.1016/j.anbehav.2004.02.009

56.

Daltorio

, Mirletz

, Sterenstein

, et al. How cockroaches exploit tactile boundaries to find new shelters. Bioinspir Biomim, 2015; 10(6):65002; doi: 10.1088/1748-3190/10/6/065002

57.

Jeanson

, Blanco

, Fournier

, et al. A model of animal movements in a bounded space. J Theor Biol, 2003; 225(4):443–451; doi: 10.1016/S0022-5193(03)00277-7

58.

van Casteren

, Codd

. Foot morphology and substrate adhesion in the Madagascan hissing cockroach, Gromphadorhina portentosa. J Insect Sci, 2010; 10(1):40; doi: 10.1673/031.010.4001

59.

Jayaram

, Full

. Cockroaches traverse crevices, crawl rapidly in confined spaces, and inspire a soft, legged robot. Proc Natl Acad Sci U S A, 2016; 113(8):E950–E957; doi: 10.1073/pnas.1514591113

60.

Latif

, McKnight

, Dickey

, et al. In vitro electrochemical assessment of electrodes for neurostimulation in roach biobots. PLoS One, 2018; 13(10):e0203880; doi: 10.1371/journal.pone.0203880

61.

Jianhui

, Xiaoming

, Pengsheng

, et al. Impedance spectroscopy analysis of cell-electrode interface. In: 2005 IEEE Engineering in Medicine and Biology 27th Annual Conference, 2005; pp. 7608–7611; doi: 10.1109/IEMBS.2005.1616273

62.

Onaral

, Schwan

. Linear and nonlinear properties of platinum electrode polarisation. Part 1: Frequency dependence at very low frequencies. Med Biol Eng Comput, 1982; 20(3):299–306; doi: 10.1007/BF02442796

63.

Rubinstein

, Spelman

, Soma

, et al. Current density profiles of surface mounted and recessed electrodes for neural prostheses. IEEE Trans Biomed Eng, 1987; 34(11):864–875; doi: 10.1109/TBME.1987.326007

64.

Latif

, Bozkurt

. A wireless system for longitudinal assessment of tissue-electrode interface in Biobots. In: 2015 IEEE Biomedical Circuits and Systems Conference (BioCAS), 2015; pp. 1–4; doi: 10.1109/BioCAS.2015.7348433

65.

Latif

, Bozkurt

. Roach Biobots: Toward reliability and optimization of control. IEEE Pulse, 2017; 8(5):27–30; doi: 10.1109/MPUL.2017.2729413

66.

, Lin

, Tran-Ngoc

, et al. Smart insect-computer hybrid robots empowered with enhanced obstacle avoidance capabilities using onboard monocular camera. Npj Robot, 2024; 2(1):1–10; doi: 10.1038/s44182-024-00010-3

67.

Montagut Marques

, Yuxuan

, Sato

, et al. Cyborg insect repeatable self-righting locomotion assistance using bio-inspired 3D printed artificial limb. Npj Robot, 2024; 2(1):1–14; doi: 10.1038/s44182-024-00009-w

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.06 MB

0.05 MB

0.17 MB

0.26 MB

0.29 MB

0.09 MB

0.10 MB

0.61 MB

2.00 MB

0.05 MB

0.01 MB

Biohybrid Behavior-Based Navigation with Obstacle Avoidance for Cyborg Insect in Complex Environment

Abstract

Introduction

Material and Methods

Cyborg insect

Wireless backpack stimulator

Experimental setup

Biohybrid behavior-based navigation

Reaching goal behavior

Avoiding obstacle behavior

Extended avoiding obstacle behavior/wall-following behavior

Results

Stimulation comparison between flat and sandy surfaces

Reach-avoid navigation experiment

Adaptive reach-avoid navigation experiment

Discussion

Navigation performance

Agile locomotion during autonomous navigation

Future study

Conclusions

Footnotes

Authors’ Contributions

Data Availability Statement

Author Disclosure Statement

Funding Information

Supplementary Material

References

Supplementary Material