Sage Journals: Discover world-class research

Abstract

In this article, we present an obstacle avoidance controller implemented in a field programmable gate array for an electric wheelchair. It is based on a traditional approach with ultrasonic sensors and fuzzy logic. Various tests were conducted to characterize the prototype and to evaluate the controller performance. The results showed that the system is able to acquire data from sensors and make decisions 46.16 times per second. The sensors’ coverage extends 3 m to the front, rear, left, and right sides of the wheelchair; moreover, the sensors detect 0.95-cm diameter objects at 40 cm. The power consumption was evaluated, and it was found that the hardware architecture reduces the battery life by only 0.87%. Furthermore, the controller helped to navigate in confined areas, avoiding obstacles with cautious movements and decreasing the likelihood of collision. The proposed methodology uses data from eight sonars distributed around the wheelchair to make navigation decisions, besides the hardware-based architecture guarantees real-time control and on-time response.

Keywords

Artificial intelligence assistive technology collision avoidance fuzzy logic FPGA smart wheelchairs ultrasonic sensors

Introduction

According to the study by Fehr et al.,¹ a survey about wheelchair users in the United States reported that up to 40% find difficulties in maneuvering as a consequence of disabilities. Those individuals experiment limited mobility due to visual impairments, spasticity, tremors, or cognitive deficits. Indeed, the development of automated navigation systems would benefit these persons in their daily activities.

“Smart wheelchairs” integrate navigation systems and add-on modules in conventional power wheelchairs, as presented by Simpson.² Nowadays, the state of the art and other trends have been presented by Leaman and La³ and Desai et al.⁴ It is recommended the use of a collaborative scheme or semiautonomous control by Nisbet⁵ and Hsu et al.,⁶ and hence, the user is still able to drive the wheelchair by himself and the semiautonomous system complements the driver’s skills.

Within the developed semiautonomous prototypes, a general-purpose task is used to avoid obstacles in unknown and dynamic scenarios. The traditional solution relies on the use of sensory systems for detection and range measurements, such as bumpers, infrared (IR) sensors, and ultrasonic sensors. Particularly, smart wheelchairs that use sonars have been devised by Levine et al.,⁷ Bourhis et al.,⁸ Simpson et al.,⁹ and Tyagi et al.¹⁰ Those sensors are low cost, besides they provide reliable measurements and a quick response. The sonars determine distances to objects by measuring the time-of-flight (TOF) between an emitted ultrasonic burst and its echo. However, sonars commit errors when the incidence angle is bigger than 45° or when the reflecting surface is too small to return enough signal. In order to overcome those problems, a combination of sonic and IR sensors was explored.^11

–14 The IRs emit light pulses, but this could be ineffective on absorbing materials or transparent surfaces. Furthermore, the utilization of laser range finders (LRs) has been widely used in other smart wheelchairs, presented in the literature.^{15

–22} The LRs offer the best accuracy for distance measurementsand a two-dimensional scan of the environment. Nonetheless, those sensors are far from being economical, their power consumption is high (a big problem for the wheelchair batteries), and their size limits mounting too many on any wheelchair structure.

Late detection systems integrate a combination of cameras and range sensors for collision avoidance. For instance, machine vision technology with cameras and IR sensors is implemented by Gomi and Griffith²³; meanwhile, an anti-collision system based on a stereovision camera is presented by How et al.²⁴ and stereoscopic cameras are used for three-dimensional (3-D) depth perception in the study by Nguyen et al.²⁵ The prototypes^26,27 dealt with the problem by combining an LR and a kinect sensor for 3-D mapping and also the kinect was used to detect obstacles²⁸ Furthermore, a collection of sensors has been implemented in other prototypes^29,30 including the kinect, LRs, IRs, and ultrasonics. Finally, in the study by Zhang et al.,³¹ there were used LRs, ultrasonics, and two webcams for obstacle localization. Other prototypes have used specialized software for machine vision and robot control; for example, Baya et al.³² and Cavanini et al.³³ took advantage of the robot operating system (ROS) combined with cameras and different sensory systems.

Furthermore, the use of specialized software for machine vision and robot control is now a trend because of its versatility; for example, Baya et al.,³² Cavanini et al.,³³ and TalebiFard et al.³⁴ used the ROS combined with cameras and different sensory systems. Despite the promising advantages of the vision hardware, its multiple functions are highly attached to the software and the processing core. Cameras share data by serial protocols with a computer running the machine vision software. Under this scheme, the overall system response relies on the computer’s processor resources, which are responsible for handling images, obtaining environmental information, calculating actions, and making the navigation decisions. Finally, the problem is related to the collision-free navigation of intelligent hospital beds. It was developed by Wang et al.,^35,36 a methodology for a robotic hospital bed with autonomous navigation.

At this point, all the reviewed systems have demonstrated to be useful, but very few of them preserve secure conditions against software crashes. Those software-based systems are highly vulnerable to crash at multiple levels, for example, the operative system or the application information interfaces. The main weakness in those smart wheelchairs is the processing core: even if it is fast, it does not assure a real-time control or on-time wheelchair response. Consequently, the user’s security is compromised.

A more reliable approach is the use of hardware-based architecture, which involves the application of field programmable gate arrays (FPGAs). These devices are silicon chips with reconfigurable logic gates, and their functionality is defined by the user. FPGAs have proven to be very efficient, as they execute calculations in hardware and do not depend on any operating system. Other processor cores run one instruction at a time, while FPGAs execute parallel instructions within a single clock cycle. There are many advantages about configuring a controller in an FPGA instead of using a computer or a microcontroller, as explained by Bryner and Parnell.³⁷

Admittedly, FPGA applications for electric wheelchairs have been presented in the literature^38,39; however, they are not related to the navigation control. The former is related to the energy driving of the direct current rim motors using a fuzzy controller, and the second is related to a digital parallel fuzzy controller for steering an electric wheelchair. Moreover, the system presented by How et al.²⁴ involves an FPGA for image processing, but the data are finally analyzed by a single-board computer to make the navigation decisions.

For deploying an automatic navigation control for a smart wheelchair, with real-time response and high-speed processing, this article proposes a fuzzy logic controller (FLC) implemented in an FPGA. As mentioned by Hong et al.,⁴⁰ fuzzy logic is a robust and reliable methodology that tackles environment and sensor uncertainties. A navigation control system for a mobile robot is described by Melendez et al.⁴¹ Besides, fuzzy logic requires a low computational complexity compared with other navigation schemes used for avoiding obstacles, such as vector field histogram (VHF).⁴² As presented by Baklouti et al.,²⁸ VHF is computationally expensive and its convergence is not always ensured such as in U-shaped corridors. Furthermore, fuzzy logic does not need local or global maps, besides it can be used to prevent collisions in unknown and dynamic environments in a real-world application. The FLC methodology for mobile robots is described by Omrane et al.,⁴³ and there are several smart wheelchair prototypes that use an FLC as the main control technique and range sensors as inputs.^{10,11,28,44
–46} The proposed FLC runs in the FPGA at a high-speed rate and guarantees the wheelchairs reactions on-time, as this system acquires data from sensors and makes decisions within 46.16 times per second.

Methods

The electric wheelchair architecture

The overall architecture is presented in Figure 1. The hardware was incorporated into a commercial wheelchair Quickie P-222 SE (Sunrise Medical (US) LLC, Fresno, CA). The main core is the FPGA, included in the National Instruments CompactRIO (cRIO-9103). The FGPA is a Virtex-II (Xilinx) chip with 3 M system gates, 14,336 slices, 448 Kbits RAM, 96 multiplier blocks, and a maximum of 720 available I/O pads. The code, programmed with the LabVIEW FPGA [version 2015] toolkit, is divided into two blocks: the TOF for driving the sensors and the fuzzy controller for avoiding collisions.

Figure 1.

The components of the wheelchair.

For acquiring the distance data, two high-speed digital modules NI 9401 were interfaced with eight Parallax PING))) sensors. The FPGA has a 40-MHz default clock, which means that the processor is able to respond within 25 ns cycles; however, the distance scanning process is limited by the sensor’s operation to 20 ms. Because of the FPGA parallel architecture and the high-speed response from the digital modules, that sampling rate is guaranteed for each sensor simultaneously.

The wheelchair movement is controlled by two analog voltage channels, provided by the NI 9263 module. There are used two analog channels, with a sampling rate of 100 kS/s and 16-bit resolution for each output. Therefore, this module provides accurate signal generation for controlling the wheelchair. Finally, the cRIO-9013 communicates, by a standard Ethernet connection, with a host laptop running the human–machine interface (HMI).

The FLC strategy

The FLC changes the wheelchair speed and orientation according to the distance measurements and the “change rate” inputs, which detect abrupt changes in distance measurements. The change rate inputs are calculated from the current and past distance samples, as presented by

d S_{n} (k) = S_{n} (k) - S_{n} (k - 1)

where S_n(k) is the current distance measured by the sensor n and S_n(k − 1) corresponds to the past sample.

According to the proposed architecture, data from eight sensors are used as inputs for the FLC; however, including many inputs outcomes in a complex and nonpractical combination of rules. To make possible the implementation, four individual cases with few inputs were considered: forward obstacle avoidance (FOA), left collision detector (LCD), right collision detector (RCD), and backward collision detector (BCD). Those cases are activated one at a time according to the virtual joystick position. Moreover, it is proposed the combination of two inputs using the fuzzy conjunction AND/MIN. This simplification helps to widespread the coverage range without adding more sensors or rules. The FLC structure is shown in Figure 2.

Figure 2.

The sensors distribution and the fuzzy logic controller structure. The distance inputs are labeled from S1 to S8.

Fuzzification of the variables

The input variables S_n (distance) are defined by the trapezoidal fuzzy sets C and F (“close” and “far”). Those membership functions are shown in Figure 3(a). The dS_n (“change rate”) inputs are defined by a trapezoidal set named GF (“getting fast”), presented in Figure 3(b). Further, the output variables M and D (“movement” and “direction”) are presented in Figure 3(c) and (d). The M output is defined by the four linguistic variables: B, N, SF, and F (“back,” “null,” “small forward,” and “forward”). The D output is defined by the five linguistic variables: L, SL, N, SR, and R (“left,” “small left,” “null,” “small right,” and “right”).

Figure 3.

Membership functions for the obstacle avoidance controller: (a) input sensors S1, S3, S6, and S7; (b) input sensors S2, S4, S5, and S8; (c) output movement; and (d) output direction.

The rules base and the inference engine

The wheelchair must prevent collisions with static and dynamic objects (like a person walking) in any environment. The main idea is simple; when a sensor detects an object, the wheelchair must turn, change its speed, or stop if it is necessary. In Figure 4, there are four configurations to analyze how the system behaves under different conditions.

Figure 4.

Navigation scenarios: (a) static obstacles, (b) dynamic obstacles, (c) lateral obstacles, and (d) rear obstacles.

The next step is considering the possible navigation cases with different input conditions and what actions are needed to do. Those situations are enlisted in Table 1. Finally, the rules are defined formally with the input and the output variables.

Table 1.

Navigation cases for the fuzzy rules.

Sensors	Conditions	Wheelchair response
1–5	Blocked	Change speed, stop
1–5	One or more are blocked	Avoid collision
1–5	Clear	Move forward
1–3	Changed state suddenly	Stop moving
1, 3–5	Clear	Keep turning
1, 3–5	Blocked	Stop turning
6–8	All clear	Move backwards
6–8	All blocked	Stop moving

The complete rule set for the FOA, LCD, RCD, and BCD cases is shown in Table 2. Each case possess its own rules: FOA is an obstacle avoidance controller with nine rules, which uses sensors from S1 to S5; the LCD and RCD are used to detect obstacles on both sides of the wheelchair, using as inputs S4 and S5; and finally, the BCD is used to stop the backward movement if any of the rear sensors S6, S7, or S8 is blocked.

Table 2.

Rule set for the fuzzy navigation assistant.

Case	n	Rules
	1.	If MIN(S1, S4) is C and S2 is C and MIN(S3, S5) is C then M is N and also D is N
	2.	If MIN(S1, S4) is C and S2 is C and MIN(S3, S5) is F then M is N and also D is R
	3.	If MIN(S1, S4) is C and S2 is F and MIN(S3, S5) is C then M is SF and also D is N
	4.	If MIN(S1, S4) is C and S2 is F and MIN(S3, S5) is F then M is SF and also D is SR
FOA	5.	If MIN(S1, S4) is F and S2 is C and MIN(S3, S5) is C then M is N and also D is L
	6.	If MIN(S1, S4) is F and S2 is C and MIN(S3, S5) is F then M is N and also D is N
	7.	If MIN(S1, S4) is F and S2 is F and MIN(S3, S5) is C then M is SF and also D is SL
	8.	If MIN(S1, S4) is F and S2 is F and MIN(S3, S5) is F then M is F and also D is N
	9.	If dS1 is GFor dS2 is GF or dS3 is GF then M is N and also D is N
LCD	10.	If MIN(S1, S4) is F then M is N and D is L
	11.	If MIN(S1, S4) is C then M is N and D is SR
RCD	12.	If MIN(S3, S5) is F then M is N and D is R
	13.	If MIN(S3, S5) is C then M is N and D is SL
BCD	14.	If S6 is F and S7 is F and S8 is F then M is B and D is N
	15.	If S6 is C or S7 is C or S8 is C then M is N and D is N

FOA: forward obstacle avoidance; LCD: left collision detector; RCD: right collision detector; BCD: backward collision detector.

Defuzzification of the outputs

The Mamdani model uses the inferred set shape for calculating the crisp outputs of the system. The defuzzification block uses the center of gravity to evaluate the area under the scaled membership functions.

The semiautonomous smart wheelchair operation

The smart wheelchair can be maneuvered with a virtual joystick, implemented in the HMI. The virtual joystick functions similarly to a physical one: if the individual drags the cursor from its home position, the wheelchair starts moving with that speed and orientation. When the user releases the cursor, it returns to the home position and the wheelchair stops.

The user is able to maneuver in normal or assisted mode. In normal mode, the wheelchair steers according to the user; however, in the assisted mode, the control is shared with the FLC. The four controller modes are selected according to the virtual joystick position. In particular, the FOA is activated by dragging the cursor along the positive y-axis and the BCD is activated by dragging the cursor along the negative y-axis. Similarly, the LCD is activated when the cursor is dragged to the negative side of the x-axis and the RCD controller is activated when the user drags the cursor to the positive side of the x-axis. Moreover, when the cursor is dragged diagonally to any axis, the wheelchair returns to normal mode. Under this scheme, the user is able to switch between collision assistants and maneuver manually when it is necessary.

Tests and results

Several tests were done in order to characterize the sensory system and the hardware implemented in the wheelchair. In addition, the smart wheelchair was steered by one of the authors to observe the FLC utility in different scenarios. Both evaluation sections are described next.

Prototype characterization

Sensor’s sampling rate

A sample rate of 50 Hz was configured for data acquisition; however, this value changes when the FLC is running. The evaluation of this parameter was done using the LabVIEW tick count function, which measures the cycle length. When the FLC was active, the measured cycle length was 21.66 ms, which represents a real execution rate of 46.16 Hz. Within this cycle time, data acquisition and processing are executed.

Sensor’s coverage

This test was done to determine the sonar’s coverage and the wheelchair’s blind spots. A mapping of positions was drawn, using a 1.8 × 2.4 m (5.9 × 7.87 ft) grid, with a resolution square of 5 × 5 cm (0.16 × 0.16 ft). The wheelchair was placed in the middle of the grid, filling 288 of the 1728 squares. A square wood dowel, whose dimensions were 40 × 40 × 62 cm (1.31 × 1.31 × 2.03 ft), was placed in all the grid positions to verify detection. The measured distance was considered valid or invalid by comparing it with the grid dimensions. The process was repeated for the 1440 positions and the results are presented in Figure 5, where the sensory system’s coverage is indicated by the dark areas. The coverage mapping was extended to 60 cm from the stationary wheelchair to the left, right, front, and rear sides.

Figure 5.

The sensor’s coverage map. The dark areas indicate the detection positions in a grid resolution of 5 × 5 cm.

Detection of small objects

The evaluation was done using three tubes with different diameters. Those objects were placed in front of every sensor at different distances. The maximum distance for detection was considered the limit where the sensor loses the object. The collected data from the eight sensors were averaged, thus the values of maximum mean distance and standard deviation (SD) are presented in Table 3.

Table 3.

Obstacle detection limits.

Obstacle diameter(cm)	Maximum mean distance (cm)
0.95	40.13 ± 4.64
2.2	63.15 ± 13.86
5	98.11 ± 16.97

Wheelchair’s safe power level

As indicated on the Quickie P-222 SE datasheet, the wheelchair’s speed at full power is 3.79 m/s (8.5 mph). This power can be adjusted manually with a knob control. The FLC adjusts speed according to the space conditions; however, it is important to estimate a safe power level for avoiding dangerous collisions.

For this test, the wheelchair was placed 3 m away from a foam wall, as shown in Figure 6, then it was steered forward with different speeds. Because of the controller’s action, the wheelchair must slow down and stop before colliding. This process was repeated 10 times for each power level. In order to register the moment when the wheelchair was stopped, it was used a software chronometer activated automatically by the controller outputs. When the front sensor detects a close distance, the wheelchair must be stopped.

Figure 6.

Experimental scenario for determining the wheelchair’s safe speed for navigation.

The results obtained are presented in Table 4. There are shown different speeds versus the distance at which the wheelchair was commanded to stop, the final distance when the wheelchair really stopped and the speed versus the trial time and the number of collisions. The mean and the SD of the repetition values were calculated and presented in Table 4.

Table 4.

Collision test results.

Speed (m/s)	Stop value (cm)	Final value (cm)
0.65	47.21 ± 0.18	41.15 ± 0.78
0.86	46.66 ± 0.85	43.96 ± 0.85
0.06	43.96 ± 1.98	12.9 ± 1.7
0.25	44.11 ± 0.76	4.59 ± 0.5
0.47	44.11 ± 1.23	4.09 ± 1.05
Speed (m/s)	Trial time (s)	Collisions
0.65	6.21 ± 0.11	0/10
0.86	4.72 ± 0.04	0/10
1.06	4.03 ± 0.07	0/10
1.25	3.31 ± 0.02	10/10
1.47	3.11 ± 0.03	10/10

For the speeds 0.65, 0.86, and 1.06 m/s (1.45, 1.92, and 2.37 mph), no collisions occurred as the wheelchair was able to slow down when the wall was nearby, thus those power levels are considered safe. However, for 1.25 and 1.47 m/s (2.79 and 3.28 mph), there were collisions during all the attempts. As a consequence, those power levels are considered unsafe. To prevent accidents in other trials, it was important to use the minimum power level and the safe speed of 0.65 m/s (1.45 mph). In addition, the navigation scenarios were built with foam blocks to prevent damages.

Hardware power consumption

The power consumption of the hardware presented in Figure 1 was measured. There are two parts, which operate at different voltages: the first part uses a 12 V and it is integrated by the power supply circuit for the sensors and the bidirectional interface; and the second part is the cRIO-9103, which uses a 24 V supply. The power consumption was calculated by measuring the currents and voltages when the FLC was active and inactive.

The power consumption results are summarized in Table 5. It is clear that the maximum power consumption occurs when the controller is active.

Table 5.

Power consumption of the assistance system.

	12 V circuit (W)	24 V cRIO (W)	Total power (W)
FLC inactive	2.39	3.71	6.10
FLC active	3.01	4.75	7.76

FLC: fuzzy logic controller.

FPGA device utilization

After the compilation, the device utilization report for the implemented system was obtained. The report is presented in Table 6.

Table 6.

Device utilization report.

Total slices	100% (24,334 of 14,336)
Slice registers	52.7% (15,100 of 28,672)
Slice LUTs	79.1% (22,667 of 28,672)
Block RAMs	0.0% (0 of 96)

LUTs: Look Up Table; RAM: Random access memory.

FLC navigation evaluation

FOA considers many rules for avoiding obstacles, as presented in Table 2. In comparison, LCD, RCD, and BCD only stop the movement if obstacles are detected as the user turns the wheelchair or moves backward. According to the sensory system characterization, the limitations for those tasks are the coverage area shown in Figure 5, the size of the objects, and the wheelchair speed. For that reason, navigation tests were done only to evaluate the FOA performance. The test was developed in different scenarios, described as follows.

Corridor navigation

It is expected that the FOA assistant will help the users to navigate through long and narrow corridors. For this case, a 6-m long corridor (19.68 ft) was built. Five navigation repetitions were done; however, different widths were used: 2.5, 2, 1.5, 1, 0.8, and 0.7 m (8.2, 6.56, 4.92, 3.28, and 2.62 ft). The widest and the narrowest corridors are shown in Figure 7.

Figure 7.

Experimental scenarios: (a) 2.5-m corridor and (b) 0.7-m corridor.

During the five iterations, the wheelchair was able to complete the trial without colliding, even in the narrowest corridor. The relation between the base speed (0.65 m/s) and the speed observed at the different corridors is presented in Table 7. It was observed that the speed changed according to the corridor space. For the 2.5 and 2 m corridors, the speed was unaffected; however, the speed decreased to almost the half in the narrowest corridor. No collisions were observed during this test.

Table 7.

Speed reduction in different corridor widths.

Corridor width (m)	Trial time (s)	Speed (m/s)	Speed reduction (%)
2.5	12.75	0.65	100
2	12.79	0.65	100
1.5	13.89	0.56	86.15
1	17.25	0.46	10.76
0.8	19.94	0.37	56.92
0.7	20.12	0.33	50.76

Corridor with obstacles

There were done five navigation attempts assisted automatically by the FOA. For comparison, other five trials were done by steering manually with the virtual joystick. The trajectories were recorded on video and marked using the VideoPoint Physics Fundamentals software [version 1.0]. In the first case, it was expected that the wheelchair could steer inside a corridor to avoid collisions automatically. The corridor, whose dimensions are 4.8 m length and 2 m width (15.74 × 6.56 ft), contains square obstacles of 0.6 × 0.6 m (1.96 × 1.96 ft).

Figure 8(a) shows the obtained trajectory during one attempt. Moreover, in Figure 8(b), it is shown the trajectory obtained by manual control. The average time for both cases and the total number of collisions registered are presented in Table 8. The time corresponding to each trial was measured, besides the mean and the SD were calculated. The SD for the automated navigation was small, which indicates that the trajectories were similar during all repetitions. However, the SD for the manual control indicates more variations in the time registered between each test.

Figure 8.

The obstacle avoidance action in a corridor: (a) assisted automatically by the fuzzy controller and (b) controlled manually by the user.

Table 8.

Test time and number of collisions for the navigation experiments.

Experiment	Mean time (s)	Col.
Obstacle avoidance (fuzzy)	15.466 ± 0.43	0
Obstacle avoidance (human)	15.95 ± 2.16	0
Curved corridor (fuzzy)	25.78 ± 0.92	0
Curved corridor (human)	22.58 ± 4.42	13
Maze navigation (fuzzy)	21.14 ± 0.16	0
Maze navigation (human)	19.45 ± 3.05	0

No collisions were registered during the trials; however, using the manual control, some movements were abrupt and hesitant.

Curved corridor

The FOA allows the wheelchair to navigate inside an irregular shape corridor, like the case presented in Figure 9. The space is narrow; thus, it is difficult to take the curve.

Figure 9.

Navigation through differently shaped corridors: (a) assisted automatically by the fuzzy controller and (b) controlled manually by the user.

The fuzzy assistant helped to complete the navigation without difficulties, as shown in Figure 9(a). However, in the manual case presented in Figure 9(b), it was hard for the user to avoid abrupt movements. Moreover, it was easy to lose control and several collisions were registered.

Navigation in a small maze

For this test, a small maze was built inside a square room of 3.6 × 3.6 m (11.81 × 11.81 ft). The wheelchair, starting from one corner, had the objective to leave the maze. The obtained trajectory for the case assisted by the FLC is presented in Figure 10(a). No collisions were registered within this mode and the user was able to leave the room in every attempt. However, in the trajectory obtained during the manual control, shown in Figure 10(b), there were collisions with the walls. This trajectory shows that the user had difficulties in controlling the wheelchair because of the maze shape and the narrow space. The consumed time, the mean, and the SD were obtained for each scenario, as shown in Table 8. The SD for the automated attempt is small, thus the trajectories were similar during all the iterations. However, the SD for the manual tests indicates variations in the time registered between each test, which indicates that the attempts were irregular.

Figure 10.

Navigation through the maze: (a) assisted automatically by the fuzzy controller and (b) controlled manually by the user.

Discussion

Prototype characterization

As mentioned in the study by Lopresti et al.,¹³ the sensor’s sampling rate is a major aspect for robotic wheelchairs. Microcontrollers and computers use sequential processing architectures, thus the consumed time is accumulated from each line of the code. On the other hand, the parallel architecture implemented in this prototype allows to get data from the eight sensors, simultaneously, to process the information, and to make navigation decisions every 21.66 ms. According to the sonar datasheet, the fastest time for taking measurements is 19.255 ms, however, to drive eight sensors in any software architecture (even a real-time software) that value is accumulated by each sensor, besides the time for passing data to the computer and the processing time. As a consequence, it is very difficult to consider sampling eight sensors with any software-based system at a high-speed rate.

The sensor coverage map presented in Figure 5 shows representative data about the sensor’s coverage at short distance. It shows a cone shape extending from the emitter to 3 m, according to the sensor’s range. Besides, the mapping shows clear areas at short distances and the coverage overlap in some areas. To augment the chances of detection, some fuzzy rules from Table 2 were designed to join the coverage cones together. As an example, the FOA controller is able to detect objects in the dark areas covered by sensors S1–S5, thus the object is likely to be detected before reaching the clear positions. In addition, the high sample rate reduces the probability of missing an object during the navigation.

The results presented in Table 3 show that the sensors are able to detect small objects at close distances. The smallest object was detected at approximately 40 cm (1.31 ft), and the SD confirms that this is a regular value for the eight sensors. Furthermore, the object size is not a problem, but its shape affects the signal reflection. Some objects are invisible to the sensors because of their orientation not because of their dimensions. Finally, another limitation observed is that the PING))) sensors cannot measure accurate distances to objects behind 3 m (9.84 ft). Thus, detection for this prototype is limited to 3 m.

According to the data presented in Table 4, the speeds at 0.65, 0.86, and 1.06 m/s are safe as no collisions occurred. However, the speeds at 1.25 and 1.47 m/s are unsafe, because all the trials ended with collisions. The registered stop value is similar for all the power levels, which implies that the controller reacted at the same moment during all the tests. Likewise, after comparing the stop and the final values, it is clear that the wheelchair kept moving after the controller stopped the motors but slight variations occurred because of the acceleration. This difference is more prominent in the upper power levels. Despite the controller’s capacity to react when an object is detected, the wheelchair’s mechanical system is not fast enough to stop the movement. At the unsafe speed levels, the difference between the stop distance and the final distance is more than 40 cm (1.31 ft), which means the wheelchair kept running after the controller slowed the motors. On the contrary, for the safe speed levels, the stop distance and the final distance present similar values as the wheelchair was able to stop before colliding. The Semi-autonomous Smart Wheelchair immediate reaction is not related to the sensor rate, but it depends on the wheelchair’s physical and mechanical characteristics.

In relation to the power consumption, the wheelchair uses rechargeable sealed lead-acid batteries, whose nominal voltage is 12 V and 35AH. The maximum consumption of the additional hardware in our prototype is 7.76 W, which compared to the 26.6 W calculated in the study by Lopresti et al.,¹³ do not represent a significant consumption for the batteries. The energy consumption of the installed hardware in our prototype is 7.76W. As indicated in that article, 26.6 W reduces battery life by 3%; meanwhile, our power consumption reduces battery life only by 0.87%. Consequently, the use of the FLC will not affect the battery life or the daily charges. Power consumption is an important factor to be considered when adapting commercial wheelchairs as indicated by Mazo et al.¹²

Finally, for the device utilization, the implemented code uses a lot of the FPGA resources as seen in Table 6. A code optimization will be necessary for a future version.

FLC navigation evaluation

According to the corridor navigation with FOA, the fuzzy controller fits the wheelchair speed according to the space in corridors. Despite the speed reduction of 50% for the narrowest corridor, the wheelchair was able to complete the task without colliding. In that corridor, the space between the wheelchair and the walls was minimal.

For the navigation comparison between the manual and the automatic assistance, the tests showed that it is unsafe to complete the tasks solely by human skills. After several repetitions using the manual control, the task is likely to be completed with less problems. However, the user tends to make abrupt movements when he tries to correct his orientation. In confined areas, the user has to be very accurate with the joystick operation. However, for users with disabilities, this is complicated. The comparison between the trajectories presented in Figures 8 to 10 showed that the FLC helped to complete the task without collisions during the attempts.

Limitations

The sensor coverage was studied in two dimensions, but the signal is actually omnidirectional. As a consequence, the sensors could have different behaviors depending on its height position. This effect was not considered in the research.

The other prototypes presented in the literature^7,44,47,48 included additional behaviors such as door passing or follow walls. For this controller, it will be important to optimize the FPGA code to include more behaviors.

It was observed that the fast data acquisition and processing do not impede collisions because the mechanical system is not able to react immediately. Moreover, the wheelchair does not possess a braking engine to stop instantaneously.

The LCD, RCD, and BCD were not evaluated with the similar tests presented for the FOA. However, their operation is related to the sensor’s coverage and response.

Finally, the tests were done inside the laboratory under specific temperature and lighting conditions. No navigation tests were done with rehabilitation patients or user with disabilities.

Future work

In order to implement more functions for the navigation assistant, it will be important to optimize the FPGA code. It is necessary to develop a test protocol for the different navigation modes and with other users. Finally, it is very important to test the prototype with end users and specialized rehabilitation personal counselors to show the system usability and improve its functionality.

Conclusion

A navigation assistant was implemented in a commercial wheelchair. The tests indicated that the system’s architecture based on the FPGA, a fuzzy logic methodology, and ultrasonic sensors provide secure navigation. Furthermore, the controller is independent from software execution, which assures data acquisition from eight channels and data calculations within 21.66 ms. The sonars are able to detect different objects around the wheelchair, besides the fuzzy rules merge the sensor’s coverage to detect objects in critical points. Despite the immediate reaction of the controller, the wheelchair mechanical response is not immediate. Consequently, a safe speed of 0.65 m/s was determined. There were done tests within confined areas, which showed that the FLC modifies this speed in order to make cautious movements. Furthermore, the use of the FLC to maneuver in three different unstructured scenarios was validated. It was shown that the implemented system can help users who find it difficult to operate a wheelchair, as the assistant prevents collisions with objects when the user navigates in all directions.

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Mario Rojas

References

Fehr

Langbein

Skaar

. Adequacy of power wheelchair control interfaces for persons with severe disabilities: a clinical survey. J Rehabil Res Dev 2000; 37(3): 353–360.

Simpson

. Smart wheelchairs: a literature review. J Rehabil Res Dev 2005; 42(4): 423–436.

Leaman

. A comprehensive review of smart wheelchairs: past, present, and future. IEEE Trans Human–Mach Syst 2017; 47(4): 486–499. DOI: 10.1109/THMS.2017.2706727.

Desai

Mantha

Phalle

. Advances in smart wheelchair technology. In: 2017 international conference on nascent technologies in engineering (ICNTE), Navi Mumbai, India, 27–28 January 2017, pp. 1–7. IEEE. DOI: 10.1109/ICNTE.2017.7947914.

Nisbet

. Who’s intelligent? Wheelchair, driver or both? In: Proceedings of the 2002 international conference on control applications, 2002, vol. 2, Glasgow, UK, UK, 18–20 September 2002, pp. 760–765. IEEE. DOI: 10.1109/CCA.2002.1038696.

Hsu

Chang

. Mobility assistance design of the intelligent robotic wheelchair. Int J Adv Rob Syst 2012; 9(6): 244. DOI: 10.5772/54819. URL https://doi.org/10.5772/54819.

Levine

Bell

Jaros

. The NavChair assistive wheelchair navigation system. IEEE Trans Rehabil Eng 1999; 7(4): 443–451. DOI: 10.1109/86.808948.

Bourhis

Horn

Habert

. An autonomous vehicle for people with motor disabilities. IEEE Robot Autom Mag 2001; 8(1): 20–28. DOI: 10.1109/100.924353.

Simpson

Poirot

Baxter

. The Hephaestus smart wheelchair system. IEEE Trans Neural Syst Rehabil Eng 2002; 10(2): 118–122. DOI: 10.1109/TNSRE.2002.1031980. VHP VFF Appliction.

10.

Tyagi

Gupta

Tyagi

. Smart wheelchair using fuzzy inference system. In: 2013 IEEE global humanitarian technology conference: South Asia satellite (GHTC-SAS), Trivandrum, India UK, 23–24 August 2013, pp. 175–180. IEEE. DOI: 10.1109/GHTC-SAS.2013.6629911.

11.

Luis Conde Bento

. A behavior based fuzzy control architecture for path tracking and obstacle avoidance. In: Proceedings of the 5th Portuguese conference on automatic control (Controlo 2002), Aveiro. PT, 5–7 September 2002, pp. 341–346. University of Aveiro Portugal: IEETA.

12.

Mazo

Rodrguez

Lzaro

. Wheelchair for physically disabled people with voice, ultrasonic and infrared sensor control. Autom Rob 1995; 2(3): 203–224. DOI: 10.1007/BF00710857. URL http://link.springer.com/article/10.1007/BF00710857.Simps.

13.

Lopresti

Sharma

Simpson

. Performance testing of collision-avoidance system for power wheelchairs. J Rehabil Res Dev 2011; 48(5): 529–544.

14.

Braga

RAM

Petry

Reis

. Intell-wheels: modular development platform for intelligent wheelchairs. J Rehabil Res Dev 2011; 48(9): 1061. DOI: 10.1682/ JRRD.2010.08.0139.

15.

Boucher

Atrash

Kelouwani

. Design and validation of an intelligent wheelchair towards a clinically functional outcome. J Neuroeng Rehabil 2013; 10(1): 58. DOI: 10.1186/1743-0003-10-58.

16.

Urdiales

Peula

Fdez-Carmona

. A new multi-criteria optimization strategy for shared control in wheelchair assisted navigation. Autonom Rob 2010; 30(2): 179–197. DOI: 10.1007/s10514-010-9211-2. URL http://link.springer.com/article/10.1007/s10514-010-9211-2.

17.

Montesano

Diaz

Bhaskar

. Towards an intelligent wheelchair system for users with cerebral palsy. IEEE Trans Neural Syst Rehabil Eng 2010; 18(2): 193–202. DOI: 10.1109/TNSRE.2009.2039592. Test.

18.

Rofer

Mandel

Laue

. Controlling an automated wheelchair via joystick/head-joystick supported by smart driving assistance. In: IEEE international conference on rehabilitation robotics, 2009. ICORR 2009, Kyoto, Japan, 23–26 June 2009, pp. 743–748. IEEE. DOI: 10.1109/ICORR.2009.5209506. Interface/test.

19.

Hadj-Abdelkader

Bourhis

Cherki

. Haptic feedback control of a smart wheelchair. Appl Bion Biomech 2012; 9(2): 181–192. DOI: 10.3233/ABB-2012-0067. URL http://www.hindawi.com/journals/abb/2012/921982/abs/.Interface.

20.

Rao

Conn

Jung

. Human robot interaction: application to smart wheelchairs. In: Proceedings 2002 IEEE international conference on robotics and automation (Cat. No.02CH37292), vol. 4, Washington, DC, USA, USA, 11–15 May 2002, pp. 3583–3588. IEEE. DOI: 10.1109/ROBOT. 2002.1014265.

21.

Leishman

Horn

Bourhis

. Smart wheelchair control through a deictic approach. Rob Auton Syst 2010; 58(10): 1149–1158. DOI: 10.1016/j.robot.2010.06.007. URL http://www.sciencedirect.com/science/article/pii/S0921889010001272.

22.

Ito

Kamata

. Interactive collision avoidance based on surrounding mobility type for an intelligent powered wheelchair. Int J Adv Rob Syst 2013; 10(5): 241. DOI: 10. 5772/56450. URL https://doi.org/10.5772/56450.

23.

Gomi

Griffith

. Developing intelligent wheelchairs for the handicapped. In: Mittal

Yanco

Aronis

. (eds) Assistive technology and artificial intelligence. Number 1458 in Lecture Notes in Computer Science, Berlin/Heidelberg: Springer, 1998, pp. 150–178. ISBN 978-3-540-64790-4 978-3-540-68678 -1, http://link.springer.com/chapter/10.1007/BFb0055977.Simps (accessed 25 February 2017).

24.

How

Wang

Mihailidis

. Evaluation of an intelligent wheelchair system for older adults with cognitive impairments. J Neuroeng Rehabil 2013; 10(1): 90. DOI: 10.1186/1743-0003-10-90.

25.

Nguyen

. Experimental study on a smart wheelchair system using a combination of stereoscopic and spherical vision. In: 2013 35th annual international conference of the IEEE engineering in medicine and biology society (EMBC), Osaka, Japan, 3–7 July 2013, pp. 4597–4600. IEEE. DOI: 10.1109/EMBC.2013.6610571. Test/interface.

26.

Tomari

Kobayashi

Kuno

. Enhancing wheelchair manoeuvrability for severe impairment users. Int J Adv Rob Syst 2013; 10(2): 92–104. DOI: 10.5772/55477. Shared control similar.

27.

Zhao

Duan

. Human cooperative wheelchair with brain–machine interaction based on shared control strategy. IEEE/ASME Trans Mechatronics 2017; 22(1): 185–195. DOI: 10.1109/TMECH.2016.2606642.

28.

Baklouti

Amor

Jallouli

. Reactive control architecture for mobile robot autonomous navigation. Rob Autonom Syst 2017; 89: 9–14. DOI: 10.1016/j.robot.2016.09.001.

29.

Sinyukov

Desmond

Dickerman

. Multi-modal control framework for a semiautonomous wheelchair using modular sensor designs. Intell Serv Rob 2014; 7(3): 145–155. DOI: 10.1007/s11370-014-0149-7. URL http://link.springer.com/article/10.1007/s11370-014-0149-7.Sensors.

30.

Eid

Giakoumidis

Saddik

. A novel eye-gaze-controlled wheelchair system for navigating unknown environments: case study with a person with ALS. IEEE Access 2016; 4: 558–573. DOI: 10.1109/ACCESS.2016.2520093.

31.

Zhang

Yan

. Control of a wheelchair in an indoor environment based on a brain–computer interface and automated navigation. IEEE Trans Neural Syst Rehabil Eng 2016; 24(1): 128–139. DOI: 10.1109/TNSRE.2015.2439298.

32.

Bayar

Akar

Yayan

. Fuzzy logic based design of classical behaviors for mobile robots in ROS middleware. In: 2014 IEEE international symposium on innovations in intelligent systems and applications (INISTA) proceedings, Alberobello, Italy, 23–25 June 2014, pp. 162–169. IEEE. DOI: 10.1109/INISTA.2014.6873613. Fuzzy/behaviour.

33.

Cavanini

Benetazzo

Freddi

. SLAM-based autonomous wheelchair navigation system for AAL scenarios. In: 2014 IEEE/ASME 10th international conference on mechatronic and embedded systems and applications (MESA), Senigallia, Italy, 10–12 September 2014, pp. 1–5. IEEE. DOI: 10.1109/MESA.2014.6935628. Scan and map.

34.

TalebiFard

Sattar

Mitchell

. A risk assessment infrastructure for powered wheelchair motion commands without full sensor coverage. In: 2014 IEEE/RSJ international conference on intelligent robots and systems, Chicago, IL, USA, 14–18 September 2014, pp. 3592–3597. DOI: 10.1109/IROS.2014.6943065.

35.

Wang

Savkin

Clout

. An intelligent robotic hospital bed for safe transportation of critical neurosurgery patients along crowded hospital corridors. IEEE Trans Neural Syst Rehabil Eng 2015; 23(5): 744–754. DOI: 10.1109/TNSRE.2014.2347377.

36.

Wang

Matveev

Savkin

. A semi-autonomous motorized mobile hospital bed for safe transportation of head injury patients in dynamic hospital environments without bed switching. Robotica 2016; 34(8): 1880–1897. DOI: 10.1017/S0263574714002641.

37.

Bryner

Parnell

. Comparing and contrasting FPGA and microprocessor system design and development. White Paper, XILINX, 2004.

38.

Kung

Wang

Yang

. Realization of a motion control IC for electric-powered wheelchair. In: 2010 8th IEEE international conference on industrial informatics (INDIN), Osaka, Japan, Japan, 13–16 July 2010, pp. 523–528. IEEE. DOI: 10.1109/INDIN.2010.5549687. Fpga.

39.

Poplawski

Bialko

. Implementation of parallel fuzzy logic controller in FPGA circuit for guiding electric wheelchair. In: 2008 conference on human system interactions, Krakow, Poland, 25–27 May 2008, pp. 405–408. IEEE. DOI: 10.1109/HSI.2008.4581473.

40.

Hong

Nakhaeinia

Karasfi

. Application of fuzzy logic in mobile robot navigation, fuzzy logic. Controls, Concepts, Theories and Applications 2012; 1: 21–36. DOI: 10.5772/36358.

41.

Melendez

Castillo

Valdez

. Optimal design of the fuzzy navigation system for a mobile robot using evolutionary algorithms. Int J Adv Rob Syst 2013; 10(2): 139. DOI: 10.5772/55561. URL https://doi.org/10.5772/55561.

42.

Borenstein

Koren

. The vector field histogram-fast obstacle avoidance for mobile robots. IEEE Trans Robot Autom 1991; 7(3): 278–288. DOI: 10.1109/70.88137.

43.

Omrane

Masmoudi

. Fuzzy logic based control for autonomous mobile robot navigation. Comput Intell Neurosci 2016; 2016: 10. DOI: 10.1155/2016/9548482. URL https://www.hindawi.com/journals/cin/2016/9548482/.

44.

Kuo

Yeh

. Development of autonomous navigation robotic wheelchairs using programmable system-on-chip based distributed computing architecture. In: IEEE international conference on systems, man and cybernetics, 2007. ISIC, Montreal, Que., Canada, 7–10 October 2007, pp. 2939–2944. IEEE. DOI: 10.1109/ICSMC.2007.4414123. Fuzzy/behaviour/low cost.

45.

Thongchai

Kawamura

. Application of fuzzy control to a sonar-based obstacle avoidance mobile robot. In: Proceedings of the 2000 IEEE international conference on control applications, 2000, Anchorage, AK, USA, USA, 27–27 September 2000, pp. 425–430. IEEE. DOI: 10.1109/CCA.2000.897461Fuzzy

46.

Zal

Chen

Chi

. Fuzzy controller based subsumption behavior architecture for autonomous robotic wheelchair. In: 2013 International conference on advanced robotics and intelligent systems (ARIS), Tainan, Taiwan, 31 May–2 June 2013, pp. 158–163. IEEE. DOI: 10.1109/ARIS.2013.6573552. Fuzzy/behaviour.

47.

Pires

Nunes

. A Wheelchair steered through voice commands and assisted by a reactive fuzzy-logic controller. J Intell Rob Syst 2002; 34(3): 301–314. DOI: 10.1023/A:1016363605613. URL http://link.springer.com/article/10.1023/A%3A1016363605613.Fuzzy.

48.

Rahman

Jafri

. Two layered behaviour based navigation of a mobile robot in an unstructured environment using fuzzy logic. In: Proceedings of the IEEE symposium on emerging technologies, 2005, Islamabad, Pakistan, 18–18 September 2005, pp. 230–235. IEEE. DOI: 10.1109/ICET.2005.1558886. Fuzzy.

A fuzzy logic navigation controller implemented in hardware for an electric wheelchair

Abstract

Keywords

Introduction

Methods

The electric wheelchair architecture

The FLC strategy

Fuzzification of the variables

The rules base and the inference engine

Defuzzification of the outputs

The semiautonomous smart wheelchair operation

Tests and results

Prototype characterization

Sensor’s sampling rate

Sensor’s coverage

Detection of small objects

Wheelchair’s safe power level

Hardware power consumption

FPGA device utilization

FLC navigation evaluation

Corridor navigation

Corridor with obstacles

Curved corridor

Navigation in a small maze

Discussion

Prototype characterization

FLC navigation evaluation

Limitations

Future work

Conclusion

Footnotes

Declaration of conflicting interests

Funding

ORCID iD

References