Sage Journals: Discover world-class research

Abstract

Search and rescue robots gained a significant attention in the past, as they assist firefighters during their rescue missions. The opportunity to move autonomously or remotely controlled with intelligent sensor technology, to detect victims in unknown fire smoke environments, introduces a growing technology in fire engineering. Since sensor systems are a component of mobile robots, there is a demand for intelligent robot vision, especially for human detection in fire smoke environments. In this article, an overview of sensor technologies and their algorithms for human detection in regular and smoky environments is presented. These sensor technologies are categorized into single sensor and multi-sensor systems. Novel sensor approaches are led by artificial intelligence, 3D mapping and multi-sensor fusion. The article provides a contribution for future research directions in algorithms and applications and supports decision-makers in fire engineering to get knowledge in trends, novel applications and challenges in this field of research.

Keywords

Computer vision deep learning firefighting human detection mobile robot search and rescue sensor fusion smoky environment

Introduction

The increasing number of catastrophes and numerous apartment fires over the last years caused a tremendous loss of human life (19,178 fire deaths worldwide in 2019¹). This results that search and rescue robots gained a significant attention in the past, as they assist firefighters during their search and rescue mission.² Especially novel sensor approaches equipped on these mobile robots support firefighters in human detection in hazardous environments. This can be explained by the fact that firefighters are on the one hand exposed to unstable building structures and on the other hand their cognitive fatigue, due to long search and rescue missions, reduce the efficient and time-critical human detection in smoky indoor environments.³

Teixeira et al.⁴ classify sensor systems into single sensor and multi-sensor (more than one sensor) systems. The physical intrinsic traits of a human that can be measured are shape, emissivity, scent or internal motion. The shape of a human is detectable with visual cameras or range finding sensors, whereas the emissivity of a human can be detected with thermal cameras. Scent is detectable with chemical sensors and radar sensors are suitable for internal motion, for example, heartbeat or respiration.⁴

Figure 1 illustrates the human detection approach, defined by the author of this work, during search and rescue missions in more detail.

Figure 1.

Human detection approach during search and rescue missions.

The sensor system (single or multi) is equipped on a mobile robot that moves autonomously or remotely controlled by an operator outside the scene. The sensors detect humans in a regular or smoky indoor environment. The data are sent to a human–machine interface and the operator is enabled to better assess whether a human is in the indoor environment or not.

The article has the following structure: At the beginning, the vision and range finding single sensor approaches for human detection in regular and smoky environments are presented. The following section describes the human detection methods with multi-sensor approaches and their application in smoky environments. Afterwards, a brief overview of the reliability of the communication between the operator and the mobile robot in the area affected by smoke is then given. Finally, a conclusion of the work and future research directions are provided in the last section.

Single sensor approaches

In this section, an overview of various single sensor approaches in regular and smoky environments is given. This includes imaging and distance measuring sensors, as well as touch and chemical sensors.

Thermal camera

Thermal cameras are common for imaging physiological conditions of the skin surface to analyse its condition. For clinical diagnosis or monitoring, Yoon et al.⁵ examine the temperature of regions of the forearms. The objective of the work⁵ is to segment regions of the forearms automatically in an infrared image and locate regions of importance. Herry et al.⁶ also point out ways to segment and register (automatically) hands and arms on infrared images. The basis of this segmentation are mathematical morphological operations. Human detection with a thermal camera during search and rescue missions is introduced by Königs and Schulz.⁷ The authors⁷ figure out that a simple threshold approach detects high temperature blobs indoor and outdoor. They also describe that the results of the approach become worse on warmer days and indoor during unusual circumstances, for example, in fire smoke contaminated environments. The work of Sharma et al.⁸ describes two algorithms for detecting humans in visibility at night. The authors⁸ propose an algorithm, which uses the theory of the radiation of black bodies (hot spot), and an algorithm which subtracts the background. Average accuracy of the first algorithm is 69% and of the second algorithm is 79%. The authors⁸ note that the result depends on the background conditions, for example, illumination or brightness. Compared to this, another work⁹ suggests an algorithm for human detection based on thermal pixel values with an accuracy of up to 90%. An alternative approach of sensing humans in infrared illustrations is introduced by Setjo et al.¹⁰ The authors¹⁰ apply the classifier by Haar to sense the presence of humans in infrared images. The results are: The more distance between the object and the camera, the less accurate the detection of the human. Positions that are not turned facing to the front are also difficult to detect. The approach can sense two or more persons in the infrared image with the following postures: overlap each other, in front of one another or side by side. The approach reaches false detection (e.g. infrared shadow formed by reflecting glass) if an object with attributes like a person is sensed. In the work of Cerutti et al.,¹¹ the authors conduct an exploratory study on human detection in outdoor scenes by using an 8 × 8 pixels thermal camera. Experiments show that the described approach achieves 97% of accuracy. Ivašić-Kos et al.¹² present findings of human sensing on a custom data set of infrared images with YOLOv3 convolutional neural network (CNN). Experimental tests result in an improved performance for the trained model (30% average precision) compared to the genuine model (7% average precision). Also, Perdana et al.¹³ use a CNN (MobileNet) to sense humans on infrared images taken from an unmanned aerial vehicle (UAV). The approach reaches an average precision of 82.49%. Further, the study of Hoshino et al.¹⁴ examines the human detection with infrared images taken from an UAV using a trained single shot detector (SSD) CNN. Jaradat and Valles¹⁵ propose a human detection approach with trained thermal images on a CNN and perform an average precision of 96.3%. Cruz Ulloa et al.¹⁶ determine an average precision of 85% using a YOLOv3 CNN based on thermal images. With thermal images trained on a YOLOv4 CNN, Tsai et al.¹⁷ achieve an average precision of greater than 95% for the standing, sitting, lying and squatting postures of humans.

The two works of Starr and Lattimer^18,19 evaluate thermal cameras and discuss the application and capabilities of thermal cameras for distance mapping and navigation in fire smoke contaminated environments. The authors^18,19 point out that thermal cameras outperform optical cameras in the presence of thick smoke and light smoke with high temperature. However, thermal cameras are noisier. Image processing is able to reduce the noise. Also, Kim et al.²⁰ focus on a method to find hot spots in a fire smoke contaminated environment. The authors²⁰ solve this by analysing the statistical characteristics in infrared images and fusing the data with the Bayesian estimation (calculation of fire sections). Afterwards, the method is validated in large-scale fire tests (with several conditions in temperature and smoke). In a smoky environment (indoor), the approach shows an accurate hotspot on the infrared image.

An approach of human detection in challenging visibility conditions is provided by Kapusta and Beeson.²¹ The authors²¹ use a 3D thermal camera and reach an accuracy of up to 90% at 20 fps inside a smoky environment. Also, Krišto et al.²² submit a work using a thermal camera (320 × 240 pixels) and the YOLO CNN to detect humans in a foggy environment. The trained model achieves an average precision of 97.85% in human detection.

Optical camera

Prior work for human detection with an optical camera is mainly in the field of UAVs.^14,23

–26 Also, the two works of Liu et al.^27,28 propose a novel approach to automatically detect humans in real time with an optical camera. Experimental tests show that the system is suitable in regular environments. Xiao et al.²⁹ show a new human detection methodology by using a learning classifier (online). Experiments show that the approach achieves higher rates of detection compared to 3D human sensing approaches. Ghidary et al.³⁰ introduce another method of human detection in indoor environments. The work³⁰ determines the 3D position of face and head with the method depth from focus. The authors calculate the position of the person in the scene with the position of the camera and the tilt and pan angles. The approach measures with an error rate of less than 0.1 m in a distance range from 0.9 m to 3.4 m. Myrzin et al.³¹ present a current human sensing approach that connects rotation invariant histogram of oriented gradients (RIHOG) characteristics, binarized normed gradients processing, as well as segmentation of skin. For evaluation, RGB body images are used. The quadratic support vector machines model demonstrates an average precision of 90.4% in human detection and shows the performance of RIHOG characteristics for human sensing.

A further study¹⁹ evaluates the efficiency of visual cameras working in the visible area. The work¹⁹ uses an edge detection method and considers the value switch between each pixel. Next, it assesses each of these differences to a standard value. Experiments result that the image is substantially mitigated due to smoke if the viewability is about 4 m. Below a visibility of about 1 m, the method is unable to locate any features. Tests during light smoke with high temperature show no influence on the identified edges.

Radar

A novel approach of human detection in a cluttered environment using passive millimetre wave radar is introduced by Nanzer and Rogers.³² The method consists of a Bayesian classification formulation and shows promising results in classification of humans and objects. An algorithm for human detection using an ultra-wideband impulse-based mono static radar is demonstrated by Chang et al.³³ Also, Stephan and Santra³⁴ show a framework for precise human sensing in an environment (indoor) based on radar. In another work, Chen³⁵ discusses radar backscattering from a human, as well as how to analyse the radar backscattering from a subject of interest. Finally, an example of the radar backscattering from human arm and leg motions is introduced. Cui and Dahnoun³⁶ discuss a framework for person sensing with a millimetre wave radar. The authors³⁶ show a sensitivity of over 90% in indoor environments and mention that the correctness can be enhanced with two radars. Bandala et al.³⁷ use an ultra-wideband radar to sense the existence of persons based on their movement of respiration. The researchers demonstrate an accuracy of 97.28%.

A further work³⁸ demonstrates how ultra-wideband radar is able to sense objects in hazardous environments. The researcher³⁸ shows that ultra-wideband radar is able to pass through bad weather (e.g. fog) and sense objects compared to other vision or range finding systems. Yamauchi³⁸ develops algorithms that process radar data to reduce ground reflections. In the research work ‘Evaluation of Navigation Sensors in Fire Smoke Environments’, the authors Starr and Lattimer¹⁹ examine radar (26 GHz) to quantify the performance of this range finding technology. The system delivers a small change in distance (6% with outliers) during a thick smoke, as well as during a light smoke with high temperature test. Mandischer et al.³⁹ introduce a strategy to steer in low visibility environments with radar and an efficient radar filtering process. The process consists of three steps and enables a consistent noise reduction.

In the work of Sakai and Aoki,⁴⁰ the authors focus on fire rescue operations and the issue that the view of firefighters is blocked by thick smoke. The authors⁴⁰ use a millimetre wave radar to design a 3D map from the reflection data. Afterwards, Sakai and Aoki⁴⁰ execute a 3D grouping and labelling process for the human detection. An experiment verifies the detection of humans and objects in a smoky environment.

LIDAR

Taipalus and Ahtiainen⁴¹ introduce a novel algorithm for human detection with a 2D light detection and ranging (LIDAR) system. The LIDAR receives echoes of human legs captured from the level of knees. With a catalogue of predefined characteristics, the authors⁴¹ use these for clustering the reflections. If two clusters meet the defined requirements (close together, leg classification), a human object is created.

Experimental tests demonstrate that the algorithm can detect humans in regular indoor environments. Lucian et al.⁴² also present an approach for human detection by counting pairs of legs. The authors⁴² achieve a detection rate of 96.8%. Another work⁴³ describes the human detection task with a 3D LIDAR system mounted on an UAV. Based on a CNN, the authors⁴³ propose an approach which can evaluate point clouds sent by a LIDAR system (3D). The algorithm consists of three elements: pre-processing data, post-processing data and person classification. An enhanced learning framework for human sensing with LIDAR (3D) is introduced by Yan et al.⁴⁴ Experiments show that the human classifier, created by the authors,⁴⁴ performs better than other solutions in terms of accuracy.

In the work of Yamauchi,³⁸ the author shows during an experiment that LIDAR cannot penetrate thick smoke for object detection. A further work¹⁹ evaluates two LIDAR systems (single-echo and multi-echo) operating at 905 nm. The results show that if smoke affects a viewability of 4 m, the measurement of the distance is involved. If smoke affects a viewability of around 1 m, a LIDAR system reverts the distance to the bottom of the smoke and cannot sense the object’s upper edges. Within light smoke with high temperature, a LIDAR system returns a result comparable as under regular environments. The outcomes show that LIDAR is not appropriate of providing accurate range finding in thick smoke environments. Similarly, Pascoal et al.⁴⁵ review various LIDAR systems. In regular environments (adequate viewability), LIDAR systems have a high accuracy and linearity, particularly after warming up. In thick smoke environments, LIDAR provides erroneous or saturated outputs.

Sonar

Blumrosen et al.⁴⁶ show a wideband sonar system for human detection in indoor environments. The technology manages to classify different human activities in human detection. An alternative method for face detection using sonar is presented by Miao et al.⁴⁷ The authors⁴⁷ use different configurations of transmitter–receiver pairs and demonstrate that the system can achieve a high detection rate of up to 99%.

Another work¹⁹ evaluates sonar systems at 42 and 50 kHz. In thick smoke or light smoke with high temperature, the sonar systems result with an error rate of 10%–20%.

Tactile sensor

Omata and Terunuma⁴⁸ develop a tactile sensor with a piezoelectric element for human detection and demonstrate an excellent performance. The sensor distinguishes the difference in hardness of a soft tissue, just as the human touch does. Also, Moromugi et al.⁴⁹ develop a tactile sensor to estimate human’s soft tissue hardness. With the tactile sensor, a measurement of the hardness with any pressing force is possible. Also, the tactile sensor removes automatically unwanted effects from the variability of the pressing force. The tactile sensor demonstrates a standard deviation of 0.447 under fluctuating pressing force, compared to 0.7 with a conventional durometer measurement under controlled pressing force. Two other works^50,51 also examine tactile sensors. The authors^50,51 successfully demonstrate the proposed detection principles through evaluation experiments. They reduce the contact force and optimize the layout of the reference plane. The hardness error is suppressed to a shore A hardness of 4 (contact force below 1 N). This outcome relates to the detection ability of the hardness of fat.

Chemical sensor

Güntner et al.⁵² introduce a research paper with sensor arrays (compact and orthogonal) to sense the skin- and breath-emitted metabolic tracers’ ammonia, acetone, CO₂ and isoprene. It contains three metal-oxide sensors (nanostructured), each particularly customized at the nanoscale (selective and sensitive tracer detection), additionally with commercial humidity and CO₂ sensors. The sensor array rapidly detects ammonia, isoprene as well as acetone with high precision. Another human sensing approach based on various ambient gas parameters is introduced by Kamal et al.⁵³ The authors⁵³ perform various experiments under controlled situations and accomplish the task with several machine learning algorithms: RandomForest, Bagging, J48 and IBK. The method achieves an accuracy of up to 95%.

Table 1 summarizes the prior work of single sensor approaches reviewed in this section.

Table 1.

Overview of single sensor approaches.

Sensor	Regular environment	Smoky environment
	Human detection	Object detection	Human detection
Thermal camera	5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17	18, 19, 20	21, 22
Optical camera	14, 23, 24, 25, 26, 27, 28, 29, 30, 31	19	–
Radar	32, 33, 34, 35, 36, 37	19, 38, 39	40
LIDAR	41, 42, 43, 44	19, 38, 45	–
Sonar	46, 47	19	–
Tactile sensor	48, 49, 50, 51	–	–
Chemical sensor	52, 53	×	–

– no prior work; × not possible.

Table 1 shows that single sensor approaches are mainly examined in regular environments. A comparison between the number of prior work in regular and smoky environments shows the focus of research.

Overall, the most suitable single sensor approach for human detection in regular environments is a thermal camera. A thermal camera offers the advantage that the human temperature can be clearly distinguished to the background temperature. Although the results show that accuracy in human detection decreases with distance from the human, this approach offers an accuracy of up to 90% at 13 m. Future trends tend to use deep learning approaches to automatically detect humans in various environments.

An optical camera is another promising approach for human detection in regular environments. The latest approaches examine automatic human detection in real time by using online learning classifier. At this time, the prior work does not characterize the effects of different real-world factors on system performance, such as scattered background conditions.

LIDAR is, in comparison to radar and sonar, the most accurate single range finding approach for human detection in regular environments. Experiments show that the shape of a human can be detected with this approach and result in an accuracy of greater than 90%. However, this range finding approach has several disadvantages. For instance, an ideal human detection system should be able to distinguish human shapes with similar shapes and surface textures in the same scene.

Radar is the most accurate single range finding approach for human detection in smoky environments. Experiments show that radar provides less than 6% change in distance during thick smoke, as well as during light smoke with high temperature. However, the approach has the disadvantage that there are none out of the shelf radar systems available.

In smoky environments, thermal cameras are the most promising single sensor approach for detecting hot objects. Various experiments show that thermal cameras detect hot spots in thick smoke. This type of sensor outperforms optical camera systems in the presence of thick smoke and light smoke with high temperature. However, the approach has various disadvantages. For instance, thermal cameras generate more noise and they cannot distinguish between objects and humans in the same temperature range.

Multi-sensor approaches

This section gives an overview of multi-sensor approaches in regular and smoky environments. This includes the multi-sensor fusion of imaging and distance measuring sensors.

Radar and LIDAR

A novel work⁵⁴ focuses on a fusion approach by using radar and LIDAR, particularly in human detection. The approach contains object detection, human sensing and occluded depth creation. The multi-sensor system detects humans by merging the human features by LIDAR as well as Doppler radar distribution. Another work⁵⁵ presents a sensor fusion scheme for detecting partially concealed humans.

A further work⁵⁶ investigates techniques in which sensor data from LIDAR and ultra-wideband radar can be fused. The author⁵⁶ examines the fusion of sensors, where data from numerous sensors are blended in a single representation. The smoke detection algorithm makes use of LIDAR to verify, whether it is enclosed by smoke. If yes, the ‘obstacle avoidance behaviour’ (radar-based) is activated. If not, the LIDAR algorithm with higher performance is used. Compared to a single sensor approach, this approach allows robustness to environmental unpredictability. Further works^57
–59 examine the fusion of radar and LIDAR data in more detail. The first work⁵⁷ describes a fusion strategy to reduce the negative influence on LIDAR in thick smoke by introducing a high bandwidth radar (2D). The authors⁵⁷ also introduce a LIDAR–radar ratio, which correlates with the number of aerosols in the surroundings. The second work⁵⁸ introduces two ways for merging LIDAR and radar data to achieve simultaneous localization and mapping in an environment with low viewability. The third work⁵⁹ focuses in optimizing the quality of the map. In addition, the authors⁵⁹ reach to model the concentration of aerosols with combined radar and LIDAR data. The authors⁵⁹ use a finite model.

Majer et al.⁶⁰ present another multi-sensor system with ultra-wideband radar and LIDAR. In numerous experiments, the authors⁶⁰ demonstrate that the merging approach not only improves the performance, it also efficiently overcomes sensing malfunctions by thick smoke.

Thermal camera and optical camera

The work of Han and Bhanu⁶¹ proposes a scheme to automatically locate the correspondence between the human shape extracted from infrared and RGB images. The method reaches appropriate results for human shape extraction and image registration, as well as demonstrates improvements in performance compared to none fused sensors. The two works of Kang et al.^62,63 focus on a new method for human sensing by merging RGB and infrared images. The authors^62,63 use a normalized cross-correlation of average histogram algorithm to analyse the relationship between infrared and RGB data. Experiments show that the approach yields multiple human detection in real time. Correa et al.⁶⁴ present a robot that uses an infrared camera to sense humans and an optical camera to acquire a video of the scene. In the work of Scebba et al.,⁶⁵ the authors explore a method for the automatic sensing of the human’s nose. The approach is based on facial landmark sensing from merged infrared and RGB images. The authors⁶⁵ evaluate the detection rate as well as the precision of the algorithm acquired from several objects under challenging situations. Experiments result in an appropriate spatial accuracy (two pixels average root-mean-square error) and a high detection rate (up to 92%). Brunner et al.⁶⁶ figure out that optical cameras, compared to thermal cameras, degrade the detection accuracy. In the work of Gupta,⁶⁷ infrared images are used for extracting local characteristics and RGB images are utilized for skin detection. The skin detection is conducted with a ‘non-parametric histogram-based trained skin pixel likelihood model’. Three feature descriptors (SIFT, SURF, HOG) are examined in the scenery of human sensing in infrared images. Results illustrate that human sensing is more suitable in thermal images than skin sensing in RGB images. Dawdi et al.⁶⁸ submit a research work for human detection using shape template matching. In real time, the authors⁶⁸ blend RGB images into thermal images, processed on board of an UAV.

Thermal camera and LIDAR

Gleichauf et al.⁶⁹ present a triangular calibration approach with a thermal camera and a 2D LIDAR. The authors⁶⁹ assign every laser measurement within the field of view with a corresponding thermal pixel. The system is verified for distances up to 10 m. Another work⁷⁰ proposes a special application of a thermal camera and a LIDAR by mounting the sensors on an UAV.

Starr⁷¹ develops a sensor fusion system with long-wavelength stereo infrared and a spinning LIDAR. The data are fused in a multi-resolution 3D voxel domain and use evidential theory to model free space and occupied states. Furthermore, Starr⁷¹ presents a heuristic approach to split attenuated LIDAR signals from low attenuation signals. Additional works^72,73 submit an approach of sensor combination with LIDAR and stereo infrared for enhanced range finding in smoky environments. The approach allows range finding in regular and smoky environments. It relies on the ability of thermal cameras in smoky environments and the accuracy of LIDAR in regular environments. The authors⁷³ combine sensor data in a multi-resolution voxel domain (3D) for free space and occupied states. Experiments in a scenario (indoor) with low smoke, thick smoke and fire illustrate that the system is suitable in regular and smoky environments. LIDAR does not provide any data in thick smoke.

Thermal camera and radar

The work of Davidson et al.⁷⁴ demonstrates the fusion of real data from a thermal camera and radar. The authors⁷⁴ describe a solution to detect objects in 3D, including signal processing of subsystems and fusion. Quaranta et al.⁷⁵ propose an algorithm for the fusion of 2D radar and thermal camera data. Another work on human detection based on radar and thermal imaging is examined by Ulrich et al.⁷⁶ The authors⁷⁶ use a multi-sensor fusion approach that first conducts a process step, where frames of candidates for mirrored or real humans are defined in thermal images. These frames are associated to the radar objectives and then classified. Radar is used to distinguish mirrored and real humans, which is challenging by thermal cameras only. In summary, the authors⁷⁶ demonstrate the performance of the approach with real measurements. Bao et al.⁷⁷ propose an algorithm with a radar and a thermal camera based on an interacting multiple model and unscented Kalman filter, which reduces the radar radiation time.

Kim et al.⁷⁸ develop a multi-sensor system that utilizes sensor fusion between an infrared camera (stereo) and a radar (frequency modulated continuous wave (FMCW)) to detect targets through low viewability in real time. The infrared camera (stereo) is applied to obtain information (3D) about the location, while the radar offers accurate distance data. Matching radar data with those in the thermal image (3D), the accuracy of the infrared camera (stereo) is updated. Fire tests with and without smoke demonstrate that the error of distance for the infrared camera (stereo) is lowered due to sensor fusion of the infrared camera (stereo) with FMCW radar.

Optical camera and radar

Streubel et al.⁷⁹ introduce another novel approach by fusing stereo optical and radar data to significantly improve the tracking of humans in an indoor environment. The authors⁷⁹ demonstrate that the precision of the proposed fusion system achieves an accuracy of 90.3% compared to 83.4% and 81.8% obtained by an optical camera and a radar only. Ivanovs et al.⁸⁰ explore a new sensor configuration for autonomous human sensing and contact-free monitoring of respiration using a combination of an optical camera and an ultra-wideband radar. In the proposed approach, a CNN is used to process camera data and sense humans in real time. After the human is detected, radar is utilized to execute the monitoring of respiration. Experiments with humans (different poses) illustrate that the suitable detection range is 4 m in regular environments.

Jiang et al.⁸¹ present an object detection approach with collected data from a smoky environment by fusing millimetre wave radar with visual data. Experimental results prove the algorithm’s feasibility.

Thermal camera, radar and LIDAR

The merging of LIDAR, thermal and radar data for object detection in environments with a minimal visibility is described by Fritsche et al.⁸² The authors⁸² merge radar and LIDAR scans for a simultaneous localization and mapping (SLAM) approach that generates an environmental map. Additionally, the authors create heated obstacle scans with merged radar and LIDAR data in aggregation with infrared images.

Table 2 summarizes the prior work of multi-sensor approaches reviewed in this section.

Table 2.

Overview of multi-sensor approaches.

Sensor	Regular environment	Smoky environment
	Human detection	Object detection	Human detection
Radar + LIDAR	54, 55	56, 57, 59, 58	60
Thermal camera + Optical camera	61, 62, 63, 64, 65, 66, 67, 68		–
Thermal camera + LIDAR	69, 70	71, 72, 73	–
Thermal camera + Radar	74, 75,76, 77	78	–
Optical camera + Radar	79, 80	–	81
Thermal camera + LIDAR + Radar	–	82	–

– no prior work.

Thermal cameras, in combination with an additional sensor, seems to be the most used approach for human detection in regular environments. Results show a detection rate of up to 92% while using a thermal camera to detect humans and an optical camera to acquire the scene. Thermal cameras deliver 3D scene information in smoky environments, while radar provides accurate distance information in the field of view.

Comparing Table 2 with the single sensor approaches listed in Table 1, it becomes clear that the extract of current prior work in sensor fusion does not leverage the full potential of their specific sensor combinations. Therefore, human detection, especially in smoky environments, is a research field which needs to be considered more closely and characterizes a gap in research.

Challenges in communication

In this section, a short overview of challenges in communication between the operator and the mobile robot is given.

A status and challenges in human–robot interaction is given by Sheridan.⁸³ The author⁸³ indicates among others the challenge in compensation for dropouts and delays in communication.

Tadokoro⁸⁴ presents challenges of future mobile robots. Besides several challenges in robotics, Tadokoro⁸⁴ points out that wireless communication is one of these challenges. The author⁸⁴ mentions in this context stability, bandwidth and latency to transfer large quantity of data between the operator and the mobile robot.

Also Liu et al.⁸⁵ identify the challenges in robot-assisted firefighting as well as future trends for efficient and smart operations. The authors⁸⁵ mention for instance a challenge in precise information in real time for targeted decision-making.

A design of a wireless communication device for use in a smoky environment is given by Orita et al.⁸⁶ The authors⁸⁶ design a wireless communicator at which multiple dropping devices create a wireless ad hoc network along the way of the mobile robot.

Conclusion and future work

The research article provides a compact overview about sensors for search and rescue robots assisting firefighters in their mission of detecting humans. The main areas involved in this article are single sensor and multi-sensor approaches used for human detection in regular and smoky environments. The prior work and novel algorithms are presented. Also a brief overview of challenges in communication between the operator and the mobile robot is given.

In general, methods to detect humans during search and rescue missions in hazardous environments are increasing in recent years, which indicates a growing field of research in this area. Especially, the latest trend in deep learning approaches with thermal cameras as well as multi-sensor systems confirm the demand of rescue robots with intelligent sensors to accomplish the automatic human detection in smoky indoor environments. Furthermore, an overview of sensors that detect humans under rubble in earthquake-affected areas with dust would be of interest.

Footnotes

Declaration of conflicting interests

The author declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Sebastian Gelfert

References

CTIF. International Association of Fire and Rescue Services. World fire statistics. https://www.ctif.org/sites/default/files/2021-06/CTIF_Report26_0.pdf (accessed 15 July 2022).

Casper

Murphy

. Human-robot interactions during the robot-assisted urban search and rescue response at the World Trade Center. IEEE Trans Syst Man Cybern, Part B (Cybern) 2003; 33(3): 367–385.

Dadwhal

Kumar

Sardana

. Data-driven skin detection in cluttered search and rescue environments. IEEE Sens J 2020; 20: 3697–3708.

Teixeira

Dublon

Savvides

. A survey of human-sensing: methods for detecting presence, count, location, track, and identity. ACM Comput Surv 2010; 5(1): 59–69.

Yoon

Kim

Lee

, et al. Image segmentation of human forearms in infrared image. In: 2006 International conference of the IEEE Engineering in Medicine and Biology Society, New York City, NY, USA, 30 August–3 September 2006, pp. 2762–2765. IEEE. DOI: 10.1109/IEMBS.2006.260706.

Herry

Frize

Goubran

. Segmentation and landmark identification in infrared images of the human body. In: 2006 International conference of the IEEE Engineering in Medicine and Biology Society, New York City, NY, USA, 30 August–3 September 2006, pp. 957–960. IEEE. DOI: 10.1109/IEMBS.2006.260077.

Königs

Schulz

. Evaluation of thermal imaging for people detection in outdoor scenarios. In: 2012 IEEE international symposium on safety, security, and rescue robotics (SSRR), College Station, TX, USA, 5–8 November 2012, pp. 1–6. IEEE. DOI: 10.1109/SSRR.2012.6523883.

Sharma

Agrawal

Srivastava

, et al. Review of human detection techniques in night vision. In: 2017 International conference on wireless communications, signal processing and networking (WiSPNET ), Chennai, India, 22–24 March 2017, pp. 2216–2220. IEEE. DOI: 10.1109/WiSPNET.2017.8300153.

Lee

Bui

. Temperature model and human detection in thermal image. In: 2018 IEEE seventh global conference on consumer electronics (GCCE), Nara, Japan, 9–12 October 2018, pp. 57–59. IEEE. DOI: 10.1109/GCCE.2018.8574800.

10.

Setjo

Achmad

Faridah . Thermal image human detection using Haar-cascade classifier. In: 2017 Seventh international annual engineering seminar (InAES), Yogyakarta, Indonesia, 1–2 August 2017, pp. 1–6. IEEE. DOI: 10.1109/INAES.2017.8068554.

11.

Cerutti

Prasad

Farella

. Convolutional neural network on embedded platform for people presence detection in low resolution thermal images. In: 2019 IEEE international conference on acoustics, speech and signal processing (ICASSP), Brighton, UK, 12–17 May 2019, pp. 7610–7614. IEEE. DOI: 10.1109/ICASSP.2019.8682998.

12.

Ivašić-Kos

Krišto

Pobar

. Human detection in thermal imaging using YOLO. In: 2019 Fifth international conference on computer and technology applications, Istanbul, Turkey, 16–17 April 2019, pp. 20–24. Association for Computing Machinery. DOI: 10.1145/3323933.3324076.

13.

Perdana

Risnumawan

Sulistijono

. Automatic aerial victim detection on low-cost thermal camera using convolutional neural network. In: 2020 International symposium on community-centric systems (CcS), Tokyo, Japan, 23–26 September 2020, pp. 1–5. IEEE. DOI: 10.1109/CcS49175.2020.9231433.

14.

Hoshino

Seo

Yamazaki

. A study for detecting disaster victims using multi-copter drone with a thermographic camera and image object recognition by SSD. In: 2021 IEEE/ASME international conference on advanced intelligent mechatronics (AIM), Delft, Netherlands, 12–16 July 2021, pp. 162–167. IEEE. DOI: 10.1109/AIM46487.2021.9517524.

15.

Jaradat

Valles

. A victims detection approach for burning building sites using convolutional neural networks. In: 2020 10th annual computing and communication workshop and conference (CCWC), Las Vegas, NV, USA, 6–8 January 2020, pp. 280–286. IEEE. DOI: 10.1109/CCWC47524.2020.9031275.

16.

Cruz Ulloa

Prieto Sánchez

Barrientos

, et al. Autonomous thermal vision robotic system for victims recognition in search and rescue missions. Sensors 2021; 21(21): 7346.

17.

Tsai

Liao

Yuan

. Using deep learning with thermal imaging for human detection in heavy smoke scenarios. Sensors 2022; 22(14): 5351.

18.

Starr

Lattimer

. Application of thermal infrared stereo vision in fire environments. In: 2013 IEEE/ASME international conference on advanced intelligent mechatronics, Wollongong, Australia, 9–12 July 2013, pp. 1675–1680. IEEE. DOI: 10.1109/AIM.2013.6584337.

19.

Starr

Lattimer

. Evaluation of navigation sensors in fire smoke environments. Fire Technol 2013; 50(6): 1459–1481.

20.

Kim

Sung

Lattimer

. Bayesian estimation based real-time fire-heading in smoke-filled indoor environments using thermal imagery. In: 2017 IEEE international conference on robotics and automation (ICRA), Singapore, 29 May–3 June 2017, pp. 5231–5236. IEEE. DOI: 10.1109/ICRA.2017.7989613.

21.

Kapusta

Beeson

. Person tracking and gesture recognition in challenging visibility conditions using 3D thermal sensing. In: 2016 25th IEEE international symposium on robot and human interactive communication (RO-MAN), Columbia, NY, USA, 26–31 August 2016, pp. 1112–1119. IEEE. DOI: 10.1109/ROMAN.2016.7745247.

22.

Krišto

Ivasic-Kos

Pobar

. Thermal object detection in difficult weather conditions using YOLO. IEEE Access 2020; 8: 125459–125476.

23.

Lygouras

Santavas

Taitzoglou

, et al. Unsupervised human detection with an embedded vision system on a fully autonomous UAV for search and rescue operations. Sensors 2019; 19: 1–20.

24.

Liu

Szirányi

. Real-time human detection and gesture recognition for on-board UAV rescue. Sensors 2021; 21(6): 2180.

25.

Safadinho

Ramos

Ribeiro

, et al. UAV landing using computer vision techniques for human detection. Sensors 2020; 20(3): 613.

26.

Cardona

Ramirez-Rugeles

Mojica-Nava

, et al. Visual victim detection and quadrotor-swarm coordination control in search and rescue environment. Int J Electr Comput Eng 2021; 11(3): 2079.

27.

Liu

Cui

, et al. Real-time human detection and tracking in complex environments using single RGBD camera. In: 2013 IEEE international conference on image processing, Melbourne, Australia, 15–18 September 2013, pp. 3088–3092. IEEE. DOI: 10.1109/ICIP.2013.6738636.

28.

Liu

Zhang

, et al. Detecting and tracking people in real time with RGB-D camera. Pattern Recognit Lett 2015; 53: 16–23.

29.

Xiao

Kamat

Menassa

. Human tracking from single RGB-D camera using online learning. Image Vis Comput 2019; 88: 67–75.

30.

Ghidary

Nakata

Takamori

, et al. Human detection and localization at indoor environment by home robot. In: 2000 IEEE international conference on systems, man and cybernetics, Nashville, TN, USA, 8–11 October 2000, pp. 1360–1365. IEEE. DOI: 10.1109/ICSMC.2000.886043.

31.

Myrzin

Tsoy

Bai

, et al. Visual data processing framework for a skin-based human detection. In: Ronzhin

Rigoll

Meshcheryakov

(eds) Interactive collaborative robotics, Cham, Germany: Springer International Publishing, 2021, pp. 138–149. DOI: 10.1007/978-3-030-87725-5_12.

32.

Nanzer

Rogers

. Human presence detection using millimeter-wave radiometry. IEEE Trans Microw Theory Tech 2007; 55(12): 2727–2733.

33.

Chang

Wolf

Burdick

. Human detection and tracking via Ultra-Wideband (UWB) radar. In: 2010 IEEE international conference on robotics and automation, Anchorage, AK, USA, 3–8 May 2010, pp. 452–457. IEEE. DOI: 10.1109/ROBOT.2010.5509451.

34.

Stephan

Santra

. Radar-based human target detection using deep residual U-net for smart home applications. In: 2019 18th IEEE international conference on machine learning and applications (ICMLA), Boca Raton, FL, USA, 16–19 December 2019, pp. 175–182. IEEE. DOI: 10.1109/ICMLA.2019.00035.

35.

Chen

. Detection and analysis of human motion by radar. In: 2008 IEEE radar conference, Rome, Italy, 26–30 May 2008, pp. 1–4. IEEE. DOI: 10.1109/RADAR.2008.4721059.

36.

Cui

Dahnoun

. High precision human detection and tracking using millimeter-wave radars. IEEE Aerosp Electron Syst Mag 2021; 36(1): 22–32.

37.

Bandala

Sybingco

Maningo

JMZ

, et al. Human presence detection using ultra wide band signal for fire extinguishing robot. In: 2020 IEEE Region 10 conference (TENCON), Osaka, Japan, 16–19 November 2020, pp. 1176–1181. IEEE. DOI: 10.1109/TENCON50793.2020.9293893.

38.

Yamauchi

. All-weather perception for man-portable robots using ultra-wideband radar. In: 2010 IEEE international conference on robotics and automation, Anchorage, AK, USA, 3–8 May 2010, pp. 3610–3615. IEEE. DOI: 10.1109/ROBOT.2010.5509804.

39.

Mandischer

Eddine

Huesing

, et al. Bots2ReC: radar localization in low visibility indoor environments. In: 2019 IEEE international symposium on safety, security, and rescue robotics (SSRR), Würzburg, Germany, 2–4 September 2019, pp. 158–163. IEEE. DOI: 10.1109/SSRR.2019.8848981.

40.

Sakai

Aoki

. Human and object detection in smoke-filled space using millimeter-wave radar based measurement system. In: 18th international conference on pattern recognition (ICPR), Hong Kong, 20–24 August 2006, pp. 750–750. IEEE. DOI: 10.1109/ICPR.2006.634.

41.

Taipalus

Ahtiainen

. Human detection and tracking with knee-high mobile 2D LIDAR. In: 2011 IEEE international conference on robotics and biomimetics, Phuket, Thailand, 7–11 December 2011, pp. 1672–1677. IEEE. DOI: 10.1109/ROBIO.2011.6181529.

42.

Lucian

Sandu

Orghidan

, et al. Human leg detection from depth sensing. In: 2018 IEEE international conference on automation, quality and testing, robotics (AQTR), Cluj-Napoca, Romania, 24–26 May 2018, pp. 1–5. IEEE. DOI: 10.1109/AQTR.2018.8402735.

43.

Hayton

JNC

Barros

Premebida

, et al. CNN-based human detection using a 3D LiDAR onboard a UAV. In: 2020 IEEE international conference on autonomous robot systems and competitions (ICARSC), Ponta Delgada, Portugal, 15–16 April 2020, pp. 312–318. IEEE. DOI: 10.1109/ICARSC49921.2020.9096075.

44.

Yan

Duckett

Bellotto

. Online learning for 3D LiDAR-based human detection: experimental analysis of point cloud clustering and classification methods. Auton Robots 2020; 44: 147–164.

45.

Pascoal

Marques

de Almeida

. Assessment of laser range finders in risky environments. In: 2008 IEEE/RSJ international conference on intelligent robots and systems, Nice, France, 22–26 September 2008, pp. 3533–3538. IEEE. DOI: 10.1109/IROS.2008.4650961.

46.

Blumrosen

Fishman

Yovel

. Noncontact wideband sonar for human activity detection and classification. IEEE Sens J 2014; 14: 4043–4054.

47.

Miao

, et al. A novel ultrasonic sensing based human face recognition. In: 2008 IEEE ultrasonics symposium, Beijing, China, 2–5 November 2008, pp. 1873–1876. IEEE. DOI: 10.1109/ULTSYM.2008.0461.

48.

Omata

Terunuma

. Development of new type tactile sensor for detecting hardness and/or software of an object like the human hand. In: TRANSDUCERS ‘91: 1991 International conference on solid-state sensors and actuators. Digest of technical papers, San Francisco, CA, USA, 24–27 June 1991, pp. 868–871. IEEE. DOI: 10.1109/SENSOR.1991.149023.

49.

Moromugi

Kumano

Ueda

, et al. A sensor to measure hardness of human tissue. In: 2006 IEEE SENSORS, Daegu, Korea, 22–25 October 2006, pp. 388–391. IEEE. DOI: 10.1109/ICSENS.2007.355487.

50.

Maeda

Terao

Suzuki

, et al. A tactile sensor with the reference plane for detection abilities of frictional force and human body hardness aimed to medical applications. In: 2015 28th IEEE international conference on micro electro mechanical systems (MEMS), Estoril, Portugal, 18–22 January 2015, pp. 253–256. IEEE. DOI: 10.1109/MEMSYS.2015.7050936.

51.

Maeda

Terao

Shimokawa

, et al. Reduction of contact force dependence on the MEMS hardness sensor using reference plane to detect human body hardness. In: 2015 IEEE SENSORS, Busan, Korea, 1–4 November 2015, pp. 1–4. IEEE. DOI: 10.1109/ICSENS.2015.7370231.

52.

Güntner

Pineau

Mochalski

, et al. Sniffing entrapped humans with sensor arrays. Anal Chem 2018; 90(8): 4940–4945.

53.

Kamal

Ahmed

Toha

, et al. Intelligent human counting through environmental sensing in closed indoor settings. Mob Netw Appl 2019; 25(2): 474–490.

54.

Kwon

Son

Hyun

, et al. Radar-Lidar sensor fusion scheme using occluded depth generation for pedestrian detection. In: 2017 International conference on computational science and computational intelligence (CSCI), Las Vegas, NV, USA, 14–16 December 2017, pp. 1811–1812. IEEE. DOI: 10.1109/CSCI.2017.322.

55.

Kwon

Hyun

Lee

, et al. A low-complexity scheme for partially occluded pedestrian detection using LIDAR-RADAR sensor fusion. In: 2016 IEEE 22nd international conference on embedded and real-time computing systems and applications (RTCSA), Daegu, Korea, 17–19 August 2016, p. 104. IEEE. DOI: 10.1109/RTCSA.2016.20.

56.

Yamauchi

. Fusing ultra-wideband radar and LIDAR for small UGV navigation in all-weather conditions. In: Gerhart

Gage

Shoemaker

(eds) Unmanned systems technology XII, Orlando, FL, USA: SPIE, 2010, pp. 250–259. DOI: 10.1117/12.850386.

57.

Fritsche

Kueppers

Briese

, et al. Radar and LiDAR sensor fusion in low visibility environments. In: ICINCO, Vol. 2, Lisbon, Portugal, 2016, pp. 30–36. DOI: 10.5220/0005960200300036.

58.

Fritsche

Wagner

. Modeling structure and aerosol concentration with fused radar and LiDAR data in environments with changing visibility. In: 2017 IEEE/RSJ international conference on intelligent robots and systems (IROS), Vancouver, Canada, 24–28 September 2017, pp. 2685–2690. IEEE. DOI: 10.1109/IROS.2017.8206093.

59.

Fritsche

Kueppers

Briese

, et al. Fusing LiDAR and radar data to perform SLAM in harsh environments. In: Madani

Peaucelle

Gusikhin

(eds) Informatics in control, automation and robotics. Lecture notes in electrical engineering, Vol. 430. Springer, Cham, 2018, pp. 175–189. DOI: 10.1007/978-3-319-55011-4_9.

60.

Majer

Yan

Broughton

, et al. Learning to see through haze: radar-based human detection for adverse weather conditions. In: 2019 European conference on mobile robots (ECMR), Prague, Czech Republic, 4–6 September 2019, pp. 1–7. IEEE. DOI: 10.1109/ECMR.2019.8870954.

61.

Han

Bhanu

. Fusion of color and infrared video for moving human detection. Pattern Recognit 2007; 40(6): 1771–1784.

62.

Kang

Jeon

Kyoung

, et al. Multiple human body tracking using the fusion of CCD and thermal image sensor. In: 2011 IEEE applied imagery pattern recognition workshop (AIPR), Washington, DC, USA, 11–13 October 2011, pp. 1–4. IEEE. DOI: 10.1109/AIPR.2011.6176356.

63.

Kang

Jeon

Kyoung

, et al. Multiple human body tracking based on normalized cross-correlation of average histogram using the fusion of color and thermal image sensor. In: 2012 IEEE RO-MAN: the 21st IEEE international symposium on robot and human interactive communication, Paris, France, 9–13 September 2012, pp. 398–403. IEEE. DOI: 10.1109/ROMAN.2012.6343785.

64.

Correa

Hermosilla

Verschae

, et al. Human detection and identification by robots using thermal and visual information in domestic environments. J Intell Robot Syst 2011; 66(1–2): 223–243.

65.

Scebba

Tüshaus

Karlen

. Multispectral camera fusion increases robustness of ROI detection for biosignal estimation with nearables in real-world scenarios. In: 2018 40th annual international conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Honolulu, HI, USA, 18–21 July 2018, pp. 5672–5675. IEEE. DOI: 10.1109/EMBC.2018.8513501.

66.

Brunner

Peynot

Vidal-Calleja

. Combining multiple sensor modalities for a localisation robust to smoke. In: 2011 IEEE/RSJ international conference on intelligent robots and systems (IROS), San Francisco, CA, USA, 25–30 September 2011, pp. 2489–2496. IEEE. DOI: 10.1109/IROS.2011.6094614.

67.

Gupta

. A fusion of visible and infrared images for victim detection. In: High performance vision intelligence. Singapore: Springer, 2020, pp. 171–183. DOI: 10.1007/978-981-15-6844-2_12.

68.

Dawdi

Abdalla

Elkalyoubi

, et al. Locating victims in hot environments using combined thermal and optical imaging. Comput Electr Eng 2020; 85: 106697. DOI: 10.1016/j.compeleceng.2020.106697.

69.

Gleichauf

Pfitzner

May

. Sensor fusion of a 2D laser scanner and a thermal camera. In: 14th International conference on informatics in control, automation and robotics (ICINCO), Madrid, Spain, 2017, pp. 398–405. DOI: 10.5220/0006397303980405.

70.

Lee

Har

Kum

. Drone-assisted disaster management: finding victims via infrared camera and LIDAR sensor fusion. In: 2016 Third Asia-Pacific world congress on computer science and engineering (APWC on CSE), Nadi, Fiji, 5–6 December 2016, pp. 84–89. IEEE. DOI: 10.1109/APWC-on-CSE.2016.025.

71.

Starr

. Rangefinding in fire smoke environments. PhD Thesis. Virginia Tech, 2016.

72.

Starr

Lattimer

. A comparison of IR stereo vision and LIDAR for use in fire environments. In: IEEE SENSORS, Taipei, Taiwan, 28–31 October 2012, pp. 1–4. IEEE. DOI: 10.1109/ICSENS.2012.6411591.

73.

Starr

Lattimer

. Evidential sensor fusion of long-wavelength infrared stereo vision and 3D-LIDAR for rangefinding in fire environments. Fire Technol 2017; 53(6): 1961–1983.

74.

Davidson

Symons

Everett

. Sensor fusion system for infrared and radar. In: The IEE seminar on signal processing solutions, London, UK, 11 October 2005, pp. 1–5. IEEE. DOI: 10.1049/ic:20050271.

75.

Quaranta

Balzarotti

. Technique for radar and infrared search and track data fusion. Opt Eng 2013; 52(4): 1–14.

76.

Ulrich

Hess

Abdulatif

, et al. Person recognition based on micro-Doppler and thermal infrared camera fusion for firefighting. In: 2018 21st international conference on information fusion (FUSION), Cambridge, UK, 10–13 July 2018, pp. 919–926. IEEE. DOI: 10.23919/ICIF.2018.8455723.

77.

Bao

Zhang

Sabahi

. An improved radar and infrared sensor tracking fusion algorithm based on IMM-UKF. In: 2019 IEEE 16th international conference on networking, sensing and control (ICNSC), Banff, Canada, 9–11 May 2019, pp. 420–423. IEEE. DOI: 10.1109/ICNSC.2019.8743212.

78.

Kim

Starr

Lattimer

. Firefighting robot stereo infrared vision and radar sensor fusion for imaging through smoke. Fire Technol 2014; 51(4): 823–845.

79.

Streubel

Yang

. Fusion of stereo camera and MIMO-FMCW radar for pedestrian tracking in indoor environments. In: 2016 19th international conference on information fusion (FUSION), Heidelberg, Germany, 5–8 July 2016, pp. 565–572. IEEE.

80.

Ivanovs

Nikitenko

Di Castro

, et al. Multisensor low-cost system for real time human detection and remote respiration monitoring. In: 2019 Third IEEE international conference on robotic computing (IRC), Naples, Italy, 25–27 February 2019, pp. 254–257. IEEE. DOI: 10.1109/IRC.2019.00047.

81.

Jiang

Wang

Liang

. Target detection and adaptive tracking based on multisensor data fusion in a smoke environment. In: 2022 Eighth international conference on control, automation and robotics (ICCAR), Xiamen, China, 8–10 April 2022, pp. 420–425. IEEE. DOI: 10.1109/ICCAR55106.2022.9782600.

82.

Fritsche

Zeise

Hemme

, et al. Fusion of radar, LiDAR and thermal information for hazard detection in low visibility environments. In: 2017 IEEE international symposium on safety, security and rescue robotics (SSRR), Shanghai, China, 11–13 October 2017, pp. 96–101. IEEE. DOI: 10.1109/SSRR.2017.8088146.

83.

Sheridan

. Human-robot interaction: status and challenges. Hum Factors 2016; 58(4): 525–532.

84.

Tadokoro

. Rescue robotics challenge. In: 2010 IEEE workshop on advanced robotics and its social impacts, Seoul, Korea (South), 26–28 October 2010, pp. 92–98. IEEE. DOI: 10.1109/ARSO.2010.5680049.

85.

Liu

Cang

, et al. Robot-assisted smart firefighting and interdisciplinary perspectives. In: 2016 22nd International conference on automation and computing (ICAC), Colchester, UK, 7–8 September 2016, pp. 395–401. IEEE. DOI: 10.1109/IConAC.2016.7604952.

86.

Orita

Takaba

Fukao

. Design and installation of waterproof wireless communication device to support firefighting activities. In: 2021 21st International conference on control, automation and systems (ICCAS), Jeju, Korea (South), 12–15 October 2021, pp. 1883–1888. IEEE. DOI: 10.23919/ICCAS52745.2021.9649987.

A sensor review for human detection with robotic systems in regular and smoky environments

Abstract

Keywords

Introduction

Single sensor approaches

Thermal camera

Optical camera

Radar

LIDAR

Sonar

Tactile sensor

Chemical sensor

Multi-sensor approaches

Radar and LIDAR

Thermal camera and optical camera

Thermal camera and LIDAR

Thermal camera and radar

Optical camera and radar

Thermal camera, radar and LIDAR

Challenges in communication

Conclusion and future work

Footnotes

Declaration of conflicting interests

Funding

ORCID iD

References