Abstract
This paper describes the underwater data collection utilizing stationary sensor node and movable sensor node. We consider applying the system to investigate the conditions of marine ecosystem around the coral reef at Okinawa, Japan. The framework and fundamental design of our proposed underwater information gathering system under development are introduced. It is also reported the development of the prototypes of the sensory node and movable sensor node. Basic examinations utilizing such prototype at underwater environment are also described. This paper also describes experimental results of image gathering from via wireless local area network communication. Our future trials based on the current developmental situation are also discussed.
1. Introduction
The water covers the earth with over 70%. It includes oceans, rivers, lakes, ponds, and glaciers. The ocean is the big one among them. In recent reports related to climate changes [1, 2], the changes in the ocean environment will be affecting our society. Currently, actual condition surveys on underwater environment and marine ecosystems are conducted in the world. In Japan, there are coral reefs [3] around Okinawa islands (Figure 1) and coral reef check [4] is also conducting to investigate their status. Currently, the divers and the experts are doing such investigations. In general investigation process, water status data and observation data related to sea animals (species, number, size, etc.) around the coral are collected. However, the works are a heavy physical burden to the investigators and it makes it difficult to carry out a continuous or frequent survey. Therefore, it is very important to make such survey more effective and support investigation activities.
Coral and symbionts in Motobu, Okinawa, Japan.
In recent years, sensor network technologies [5, 6] are currently developed to be used in various practical applications. A typical application of sensor networks is to monitor the environmental situation. LUSTER (light under shrub thicket for environmental research) is a typical environmental research by utilizing the sensor network technology and it reported significant works. Selavo et al. [7] reported the design frameworks of LUSTER and considerable issues to realize reliable long-term data collection. Moreover, robotics for environmental monitoring and field robotics become important research topics of recent robotics [8, 9].
As previous works, Yang et al. [10] proposed the concept of restricted mobile sensor (RMS) network and a sea depth measurement method utilizing sensor nodes. Wada et al. [11] reported the development of an intelligent buoy and the experiments of seawater temperature measurement. Watanabe et al. [12] proposed the concept of a coral reef monitoring system and developed a prototype of a compact autonomous unmanned vehicle (AUV). It was for measuring the shape of the environment like the bottom of the ocean based on ultrasonic sonar sensor. Vasilescu et al. [13] proposed an underwater sensor network system with stationary nodes and an underwater vehicle. Their system utilized acoustic communication for collecting temperature and pressure data at underwater environment. Ultrasonic and acoustic communications are standard approach for underwater application. Cella et al. [14] proposed to introduce the radio frequency (RF) communication for the underwater data transmission. They discussed the theoretical analysis of underwater RF communication and showed its practical possibility of conducting the experimental results. The conditions related to the seawater are important barometers for understanding the status or the trend of the underwater environment. However, for investigation of coral reef ecosystem, visual observation data are highly significant. The status of coral bleaching [15] and the symbiont data (species, a number, size, etc.) are significant investigation targets [4]. Thus, it is required to collect and accumulate underwater image data for supporting the reef check missions and to construct a system that can be capable of transmitting the image data.
Therefore, we also have started to conduct the research and development to contribute to survey activities the coral reef by using image data. Our proposed system is also a sort of wireless sensor network that consists of stationary sensor node and movable sensor node that installs the camera.
In this paper, we introduce our research and development activities of our wireless sensor network system for collecting underwater data. Also, we report current experimental results by utilizing the prototypes of the sensor nodes.
In Section 2, the concept of our proposed underwater data collection system is described and Section 3 describes the prototypes of developed sensor nodes. In Section 4, the results of the data collection experiment with the prototypes are shown. Discussions related to further subjects are done in Section 5. Finally, in Section 6, we summarize this paper.
2. System Concept
In this section, we describe the concept of our proposed data collection system for the underwater environment. The conceptual figure of our proposed underwater data collection system is shown in Figure 2. Deployed stationary sensor node and movable sensor node are mutually connected to each other via wireless communication. They organize a temporal information infrastructure and realize and collect underwater data at each location. All collected data are transmitted to the host computer through the network that is organized by the sensor nodes.
Concept of underwater information gathering system utilizing stationary point sensor node and movable sensor node.
Stationary sensor node contributes to the collection of the underwater environment data at deployed location for medium and long term. By deploying plural ones and extending their connection, simultaneous data collection to cover the wider region is also realized. It also can contribute to on-demand observation on designated stationary sensor node through the network. Movable sensor node utilizes its mobility in the underwater environment for collecting the data at the blind spot of stationary sensor nodes. It also can approach the observation target and collect detailed data by controlling own position and posture. For supporting observation activities, image is important data. Therefore, basically, each sensor node installs a camera for collecting underwater image. The other sensors for measuring underwater conditions can be also installed to each sensor node depending on the mission.
Our motivation is to realize spatial-temporal underwater data collection based on mutual connection between these sensor nodes for supporting the investigation activities on marine ecosystems. Realization of this system would be to expect to provide novel marine observation and investigation method that exceeds the limits of the present method. It would be also expected to detect the signs of the environmental changes in the early stage and discover new facts in marine biology, ecology, and environmental science. Especially, our current target is to support observation and investigation at the coral reef of Okinawa, Japan. The state of coral reef is one of the barometers on the climate change of the oceanic environment. Typical phenomena in the coral reef are, for example, coral bleaching under environmental stress and leading to widespread devastation of the corals by the outbreak of a starfish Acanthaster planci that feed on corals. In order to notice symptom of such phenomena, continuous and seamless observation is required. Thus, it is important to contribute visual survey and investigation by accumulating the image data at the coral reef. Therefore, the sensor nodes organized the proposed system and installed the camera and it can be utilized to monitor and record visual data for not only investigation activities but also secure mission execution. Moreover, stationary sensor nodes can provide bird's-eye view for controlling a movable sensor node. It would contribute efficient navigation of a movable sensor node to the target site.
In this paper, we describe the elemental technologies and the framework for the on-line underwater image collection based on the above-mentioned concept.
3. Prototype of Sensor Nodes
We are currently developing the prototypes for realizing the system for underwater data collection. In this section, our current prototypes of the sensor nodes utilizing the wireless LAN are described.
3.1. Stationary Sensor Node
The stationary sensor node is to collect sensory data, including underwater image at the fixed point. For deploying the stationary sensor node at demanded location, self-contained type system configuration is required, including the wireless communication device and the built-in battery for power supply to the sensor node. Moreover, the stationary sensor node must automatically provide underwater data every designated interval. Therefore, in order to capture underwater images and transfer them via wireless communication, we design a basic architecture of the stationary sensor node [16] that consists of a communication unit, sensor unit, and a tether. Figure 3 shows the first prototype of the stationary sensor node and its configuration. Communication unit is equipped with a computer with wireless LAN adaptor, a GPS module (Akizuki Co. GT-703F/L), and the batteries and it floats on the seawater surface for communicating with the others via wireless network. Sensor unit is the part for mooring underwater. It is equipped with a camera, a temperature sensor, and a 3-axis acceleration sensor for measuring underwater conditions, temporally. The tether is the part that connects between communication unit and the sensor unit for power supply and data/command transmissions. In Figure 3, thinner lines indicate the information connection and thicker lines indicate the power supply connection. The depth of the sensor unit is adjustable depending on the tether length. Thus, the sensor unit does not carry the battery.
First prototype of stationary sensor node.
Figure 4 shows the outlook of preliminary image capturing experiment and captured images (640 × 480 [pixel]) by using the first prototype of the stationary sensor node. In this experiment, the sensor unit did not equip the anchor. As the experimental results, we discussed these preliminary results with the coral reef researchers and found that the posture of the sensor unit was changed by the water flow. Especially, the motion in the yaw orientation was slow but the range was large and it was not sometime suitable to observe the captured underwater image. Moreover, it was also found that higher resolution of the image would be suitable for detailed visual recognition in long distance.
Captured image by using first prototype of stationary sensor node.
Figure 5 shows second prototypes of the stationary sensor node. First prototype is for preliminary examination to collect the underwater conditions. Communication unit of this first prototype floated on the sea surface for image providing via wireless LAN and the sensor unit is to be set into sea water for capturing the images. At every designated time period, captured image by the sensor unit is transferred to the host computer for data collection via the communication unit of the sensor node.
Second prototype of the stationary sensor node.
Based on the result by utilizing first prototype of stationary sensor node, we designed second prototype (right picture in Figure 4) that has the sensor unit with three thrusters (RoboPlus Co., 90 [w], 24.5 [N]). The basic system structure is common to both of the prototypes, and the dotted rectangle part in Figure 5 indicates additional part for controlling the thrusters. These thrusters are utilized for adjusting actively the posture of the sensor unit. This sensor unit is also equipped with a camera (Logicool C905 m, maximum resolution 1600 × 1200 [pixel]), a temperature sensor (Strawberry Linux Co.), and an acceleration sensor (Vitec Co., Ltd., TDS01V).
3.2. Movable Sensor Node
A major role of stationary sensor node is to collect the data deployed location for medium and long term. However, there are the blind spots and areas that can not observed by the stationary sensor nodes. Data collected by getting closer and moving around the target are not suitable for the stationary sensor nodes. Such investigations are sometimes required for detailed survey on interesting issues. Therefore, our system introduces a movable sensor node for the subjects. For our objective, autonomous operation is not suitable. Thus, in our system, the movable sensor node is remotely controlled based on the operator's commands.
Left picture in Figure 6 shows a prototype of the movable sensor node that is equipped with the pairs of the thrusters (RoboPlus Co., 150 [w], 38 [N]) in vertical and horizontal direction of its main body. Right picture in Figure 6 shows the device for maneuvering the movable sensor node. Figure 7 shows an example view of the operation interface and Figure 8 indicates the fundamental system configuration for control and sensor data collection including underwater images. The operation computer and the movable sensor unit are connected by wired ethernet cable (tether). As mentioned above, in this prototype, the operator watches the image which is provided by built-in camera of the movable sensor node and sends the motion control command by using the game pad. Based on the motion control command, the rotational velocity command on each thruster is derived kinematically. A network connected Pan-Tilt-Zoom camera (Panasonic BB-TS165) for capturing the images is mounted on the anterior part of the sensor node. The camera is connected to the network hub with wireless antenna and can directly provide the captured image for the external network to which the operation computer connects. A depth meter (KELLER PR-36XW), a 3-axis attitude sensor (Silicon Sensing Systems TruePoint), and a gyro (Silicon Sensing Systems CRS03-04R) for the thrusters are also equipped to measure and estimate the state of the movable sensor node. We assume that we operate the movable sensor node from the vessel or boat. Thus, power supply and control command transmissions are done by wired cable. The operator monitors transferred image from the on-mount network camera and maneuvers the movable sensor node by using an input device.
A prototype of movable sensor node and operation devices.
An example view of the operation interface.
Fundamental control system of movable sensor node.
Figure 9 shows an extended system configuration of the movable sensor node with a subsystem for transferring captured images. Additional subsystem is the computer for data forwarding and it joined to the same network that the operation computer and the movable sensor node connected. The subsystem captures the images from the camera on the movable sensor unit via wired network and transfers them via wireless network sequentially. The subsystem executes such process independently of fundamental operation and control system of the movable sensor node. Figure 10 shows practical setup of the subsystem for transferring the image data. Right-side computer captures the images from the movable sensor node and transfers them to left-side computer via a wireless network. We conducted preliminary experiments of capturing underwater images by using movable sensor node at the swimming pool (Figure 11). We reorganized captured image transferring subsystem after the examinations. In this experiment, the mission was to follow the line on the bottom of the swimming pool and the operator executed it by monitoring camera data. Figure 12 shows collected images (640 × 480 [pixel]) from a movable sensor node. In this experiment, a JPEG file is utilized for image capture. Collected sensory data can be monitored on the computer at the operation site in real time. The movable sensor node also can provide captured images for the client computer during the mission via wireless network.
Extended system configuration with a subsystem for transferring captured images via wireless network.
Captured data transferring subsystem (left: client computer to receive and store transferred images, right: data forwarding computer).
Outlook of preliminary experiment with movable sensor node at the pool.
Transferred and stored images.
3.3. Wireless Communication
In order to collect underwater data online, wireless communication among the sensor nodes should be introduced. Some of the ad hoc and mesh network protocols are developed and we will select an adequate one for our purpose. We had experimented communication performance of wireless local area network (LAN) and confirmed that it works well on the sea and around the coastal area [6]. At present, the usual IP protocol is utilized for fundamental examinations and experiments.
4. Image Gathering Experiment at Sea
In this section, we describe the experimental results and current trials with developed prototypes of sensor nodes.
Utilizing prototype of stationary sensor node, we have conducted the fundamental experiments in the swimming pool and also practical experiments at the coastal area near, Motobu, Okinawa. Figure 13 shows outlooks of the experiments with our second prototype of the stationary sensor node. In this experiment, sensor node was fixed to the anchor at the sea bottom. The depth of seawater was 2.0 [m] and the sensor unit was set at a depth of 0.5 [m] (distance between the sensor unit and the sea bottom was approximately 1.5 [m]). For movable sensor node, the experiment has been conducted in the swimming pool. Figure 14 shows the outlook of the experiment.
Outlooks of data collection experiments with the second prototype of stationary sensor node.
Outlook of the experiment utilizing movable sensor node in the coastal area of Motobu, Okinawa.
Figure 15 shows captured images (resolution 1600 × 1200 [pixel]) that were transferred every 1.0 [sec] from stationary sensor node to the host computer at the coast (the duration of experiment: approximately 30 [min]). The observation target was a colony of acroporid coral and its size was approximately 0.6 × 0.8 [m]. Collected time data were described below each image and its format is “year (4 digits) + month (2 digits) + day (2 digits) + hour (2 digits) + minute (2 digits) + second (2 digits) +. extension (file type indicator)” in a similar manner to that in [17]. During the experiments, the thrusters were not utilized because the posture of the sensor unit was not affected by the flow of the seawater.
Collected images of the coral from stationary sensor node.
Figure 16 shows collected images from movable sensor node and a period of the collection was not fixed. Here, collected time data were also described below each image and the format is “year (4 digits) + month (2 digits) + day (2 digits) + hour (2 digits) + minute (2 digits) + second (2 digits) + millisecond (3 digits) +. extension (file type indicator)”. In this experiment, a JPEG file is utilized for image capture.
Transferred and stored images with their filename included captured time data.
These experimental results show the potential of our developed prototypes and there is a good possibility of contributing online underwater data collection by integrating them into the system.
5. Discussions
We had described our current research and development for the platforms and experimental examinations in the swimming pool or coastal area. However, there are the subjects on each elemental technology.
For stationary sensor node, it is required to realize image collection with spatiotemporal data. We are now implementing it on the stationary sensor node platform and we will have experimental examinations. Also, data collection experiments with plural sensor nodes should be conducted and evaluated. Power supply for long-term data collection is still a technical subject. On-demand underwater observation would be interesting experimental subject by utilizing developed platform. For movable sensor node, we will attempt image data collection experiments by a teleoperated movable sensor node in the coastal area. Conducting those, we will able to obtain some subjects on control method for underwater navigation and stable observation. Image data collection with temporal data was already experimented and localization of the sensor node is also another important subject. A localization method based on image processing with bird's-eye view is now being developed [18]. We also utilized internal sensors like attitude sensor and gyro for real-time localization.
Not only we are studying the above-mentioned subjects, but also integration of such elemental developments realizes the underwater data collection systems. System integration with wireless network communication will be done and experimental confirmation will also be done for on-line underwater data collection. Moreover, cooperative data collection experiment with the sensor nodes will be conducted in the coastal area (Figure 17).
A plan of underwater data collection experiment with sensor nodes.
Our target is not only to develop the elemental technologies and the sensor network system, but also to apply them to support biological and environmental observations. For example, coral bleaching is the loss of the zooxanthellae under environmental stress. In order to inspect the situation of coral bleaching, it is suitable to compare the present state with the past one. In such case, accumulated image data by using the stationary sensor node are useful. We will attempt long-term image data collection at the coral reef by the stationary sensor node and measure how the state of coral bleaching is advanced. Early detection of the starfish Acanthaster planci is also helpful to prevent the starfish from critical feeding damage. For this purpose, long-term stationary observation data can be utilized to notice the existence of it and the moving survey by the movable sensor node contributes to preinspection to find how widespread the starfish appears and exits. Such data are useful to plan the missions for removing harmful starfish from the coral reef. Now we are examining object tracking and pose estimation of the sensor nodes based on image processing [19, 20]. Underwater view is generally dynamically changing. Therefore, we will require detecting the moving objects like fish and sea animal for more detailed investigation and also estimate the pose of the sensor node for getting an accurate image capturing the location of captured images. This kind of technology would contribute to automated object tracking and the registration based on visual matching. Now for practical developments, we are discussing what kind of visual features is utilized for the visual survey by the researchers and the divers.
We described the elementary technologies and the system to support the visual surveys at the coral reef and mainly discussed the image collection with the time stamp by the sensor nodes. We will extend the functions to collect various kinds of the condition data of the seawater by installing the sensors to the sensor nodes. Such technology will contribute to the organization of the big data to preserve the coral reef.
6. Conclusion
In this paper, we described an underwater data collection system with stationary and movable sensor nodes. Our objective is to contribute to survey mission for the observation/record of the coral reef and their symbionts and also understand the marine ecosystems. The concept design of our proposed system was described and also our current research and development issues were introduced. Especially, we explained the prototypes of stationary and movable node and the experimental trials with them. Based on current situations and results, we expect to integrate developed elemental technologies into an underwater collection system and therefore we discussed our future subjects and plans for this research and development.
In the near future, we are planning to have data collection experiments with the prototype system in the sea of Okinawa, Japan, and evaluate the system's performance by interviewing with the experts on the marine ecosystems.
Footnotes
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
Acknowledgments
This work was partially supported by KAKENHI, MEXT, Japan (no. 23300074 and no. 25540118) and NSFC Project 90920301 in China.