Sage Journals: Discover world-class research

Abstract

A visual sensor network (VSN) is a distributed system of a large number of camera nodes and has useful applications in many areas. The primary difference between a VSN and an ordinary scalar sensor network is the nature and volume of the information. In contrast to scalar sensor networks, a VSN generates two-dimensional data in the form of images. In this paper, we design a heterogeneous VSN to reduce the implementation cost required for the surveillance of a given area between two altitude limits. The VSN is designed by combining three sub-VSNs, which results in a heterogeneous VSN. Measurements are performed to verify full coverage and minimum achieved object image resolution at the lower and higher altitudes, respectively, for each sub-VSN. Verification of the sub-VSNs also verifies the full coverage of the heterogeneous VSN, between the given altitudes limits. Results show that the heterogeneous VSN is very effective to decrease the implementation cost required for the coverage of a given area. More than 70% decrease in cost is achieved by using a heterogeneous VSN to cover a given area, in comparison to homogeneous VSN.

1. Introduction

Visual sensor networks (VSNs) are the image sensor based distributed systems. They consist of a large number of low-power camera nodes which collect image data from environment and perform distributed and collaborative processing on this data [1, 2]. These nodes extract useful information from collected images by processing the image data locally and also collaborate with other neighboring nodes to create useful information about the captured events. The large amount of image data produced by camera nodes and the limited network resources demand the exploration of new techniques and methods of sensor management and processing and communication of large data. It demands interdisciplinary approach with the disciplines such as vision processing, communication, networking, and embedded processing [3]. Camera sensor nodes with different types and costs are available in the market today. These nodes have different optical and mechanical properties. They have varied ranges of computation capabilities, communication powers, and energy requirements. The choice of camera nodes to design a VSN depends on the requirements and constraints of a given application. Depending on the type of the camera nodes, VSNs can be divided into two major categories, homogeneous and heterogeneous VSNs [4]. A homogeneous VSN uses similar types of nodes while a heterogeneous VSN uses different types of nodes.

VSNs have a large number of potential applications. They can be used in surveillance applications for intrusion detection, building monitoring, home security, and so forth. These applications capture images of the surroundings and process them for recognition and classification of the suspicious objects [5, 6]. Visual monitoring is being used in retail stores and in homes for security purposes [7]. Modern VSNs can provide surveillance coverage across a wide area, ensuring object visibility over a large range of depths [8, 9]. They can be used in traffic monitoring applications [10–14] for vehicle count, speed measurements, vehicle classifications, travel times of the city links, and so forth. VSNs have a number of applications in sports and gaming field [15, 16]. They can collect statistics about the players or the game to design more specific training to suit the individual players. VSNs have applications in environment monitoring to collect data about animals and the environment which can be helpful to solve the nature conservation issues [17, 18]. Video monitoring has important applications in searching empty parking lots [19]. The PSF application [20] is designed to locate and direct a driver to the available parking spaces, near a desired destination. Video tracking techniques are used for people tracking [21].

A homogeneous VSN is formed by combining similar camera nodes. In contrast to this, a heterogeneous VSN contains different types of nodes [4, 22]. A heterogeneous VSN provides many advantages over homogeneous VSN such as resolution of conflicting design issues, reduction in energy consumption without degrading surveillance accuracy, higher coverage, lower cost, and reliability. The tasks allocation strategy in heterogeneous VSNs is designed such that the simpler tasks are assigned to resource-constrained nodes while more complex tasks are assigned to high performance nodes. For example, in a surveillance application, a motion detection task can be assigned to lower-resolution nodes while object recognition task can be assigned to high-resolution nodes. This strategy results in optimized use of node resources. This optimization in heterogeneous VSNs results in maximization of network lifetime as compared to homogeneous VSNs. In designing homogeneous VSNs, the sensor selection and node design are finalized on the basis of most-demanding application tasks. Assigning such a node to simpler tasks wastes precious node resources. A homogeneous VSN design is unable to provide all the desired features, such as higher coverage, low cost, reliability, and energy reduction, at the same time. It optimizes one or more parameters at the expense of other parameters [9, 23].

A single camera has limited field of view and is unable to provide coverage of a large area. To cope with this problem, arrays of cameras are used. The decreasing cost and increasing quality of cameras have made the visual surveillance of an area common by using a network of cameras [24]. An important research area is the placement of visual sensors to achieve a specific goal. A common goal is to place the image sensors in the arrays for complete coverage of a given area. The other goal can be to minimize the cost of visual sensor arrays, while maintaining the required resolution. Currently, the designers place cameras by hand due to the lack of theoretical research on planning visual sensor placement. In future, the number of cameras in smart surveillance applications is expected to increase to hundreds or even thousands. Thus, it is extremely important to develop camera placement strategies [25, 26]. In recent years, a large number of smart camera networks are deployed for a variety of applications. An important design issue in these distributed environments is the proper placement of cameras. Optimum camera placement increases the surveillance coverage and also improves the appearance of objects in cameras [24].

This paper discusses the design of a heterogeneous VSN, to be used for application of monitoring large birds, such as eagle, in the sky. The heterogeneous VSN will be able to provide coverage of each point in the area with the required minimum resolution. The heterogeneous VSN is designed by combining three sub VSNs for the surveillance of an area between two altitude limits. The VSN will be implemented with real monitoring nodes, which will be designed by following the model. Images will be captured and measurements will be performed to verify the full coverage at the given lower altitude and minimum achieved resolution at the given higher altitude. The heterogeneous VSN will reduce the implementation cost in comparison to homogeneous VSN.

The remaining of this paper is organized as follows. Section 2 describes the necessary theory for designing homogeneous and heterogeneous VSNs. Section 3 discusses the camera models and optics used in the experiment. Section 4 describes the heterogeneous VSN coverage model. Section 5 discusses the experimental measurement performed to verify VSN parameters. Section 6 discusses the results, and finally Section 7 describes the conclusion.

2. Theory

This section describes a brief theory about designing VSNs and discusses the homogeneous and heterogeneous VSN models. The heterogeneous VSN is a combination of a number of homogeneous VSNs and is effective in reducing the implementation cost for the surveillance of an area. An example of heterogeneous VSN is presented, and cost comparison between homogeneous and heterogeneous VSNs is made to see the cost reduction capability of the heterogeneous VSN.

2.1. Homogeneous VSN Model

The design of a VSN for the surveillance of golden eagles is presented in [27]. The VSN is formed with a number of cameras. A 3D visualization of the coverage with a matrix of cameras is shown in Figure 1. A sharp tip at the bottom of the figure represents a camera node. A number of such nodes form a matrix which is used to monitor eagles in the sky between two altitude limits above the ground, the higher altitude $a_{h}$ and the lower altitude $a_{l}$ . The area covered by the VSN is increased as the altitude from the camera nodes is increased. The area coverage increases till lower altitude $a_{l}$ , where VSN is able to fully cover a given area. Above altitude $a_{l}$ the coverage starts to overlap. This overlap is used to apply triangulation technique to find the exact size and altitude of an eagle.

Figure 1

3D visualization of coverage with a matrix of cameras.

As the altitude of an eagle is increased from camera nodes, the resolution of its image on the camera sensor is decreased. A minimum image resolution is necessary to recognize an object moving at altitude $a_{h}$ and is assumed to be 10 pixels per meter. For altitudes higher than $a_{h}$ , the resolution of the object image drops below the minimum required criterion, so the VSN coverage is assumed zero above this altitude. The dark surface at the top of Figure 1 represents the altitude $a_{h}$ .

The first step to design a VSN is the selection of a camera sensor. A number of camera sensors are used in this study, and the details about their type, resolution, and size are given in Table 1. The second step of VSN design is the calculation of focal length of lens to be used with the chosen camera, which ensures the minimum required resolution at altitude $a_{h}$ and can be calculated by the following equation:

\begin{matrix} f = \frac{p_{x}}{l_{x}} \cdot \frac{l_{h}}{r_{h}} \cdot a_{h} or f = \frac{p_{y}}{l_{y}} \cdot \frac{l_{v}}{r_{v}} \cdot a_{h}, \end{matrix}

(1)

where $p_{x}$ and $p_{y}$ are the minimum number of pixels of the object image along width and length, $l_{x}$ and $l_{y}$ are width and length of the object, $a_{h}$ is the higher altitude, $l_{h}$ and $l_{v}$ are the image sensor lengths, and $r_{h}$ and $r_{v}$ are the image sensor resolutions along horizontal and vertical directions, respectively. The angle of view (AoV) of a node, using a particular image sensor and a lens of specific focal length, can be calculated by the following equation:

\begin{matrix} α = 2 \arctan \frac{K}{2 f}, \end{matrix}

(2)

where K is the chosen length (horizontal, vertical, or diagonal) of the camera sensor and f is the lens focal length. The combination of camera and lens forms an individual node, and a number of such nodes are required to surveil a given area.

Table 1

Camera sensors used in study.

Number	Sensor	Type	Resolution		Size (mm)
Number	Sensor	Type	$r_{h}$	$r_{v}$	$l_{h}$	$l_{v}$
1	MT9V011	VGA	640	480	3.58	2.69
2	MT9V032	WVGA	752	480	4.51	2.88
3	MT9M033	1.2Mp	1280	960	3.54	2.69
4	MT9M001	1.3Mp	1280	1024	6.66	5.32
5	MT9M032	1.6Mp	1472	1096	3.24	2.41
6	MT9D012	2Mp	1600	1200	3.56	2.68
7	MT9T031	3Mp	2048	1536	6.55	4.92
8	MT9P031	5Mp	2592	1944	5.70	4.28
9	MT9E013	8Mp	3264	2448	4.57	3.43
10	MT9J003	10Mp	3664	2748	6.119	4.589
11	MT9F002	14Mp	4384	3288	6.14	4.6

The third step of VSN design is the placement of nodes which ensures full coverage at the given lower altitude $a_{l}$ . The placement of nodes is calculated by calculating the distances $d_{x}$ and $d_{y}$ between the two neighboring nodes, along x and y directions, respectively. These distances can be calculated by the following equations:

\begin{matrix} d_{x} = \frac{a_{l} \cdot l_{h}}{f} {, d}_{y} = \frac{a_{l} \cdot l_{v}}{f} . \end{matrix}

(3)

The fourth step of VSN design is to calculate the total number of nodes n (called cost) to cover a given area. The cost to cover a given area A, can be calculated by the following equation:

\begin{matrix} n = \frac{A}{(l_{v} / l_{h}) \cdot {((a_{l} \cdot r_{h} \cdot l_{x}) / (a_{h} \cdot p_{x}))}^{2}}, \end{matrix}

(4)

where $l_{h}$ and $l_{v}$ , $r_{h}$ , $r_{v}$ , $p_{x}$ , $l_{x}$ , $a_{l}$ , and $a_{h}$ are as defined above.

By using camera sensors given in Table 1 and the equations from (1) to (4), a VSN can be designed to cover a given area in the given altitude ranges. The favorable VSN design objective is to increase the coverage of an area but decrease its implementation cost. However, in actual practice, the cost of a VSN is greatly increased when the covered area is increased. For example, to cover an area of 1 km², between altitude limits from 3000 to 5000 m above the ground, a homogeneous VSN requires a cost of 20 nodes. This cost calculation assumes minimum resolution of 10 pixels per meter and 14 Mp camera sensor. If the area of 1 km² is increased by extending the altitude limits from 500 to 5000 m, the cost of the VSN is increased to 694. In this example, the increase in area is about 2.25 times, but the corresponding increase in cost is about 35 times. This result indicates the need for a new technique which is able to increase the area coverage but at the same time decreases the implementation cost. In the following paragraphs the design of a heterogeneous VSN is presented which fulfills this purpose.

2.2. Heterogeneous VSN Model

The input parameters to design a heterogeneous VSN include the area A to be covered, lower altitude $a_{l}$ , higher altitude $a_{h}$ , and the object size, $l_{x}$ and $l_{y}$ , to be monitored. The VSN is designed with three sub-VSNs: VSN₁, VSN₂, and VSN₃, as shown in Figure 2. The VSN₁ is designed for lower altitude $a_{l 1}$ and higher altitude $a_{h 1}$ and covering the top most part of the given altitudes range, the VSN₂ is designed for lower altitude $a_{l 2}$ and higher altitude $a_{h 2}$ and covering the middle part of the given altitudes range, and the VSN₃ is designed for lower altitude $a_{l 3}$ and higher altitude $a_{h 3}$ and covering the lower part of the given altitudes range.

Figure 2

Heterogeneous VSN model.

The given higher altitude $a_{h}$ is treated as the higher altitude $a_{h 1}$ for VSN₁. Suppose an image sensor $S_{1}$ , having parameters $l_{h 1}$ , $l_{v 1}$ , $r_{h 1}$ , and $r_{v 1}$ , is chosen to implement the nodes for VSN₁ along with a lens of focal length $f_{1}$ . The lens focal length must ensure the minimum required resolution for the given object, when moving at the altitude $a_{h 1}$ . The value of $f_{1}$ can be calculated by using (1) for sensor $S_{1}$ and altitude $a_{h 1}$ .

The VSN₂ is designed to cover an area A between the higher altitude $a_{h 2}$ and the lower altitude $a_{l 2}$ . Suppose an image sensor $S_{2}$ , having parameters $l_{h 2}$ , $l_{v 2}$ , $r_{h 2}$ , and $r_{v 2}$ , is chosen to implement the nodes for VSN₂ along with a lens of focal length $f_{2}$ . The value of the altitude $a_{h 2}$ , where the combination of the sensor $S_{2}$ and the lens of focal length $f_{2}$ can provide the minimum required resolution for the given object, can be derived from (1) and is given below:

\begin{matrix} a_{h 2} = \frac{l_{x}}{p_{x}} \cdot \frac{r_{h 2}}{l_{h 2}} \cdot f_{2} . \end{matrix}

(5)

The VSN₂ will cover the altitudes range from $a_{h 2}$ and below. The VSN₁ must ensure the coverage of the altitudes range from $a_{h 1}$ to $a_{h 2}$ . The higher altitude $a_{h 2}$ for VSN₂ will serve as the lower altitude $a_{l 1}$ for VSN₁, as shown in Figure 2. The value of $a_{h 2}$ is used to calculate the distance $d_{x 1}$ between VSN₁ nodes by using (3), for focal length $f_{1}$ and sensor $S_{1}$ . Suppose the same distance $d_{x 1}$ is set between the nodes of VSN₂, such that the nodes of VSN₂ are placed at the midpoint between the nodes of VSN₁. The value of the lower altitude $a_{l 2}$ for VSN₂ is calculated by solving the triangles, connected to VSN₁ and VSN₂. The value of $a_{l 2}$ can be calculated by using the following equation:

\begin{matrix} a_{l 2} = \frac{a_{h 2} \cdot l_{h 1}}{f_{1}} \cdot \frac{f_{1} \cdot f_{2}}{f_{1} l_{h 2} + f_{2} l_{h 1}} . \end{matrix}

(6)

If the same camera sensor is used for both VSN₁ and VSN₂, then the above equation is reduced to the following equation:

\begin{matrix} a_{l 2} = \frac{a_{h 2} \cdot f_{2}}{f_{1} + f_{2}} . \end{matrix}

(7)

The VSN₃ is designed to cover the altitudes range from $a_{l 2}$ to the given lower altitude $a_{l}$ . The value $a_{l 2}$ of VSN₂ will serve as the higher altitude $a_{h 3}$ , and the given value of lower altitude $a_{l}$ will serve as the lower altitude $a_{l 3}$ for VSN₃, as shown in Figure 2. Suppose an image sensor $S_{3}$ , having parameters $l_{h 3}$ , $l_{v 3}, r_{h 3}$ , and $r_{v 3}$ , is chosen to implement the nodes for VSN₃ along with a lens of focal length $f_{3}$ . The lens focal length must ensure the minimum required resolution for the given object, when moving at the altitude $a_{h 3}$ . The value of $f_{3}$ can be calculated by using (1) for sensor $S_{3}$ and altitude $a_{h 3}$ . To verify whether the VSN₃ has the lower altitude the same as that given in the specification, the following equation can be used:

\begin{matrix} a_{l} = \frac{d_{x}}{((l_{h 1} / f_{1}) + (l_{h 2} / f_{2}) + (2 l_{h 3} / f_{3}))}, \end{matrix}

(8)

where $l_{h 1}$ , $l_{h 2}$ , and $l_{h 3}$ are sensors and $f_{1}$ , $f_{2}$ , and $f_{3}$ are lenses focal lengths for VSN₁,VSN₂ and VSN₃, respectively.

The lower altitude value $a_{l 3}$ can be used to calculate the placement of VSN₃ nodes with respect to the nodes of VSN₁ and VSN₂. The proper placement of VSN₃ will ensure the full coverage of the area for the given lower altitude value. If $d_{x 31}$ and $d_{x 32}$ are the distances of VSN₃ nodes with respect to VSN₁ and VSN₂ nodes (Figure 6), respectively, their values can be calculated by using the following equations:

\begin{matrix} d_{x 31} = \frac{1}{2} (\frac{l_{h 1}}{f_{1}} + \frac{l_{h 3}}{f_{3}}) a_{l}, \end{matrix}

(9)

\begin{matrix} d_{x 32} = \frac{1}{2} (\frac{l_{h 2}}{f_{2}} + \frac{l_{h 3}}{f_{3}}) a_{l} . \end{matrix}

(10)

After describing the theory to design homogeneous and heterogeneous VSNs, an example of heterogeneous VSN is designed in the following paragraphs, by using the concepts described above.

2.3. Example of Heterogeneous VSN

Suppose coverage of an area is required between altitudes range from 293 to 47.3 cm. Thus, the values of $a_{h}$ and $a_{l}$ for the desired heterogeneous VSN are 293 and 47.3 cm, respectively. Suppose this VSN is designed to monitor an object, having length and width of 0.54 and 1.08 cm, respectively, shown in Figure 3. First of all, the design of the network VSN₁ is presented. The higher altitude $a_{h 1}$ of VSN₁ is equal to the given higher altitude $a_{h}$ , 293 cm. Suppose the sensor chosen to implement the nodes of this VSN is 5 Mp. The parameters of this sensor are given in Table 1. The value of $f_{1}$ to be used with sensor $S_{1}$ is calculated by using (1) and is found to be 12 mm.

Figure 3

Bird to be monitored by VSN.

Suppose the components chosen to implement the nodes for VSN₂ are 5 Mp image sensor and lens with focal length 8.5 mm. The value of the altitude $a_{h 2}$ , where this combination of the sensor and the lens can provide the minimum required resolution for the given object, is calculated by using (5) and is found to be 208.6 cm. This higher altitude value $a_{h 2}$ for VSN₂ will also serve as the lower altitude value $a_{l 1}$ for VSN₁. The VSN₁ must ensure the coverage of the altitudes range from 293 to 208.6 cm. The value of 208.6 m is used to calculate the distance $d_{x 1}$ between VSN₁ nodes, along x side, by using (3). The values for focal length and sensor used to calculate $d_{x 1}$ are the same as for VSN₁, that is, 12 mm focal length and 5 Mp image sensor. The value of $d_{x 1}$ is found to be 97.9 cm. Similarly, the value of $d_{y 1}$ can be found along y side. The same distance of 97.9 cm is set between the nodes of VSN₂ along x side, as discussed before. The lower value of $a_{l 2}$ for VSN₂ is calculated by using (7) and is found to be 86.5 cm.

After discussing the design of VSN₂, the design for the network VSN₃ is considered. This network is designed to cover the altitudes range from $a_{l 2}$ to the given lower altitude $a_{l}$ . The value $a_{l 2}$ of VSN₂ serves as the higher altitude $a_{h 3}$ for VSN₃ while the value of lower altitude $a_{l}$ serves as the lower altitude $a_{l 3}$ . VSN₃ must ensure the coverage of the altitudes range from 86.5 to 47.3 cm. Suppose WVGA image sensor is chosen to implement the nodes for VSN₃. The value of $f_{3}$ to be used with WVGA sensor, to ensure the minimum required image resolution for the given object, when it is moving at altitude $a_{h 3}$ , is calculated by using (1) and is found to be 9.6 mm. The placement of the VSN₃ nodes with respect to the VSN₁ and VSN₂ nodes is finalized by calculating the distances $d_{x 13}$ and $d_{x 23}$ , by using (9) and (10), and are found to be 22.19 and 22.76 cm, respectively.

The summary of the sub-VSNs parameters, discussed above, is given in Table 2.

Table 2

Summary of sub=VSNs.

Parameter	$VS N_{1}$	$VS N_{2}$	$VS N_{3}$
Higher altitude (cm)	293	208.6	86.5
Lower altitude (cm)	208.6	86.5	47.3
Image sensor	5 MP	5 MP	WVGA
Focal length (mm)	12	8.5	9.6

2.4. Cost Reduction by Heterogeneous VSN

Suppose an area of 1000 cm by 1000 cm is required to be covered between altitude limits from 293 to 47.3 cm. The area is covered by using two different types of VSNs: homogeneous VSN and heterogeneous VSN. Suppose 5 Mp image sensor is used to implement both types of VSNs. The cost required to implement the homogeneous VSN is calculated by using (4) and is found to be 793 nodes. In case of heterogeneous VSN, the cost required to implement VSN₁ for altitudes range from 293 to 208.6 cm is 40 nodes, for VSN₂ from 208.6 to 86.5 is 117 nodes, and for VSN₃ from 86.5 to 47.3 is 69 nodes. The total cost required to implement the complete heterogeneous VSN is 226 nodes. If we compare the costs for homogeneous VSN and heterogeneous VSN, it is obvious that heterogeneous VSN is offering more than 70% decrease in cost. Similar results can be verified by using WVGA image sensor in place of 5 Mp sensor. Although in case of using WVGA sensor more cost will be required to implement the same ranges of altitudes, but the decrease in cost will be more than 70% in case of using heterogeneous VSN.

In the remaining of this paper, the heterogeneous VSN which is described in the above example, is implemented by using actual cameras and optics.

3. Camera Models and Optics

The image sensors chosen to implement the monitoring nodes for sub VSNs in the above example include 5 Mp and WVGAs. The 5 Mp image sensor MT9P031 is used in camera model UI-5480CP-M from UEye and is shown in Figure 4. The horizontal length $l_{h}$ and the vertical length $l_{v}$ of this sensor are 5.632 and 4.224 mm, respectively. Also, the horizontal resolution $r_{h}$ and vertical resolution $r_{v}$ of this sensor are 2560 and 1920, respectively. These parameters are shown in Table 3. The power is applied to this camera module through Hirose connector while the data is received via Ethernet link.

Table 3

Parameters of camera sensors.

Parameter	MT9P031	MT9V032
Image sensor	5 Mp	WVGA
Horizontal length (mm)	5.632	4.512
Vertical length (mm)	4.224	2.880
Horizontal resolution	2560	752
Vertical resolution	1920	480

Figure 4

Cameras and lenses used in VSN design.

The WVGA image sensor MT9V032 is used in camera model UI-1220SE-M from UEye and is shown in Figure 4. The horizontal length $l_{h}$ and the vertical length $l_{v}$ of this sensor are 4.512 and 2.880 mm, respectively. Also, the horizontal resolution $r_{h}$ and vertical resolution $r_{v}$ of this sensor are 752 and 480, respectively. These parameters are shown in Table 3. The camera is using a USB interface which is used to apply power as well as receive data from the camera module.

The cameras and the accompanied optics used to implement the heterogeneous VSN are shown in Figure 4. The combination of a camera and the accompanied lens of relevant focal length form a monitoring node. An individual laptop machine is used with each monitoring node as data gathering and analysis platform. The 5 Mp camera module is connected to the laptop by using CAT 6 Ethernet cable. The power is supplied to the camera module by using 6 pin Hirose connector. The camera is connected to the laptop as client while the laptop is used as server. The WVGA camera is connected to the laptop by using a USB cable. The power to a camera node and laptop is supplied by using a 12 Volt rechargeable battery. A complete node is shown in Figure 5. The images of the bird are captured by UEye cockpit software. This is the camera interface to capture images/videos or to change the controlling parameters of the camera.

Figure 5

A complete camera node.

Figure 6

VSN model.

4. Coverage Model

The camera coverage model based on all the above design parameters is shown in Figure 6. The model contains five altitude lines A, B, C, D, and E. The altitude line A represents the ground plane and monitoring nodes are placed on this line. This line contains seven points from $A_{1}$ to $A_{7}$ . The camera nodes of VSN₁ are placed at points $A_{2}$ and $A_{6}$ , the camera node of VSN₂ is placed at point $A_{4}$ , and the camera nodes for VSN₃ are placed at points $A_{3}$ and $A_{5}$ . The second altitude line is B which contains eight points from $B_{1}$ to $B_{8}$ . The network VSN₃ provides full coverage on this line by making intersections with VSN₁ and VSN₂. The third altitude line is C which contains ten points from $C_{1}$ to $C_{10}$ . The VSN₂ provides full coverage on this line by making intersections with VSN₁. This altitude line also acts as the highest altitude boundary for VSN₃. This is because the VSN₃ is not able to provide the required resolution above this altitude. The fourth altitude line is D which contains seven points from $D_{1}$ to $D_{7}$ . This altitude line acts as the highest altitude boundary for VSN₂. The VSN₂ is not able to provide the required resolution above this altitude. The last altitude line is E which contains four points from $E_{1}$ to $E_{4}$ . This altitude line represents the highest altitude for VSN₁ as well as for the entire heterogeneous VSN, where an object can be monitored with the required minimum resolution.

To perform coverage and resolution measurements, it is necessary to find the altitudes of the lines above ground and the distances of the points from the edge of the relevant lines. The altitude line A represents the ground so it has altitude 0 cm. The altitude line B is the lower altitude for VSN₃; it is at the altitude 47.3 cm above the ground, as calculated before. The altitude line C is the higher altitude for VSN₃ or the lower altitude for VSN₂. Thus, it is at altitude 86.5 cm above the ground. The altitude line D is the higher altitude for VSN₂ or the lower altitude for VSN₁. Thus, it is at the altitude 208.6 cm above the ground. The altitude line E represents the higher altitude for VSN₁. Thus, it is at the altitude 293 cm above the ground. All these altitudes are summarized in Figure 7.

Figure 7

VSN altitudes.

After finding the altitudes of the lines, the distances of the points from the edge of the respective altitude lines are calculated. For altitude line A, the distance of point $A_{2}$ from point $A_{1}$ can be calculated by solving the triangle $E_{1} A_{1} A_{2}$ in Figure 6. The distance of point $A_{2}$ from point $A_{1}$ is calculated and is found to be 68.8 cm. The distance between points $A_{2}$ and $A_{6}$ is $d_{x}$ which is 97.9 cm. The distance of point $A_{6}$ from point $A_{1}$ is calculated by adding the distance of point $A_{1}$ from $A_{2}$ and the distance of point $A_{2}$ from $A_{6}$ and is found to be 166.6 cm. The distance of point $A_{7}$ from $A_{1}$ is 235.4 cm. The point $A_{4}$ is $d_{x}$ /2 distance away from $A_{2}$ . Thus, the point $A_{4}$ is at 117.7 cm away from $A_{1}$ . The point $A_{3}$ is at 22.19 cm distance from $A_{2}$ , and point $A_{5}$ is at 22.19 cm distance from $A_{6}$ . Thus, the points $A_{3}$ and $A_{5}$ are at distances 91.0 and 144.5 cm away from $A_{1}$ , respectively.

Similarly, the distances of the points on other altitude lines are calculated. For altitude line B, the distances of the points $B_{2}$ , $B_{3}$ , $B_{4}$ , $B_{5}$ , $B_{6}$ , $B_{7}$ , and $B_{8}$ from $B_{1}$ are 57.7, 79.9, 102.1, 133.3, 155.5, 177.7, and 235.4 cm, respectively. For altitude line C, the distances of the points $C_{2}$ , $C_{3}$ , $C_{4}$ , $C_{5}$ , $C_{6}$ , $C_{7}$ , $C_{8}$ , $C_{9}$ , and $C_{10}$ from $C_{1}$ are 48.5, 70.7, 89.1, 111.3, 124.2, 146.3, 164.8, 186.9, and 235.4 cm, respectively. For altitude line D, the distances of points $D_{2}$ , $D_{3}$ , $D_{4}$ , $D_{5}$ , $D_{6}$ , and $D_{7}$ from $D_{1}$ are 19.8, 48.6, 117.7, 186.8, 215.6, and 235.4 cm, respectively. For altitude line E, the distances of the points $E_{2}$ , $E_{3}$ , and $E_{4}$ from $E_{1}$ are 97.9, 137.5, and 235.4 cm, respectively. The values of all these points are given in Table 4.

Table 4

Values of the points in VSN model.

No.	i (cm)	B (cm)	C (cm)	D (cm)	E (cm)
1	0	0	0	0	0
2	68.8	57.7	48.5	19.8	97.9
3	91.0	79.9	70.7	48.6	137.5
4	117.7	102.1	89.1	117.7	235.4
5	144.5	133.3	111.3	186.8	—
6	166.6	155.5	124.2	215.6	—
7	235.4	177.7	146.3	235.4	—
8	—	235.4	164.8	—	—
9	—	—	186.9	—	—
10	—	—	235.4	—	—

5. Measurements of VSN Parameters

Two important parameters of a VSN which must be ensured are the full area coverage at a given lower altitude and the minimum required image resolution for an object, when moving at the given higher altitude. The remaining part of the paper describes the measurements to verify these parameters. The measurements are performed in a room. To measure all the distances, a 240 cm long scale is constructed on a long strip of paper. The markings on the scale are marked after every 5 cm. The scale is glued on a wall of the room. The wall is treated as sky for this experiment. A bird of size (1.08 × 0.54) cm, shown in Figure 3, is printed on a card board and is used to measure the minimum required image resolution.

5.1. Coverage Measurements

This section measures coverage at lower altitude of a given sub-VSN to verify whether this network is able to provide full coverage or not. The measurements are performed for altitudes D, C, and B, which are lower altitudes for the networks VSN₁, VSN₂, and VSN₃, respectively. For coverage measurement, the images are captured at the respective altitude line with the concerned camera nodes. The coverage of each node is measured and compared with the calculated coverage for that node to verify whether these nodes are covering the complete required area or there is some gap in the coverage.

5.1.1. Altitude D

It is the lower altitude of VSN₁, where full coverage of this network is assumed. For more clear visualization of this altitude, a separate model which is extracted from heterogeneous VSN model (Figure 6) is shown in Figure 8(a). To perform measurements for coverage verification at lower altitude of VSN₁, the wall is treated as altitude line D. The ground line A of length 235.4 cm is drawn parallel to the wall, 208.6 cm away from the wall. The points $A_{2}$ and $A_{6}$ are marked at distances 68.8 cm and 166.6 cm from the edge of the line, according to Figure 8(a). Node 1 is placed at point $A_{2}$ while node 2 is placed at point $A_{6}$ . Both nodes are using similar 5 Mp image sensors and lenses of focal length 12 mm, as discussed before. The complete setup is shown in Figure 9. The calculated distances expected to be covered by nodes 1 and 2 are from point $D_{2}$ (19.8 cm) to $D_{4}$ (117.7 cm) and from point $D_{4}$ (117.7 cm) to $D_{6}$ (215.6 cm). Images are captured with both monitoring nodes, and pixels are counted to measure the distances covered by these nodes.

Figure 8

VSN models, (a) VSN₁, (b) VSN₂, and (c) VSN₃.

Figure 9

VSN installation.

The image captured by node 1 is shown in Figure 10(a). The pixel measurements in this figure show that about 131 pixels represent 5 cm distance. There are about 145 pixels from the left edge of the figure to the 25 cm point, which represents about 5.5 cm distance. Thus, $D_{2}$ point has value of 19.5 cm. Similarly, there are about 84 pixels from right edge of the figure to the 115 cm point, which represent about 3.2 cm distance. Thus, $D_{4}$ point has value of 118.2 cm.

Figure 10

VSN measurements at different altitudes with different nodes: (a–c) VSN₁, altitude D node 1, altitude D node 2, and altitude E, node 1; (d–f) VSN₂, altitude C node 1, altitude C node 2, and altitude D node 1; (g–j) VSN₃, altitude B node 1, altitude B node 2, altitude B node 3, altitude C node 1.

The image captured by node 2 is shown in Figure 10(b). The pixel measurements in this figure show that about 131 pixels represent 5 cm distance. There are about 66 pixels from the left edge of the figure to the 120 cm point, which represents about 2.5 cm distance. Thus, $D_{4}$ point has value of 117.5 cm. Similarly, there are about 146 pixels from the right edge of the figure to the 210 cm point, which represents about 5.6 cm distance. Thus, $D_{6}$ point has value of 215.6 cm. All these values are shown in Table 5.

Table 5

Coverage measurements details.

Node	Pt.	5 cm Pix.	Pix	Dist.	Ref. Pt.	Cal. Dist.	Meas. Dist.
Altitude D measurement
1	$D_{2}$	131	145	−5.5	25	19.8	19.5
1	$D_{4}$	131	84	+3.2	115	117.7	118.2
2	$D_{4}$	131	66	−2.5	120	117.7	117.5
2	$D_{6}$	131	149	+5.7	210	215.7	215.7

Altitude C measurement
1	$C_{2}$	317	108	−1.7	50	48.5	48.3
1	$C_{4}$	317	279	+4.4	85	89.1	89.4
2	$C_{4}$	220	264	−6.0	95	89.1	89.0
2	$C_{7}$	220	286	+6.5	140	146.3	146.5

Altitude B measurement
1	$B_{2}$	573	276	−2.4	60	57.7	57.6
1	$B_{3}$	573	574	+5.0	75	79.9	80.0
3	$B_{3}$	149	9	−0.3	80	79.9	79.7
3	$B_{4}$	149	66	+2.2	100	102.1	102.2
2	$B_{4}$	403	250	−3.1	105	102.1	101.9
2	$B_{5}$	403	290	+3.6	130	133.3	133.6

5.1.2. Altitude C

It is the lower altitude of VSN₂, where full coverage of this network is assumed. For more clear visualization of this altitude, a separate model which is extracted from heterogeneous VSN model, is shown in Figure 8(b). To perform measurements for coverage verification at lower altitude of VSN₂, the wall is treated as altitude line C. The ground line A of length 235.4 cm is drawn parallel to the wall at a distance of 86.5 cm away from the wall. The points $A_{2}$ and $A_{4}$ are marked at distances 68.8 and 117.7 cm from the edge of the line, according to Figure 8(b). The camera node 1 is placed at point $A_{2}$ while the camera node 2 is placed at point $A_{4}$ . Both camera nodes 1 and 2 have similar 5 Mp image sensor, but different lens focal lengths, 12 mm and 8.5 mm, respectively. The calculated distances expected to be covered by nodes 1 and 2 are from point $C_{2}$ (48.5 cm) to $C_{4}$ (89.1 cm) and from point $C_{4}$ (89.1 cm) to $C_{7}$ (146.3 cm). Images are captured with both nodes, and pixels are counted to measure the distance covered by these nodes.

The image captured by node 1 is shown in Figure 10(d). The pixel measurements in this figure show that about 317 pixels represent 5 cm distance. There are about 108 pixels from the left edge of the figure to the 50 cm point, which represents about 1.7 cm distance. Thus, $C_{2}$ point has value of 48.3 cm. Similarly, there are about 279 pixels from the right edge of the figure to the 85 cm point, which represents about 4.4 cm distance. Thus, $C_{4}$ point has value of 89.4 cm.

The image captured by node 2 is shown in Figure 10(e). The pixel measurements in this figure show that about 220 pixels represent 5 cm distance. There are about 264 pixels from the left edge of the figure to the 95 cm point, which represents about 6.0 cm distance. Thus, $C_{4}$ point has value of 89.0 cm. Similarly, there are about 286 pixels from right edge of the figure to the 140 cm point, which represents about 6.5 cm distance. Thus, $C_{7}$ point has value of 146.5 cm. All these values are given in Table 5.

5.1.3. Altitude B

It is the lower altitude of VSN₃, where full coverage of this network is assumed. For more clear visualization of this altitude, a separate model which is extracted from heterogeneous VSN model is shown in Figure 8(c). To perform measurements for coverage verification at lower altitude of VSN₃, the wall is treated as altitude line B. The ground line A of length 235.4 cm is drawn parallel to the wall at a distance of 47.3 cm away from the wall. The points $A_{2}$ , $A_{3}$ , and $A_{4}$ are marked at distances 68.8, 91.0, and 117.7 cm from the edge of the line, according to Figure 8(c). Camera node 1 is fixed at point $A_{2}$ , camera node 2 is fixed at point $A_{4}$ , while camera node 3 is fixed at point $A_{3}$ . Both camera nodes 1 and 2 have similar 5 Mp image sensors but different lens focal lengths, 12 mm and 8.5 mm, respectively. Camera node 3 has WVGA camera sensor and lens of focal length 9.6 mm. The calculated distances expected to be covered by nodes 1, 3, and 2 are from point $B_{2}$ (57.7 cm) to $B_{3}$ (79.9 cm), from point $B_{3}$ (79.9 cm) to $B_{4}$ (102.1 cm), and from point $B_{4}$ (102.1 cm) to $B_{5}$ (133.3 cm), respectively.

The image captured by node 1 is shown in Figure 10(g). The pixel measurements in this figure show that about 573 pixels represent 5 cm distance. There are about 276 pixels from the left edge of the figure to the 60 cm point, which represents about 2.4 cm distance. Thus, $B_{2}$ point has value of 57.6 cm. Similarly, there are about 574 pixels from the right edge of the figure to the 75 cm point, which represents about 5.0 cm distance. Thus, $B_{3}$ point has value of 80.0 cm.

The image captured by node 3 is shown in Figure 10(h). The pixel measurements in this figure show that about 149 pixels represent 5 cm distance. There are about 9 pixels from the left edge of the figure to the 80 cm point, which represents about 0.3 cm distance. Thus, $B_{3}$ point has value of 79.7 cm. Similarly, there are about 66 pixels from the right edge of the figure to the 100 cm point, which represents about 2.2 cm distance. Thus, $B_{4}$ point has value of 102.2 cm.

The image captured by node 2 is shown in Figure 10(i). The pixel measurements in this figure show that about 403 pixels represent 5 cm distance. There are about 250 pixels from the left edge of the figure to the 105 cm point, which represents about 3.1 cm distance. Thus, $B_{4}$ point has value of 101.9 cm. Similarly, there are about 290 pixels from the right edge of the figure to the 130 cm point, which represents about 3.6 cm distance. Thus, $B_{5}$ point has value of 133.6 cm. All these values are given in Table 5.

5.2. Resolution Measurements

After measuring the coverage at respective lower altitudes of the sub-VSNs, the measurement of the minimum achieved image resolution is performed for the given object, when moving at the higher altitude of the respective sub-VSN, to verify whether this resolution fulfills the minimum resolution criterion. The measurements are performed for altitudes E, D, and C, which are higher altitudes for the networks VSN₁, VSN₂, and VSN₃, respectively. For resolution measurement, the images are captured at the respective altitude line with the concerned camera nodes. The resolution of the object image obtained by each node is measured for verification.

5.2.1. Altitude E

It is the higher altitude of VSN₁ (Figure 8(a)) where minimum required object image resolution is assumed. To perform measurements for resolution verification for VSN₁, the wall is treated as altitude line E and the bird image of Figure 3 is fixed on the wall. The ground line A of length 235.4 cm is drawn parallel to the wall at a distance of 293 cm away from the wall. Both nodes 1 and 2 are placed the same way as for measuring the full coverage, and images are captured by these nodes. One such image captured by node 1 is shown in Figure 10(c). For more clear observation, a small portion of Figure 10(c) which contains bird image is shown magnified in Figure 11(a). The bird pixels are measured along the width side by using segmentation. The number of pixels obtained is 19. This value is very near to the calculated value of 20.

Figure 11

Bird resolution at the highest altitude: (a) VSN₁, (b) VSN₂, and (c) VSN₃.

5.2.2. Altitude D

It is the higher altitude of VSN₂ (Figure 8(b)) where minimum required object image resolution is assumed. To perform measurements for resolution verification for VSN₂, the wall is treated as altitude line D. The ground line A of length 235.4 cm is drawn parallel to the wall at a distance of 208.6 cm away from the wall. Both nodes 1 and 2 are placed the same way as for measuring the full coverage, and images are captured by these nodes. One such image captured by node 2 is shown in Figure 10(f). For more clear observation, a small portion of Figure 10(f) which contains bird image is shown magnified in Figure 11(b). The bird pixels are measured along the width side by using segmentation. The number of pixels obtained is 19. This value is very near to the calculated value of 20.

5.2.3. Altitude C

It is the higher altitude of VSN₃ (Figure 8(c)) where minimum required object image resolution is assumed. To perform measurements for resolution verification for VSN₃, the wall is treated as altitude line C. The ground line A of length 235.4 cm is drawn parallel to the wall at a distance of 86.5 cm away from the wall. The nodes 1, 3, and 2 are placed the same way as for measuring the full coverage, and images are captured by these nodes. One such image captured by node 3 is shown in Figure 10(j). For more clear observation, a small portion of Figure 10(j) which contains bird image is shown magnified in Figure 11(c). The bird pixels are measured along the width side by using segmentation. The number of pixels obtained is 19. This value is very near to the calculated value of 20.

6. Results

The coverage measurement results in Table 5 show that the measured values for different points are in accordance with the values calculated by heterogeneous VSN model. Even the measured values are broader than the calculated values. For example, the analysis of the range covered by node 1 at altitude line D between $D_{2}$ and $D_{4}$ points (first two rows of Table 5) shows that the calculated value of $D_{2}$ is 19.8 cm while the measured value is 19.5 cm. It shows that the node is providing coverage before the calculated value. Similarly, the calculated value of $D_{4}$ is 117.7 cm while the measured value is 118.2 cm, which shows that the node is providing coverage after the calculated value. Thus, the node is providing reliable coverage of the required area. Similar is the case with other values in the table. Moreover, the resolution measurement results are also in accordance with the calculated values. The implementation cost comparison for both homogeneous and heterogeneous VSNs shows that the heterogeneous VSN provides more than 70% decrease in cost as compared to homogeneous VSN.

7. Conclusion

This paper discusses the design of a heterogeneous VSN for the surveillance of a volume between two altitude limits. Images are captured and measurements are performed to verify full coverage and minimum achieved resolution at lower and higher altitudes, respectively. The measurements verify that the heterogeneous VSN is able to provide full volume coverage between the given altitude limits. The core advantage of heterogeneous VSN over homogeneous VSN is the higher cost reduction.

References

Obraczka

Manduchi

Garcia-Luna-Aceves

Managing the information flow in visual sensor networks

Proceedings of the 5th International Symposium on Wireless Personal Multimedia Communication

2002

Medeiros

Park

Kak

A light-weight event-driven protocol for sensor clustering in wireless camera networks

Proceedings of the 1st ACM/IEEE International Conference on Distributed Smart Cameras (ICDSC '07)

September 2007

Vienna, Austria

203 210

2-s2.0-47349132834

10.1109/ICDSC.2007.4357525

Soro

Heinzelman

A survey of visual sensor networks

Advances in Multimedia 2009 2009 21

2-s2.0-68949132673

10.1155/2009/640386

640386

Charfi

Wakamiya

Murata

Challenging issues in visual sensor networks

IEEE Wireless Communications 2009 16 2 44 49

2-s2.0-67349127129

10.1109/MWC.2009.4907559

Chitnis

Liang

Zheng

J. Y.

Pagano

Lipari

Wireless line sensor network for distributed visual surveillance

Proceedings of the 6th ACM International Symposium on Performance Evaluation of Wireless Ad-Hoc, Sensor, and Ubiquitous Networks (PE-WASUN '09)

October 2009

71 78

2-s2.0-74049159489

10.1145/1641876.1641890

Tseng

Y.-C.

Wang

Y.-C.

Cheng

K.-Y.

Hsieh

Y.-Y.

iMouse: an integrated mobile surveillance and wireless sensor system

Computer 2007 40 6 60 66

2-s2.0-34447327247

10.1109/MC.2007.211

Brodsky

Cohen

Cohen-Solal

Visual surveillance in retail stores and in the home

Advanced Video-Based Surveillance Systems 2001

Boston, Mass, USA

Kluwer Academic

50 61

Foresti

G. L.

Micheloni

Snidaro

Remagnino

Ellis

Active video-based surveillance system

IEEE Signal Processing Magazine 2005 22 2 25 37

2-s2.0-17244373650

10.1109/MSP.2005.1406473

Kulkarni

Ganesan

Shenoy

The case for multi-tier camera sensor networks

Proceedings of the 15th International Workshop on Network and Operating Systems Support for Digital Audio and Video (NOSSDAV '05)

June 2005

141 146

2-s2.0-29844438527

10.1145/1065983.1066016

10.

Beymer

McLauchlan

Coifman

Malik

Real-time computer vision system for measuring traffic parameters

Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition

June 1997

495 501

2-s2.0-0030683245

11.

Lim

K. H.

Ang

L. M.

Seng

K. P.

Chin

S. W.

Lane-vehicle detection and tracking

Proceedings of the International Multi Conference of Engineers and Computer Scientists

2009

12.

Ferryman

J. M.

Maybank

S. J.

Worrall

A. D.

Visual surveillance for moving vehicles

Proceedings of the IEEE Workshop on Visual Surveillance (ICCV '98)

1998

73 80

13.

Koller

Weber

Malik

Robust multiple car tracking with occlusion reasoning

Proceedings of the European Conference on Computer Vision

1994

189 196

14.

Koller

Weber

Huang

Malik

Ogasawara

Rao

Russell

Towards robust automatic traffic scene analysis in real-time

Proceeding of International Conference on Pattern Recognition

1994

15.

Orwell

Lowey

Thride

Architecture and algorithms for tracking football players with multiple cameras

Proceedings of IEE Vision, Image and Signal Processing 2005 152 2 232 241

16.

Needham

C. J.

Boyle

R. D.

Tracking multiple sports players through occlusion, congestion and scale

Proceedings of the British Machine Vision Conference

2001

2 93 102

17.

Kays

Tilak

Kranstauber

Jansen

P. A.

Carbone

Rowcliffe

Fountain

Eggert

Camera traps as sensor networks for monitoring animal communities

International Journal of Research and Review in Wireless Sensor Networks 2011 1 19 29

18.

Uchiyama

Yamamoto

Nakamura

Yamazaki

Sensor network for observation of seabirds in Awashima island

Proceedings of the International Conference on Information Networking (ICOIN '11)

January 2011

Barcelona, Spain

64 67

2-s2.0-79952928109

10.1109/ICOIN.2011.5723135

19.

Micheloni

Foresti

G. L.

Snidaro

A co-operative multi camera system for video-surveillance of parking lots

Proceedings of the IEE Workshop on Intelligent Distributed Surveillance Systems

2003

21 24

20.

Campbell

Gibbons

P. B.

Nath

Irisnet: an internet-scale architecture for multimedia sensors

Proceedings of the ACM Multimedia

2005

21.

Krumm

Harris

Meyers

Brumit

Hale

Shafer

Multi-camera multi-person tracking for easy living

Proceedings of the 3rd IEEE International Workshop on Visual Surveillance

2000

3 10

22.

Wang

Yan

A framework of heterogeneous real-time video surveillance network management system

Proceedings of the International Conference on Information Technology, Computer Engineering and Management Sciences (ICM '11)

September 2011

Nanjing, China

218 221

2-s2.0-84856638994

10.1109/ICM.2011.95

23.

Kulkarni

Ganesan

Shenoy

The case for multi-tier camera sensor networks

Proceedings of the 15th International Workshop on Network and Operating Systems Support for Digital Audio and Video (NOSSDAV '05)

June 2005

141 146

2-s2.0-29844438527

10.1145/1065983.1066016

24.

Aghajan

Cavallaro

Multi-Camera Networks, Principles and Applications 2009

Academic Press

25.

Hörster

Lienhart

Approximating optimal visual sensor placement

Proceedings of the IEEE International Conference on Multimedia and Expo (ICME '06)

July 2006

Toronto, Canada

1257 1260

2-s2.0-34247573512

10.1109/ICME.2006.262766

26.

Hörster

Lienhart

On the optimal placement of multiple visual sensors

Proceedings of the 4th ACM International Workshop on Video Surveillance and Sensor Networks

2006

27.

Ahmad

Lawal

O'Nils

Oelmann

Imran

Khursheed

Model and placement optimization of a sky surveillance visual sensor network

Proceedings of the 6th International Conference on Broadband and Wireless Computing, Communication and Applications (BWCCA '11)

October 2011

Barcelona, Spain

357 362

2-s2.0-84855849512

10.1109/BWCCA.2011.56

Modeling and Verification of a Heterogeneous Sky Surveillance Visual Sensor Network

Abstract

1. Introduction

2. Theory

2.1. Homogeneous VSN Model

2.2. Heterogeneous VSN Model

2.3. Example of Heterogeneous VSN

2.4. Cost Reduction by Heterogeneous VSN

3. Camera Models and Optics

4. Coverage Model

5. Measurements of VSN Parameters

5.1. Coverage Measurements

5.1.1. Altitude D

5.1.2. Altitude C

5.1.3. Altitude B

5.2. Resolution Measurements

5.2.1. Altitude E

5.2.2. Altitude D

5.2.3. Altitude C

6. Results

7. Conclusion

References