Abstract
Wireless sensor networks are already employed in numerous applications including military, industry, and health. Among the several inherent limitations in wireless sensor network, security is a critical concern. The declared security functions of wireless devices should be well verified. In this study, a security testing method based on security levels is proposed for wireless sensor networks. In addition, an experimental security testing platform for WIA-PA-based wireless sensor networks is implemented. The test results show that the platform is a feasible platform for assessing device security levels in wireless sensor networks.
Introduction
Wireless sensor networks (WSNs) have been widely used in many fields such as military, industry, agriculture, business, and environmental science. It has become a new driving force for the development of the information industry. 1 However, WSNs have many defects such as the openness of WSN channels, limitations of terminal resources, and the dynamic nature of network topology. In addition, traditional network security technology cannot be used in WSNs because of its inherent limitations. Several high security requirements exist for WSN applications, which face tremendous risks. 2 Therefore, security concerns represent major obstacles in the further development of WSNs.
The security of WSN protocol implementation is critical and difficult to verify. Security protocol testing is usually based on the experience of the tester and specific protocol specifications. A formal and automatic verification method and a security test platform are always required.
Implementations of security protocol are usually separated into different network devices and are in charge of security functions to guarantee communication security. The security level of a device plays a major role. A non-security device that suddenly joins a WSN will compromise the availability and reliability of a whole network, which can then be exploited by an attacker. A crucial factor in these implementations is whether the declared security functions of devices are well verified. Therefore, finding an appropriate method for assessing security levels of the WSN devices is urgent.
Several previous studies have been conducted on this issue. International Standard ISO/IEC 9646 3 stipulates common testing methodologies based on open system interconnection (OSI) and provides a framework for testing whether the consistency of a protocol conforms to its specifications. Allen et al. 4 presented a security test method that combines an available formal model and the method of fault injection used to detect security vulnerabilities in server applications. Qi et al. 5 proposed an object-oriented attack model to represent minutely a protocol attack in WSN. A security testing approach was presented for WSN protocols based on object-oriented attack models. Ji et al. 6 proposed an automated black-box testing approach for WSN security protocols. The study integrated a randomness test with conformance testing. Botella et al. 7 introduced an original security testing method guided by risk assessment, based on risk coverage, to perform and automate vulnerability testing for web applications. Harris et al. 8 developed a tool to perform security testing of Voice over IP (VoIP) applications to identify security vulnerabilities. Slijepcevic et al. 9 proposed a communication security scheme in which a corresponding security mechanism is defined for each type of data, by employing this multitier security architecture in which each mechanism has a different resource requirement. Jinwala et al. 10 proposed a novel design of link layer security architecture for WSNs. Cionca et al. 11 present a method to configure security using only paremeters taken from the application space, and a tool that implements this method, hence automating the process of security configuration for non-expert users. The principal characteristic of the design we propose is the flexible and configurable architecture.
Some established security evaluation frameworks are not available for WSN, because they do not take the specific characteristics of a WSN into account, such as the constrained resources, and the specific types of WSN devices (leaf, router, and coordinator). These studies focused on security protocol testing and involved specific applications. However, there is a dearth of common approaches for operating security protocol tests and assessing device security levels. Verifying security over protocol implementations and assessing security levels of the WSN devices remain open problems.
In this study, we address this important security problem of WSNs. In addition, we propose a security testing method that verifies the security of protocol implementations automatically and assesses the security levels of WSN devices. Finally, a security testing platform based on WIA-PA 12 is implemented, which is an International Electrotechnical Commission (IEC) international standard for wireless industrial automation networks. The remainder of this article is structured as follows. In section “Preparatory information,” related notation and knowledge are described. Section “Testing framework and method” explains the test framework and method. Section “Platform implementation and result analysis” analyzes security performance of the proposed method and reveals the results of our security test for WSN devices. A conclusion is given in section “Conclusion and future work.”
Preparatory information
Symbolic notation
In order to describe the testing framework and method adequately, Table 1 lists the symbols and notations used in this study.
Symbols.
Security levels and functions
Protocol testing represents a set of automata-based modeling and testing methods to verify the correctness of the protocol implementations, which have been proposed and widely accepted in industries since Gonenc 13 identified the distinguishing sequences in 1970. Through critical preset inputs, the unobservable system status and some internal actions can be deduced by checking the corresponding outputs. The protocol implementations can then be verified by means of testing. 14
Several specifications exist for describing WSN security levels. In this study, we consider “Chinese national standard: Information technology—sensor network—Part 601: information security general technical specifications (GB/T 30269.601),” 15 as the test reference which defines security level and some security functions. The design and implementation of the security test, however, have not been provided. The security levels of a device depend on its implemented security functions. Here, five security levels are considered. The security level increases as additional security functions are implemented by a device.
Correlations of security functions and security levels are shown in Table 2. Following the actual application requirements and GB/T 30269.601, most security functions of WSN are inspected. For example, join authentication, data integrity, data privacy, data freshness, key management, data authentication, sensitive marker, discretionary access control, mandatory access control, user authentication, and node identification are all put into consideration.
List of security levels.
Here, “+” represents the intensity of Gj. An increase in + indicates an increase in the intensity of the function. For example, the intensity of data integrity is equal to 1 at Level 1, whereas it is equal to 3 at Level 5. “×” means that the security function Gj is not required. Level 5 is the highest security level, which means that all functions with high intensity are implemented. Level 1 is the lowest security level, which means that parts of security functions are implemented. T is defined to describe Gj intensity set under a specific level (see section “Testing framework and method”).
The intensity of Gj depends on the number of sub-functions implemented. Each Gj includes several sub-functions. As additional sub-functions are implemented, the intensity of Gj increases. The related sub-functions are shown in Table 3.
Sub-functions’ symbol.
A security test case is a specification for the behaviors of a tester, which exists in an experiment to be carried out with an implementation under test. Such test cases can be specified based on the related sub-functions listed in Table 3. The test case can then be described as a specific action set with a final test verdict (pass or fail). We will discuss the manner in which to choose test cases in the following sections.
Testing framework and method
Testing framework
A security testing framework for WSNs is proposed, as shown in Figure 1.

Security testing framework.
This security testing framework mainly includes a test terminal, security test server, standard test devices (STDs), and device under test (DUT):
Test terminal is the human–machine interface, which provides a test entrance. In addition, a tester can use a browser to access the web-based database server to input the device type, declare the device security level, and check test results. The test terminal and security test server can communicate through a wired (Internet) or WiFi network.
Security test server is the core of the security testing framework, which provides web-based database services. It is used to store test cases. The server can automatically generate a test case set based on the declared security protocol, declared device security level, and declared device type. In addition, it also provides functions of querying and adding test cases for the tester, thus increasing the scalability of the platform.
STDs are placed in the test area and may play the role of coordinator, router, or sensor node (shown in Figure 1) in a WSN. STDs are configured for a specific role based on DUT type. If the DUT is a sensor node, the STDs are configured as both a router and coordinator. If the DUT is a router, the STDs are configured as both a sensor node and coordinator. STDs forward test commands and upload test responses to the security test server. The designed test platform performs an interception and engages in normal protocol operation with the DUT, to check whether the implemented protocol follows the specification.
DUT is placed in the test area (the blue device shown in Figure 1). The security functions of DUT are tested and assessed. STDs and the DUT communicate wirelessly.
Description of security testing method
A security testing method is proposed and described in this section. The security testing method is shown in Figure 2

Security testing method.
The method consists of three parts: input, output, and a processing mechanism. The specific WSN protocols for DUT are claimed and are input to the test system before the test execution. The test framework and platform are flexible and are designed for various protocols, such as 6LoWPAN, 16 WIA-PA, ZigBee, 17 and WirelessHART. 18 The processing mechanism is described in the following steps:
Step 1. After the tester inputs the declared security level and type of DUT, the security level processing module treats “device level” as the input. Then the module outputs G and T according to Table 2. The G and T sets are described by the following formulas
The value of Tj depends on the number of + in Table 2. For example, one + means that the value of Tj is 1, two + denotes a value 2, and × means the value is 0. If the device is at Level 5, then T = {1, 3, 3, 2, 4, 2, 1, 3, 1, 5, 4}.
Step 2. The case sets are selected in this step. The case set selection strategy library determines MTC for the whole test platform. Based on the relationships between the security functions and their sub-functions, the security function test cases are obtained as shown in Figure 3.

Security function test suite.
The module uses device level and “device type” as the inputs and outputs MTC that can reflect the security performance of the device from the perspective of the level
The test group selection is also related to device type. If the DUT is the coordinator, test groups including G7, G8, and G9 are considered. Other security functions are considered as wired components, which can be tested using traditional network test methods. If the DUT is a router, test groups including G0–G5 are considered because other security functions usually are not implemented in the router. The selection strategy of MTC is shown in Figure 4. The rules are generated using Table 2 and Figure 3.

Case set selection strategy library.
Step 3. The optional test case OTC is also provided to the tester. The optional test case set module in the platform provides extended interfaces in which the tester can add a new test case by means of the test terminal.
As shown in Figure 3, G11 and G12 are part of the optional test group that includes GSF27, GSF28, GSF29, and GSF30 as OTC. The OTC set is described by the following formula
Step 4. MTC and OTC as test cases are sent to the security function test suite module in the platform. The module generates a new set TC according to the inputs (MTC and OTC). The TC set is described by the following formula
Each test case maps to the unique test command (CMD) and the expected test response (ERP). The module can generate an array set B, which will be sent to test execution system. The B set is described by the following formula
Step 5. The test execution module is the core of security test. First, after the test execution module receives B, the module will begin to analyze the arrays of B to obtain the GSF and CMD. Then, the DUT is tested according to the existing CMD. Afterward, the actual test response results will be obtained, and a test response message will be structured and sent to the security function test results’ analysis module. Some test commands and responses of sub-functions for WIA-PA devices are described in Table 4.
Step 6. The security function test results are analyzed in this step. The security function test results’ analysis module generates the actual test set N. This module compares RRPn with the ERPn under each test case:
If RRPn is equal to ERPn, the test case indicates success.
If RRPn is not expected, the test case indicates failure.
The module then stores the consistency analysis report in the database for the tester to view
Step 7. The security level of the device is assessed in this step. The assessment module determines whether the DUT has coherent security functions and the security level that the tester declares.
ETj as the sum of intensity value of the test case group Gj is calculated. If the test result of the test case is successful, the intensity value of the related test case is 1; otherwise, the intensity value is 0. ETj is calculated by summing the intensity value of every test case in each test group for security function Gj. The ET set is described by the following formula
Here, a parameter ∂ is defined to describe the conformity between the declared and tested security levels.
The test command and response of sub-functions.
The parameter ∂ is calculated by the following formula
where Dj is the weighting factor of Gj. The value of Dj is related to the influence degree of security functions Gj in the whole network. Here, n is the number of Gj, as shown in Table 5.
List of weighting factors.
The ∂ is used to describe the consistency between the specified security level and the actual test result. The variable ∂ depends on the pass rate of the test case of sub-functions. It implies that as more security sub-functions pass through testing under the same security level, a lower ∂ will be obtained. Thus, ∂ reflects the degree of deviation between the specified security level and the actual test result. The device level can be assessed by ∂. A higher value of ∂ means the device is not in accordance with the declared security level. In contrast, the lower the value of ∂, the more that DUT is in accordance with the security level. In summary, after the aforementioned steps, the tester knows whether DUT conforms to the declared security level.
Platform implementation and result analysis
Platform implementation
In this section, we describe the integrated hardware and software experimental platform that we developed in this study. An actual image of our test platform is shown in Figure 5.

Experimental scenario of the test platform.
Security testing server
The security testing server was implemented using a high-performance personal computer, on which a web-based test database with all test cases was built. It provided test cases’ commands to the DUTs. The security testing server is developed based on the asp.net, which is an open-source server-side web application framework designed for the web development of dynamic web pages. It was developed by Microsoft to allow programmers to build dynamic web sites, web applications, and web services. At the same time, the SQL server is also used as a comprehensive database platform to store test commands and results.
Remote test terminal
The remote test terminal was implemented using a browser tool. The tester may use a browser tool to access the server at any PC on which Internet is available.
SDT
The SDT devices in this experimental platform include a coordinator, router, and sensor node (see Figure 5). The communication protocol in this WSN is based on WIA-PA, 12 which is an industrial WSN protocol.
DUT
The DUT device is a router in this experimental scenario. It is used for security tests in which the security is declared as Level 4.
Based on the device type and security level, we use Table 2, equations (1) and (2) to obtain the following
Based on Figure 3 and equation (3), MTC is obtained by means of the following formula
Based on equations (5) and (6), B is obtained as the following formula
After Step 6 is completed, N is obtained as follows
The test groups including G0–G5 are considered. Then, ETj can be calculated. ET and T as the following two formulas, respectively
Finally, ∂ = 0.1 can be obtained using equation (9). In this scenario, the DUT-declared security Level 4 did not pass all test cases, which means DUT is not a Level 4 device. This is reported to the tester using the security test report.
Result analysis
After the security test is finished, the tester knows whether DUT conforms to the security level and standard. The developer can then improve the DUT based on the security level report, as shown in Figures 6 and 7.

Conformance report.

Security level report.
To verify the correctness of the testing results, the message information derived from the testing is analyzed in this study. Here, we chose a key management test case (G4) to analysis the data communication process, which is the basis of the WSN security mechanism.
The security testing server sends the test command to the STD (coordinator), and the STD then forward the test command to the DUT (router). The DUT then sends a response message back to the STD. Afterward, the security testing server will receive the key distribution response message that is compared with the test command message; if so, the test case is successful. Figure 8 shows the message information in the process of key distribution.

The key distribution message information.
However, the key update is different from the key distribution. First, the test system will need to obtain the current key pair messages by sending a key request command, as shown in Figure 9. Second, the test system will send a key update command to DUT, and then a key update response message is fed back to the test system, as shown in Figure 10. Finally, the current key pair message is compared with the key update response message, and if the results are identical, the test case is deemed to be successful.

The message information of current key.

The key update response message information.
Conclusion and future work
This article analyzes security problems of the WSN field, studies some existing security protocol test methods, and proposes a security method and test platform. The proposed method is shown to help users generate probable test case commands and expected responses. The platform automatically generated the test cases to verify implementations of security functions. In the future, other international standards will be considered and additional test case sets will be added in order to improve the test platform. In addition, more complex experiments will be defined.
Footnotes
Academic Editor: Michele Amoretti
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by Institute for Information & communications Technology Promotion (IITP) grant funded by the Korean government (MSIP; B0101-15-1866, Development of autonomous management core technology and autonomous control network).
