Abstract
As steady research in wireless ad hoc and sensor networks is going on, performance evaluation through relevant network simulator becomes indispensable procedure to demonstrate superiority to comparative schemes and suitability in most literatures. Thus, it is very important to establish credibility of simulation results by investigating merits and limitations of each simulator prior to selection. Based on this motivation, in this article, we present a comprehensive survey on current network simulators for new emerging research area, three-dimensional wireless ad hoc and sensor networks which is represented by airborne ad hoc networks and underwater sensor networks by reviewing major existing simulators as well as presenting their main features in several aspects. In addition, we address the outstanding mobility models which are main components in simulation study for self-organizing ad hoc networks. Finally, open research issues and research challenges are discussed and presented.
Keywords
Introduction
In computer networks, simulation study became one of the most important research challenges since simulation is the most eligible method to evaluate performance in cost-efficient way. That is, simulation is employed to justify feasibility and performance of the new mechanisms in typically simplistic way. Moreover, simulator can provide fundamentals in performance when it comes to approach new research area specially. Based on this motivation, many simulators have been newly developed or extended to implement specific functions on typical network simulators which include NS-2, 1 NS-3, 2 QualNet, 3 OPNET, 4 and so on. Furthermore, since each simulator provides unique features in the aspects of protocols and network models, it is very important task to find out which simulator is the most suitable for corresponding work prior to performance evaluation.
Moreover, as compared to wired networks, performance evaluation with simulation in wireless networks requires more complicated functionalities for network environments such as interference and propagation model. As the credibility of simulation results is determined by these models, research efforts to improve these models are going on in each simulator continuously. This improvement contributes to enhance the accuracy as well as providing new models for communication protocols. As good case, as steady research in wireless ad hoc and sensor networks is going on, research for performance evaluation is usually conducted on well-known network simulators mentioned before as well as specific developed simulators ANSim 5 and JiST/SWAN. 6 In addition, many recent research works tend to extend current simulators to provide the eligible new models and parameters for network configuration. For example, SWANS++ 7 has been proposed to improve the existing code and new mobility models in typical wireless ad hoc networks. In the perspective of the wireless sensor networks, both mentioned existing simulators and specialized ones such as OMNeT++ 8 are mostly used for evaluation. In addition to simulation framework, implementing diverse communication protocols in existing simulators is going on now. For instance, as ad hoc networks research area is expanding to cover three-dimensional network model as well as typical two-dimensional one recently, network simulator should meet the requirements of operations on them. This implies that specific functionalities should be included in each simulator to produce the credible results.
Based on this motivation, in this article, we review these research efforts and present network simulators used in the performance evaluation. As part of review, first of all, we address the specific mobility model which greatly affects the performance and characterizes its features with others. In the next, we present current network simulators for three-dimensional wireless ad hoc and sensor networks in the aspects of key features and limitation. As for three-dimensional wireless ad hoc networks, we focus two major networks, airborne ad hoc networks (AANET) or flying ad hoc networks (FANET)9–11 and underwater sensor networks (UWSN). 12
The rest of this article is organized as follows. In the next section, we introduce the overview of network simulators and their features. We address the open issue and research challenges for network simulators in the following section. Finally, we make conclusion and present future work.
Network simulators for three-dimensional ad hoc and sensor networks
Mobility model for AANET and UWSN
There are good referred simulation studies with well-known mobility models which describe pedestrians’ movement patterns.13,14 Among them, random waypoint model is the most popular one in the simulation study. But, it cannot be applied into all types of ad hoc networks. For example, vehicular ad hoc networks with moving vehicles use different mobility model with random waypoint since cars can travel along the road. 15 To handle this difference, the representative simulation study was accomplished by the new movement generation program. 16 As for mobility model in three-dimensional network, bounded connected link and moving model are presented in Xiaoming. 17 They review the mobile node model in three-dimensional networks and analyze link dynamics by computing link lifetime, formation, and breakage. Also, they provide simulation results for connectivity in three-dimensional moving network. In addition to above paper, in the following section, we review the representative mobility models for AANET and UWSN.
Mobility model for AANET
Unlike the typical two-dimensional space, three-dimensional one possesses
The most important feature worthwhile mentioning for mobility model based on random waypoint is unsuitability for airborne networks due to sudden stop and sharp turn which are addressed by smooth turn random mobility model in Kumari et al. 21 To defeat this shortage, another mobility model called enhanced Gauss-Markov (EGM) was proposed to describe networks of UAVs. 22 Drastic change in velocity and direction in random waypoint model is prevented by Gauss-Markov (GM) mobility model. EGM mobility model adapts novel mechanism to compute direction with GM mobility model. In this model, drastic rotation is removed by advanced collision avoidance when an UAV approaches the boundary. Through this model, more realistic movement is eligible, but there is still problem of availability in that only two-dimensional model was presented at this time. So, it is required to extend current model to one in three-dimensional model.
Another mobility model is for formation or group flying where many UAVs work on a one goal in collaborative way. For more realistic model, hierarchical architecture on clustering is assumed according to altitude of the UAV. For the new mobility model, the authors proposed to combine pheromone-based model and K-HOP clustering algorithm (KHOPCA).
23
By the help of two mechanisms, a UAV can maintain connectivity to header at the upper level as well as member in the same level. Their approach is described in Figure 1. In Figure 1, the selection between pheromone-based and KHOPCA-based direction depends on two values, that is, KHOPCA probability denoted by

KHOPCA-based mobility model.
Moreover, specific mobility model was proposed to describe the tactical movement of UAVs in disaster scenarios in Snchez-Garca et al. 24 Furthermore, this mobility model is designed to maximize the number of victims detected by the UAVs while maintaining connectivity in efficient way. To achieve this, Jaccard distance and artificial intelligence algorithm are combined. But, since victims’ movement is not regular with randomness, their approach is based on several types of regions for incident site. Each area is assumed to contain some restrictions and obstacles such as structural features. Under this architecture, victims’ randomness is controlled by speed and redirection within a region. The UAV mobility model can handle sudden changes in speed and direction for more realistic scenarios through Jaccard distance and new movement rule. Also, this movement rule is affected by artificial intelligence algorithms such as simulated annealing to optimize the number of serviced victims.
In Atten et al., 25 group mobility model for special purpose was proposed. Nature-inspired mobility model, ant colony optimization (ACO), is employed to perform target discovery and tracking in an area by dropping repulsive pheromones on already scanned zones. Two types of pheromones, repulsive and attractive pheromones, are used to cover the area and follow targets, respectively. To control the movement, this model takes several steps to determine the position. First step is to ensure connectivity between UAVs. In the following step, all UAVs scan the area in order to detect targets. If the target is detected by any available UAV, it moves to target position and keeps tracking it. When target is assumed to locate within scanned area, procedure based on attractive pheromones is performed.
As other approaches for mobility model, tracing is another way to extract a mobility model as mentioned in Kim et al. 26 Since airborne node is controlled by the complex aerodynamics, it is very hard to define their behaviors by the simple mathematical equation. On the other hand, it is impossible to trace movement of aircrafts in real world due to security reasons and flying in three-dimensional space. As an alternative, the authors proposed new architecture to make use of flight simulator which is used to train the pilot by providing diverse dynamics of aircrafts. 27 With the help of automatic aviation function and collision avoidance functions, it is possible to generate multiple traces without duplication. The architecture to generate movement trace is shown in Figure 2. The type of movement and vehicle are configured and passed to mobility generator. After trace is generated for configuration, it is converted to file format of simulation platform.

Mobility generation architecture.
Mobility model for UWSN
When it comes to develop mobility model in UWSN, it shows completely different mobility patterns from AANET because the nodes in UWSN consists of static ones deployed on the sea floor.28,29 These sensor nodes can communicate with special node, called supernode, which has connection with high-speed networks and deliver data to base station. These are attached to floating and fixed buoys so their mobility pattern is limited to some regions. Therefore, as compared to AANET, movement is less dynamic and controlled by the currents. However, it has similar feature in that random waypoint model is not suitable for this network.
As for mobility model, it is closely related to localization problem which addresses the location and way to deploy nodes in the networks. Since the movement of sensor nodes makes it difficult to predict next location in UWSN, localization problem needs to take mobility into account. 30 As many literatures already mentioned, localization schemes predict the location of sensor nodes through mobility models. For example, scalable localization scheme with mobility prediction (SLMP) in Zhou et al. 31 performs localization procedure with prediction for future mobility pattern with its past location information. That is, its mobility model is built by estimating its location so mobility plays great role in localization process. Despite the motivation of mobility model was introduced in Han et al., 32 some new mobility models need to be explored in this section.
The most popular research work for mobility model for UWSN is meandering current mobility (MCM) model in Caruso et al. 33 This model consists of meandering jet and stream-function to make use of trajectories of fluid in the Gulf streams. More details, drifter for vertical movement such as strong wind-driven upwelling or down welling and stream-function with currents and vortices are used to define dynamics. Based on the model, they make strategy to maintain connectivity between nodes. Another mobility model, called Oceanic Forces Mobility Model (OFMM), was proposed to represent realistic oceanic scenarios. In order to design appropriate mobility model, the authors take major oceanic forces such as gravity, friction, and rotation of the Earth and convert them into pressure gradient force (PGF), frictional force and centrifugal force, and Coriolis force in mathematical expression. Moreover, mobility model which takes the seashore environments was proposed in Mandal et al. 34 This mobility model is based on Euler algorithm by assuming flow velocity with tidal field and remnant flow field under assumption that tidal field is oscillated in one direction homogeneously while the remnant flow field turn rotates in infinite sequence alternately. On this environment, the authors proposed how to estimate the velocity variation of an underwater node as times go in a typical seashore environment. In the next step, they use this value as their simulation parameters. On the other hand, since mobility model is very complex and complicated to implement, some literatures assume that all nodes are static without any mobility model for simplicity. 35
Network simulators for AANET and UWSN
Network simulators for AANET
AANET constructs self-organizing networks with flying multiple platforms such as command and control planes, UAVs, ocean interceptors, and ground towers. So, in the aspects of mobility model, flight objects show the obvious different patterns with existing objects on the ground. Also, network simulators should be implemented in three-dimensional space instead of typical two-dimensional one. General requirement for AANET simulators can be summarized such as (1) support various vehicle models, (2) support AANET communication simulation, (3) provide interactive graphical user interface (GUI) for users to control their experiments, and (4) provide users intuitive analysis results. While keeping this requirement in mind, we provide overview of some existing network simulators and present how they meet the requirements.
Network simulators for AANET are usually implemented as extension of two-dimensional space in mentioned general network simulators. For example, large-scale AANET was simulated by the NS-3. 36 Their new features include new models, topology control application, and visualization tool. Moreover, Aircraft Situation Display to Industry (ASDI) model is newly implemented to control position, velocity, and some important flight plan information. With the help of this ASDI, location of each vehicle is determined in a form of mobility model, for example, constant velocity model. It makes use of initial positions and scheduled velocity changes. Moreover, wireless point-to-point channel consisting of multiple wireless point-to-point devices with multiple directional antenna models was developed. In addition, both implicit and explicit topology control algorithms are provided to control connectivity. Optimized link state routing (OLSR) protocol is used for routing protocol to evaluate performance in simple scenario.
Another extension, called UAVSim, 37 is extension of OMNeT++ by including UAV modeling. They provide results analysis and experimentation control through GUI. This simulator is mainly developed for test-bed for cyber attack analysis so attack library is also developed. The software modules of UAVSim are illustrated in Figure 3. The UAVSim supports concurrent multi-user and swarm simulation, too. It has user-friendly GUI as well as high-speed simulation time.

UAVSim framework.
Due to popularity in existing research work, NS-2 is the best possible simulator for three-dimensional simulators. Aero-Sim 38 is based on NS-2 to support simulation for AANET by developing new functions and integrating them. This simulator supports three-dimensional environments and implements attributes of aeronautical nodes. In Aero-Sim, three kinds of aeronautical nodes with multiple interfaces are available in the Aero-Sim module. They are typical aircraft, airports, and satellites. Moreover, protocol stack is completely compatible with typical Transmission Control Protocol/Internet Protocol (TCP/IP) protocol. More details for protocol stacks and simulation results are also found in their work.
Another reliable simulator, OPNET, is widely used in the AANET simulation. First, the authors modeled the wireless link with 14 pipeline stage for connection between transmitter and receiver. A UAV network node is built with three layers: physical, data link, and upper layer which are slightly modified from standard module. In physical layer, a node has one omnidirectional antenna as well as three antennas. For the data link layer, two medium access control (MAC) modules, wireless local area network (LAN), and proposed UAV MAC module are implemented to work jointly. The upper layer which implements OLSR protocol is in charge of generating traffic. More details for protocol operation and simulation environments can be referred in Alshbatat and Dong. 39 Another simulation module based on OPNET was explained in Biomo et al. 40 Even though details for implementation and procedure are not explained, one study in the literature makes use of simulation package using SimJava 2. 41
Network simulators for UWSN
While few network simulators are available for AANET, more simulators are feasible now for UWSN. Dhviya and Arthi 42 and Raj and Sukumaran 43 present the current simulators and compare their strengths and weaknesses. Among them, both UWSim and Aqua-Sim get good reputations for their specificity for UWSN. But, there are more available network simulators for UWSN, so we present the outstanding features of them.
First, Simulation Emulation and real-life Testing (SUNSET) framework provides good architecture to implement and test new protocol for UWSN. SUNSET is built on the NS-2 and NS-2 Miracle. NS-2 Miracle includes the engine to process message in cross-layered architecture and enables coexistence of multiple modules. The outstanding architecture of SUNSET is multiple operations such as simulation and emulation mode. For both modes, the framework is composed of protocol stack, additional module, and channel model. Four layers are implemented in protocol stack while several modules such as timing, debug, and trace are connected to protocol stack. For the UWSN, it is feasible to use different underwater acoustic channel models such as the empirical formulas or Bellhop propagation. Through them, SUNSET has the chance to simulate the desired protocols over realistic underwater acoustic channel realizations. After SUNSET framework has been proposed, more flexible and code optimization are required. So, SUNSET 2.0 has been proposed. The new framework includes improved real-time scheduler, new communication protocols as well as enhanced functionalities to them, and new packet error modules. New features and functions are well described in Petrioli et al. 44
Another simulator called DEsign, Simulate, Emulate and Realize Test-beds (DESERT) for underwater network protocols
45
was proposed to design and test communication protocols in UWSN. DESERT is similar to SUNSET in that it is based on NS-2 Miracle. DESERT implements five-layer protocol. At first, constant bit rate (CBR) and variable bit rate (VBR) packet generators are served. Two modules, a simple module called

Protocol stack in DESERT.
Underwater Sensor Network Simulation Tool (USNeT) 46 was developed at University of Portsmouth, United Kingdom. Different from the discrete event simulator, it follows object-oriented design style by implementing all network entities as classes as well as employs threads to handle multiple tasks in parallel and concurrently. In the perspective of communication protocols, the outstanding feature of USNeT is support of clustering scheme which designates cluster header and members dynamically. Under this clustering architecture, path establishment between members, slot assignment to avoid interference, and data aggregation schemes are provided. Also, ALOHA-based MAC protocol and energy aware model are implemented in data link and physical layer. The GUI helps the researcher to display node and signal properties as well as set necessary variables such as frequency and simulation time, according to a research scenario.
Underwater acoustic WSN SIMulator (UASim) 47 is built on top of OMNeT++ and mixed simulator (MiXiM) framework 48 as shown in Figure 5. MiXim provides communication protocol in each layer for wireless and mobile networks developed using OMNeT++. This simulator includes discovery, static clustering, and simple routing which are contained in MiXim framework. For the UWSN, accurate underwater acoustic channel is modeled and implemented. A GUI provides an easy way to configure simulation scenario by enabling the user the ability to change most parameters of a simulation. The example of how to build scenario and details are explained in the references.

UASim architecture.
Jaltarang 49 and MATLAB-based Acoustic Underwater Simulator (MAcoSim) 50 were developed at Indian Institute of Technology, as part of project for robust and secure data acquisition in UWSN. The former is based on and installed on top of NS-3. The included functions are topology control, localization, mobility module, and communication protocols. Also, simulation scenario can be configured by GUI/command line access and its result is generated as trace file via command-line/file output with visualization module. The latter, MAcoSim, is identical to Jaltarang except that MAcoSim is operated by both GUI and MATLAB command line interface. Additionally, centralized parameter manger was employed to acquire easy configuration.
In addition to UWSN, underwater acoustic networks (UAN) framework 51 is well deployed in NS-3. This project offers autonomous underwater vehicle (AUV) mobility models and energy models operated by battery. Also, it provides accurate model of the underwater acoustic channel and some MAC protocols. With this framework, the user can make scenario for navigation and movement of AUV and use a specific energy model for the AUV through energy model. Moreover, in the perspective of channel model, propagation model of underwater acoustic channel includes the ideal channel model, the Thorp propagation model, and the Bellhop propagation model. For the physical layer, signal-to-noise ratio (SNR) and packet error rate (PER) models have been developed to compute Signal-to-Interference-plus-Noise Ratio (SINR) and probability of error. Similar to other simulator, as for the MAC layer, a MAC protocol which uses a slotted contention window and reserved channel MAC (RC-MAC) protocol which dynamically divides the available bandwidth into a data channel and a control channel. Also, ALOHA protocol is basically included in this framework.
Criterion of selection
Generally, there are some criteria of selection for network simulators such as types of supported simulations that is event driven or trace driven. Even though it is very difficult to get the real traces for aircrafts, trace-based mobility model leads to more reliable results than mobility one modeled by mathematical equation. However, since most of the simulators support trace-driven and event-driven models simultaneously, it does not make any difference between models. The next is to support visualization for the events. If three-dimensional visualization is not supported, additional software is demanded for analysis and monitoring. Moreover, since the three-dimensional simulations require more computing power for location, scalability is another criterion for selection. But, these scalability problems mostly depend on the framework. Besides mentioned one, traffic model for three-dimensional networks and node type for aircrafts can be another factor for selection. In other aspect, repeatability is the most important concern for diverse simulation. Since multiple scenarios are performed to monitor operations in many scenarios, easy configuration and setup for multiple operations are required for simulator in AANET and UWSN. Finally, the most important criterion of selection is to support communication protocols which are specific for the three-dimensional networks. Any protocol included in the simulators will help researchers to develop their own protocol or modify the existing protocol.
Open research issues and challenges
Scalability
As mentioned before, one of the important features in network simulators is scalability. Unlike the simulator for two-dimensional network, it is required to trace the movement with additional
Reality
Even though some simulator aims to provide realistic simulation environments including propagation and channel model, it is required to validate this model in an appropriate way. Also, since three-dimensional environments such as cloud and obstacles in the sky and wave in underwater can affect the propagation model and error rate, more realistic configuration need to get credibility of simulator. Moreover, there are more parameters to be concerned for good environmental circumstance. For example, three-dimensional terrain data contribute to enhance reality. According to terrain data, mobility of vehicle should be adjusted to avoid collision against ground or building. This means that good mobility model can lead for airborne vehicles to move along the detoured path to avoid collision. Without considering this fact, it generates wrong movement traces. In the UWSN, the antenna gain calculations on floating supernode can be affected by wave direction, length, and period on the sea surface. Also, like AANET, path loss calculation requires parameters from the ocean terrain model as well. Even though many research works focus on acoustic channel model, more works need to be conducted in propagation path loss and ocean terrain model.
Alternatives: emulation and measurement by experiments
Even though simulator is good evaluation method for AANET and UWSN, their limitations cannot be neglected. So, this limitation can be overcome by alternatives which include network emulation and measurement by experimental solutions as mentioned in Kwak et al. 52 For more details, Radio Frequency Network Emulation and Simulation Tool (RFNEST) was proposed to run large number of cases and experiments for AANET with high fidelity and low cost. It provides accurate bidirectional wireless channel and high-speed mobility management. Also, it supports autonomous routing for Unmanned Aerial Systems (UASs) and cross-layered design to validate model. Also, emulators for UWSN also were developed with the same motivation as addressed in Maxa et al. 53
On the other hand, measurement by experiments in real scenarios can overcome the shortage of simulation as well as convince the results despite its expensiveness. In the case of AANET, even if experiment in real world for aircraft is not easy, micro-UAV system is relatively easier than aircraft case. For example, experimental results for two-hop communication with quadrocopter equipped with IEEE 802.11a were presented in Yanmaz et al. 54 Two modes for communications were compared and analyzed. Even though mesh mode was implemented, more diverse experimental scenarios are expected since the number of vehicle is limited to 3. As compared to AANET, there are much more literatures for experiment on UWSN. In Pu et al., 55 the authors evaluated the performance of MAC protocols in the real sea. Random access–based underwater-Aloha (UW-Aloha), Selective ARQ with Slotted Handshaking-based Access (SASHA), and scheduling-based pipelined MAC (PMAC) were tested in the Atlantic Ocean where nine underwater acoustic nodes were deployed as a string topology. Based on experiments during 4 days, the authors can discover several problems and discuss analysis results.
Conclusion and future work
Due to their specific properties of vehicle and network environments in AANET and UWSN, it is not eligible to use existing two-dimensional network simulators to test and evaluate various communication technologies in a realistic way. To overcome this problem, extension of existing simulators and development of new frameworks are going on, respectively. In this article, we presented the research effort for network simulators which are specialized for AANET and UWSN. Even though brief survey literatures were already published a few years ago, new research trend and new features are addressed in this article. Finally, we addressed open research challenges in network simulators.
Related to new simulator, our research group will make our previous prototype concrete for AANET by combining flight simulator and existing network simulators. 56 For the implementation, OpenSceneGraph (OSG) for visualization and external interface for MATLAB are almost developed.
Footnotes
Academic Editor: Babar Shah
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by Basic Science Research Program (2015R1D1A1A010569) through the National Research Foundation of Korea (NRF) funded by the Ministry of Education. Also, this research was supported, in part, by Zayed University Research Office, Research Incentive Grant, R15121.
