Sage Journals: Discover world-class research

Abstract

Many network security systems analyze large scale data collected from multiple collaborating domains or aggregated network vantage points. Scale is clearly beneficial for these systems, however it also makes them difficult to design and test. Large scale data sets can be difficult to acquire and may not contain important meta-information (e.g. ground truth). Further, their limited availability can make it extremely difficult to understand how well experimental results would reproduce in different conditions, or at different networks. In this article, we discuss using simulation to overcome these challenges. We present an augmented version of LESS, our recently proposed agent based simulator for evaluating large scale network security systems. LESS uses publicly available data sets and high level parameters to generate synthetic traffic that models large scale, multi-network scenarios. Essentially, LESS allows researchers to “scale up” the data and statistics about networks and attacks that they have access to, so that they can be used to test large scale network security systems. Researchers can also tune LESS’s high level parameters to better understand the sensitivities of their systems, and the reproducibility of their results. The version of LESS that we discuss in this article is extended to allow researchers to study an additional factor of system performance related to reproducibility: deployment location; by modeling the global Internet topology at the Autonomous System level. We demonstrate the applicability and benefits of LESS by tuning it with publicly available traces and then using generated records to reproduce and extend results from several recently proposed large scale security systems. In new experiments, we use LESS to study how deployment location affects large scale security systems. Our results demonstrate that LESS can evoke realistic performance from these systems with minimal tuning and provide insight into the network and topological factors that may affect the reproducibility of their evaluations.

Keywords

Data challenges large scale security simulation agent based stochastic

Get full access to this article

View all access options for this article.

References

Argus , Audit records generation and utilization system, available at: http://qosient.com/argus/.

A.J.

Aviv and

Haeberlen , Challenges in experimenting with botnet detection systems, in: Proceedings of the 4th Conference On Cyber Security Experimentation And Test – CSET’11, ACM, 2011.

A.-L.

Barabási and

Albert , Emergence of scaling in random networks, Science 286(5439) (1999), 509–512. doi:10.1126/science.286.5439.509.

V.D.

Blondel ,

J.-L.

Guillaume ,

Lambiotte and

Lefebvre , Fast unfolding of communities in large networks, Journal of Statistical Mechanics: Theory and Experiment 2008(10) (2008), P10008. doi:10.1088/1742-5468/2008/10/P10008.

Boggs ,

Hiremagalore ,

Stavrou and

S.J.

Stolfo , Cross-domain collaborative anomaly detection: So far yet so close, in: RAID, Springer, 2011, pp. 142–160.

Bonabeau , Agent-based modeling: Methods and techniques for simulating human systems, PNAS 99(Suppl. 3) (2002), 7280–7287. doi:10.1073/pnas.082080899.

Caida data overview, available at: http://www.caida.org/data/overview/.

Cao ,

W.S.

Cleveland ,

Gao ,

Jeffay ,

F.D.

Smith and

Weigle , Stochastic models for generating synthetic http source traffic, in: The 23rd Annual IEEE International Conference on Computer Communications – IEEE INFOCOM 2004, Vol. 3, IEEE, 2004, pp. 1546–1557.

Chen and

R.S.

Gray , Simulating non-scanning worms on peer-to-peer networks, in: Proceedings of the 1st International Conference on Scalable Information Systems – InfoScale’06, ACM, 2006, Article no. 29. doi:10.1145/1146847.1146876.

10.

Coskun ,

Dietrich and

Memon , Friends of an enemy: Identifying local members of peer-to-peer botnets using mutual contacts, in: Proceedings of the 26th Annual Computer Security Applications Conference, 2010.

11.

Dimitropoulos ,

Krioukov ,

Riley et al., Revealing the autonomous system taxonomy: The machine learning approach, 2006, available at: arXiv:cs/0604015.

12.

S.N.

Dorogovtsev and

J.F.

Mendes , Evolution of networks, Advances in Physics 51(4) (2002), 1079–1187. doi:10.1080/00018730110112519.

13.

Dshield, available at: http://www.dshield.org/.

14.

Erdos and

Renyi , On random graphs I, Publ. Math. Debrecen 6 (1959), 290–297.

15.

Floyd and

Paxson , Difficulties in simulating the Internet, IEEE/ACM Transactions on Networking (TON) 9(4) (2001), 392–403. doi:10.1109/90.944338.

16.

J.B.

Grizzard ,

Sharma ,

Nunnery ,

B.B.

Kang and

Dagon , Peer-to-peer botnets: Overview and case study, in: Proceedings of the First Conference on First Workshop on Hot Topics in Understanding Botnets – HotBots’07, 2007, pp. 1–8, Article no. 1.

17.

Hagberg ,

Swart and

D.S.

Chult , Exploring network structure, dynamics, and function using networkX, Technical report, Los Alamos National Laboratory (LANL), 2008.

18.

Hawkinson and

Bates , Guidelines for creation, selection, and registration of an autonomous system (AS), 1996.

19.

E.M.

Hutchins ,

M.J.

Cloppert and

R.M.

Amin , Intelligence-driven computer network defense informed by analysis of adversary campaigns and intrusion kill chains, Leading Issues in Information Warfare & Security Research 1 (2011), 80–106.

20.

Katti ,

Krishnamurthy and

Katabi , Collaborating against common enemies, in: ACM IMC, 2005.

21.

Konda and

Kaur , RAPID: Shrinking the congestion-control timescale, in: The 28th Annual IEEE International Conference on Computer Communications – IEEE INFOCOM 2009, IEEE, 2009, pp. 1–9. doi:10.1109/INFCOM.2009.5061900.

22.

Lantz ,

Heller and

McKeown , A network in a laptop: Rapid prototyping for software-defined networks, in: Proceedings of the 9th ACM SIGCOMM Workshop on Hot Topics in Networks – HotNets-IX, ACM, 2010, Article no. 19. doi:10.1145/1868447.1868466.

23.

Li ,

Salour and

Su , A survey of Internet worm detection and containment, IEEE Communications Surveys & Tutorials 10(1) (2008), 20–35. doi:10.1109/COMST.2008.4483668.

24.

Moore ,

Shannon et al., Code-red: A case study on the spread and victims of an Internet worm, in: Proceedings of the 2nd ACM SIGCOMM Workshop on Internet Measurment, ACM, 2002, pp. 273–284. doi:10.1145/637201.637244.

25.

Moore ,

Shannon ,

G.M.

Voelker and

Savage , Internet quarantine: Requirements for containing self-propagating code, in: The 22nd Annual IEEE International Conference on Computer Communications – IEEE INFOCOM 2003, Vol. 3, IEEE, 2003, pp. 1901–1910.

26.

Plot digitizer, available at: http://plotdigitizer.sourceforge.net/.

27.

Provos ,

McNamee ,

Mavrommatis ,

Wang ,

Modadugu et al., The ghost in the browser: Analysis of web-based malware, in: Proceedings of the First Conference on First Workshop on Hot Topics in Understanding Botnets – HotBots’07, ACM, 2007, pp. 1–9, Article no. 4.

28.

G.F.

Riley , The Georgia tech network simulator, in: Proceedings of the ACM SIGCOMM MoMeTools Workshop, ACM, 2003, pp. 5–12. doi:10.1145/944773.944775.

29.

B.D.

Ripley , Stochastic Simulation, Wiley Series in Probability and Statistics, Vol. 316, Wiley, 1987.

30.

L.M.

Rossey ,

R.K.

Cunningham ,

D.J.

Fried ,

J.C.

Rabek ,

R.P.

Lippmann ,

J.W.

Haines and

M.A.

Zissman , LARIAT: Lincoln adaptable real-time information assurance testbed, in: 2002 IEEE Aerospace Conference Proceedings, Vol. 6, IEEE, 2002, pp. 2671–2682. doi:10.1109/AERO.2002.1036140.

31.

Sommers ,

Yegneswaran and

Barford , Recent advances in network intrusion detection system tuning, in: 40th Annual CISS, IEEE, 2006, pp. 1490–1495.

32.

Sonchack and

A.J.

Aviv , Less is more: Host-agent based simulator for large-scale evaluation of security systems, in: Computer Security – ESORICS 2014, Springer, 2014, pp. 365–382.

33.

Sonchack ,

A.J.

Aviv and

J.M.

Smith , Bridging the data gap: Data related challenges in evaluating large scale collaborative security systems, in: 6th Workshop on Cyber Security Experitmentation and Testing, 2013.

34.

Steger and

N.C.

Wormald , Generating random regular graphs quickly, Combinatorics Probability and Computing 8(4) (1999), 377–396. doi:10.1017/S0963548399003867.

35.

Tan ,

Poletto ,

J.V.

Guttag and

M.F.

Kaashoek , Role classification of hosts within enterprise networks based on connection patterns, in: Proceedings of the Annual Conference On Usenix Annual Technical Conference – ATEC’03, ACM, 2003, pp. 15–28.

36.

The caida as relationships dataset, available at: http://www.caida.org/data/as-relationships/.

37.

Wagner and

Plattner , Entropy based worm and anomaly detection in fast IP networks, in: 14th IEEE International Workshops on Enabling Technologies: Infrastructure for Collaborative Enterprise – WETICE’05, IEEE, 2005, pp. 172–177. doi:10.1109/WETICE.2005.35.

38.

M.C.

Weigle ,

Adurthi ,

Hernández-Campos ,

Jeffay and

F.D.

Smith , Tmix: A tool for generating realistic TCP application workloads in ns-2, ACM SIGCOMM Computer Communication Review 36(3) (2006), 65–76. doi:10.1145/1140086.1140094.

39.

Xie ,

Iliofotou ,

Keralapura ,

Faloutsos and

Nucci , Subflow: Towards practical flow-level traffic classification, in: The 31st Annual IEEE International Conference on Computer Communications – IEEE INFOCOM 2012, IEEE, 2012, pp. 2541–2545. doi:10.1109/INFCOM.2012.6195649.

40.

Zhang ,

Porras and

Ullrich , Highly predictive blacklisting, in: Proceedings of the 17th Conference On Security Symposium – SS’08, ACM, 2008, pp. 107–122.

Exploring large scale security system reproducibility with the LESS simulator

Abstract

Keywords

Get full access to this article

References