DeepReturn: A deep neural network can learn how to detect previously-unseen ROP payloads without using any heuristics

Abstract

Return-oriented programming (ROP) is a code reuse attack that chains short snippets of existing code to perform arbitrary operations on target machines. Existing detection methods against ROP exhibit unsatisfactory detection accuracy and/or have high runtime overhead.

In this paper, we present DeepReturn, which innovatively combines address space layout guided disassembly and deep neural networks to detect ROP payloads. The disassembler treats application input data as code pointers and aims to find any potential gadget chains, which are then classified by a deep neural network as benign or malicious. Our experiments show that DeepReturn has high detection rate (99.3%) and a very low false positive rate (0.01%). DeepReturn successfully detects all of the 100 real-world ROP exploits that are collected in-the-wild, created manually or created by ROP exploit generation tools. DeepReturn is non-intrusive and does not incur any runtime overhead to the protected program.

Keywords

Return-oriented programming intrusion detection system disassembly convolutional neural network

Get full access to this article

View all access options for this article.

References

Abadi,

Budiu,

Ú.

Erlingsson and

Ligatti, Control-flow integrity, in: ACM Conference on Computer and Communications Security (CCS ’05), 2005.

Akritidis,

Cadar,

Raiciu,

Costa and

Castro, Preventing memory error exploits with WIT, in: IEEE Symposium on Security and Privacy (Oakland ’08), 2008.

Bittau,

Belay,

Mashtizadeh,

Mazieres and

Boneh, Hacking blind, in: IEEE Symposium on Security and Privacy (Oakland ’14), 2014.

Bletsch,

Jiang and

Freeh, Mitigating code-reuse attacks with control-flow locking, in: Annual Computer Security Applications Conference (ACSAC ’11), 2011.

Bletsch,

Jiang,

V.W.

Freeh and

Liang, Jump-oriented programming: A new class of code-reuse attack, in: ACM Symposium on Information, Computer and Communications Security (ASIACCS ’11), 2011.

Böttinger,

Godefroid and

Singh, Deep reinforcement fuzzing, arXiv preprint arXiv:1801.04589 (2018).

Burow,

S.A.

Carr,

Nash,

Larsen,

Franz,

Brunthaler and

Payer, Control-flow integrity: Precision, security, and performance, ACM Computing Surveys (CSUR) 50(1) (2017), 16. doi:10.1145/3054924.

Carlini and

Wagner, ROP is still dangerous: Breaking modern defenses, in: USENIX Security Symposium (Security ’14), 2014.

Checkoway,

Davi,

Dmitrienko,

A.-R.

Sadeghi,

Shacham and

Winandy, Return-oriented programming without returns, in: ACM Conference on Computer and Communications Security (CCS ’10), 2010.

10.

Chen,

Xiao,

Shen,

Yin,

Mao and

Xie, DROP: Detecting return-oriented programming malicious code, in: International Conference on Information Systems Security (ICISS ’09), 2009.

11.

Cheng,

Zhou,

Yu,

Ding and

R.H.

Deng, ROPecker: A generic and practical approach for defending against ROP attacks, in: Proceedings of the 21th Annual Network and Distributed System Security Symposium (NDSS’14), 2014.

12.

Chollet

et al., Keras: Deep learning library for theano and tensorflow, URL: https://keras.io/ 7(8) (2015).

13.

D.C.

Ciresan,

Meier,

Masci,

L.M.

Gambardella and

Schmidhuber, Flexible, high performance convolutional neural networks for image classification, in: IJCAI Proceedings – International Joint Conference on Artificial Intelligence, Vol. 22, 2011, p. 1237, Barcelona, Spain.

14.

Davi,

Dmitrienko,

Egele,

Fischer,

Holz,

Hund,

Nürnberger and

A.-R.

Sadeghi, MoCFI: A framework to mitigate control-flow attacks on smartphones,, in: Annual Network and Distributed System Security Symposium (NDSS’12), 2012.

15.

Davi,

A.-R.

Sadeghi,

Lehmann and

Monrose, Stitching the gadgets: On the ineffectiveness of coarse-grained control-flow integrity protection, in: USENIX Security Symposium (Security ’14), 2014.

16.

Deng,

Dong,

Socher,

L.-J.

Li,

Li and

Fei-Fei, ImageNet: A large-scale hierarchical image database, in: CVPR09, 2009.

17.

Du,

Li,

Zheng and

Srikumar, Deeplog: Anomaly detection and diagnosis from system logs through deep learning, in: Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, 2017, pp. 1285–1298, ACM. doi:10.1145/3133956.3134015.

18.

Elsabagh,

Barbará,

Fleck and

Stavrou, Detecting ROP with statistical learning of program characteristics, in: Proceedings of the Seventh ACM on Conference on Data and Application Security and Privacy, 2017, pp. 219–226, ACM. doi:10.1145/3029806.3029812.

19.

Follner,

Bartel,

Peng,

Y.-C.

Chang,

Ispoglou,

Payer and

Bodden, PSHAPE: Automatically combining gadgets for arbitrary method execution, in: Security and Trust Management: 12th International Workshop, STM 2016, Heraklion, Crete, Greece, September 26–27, 2016, Proceedings, Springer International Publishing, Cham, 2016, pp. 212–228. doi:10.1007/978-3-319-46598-2_15.

20.

Ginsparg, arXiv.org e-Print archive, https://arxiv.org/, Cornell University, 1991.

21.

Göktaşs,

Athanasopoulos,

Polychronakis,

Bos and

Portokalidis, Size does matter: Why using gadget-chain length to prevent code-reuse attacks is hard, in: USENIX Security Symposium (Security’14), 2014.

22.

Goodfellow,

Bengio,

Courville and

Bengio, Deep Learning, Vol. 1, MIT Press, Cambridge, 2016.

23.

Hochreiter and

Schmidhuber, Long short-term memory, Neural computation 9(8) (1997), 1735–1780. doi:10.1162/neco.1997.9.8.1735.

24.

C.-W.

Hsu,

C.-C.

Chang,

C.-J.

Lin et al., A practical guide to support vector classification (2003).

25.

Ioffe and

Szegedy, Batch normalization: Accelerating deep network training by reducing internal covariate shift, arXiv preprint arXiv:1502.03167 (2015).

26.

Kim, Convolutional neural networks for sentence classification, arXiv preprint arXiv:1408.5882 (2014).

27.

Lai,

Liu,

He and

Zhao, How to generate a good word embedding, IEEE Intelligent Systems 31(6) (2016), 5–14. doi:10.1109/MIS.2016.45.

28.

Leek, Don’t use deep learning your data isn’t that big, https://simplystatistics.org/2017/05/31/deeplearning-vs-leekasso/, 2017.

29.

L.D.

Long , Analysis of nginx 1.3.9/1.4.0 stack buffer overflow and x64 exploitation (CVE-2013-2028), http://www.vnsecurity.net/research/2013/05/21/analysis-of-nginx-cve-2013-2028.html, VNSECURITY, 2013.

30.

Lu,

Zou,

Wen and

Gao, Packed, printable, and polymorphic return-oriented programming, in: International Workshop on Recent Advances in Intrusion Detection, 2011, pp. 101–120, Springer. doi:10.1007/978-3-642-23644-0_6.

31.

A.J.

Mashtizadeh,

Bittau,

Boneh and

Mazières, CCFI: Cryptographically enforced control flow integrity, in: Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security, CCS ’15, ACM, New York, NY, USA, 2015, pp. 941–951. ISBN 978-1-4503-3832-5. doi:10.1145/2810103.2813676.

32.

Microsoft, A detailed description of the Data Execution Prevention (DEP) feature in Windows XP Service Pack 2, 2008, http://support.microsoft.com/kb/875352.

33.

Molnar, Exec Shield, 2003, http://people.redhat.com/mingo/exec-shield/.

34.

Mountrakis,

Im and

Ogole, Support vector machines in remote sensing: A review, ISPRS Journal of Photogrammetry and Remote Sensing 66(3) (2011), 247–259. doi:10.1016/j.isprsjprs.2010.11.001.

35.

negux, Freefloat FTP Server 1.0 - DEP bypass with ROP, https://www.exploit-db.com/exploits/24944/, 2013.

36.

Nicholas,

Antonio,

Mathias,

David and

R.G.

Thomas, Control-flow bending: On the effectiveness of control-flow integrity, in: USENIX Security Symposium (Security’15), 2015.

37.

pakt, pakt/ropc: A Turing complete ROP compiler, https://github.com/pakt/ropc, GitHub, 2012.

38.

Pappas,

Polychronakis and

A.D.

Keromytis, Transparent ROP exploit mitigation using indirect branch tracing, in: Proceedings of the 22nd USENIX Security Symposium (Security’13), 2013.

39.

Payer,

Barresi and

T.R.

Gross, Fine-grained control-flow integrity through binary hardening.

40.

Pfaff,

Hack and

Hammer, Learning how to prevent return-oriented programming efficiently, in: International Symposium on Engineering Secure Software and Systems, 2015, pp. 68–85, Springer. doi:10.1007/978-3-319-15618-7_6.

41.

Polychronakis and

A.D.

Keromytis, ROP payload detection using speculative code execution, in: Malicious and Unwanted Software (MALWARE), 2011 6th International Conference on, 2011, pp. 58–65, IEEE. doi:10.1109/MALWARE.2011.6112327.

42.

Qian, On the momentum term in gradient descent learning algorithms, Neural networks 12(1) (1999), 145–151. doi:10.1016/S0893-6080(98)00116-6.

43.

N.A.

Quynh, Capstone – the ultimate disassembler, https://github.com/aquynh/capstone, GitHub, 2013.

44.

N.A.

Quynh, Unicorn – the ultimate CPU emulator, https://www.unicorn-engine.org/, Unicorn, 2015.

45.

Rudd,

Skowyra,

Bigelow,

Dedhia,

Hobson,

Crane,

Liebchen,

Larsen,

Davi,

Franz et al., Address oblivious code reuse: On the effectiveness of leakage resilient diversity, in: NDSS, 2017.

46.

Sacco, Crashmail 1.6 – stack-based buffer overflow (ROP), https://www.exploit-db.com/exploits/44331/, 2018.

47.

Sacco, PMS 0.42 – local stack-based overflow (ROP), https://www.exploit-db.com/exploits/44426/, 2018.

48.

Salwan, ROPgadget, https://github.com/JonathanSalwan/ROPgadget, GitHub, 2015.

49.

Schirra, Ropper – rop gadget finder and binary information tool, https://github.com/sashs/Ropper, GitHub, 2016.

50.

E.J.

Schwartz,

Avgerinos and

Brumley, Q: Exploit hardening made easy, in: USENIX Conference on Security (Security ’11), 2011.

51.

Seibert,

Okhravi and

Söderström, Information leaks without memory disclosures: Remote side channel attacks on diversified code, in: Proceedings of the 2014 ACM SIGSAC Conference on Computer and Communications Security, 2014, pp. 54–65, ACM.

52.

Shacham, The geometry of innocent flesh on the bone: Return-into-libc without function calls (on the x86), in: ACM Conference on Computer and Communications Security (CCS’07), 2007.

53.

Shacham, The geometry of innocent flesh on the bone: Return-into-libc without function calls (on the x86), in: ACM Conference on Computer and Communications Security (CCS ’07), 2007.

54.

K.Z.

Snow,

Monrose,

Davi,

Dmitrienko,

Liebchen and

A.-R.

Sadeghi, Just-in-time code reuse: On the effectiveness of fine-grained address space layout randomization, in: IEEE Symposium on Security and Privacy (Oakland ’13), 2013, pp. 574–588.

55.

K.Z.

Snow,

Rogowski,

Werner,

Koo,

Monrose and

Polychronakis, Return to the zombie gadgets: Undermining destructive code reads via code inference attacks, in: 2016 IEEE Symposium on Security and Privacy (SP), 2016, pp. 954–968, IEEE. doi:10.1109/SP.2016.61.

56.

Song,

Yin,

Liu and

Song, DeepMem: Learning graph neural network models for fast and robust memory forensic analysis, in: Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security, 2018, pp. 606–618, ACM. doi:10.1145/3243734.3243813.

57.

Souchet, rp++, https://github.com/0vercl0k/rp, GitHub, 2012.

58.

Srivastava,

G.E.

Hinton,

Krizhevsky,

Sutskever and

Salakhutdinov, Dropout: A simple way to prevent neural networks from overfitting., Journal of Machine Learning Research 15(1) (2014), 1929–1958.

59.

Stancill,

K.Z.

Snow,

Otterness,

Monrose,

Davi and

A.-R.

Sadeghi, Check my profile: Leveraging static analysis for fast and accurate detection of ROP gadgets, in: International Workshop on Recent Advances in Intrusion Detection, 2013, pp. 62–81, Springer.

60.

Strackx,

Younan,

Philippaerts,

Piessens,

Lachmund and

Walter, Breaking the memory secrecy assumption, in: Second European Workshop on System Security, 2009.

61.

Sue, Ropper – rop gadget finder and binary information tool, https://github.com/oblivia-simplex/roper, GitHub, 2017.

62.

R.M.

Summers, Progress in fully automated abdominal CT interpretation, American Journal of Roentgenology 207(1) (2016), 67–79. doi:10.2214/AJR.15.15996.

63.

Tanaka and

Goto, n-ROPdetector: Proposal of a method to detect the ROP attack code on the network, in: Proceedings of the 2014 Workshop on Cyber Security Analytics, Intelligence and Automation, 2014, pp. 33–36, ACM.

64.

Wang,

C.-C.

Pan,

Liu and

Zhu, Sigfree: A signature-free buffer overflow attack blocker, IEEE transactions on dependable and secure computing 7(1) (2010), 65–79. doi:10.1109/TDSC.2008.30.

65.

Wang and

Jiang, HyperSafe: A lightweight approach to provide lifetime hypervisor control-flow integrity, in: IEEE Symposium on Security and Privacy (Oakland ’10), 2010.

66.

Yao,

Torabi,

Cho,

Ballas,

Pal,

Larochelle and

Courville, Describing videos by exploiting temporal structure, in: Proceedings of the IEEE International Conference on Computer Vision, 2015, pp. 4507–4515.

67.

ZadYree, HT Editor 2.0.20 – local buffer overflow (ROP), https://www.exploit-db.com/exploits/22683/, 2012.

68.

Zhang and

Sekar, Control flow integrity for COTS binaries, in: USENIX Conference on Security (Security ’13), 2013.