Signal Detection Theory (SDT) Is Effective for Modeling User Behavior Toward Phishing and Spear-Phishing Attacks

Abstract

Objective:

To examine the utility of equal-variance signal detection theory (EVSDT) for evaluating and understanding human detection of phishing and spear-phishing e-mail scams.

Background:

Although the majority of cybersecurity breaches are due to erroneous responses to deceptive phishing e-mails, it is unclear how best to quantify performance in this context. In particular, it is unclear whether equal variances can safely be assumed in the SDT model, or, relatedly, whether degree of targeting, or threat level, primarily affects mean separation or evidence variability.

Method:

Through an online inbox simulation, the present research found that differences in susceptibility to phishing and spear-phishing e-mails could be carefully quantified with respect to detection accuracy and response bias through the use of an EVSDT framework.

Results:

The results indicated that EVSDT-based point metrics are effective for modeling and measuring phishing susceptibility in the inbox task, without the need for parameter estimation or model comparison involving unequal-variance SDT (UVSDT). Threat level modulated mean separation, with no effects on signal variances.

Conclusion:

These findings support the viability of using EVSDT to initially assess and subsequently monitor training effectiveness for phishing susceptibility, thereby providing measures that are superior to more intuitive metrics, which typically confound an individual’s bias and accuracy. Effects of threat level mapped clearly onto distribution means with no effect on variances, suggesting phishing susceptibility primarily reflects temporally stable discriminative characteristics of observers. Notably, results indicated that people are particularly poor at identifying spear-phishing e-mail threats (demonstrating only 40% accuracy).