Evaluate the effectiveness of training embedded within security warnings to identify phishing webpages.
Background:
More than 20 million malware and phishing warnings are shown to users of Google Safe Browsing every week. Substantial click-through rate is still evident, and a common issue reported is that users lack understanding of the warnings. Nevertheless, each warning provides an opportunity to train users about phishing and how to avoid phishing attacks.
Method:
To test use of phishing-warning instances as opportunities to train users’ phishing webpage detection skills, we conducted an online experiment contrasting the effectiveness of the current Chrome phishing warning with two training-embedded warning interfaces. The experiment consisted of three phases. In Phase 1, participants made login decisions on 10 webpages with the aid of warning. After a distracting task, participants made legitimacy judgments for 10 different login webpages without warnings in Phase 2. To test the long-term effect of the training, participants were invited back a week later to participate in Phase 3, which was conducted similarly as Phase 2.
Results:
Participants differentiated legitimate and fraudulent webpages better than chance. Performance was similar for all interfaces in Phase 1 for which the warning aid was present. However, training-embedded interfaces provided better protection than the Chrome phishing warning on both subsequent phases.
Conclusion:
Embedded training is a complementary strategy to compensate for lack of phishing webpage detection skill when phishing warning is absent.
Application:
Potential applications include development of training-embedded warnings to enable security training at scale.
Al-DaeefM. M.BasirN.SaudiM. M. (2017). Security awareness training: A review. In Proceedings of the World Congress on Engineering (pp. 446–451). London, U.K.
2.
AnandparaV.DingmanA.JakobssonM.LiuD.RoinestadH. (2007). Phishing IQ tests measure fear, not ability. In DietrichS.DhamijaR. (Eds.), Financial cryptography and data security. (Lecture Notes in Computer Science, vol. 4886, pp. 362–366). Berlin: Springer.
3.
AndersonJ. R. (2013). The architecture of cognition. New York, NY: Psychology Press.
4.
Bravo-LilloC.CranorL. F.DownsJ.KomanduriS. (2011). Bridging the gap in computer security warnings: A mental model approach. IEEE Security & Privacy, 9, 18–26.
5.
CanfieldC. I.FischhoffB.DavisA. (2016). Quantifying phishing susceptibility for detection and behavior decisions. Human Factors, 58, 1158–1172.
6.
CaputoD. D.PfleegerS. L.FreemanJ. D.JohnsonM. E. (2014). Going spear phishing: Exploring embedded training and awareness. IEEE Security & Privacy, 12, 28–38.
ChouN.LedesmaR.TeraguchiY.MitchellJ. C. (2004). Client-side defense against web-based identity theft. In Proceedings of the 11th Annual Network and Distributed System Security Symposium. http://crypto.stanford.edu/SpoofGuard/webspoof.pdf
9.
DhamijaR.TygarJ. D. (2005). The battle against phishing: Dynamic security skins. In Proceedings of the 2005 Symposium on Usable Privacy and Security (pp. 77–88). New York, NY: ACM.
10.
DhamijaR.TygarJ. D.HearstM. A. (2006). Why phishing works. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 581–590). New York, NY: ACM.
11.
DodgeR. C.CarverC.FergusonA. J. (2007). Phishing for user security awareness. Computers & Security, 26, 73–80.
12.
DownsJ. S.BarbagalloD.AcquistiA. (2015). Predictors of risky decisions: Improving judgment and decision making based on evidence from phishing attacks. In WilhelmsE. A.ReynaV. F. (Eds.), Neuroeconomics, judgment, and decision making (pp. 239–253). New York, NY: Psychology Press.
13.
EgelmanS.CranorL. F.HongJ. (2008). You’ve been warned: An empirical study of the effectiveness of web browser phishing warnings. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 1065–1074). New York., NY: ACM.
FeltA. P.AinslieA.ReederR. W.ConsolvoS.ThyagarajaS.BettesA.GrimesJ. (2015). Improving SSL warnings: Comprehension and adherence. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (pp. 2893–2902). New York, NY: ACM.
16.
FergusonA. J. (2005). Fostering e-mail security awareness: The West Point carronade. Educause Quarterly, 28, 54–57.
17.
FetteI.SadehN.TomasicA. (2007). Learning to detect phishing emails. In Proceedings of the 16th International Conference on World Wide Web (pp. 649–656). New York, NY: ACM.
18.
FuA. Y.LiuW. Y.DengX. (2006). Detecting phishing web pages with visual similarity assessment based on earth mover’s distance (EMD). IEEE Transactions on Dependable and Secure Computing, 3, 301–311.
19.
HautusM. J. (1995). Corrections for extreme proportions and their biasing effects on estimated values of d′. Behavior Research Methods, Instruments, & Computers, 27, 46–51.
20.
HealyA. F.BourneL. E.Jr. (Eds.) (2012). Training cognition: Optimizing efficiency, durability, and generalizability. New York, NY: Psychology Press.
21.
HerzbergA.GbaraA. (2004). Trustbar: Protecting (even naive) web users from spoofing and phishing attacks. Cryptology ePrint Archive, Report 2004/155. http://eprint.iacr.org/2004/155.
22.
HommelB.MüsselerJ.AscherslebenG.PrinzW. (2001). The theory of event coding (TEC). A framework for perception and action. Behavioral & Brain Sciences, 24, 849–937.
23.
HongJ. (2012). The state of phishing attacks. Communications of the ACM, 55, 74–81.
24.
JagaticT. N.JohnsonN. A.JakobssonM.MenczerF. (2007). Social phishing. Communications of the ACM, 50, 94–100.
25.
KelleyC. M.HongK. W.MayhornC. B.Murphy-HillE. (2012). Something smells phishy: Exploring definitions, consequences, and reactions to phishing. In Proceedings of the 56th Human Factors and Ergonomics Society Annual Meeting (pp. 2108–2112). Santa Monica, CA: Human Factors and Ergonomics Society.
26.
KhonjiM.IraqiY.JonesA. (2013). Phishing detection: A literature survey. IEEE Communications Surveys & Tutorials, 15, 2091–2121.
27.
KumaraguruP.CranshawJ.AcquistiR.CranorL.HongJ.BlairM. A.PhamT. (2009). A real-word evaluation of anti-phishing training (Technical report) Pittsburgh, PA: Carnegie Mellon University.
28.
KumaraguruP.RheeY.AcquistiA.CranorL. F.HongJ.NungeE. (2007). Protecting people from phishing: The design and evaluation of an embedded training email system. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 905–914). New York, NY: ACM.
29.
KumaraguruP.ShengS.AcquistiA.CranorL. F.HongJ. (2010). Teaching Johnny not to fall for phish. ACM Transactions on Internet Technology, 10 (2), Article 7.
30.
LaugheryK. R.WogalterM. S. (2006). Designing effective warnings. Reviews of Human Factors and Ergonomics, 2, 241-271.
31.
LeeY. S.VakochD. A. (1996). Transfer and retention of implicit and explicit learning. British Journal of Psychology, 87, 637–651.
32.
LinE.GreenbergS.TrotterE.MaD.AycockJ. (2011). Does domain highlighting help people identify phishing sites? In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 2075–2084). New York, NY: ACM.
33.
MacLeodC. M. (1991). Half a century of research on the Stroop effect: An integrative review. Psychological Bulletin, 109, 163–203.
34.
OberauerK. (2010). Declarative and procedural working memory: Common principles, common capacity limits?Psychologica Belgica, 50, 3–4.
35.
OrgillG. L.RomneyG. W.BaileyM. G.OrgillP. M. (2004). The urgency for effective user privacy-education to counter social engineering attacks on secure computer systems. In Proceedings of the 5th Conference on Information Technology Education (pp. 177–181). New York, NY: ACM.
36.
ParsonsK.McCormacA.PattinsonM.ButaviciusM.JerramC. (2015). The design of phishing studies: Challenges for researchers, Computers & Security, 52, 194–206.
37.
PerkinsD. N.SalomonG. (1992). Transfer of learning. In PostlethwaiteT. N.HusenT. (Eds.), International encyclopedia of education (2nd ed.; pp. 6452–6457). Oxford, England: Pergamon Press.
PrakashP.KumarM.KompellaR. R.GuptaM. (2010). Phishnet: Predictive blacklisting to detect phishing attacks. In Proceedings of INFOCOM, IEEE (pp. 1–5). Piscataway, NJ: IEEE.
40.
ProctorR. W.ChenJ. (2015). The role of human factors/ergonomics in the science of security: Decision making and action selection in cyberspace. Human Factors, 57, 721–727.
41.
RoedigerH. L.IIIDudaiY.FitzpatrickS. M. (Eds.) (2007). Science of memory: Concepts. New York, NY: Oxford University Press.
42.
SchmidtR. A.BjorkR. A. (1992). New conceptualizations of practice: Common principles in three paradigms suggest new concepts for training. Psychological Science, 3, 207–217.
43.
ShengS.WardmanB.WarnerG.CranorL. F.HongJ.ZhangC (2009, July). An empirical analysis of phishing blacklists. In Proceedings of the 6th Conference on Email and Anti-Spam, CEAS’09. Mountain View, CA.
44.
StoneA. (2007). Natural-language processing for intrusion detection. Computer, 40, 103–105.
45.
WhittakerC.RynerB.NazifM. (2010). Large-scale automatic classification of phishing pages. In Proceedings of the Network and Distributed System Security Symposium, NDSS 2010, San Diego, CA.
46.
WogalterM. S.GodfreyS. S.FontenelleG. A.DesaulniersD. R.RothsteinP. R.LaugheryK. R. (1987). Effectiveness of warnings. Human Factors, 29, 599–612.
47.
WogalterM. S.MayhornC. B. (2008). Trusting the internet: Cues affecting perceived credibility. International Journal of Technology and Human Interaction, 4, 75–93.
48.
WuM.MillerR. C.GarfinkelS. L. (2006). Do security toolbars actually prevent phishing attacks? In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 601–610). New York, NY: ACM.
49.
XiongA.ProctorR. W.YangW.LiN. (2017). Is domain highlighting actually helpful in identifying phishing web pages?Human Factors, 59, 640–660.
50.
YangW.XiongA.ChenJ.ProctorR. W.LiN. (2017). Use of phishing training to improve security warning compliance: Evidence from a field experiment. In Proceedings of the Hot Topics in Science of Security: Symposium and Bootcamp (pp. 52–61). New York, NY: ACM.
51.
ZhangY.HongJ. I.CranorL. F. (2007). Cantina: A content-based approach to detecting phishing web sites. In Proceedings of the 16th International Conference on World Wide Web (pp. 639–648). New York, NY: ACM.
