X-ray imaging is a crucial and ubiquitous tool for detecting threats to transport security, but interpretation of the images presents a logistical bottleneck. Recent advances in Deep Learning image classification offer hope of improving throughput through automation. However, Deep Learning methods require large quantities of labelled training data. While photographic data is cheap and plentiful, comparable training sets are seldom available for the X-ray domain.
OBJECTIVE:
To determine whether and to what extent it is feasible to exploit the availability of photo data to supplement the training of X-ray threat detectors.
METHODS:
A new dataset was collected, consisting of 1901 matched pairs of photo & X-ray images of 501 common objects. Of these, 258 pairs were of 69 objects considered threats in the context of aviation. This data was used to test a variety of transfer learning approaches. A simple model of threat cue availability was developed to understand the limits of this transferability.
RESULTS:
Appearance features learned from photos provide a useful basis for training classifiers. Some transfer from the photo to the X-ray domain is possible as ∼40% of danger cues are shared between the modalities, but the effectiveness of this transfer is limited since ∼60% of cues are not.
CONCLUSIONS:
Transfer learning is beneficial when X-ray data is very scarce—of the order of tens of training images in our experiments—but provides no significant benefit when hundreds or thousands of X-ray images are available.
AkçayS.,
KundegorskiM.E.,
DevereuxM. and
BreckonT.P.,
Transfer learning using convolutional neural networks for object classification within x-ray baggage security imagery, IEEE International Conference on Image Processing (2016), 1057–1061.
2.
AkcayS.,
KundegorskiM.E.,
WillcocksC.G. and
BreckonT.P.,
Using Deep Convolutional Neural Network Architectures for Object Classification and Detection Within X-Ray Baggage Security Imagery, IEEE Transactions on Information Forensics and Security13 (2018), 2203–2215.
3.
BiggsA.T. and MitroffS.R., Improving the Efficacy of Security Screening Tasks: A Review of Visual Search Challenges and Ways to Mitigate Their Adverse Effects, Applied Cognitive Psychology29 (2014), 142–148.
4.
BradleyA.P., The use of the area under the ROC curve in the evaluation of machine learning algorithms, Pattern Recognition30 (1997), 1145–1159.
5.
CaldwellM.,
RansleyM.,
RogersT.W. and
GriffinL.D., Transferring X-ray based automated threat detection between scanners with different energies and resolution, in: Counterterrorism, Crime Fighting, Forensics, and Surveillance,
BoumaH.,
Carlysle-DaviesF.,
StokesR.J. and
YitzhakyY., eds., SPIE, 2017, pp. 104410F1–10.
6.
CholletF.,
Xception: Deep Learning with Depthwise Separable Convolutions, arXiv.org (2016), 1–8.
7.
ChristopherM., BelghithA., BowdC., ProudfootJ.A., GoldbaumM.H., WeinrebR.N., GirkinC.A., LiebmannJ.M. and
ZangwillL.M.,
Performance of Deep Learning Architectures and Transfer Learning for Detecting Glaucomatous Optic Neuropathy in Fundus Photographs,, Scientific Reports8 (2018), 166851–13.
8.
CuiG.-B.,
ZhaoD. and
WangW., Glioma Grading on Conventional MR Images: A Deep Learning Study With Transfer Learning, Frontiers in Neuroscience12 (2018), 1–10.
9.
CutlerV. and
PaddockS.,
Use of threat image projection (TIP) to enhance security performance, 43rd Annual 2009 International Carnahan Conference on Security Technology (2009), 46–51.
10.
DengJ.,
DongW.,
SocherR.,
LiL.-J.,
LiK. and
Fei-FeiL., ImageNet: A large-scale hierarchical image database, in: IEEE Conference on Computer Vision and Pattern Recognition, IEEE, 2009, pp. 248–255.
11.
GaleaC. and
FarrugiaR.A.,
Matching Software-Generated Sketches to Face Photographs With a Very Deep CNN, Morphed Faces, and Transfer Learning, IEEE Transactions on Information Forensics and Security13 (2018), 1421–1431.
12.
HeK.,
ZhangX.,
RenS. and
SunJ.,
Deep Residual Learning for Image Recognition, arXiv.org (2015), 1–12.
JaccardN.,
RogersT.W.,
MortonE.J. and
GriffinL.D., Tackling the x-ray cargo inspection challenge using machine learning, in: SPIE Defense + Security,
AshokA.,
NeifeldM.A. and
GehmM.E., eds., SPIE, 2016, pp. 98470N1–13.
15.
JaccardN.,
RogersT.W.,
MortonE.J. and
GriffinL.D., Detection of concealed cars in complex cargo X-ray imagery using deep learning, Journal of X-ray Science and Technology25 (2017), 323–339.
16.
KingdomF.A.A. and
PrinsN., Psychophysics: A Practical Introduction, Academic Press, San Diego, second edition, 2016.
17.
KrizhevskyA.,
SutskeverI. and
HintonG.E.,
Imagenet classification with deep convolutional neural networks, Advances in Neural Information Processing Systems25 (2012), 1097–1105.
18.
KuznetsovaA., RomH., AlldrinN., UijlingsJ., KrasinI., Pont-TusetJ., KamaliS., PopovS., MallociM., DuerigT. and
FerrariV.,
The Open Images Dataset V4: Unified image classification, object detection, and visual relationship detection at scale, arXiv.org (2018), 1–20.
19.
LagunasM. and
GarcesE.,
Transfer Learning for Illustration Classification, arXiv.org (2018), 1–9.
20.
MeryD., RiffoV., ZscherpelU., MondragónG., LilloI., ZuccarI., LobelH. and
CarrascoM.,
GDXray: The database of X-ray images for nondestructive testing, Journal of Nondestructive Evaluation34 (2015), 1–12.
21.
MeryD., SvecE., AriasM., RiffoV., SaavedraJ.M. and
BanerjeeS.,
Modern Computer Vision Techniques for X-Ray Testing in Baggage Inspection, IEEE Transactions on Systems, Man, and Cybernetics: Systems47 (2017), 682–692.
22.
RainaR.,
NgA.Y., and
KollerD., Constructing informative priors using transfer learning, in: ICML ’06: Proceedings of the 23rd International Conference on Machine Learning, ACM Press, New York, USA, 2006, pp. 713–720.
23.
RazavianA.S.,
AzizpourH.,
SullivanJ. and
CarlssonS., CNN Features off-the-shelf: An Astounding Baseline for Recognition, in: Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops, IEEE, 2016, pp. 512–519.
24.
RogersT.W.,
JaccardN. and
GriffinL.D., A deep learning framework for the automated inspection of complex dual-energy x-ray cargo imagery, SPIE Defense + Commercial Sensing (2017), 101870L1–12.
25.
RussakovskyO., DengJ., SuH., KrauseJ., SatheeshS., MaS., HuangZ., KarpathyA., KhoslaA., BernsteinM., BergA.C. and
Fei-FeiL.,
ImageNet Large Scale Visual Recognition Challenge, International Journal of Computer Vision115 (2015), 211–252.
26.
SandlerM., HowardA., ZhuM., ZhmoginovA. and
ChenL.-C.,
MobileNetV2: Inverted Residuals and Linear Bottlenecks, arXiv.org (2018), 1–14.
27.
SzegedyC., LiuW., JiaY., SermanetP., ReedS., AnguelovD., ErhanD., VanhouckeV.
and
RabinovichA.,
Going Deeper with Convolutions, arXiv.org (2014), 1–9.
28.
SzegedyC., VanhouckeV., IoffeS., ShlensJ.
and
WojnaZ.,
Rethinking the Inception Architecture for Computer Vision, arXiv.org (2015), 1–10.
29.
ThrunS.,
Is Learning The n-th Thing Any Easier Than Learning The First?Advances in Neural Information Processing Systems (1996), 640–646.
30.
YosinskiJ., CluneJ., BengioY.
and
LipsonH., How transferable are features in deep neural networks?arXiv.org (2014), 1–9.
31.
ZophB.,
VasudevanV.,
ShlensJ. and
LeQ.V., Learning Transferable Architectures for Scalable Image Recognition, in: IEEE/CVF Conference on Computer Vision and Pattern Recognition, IEEE, 2018, pp. 8697–8710.
