In search, exploration, and reconnaissance operations of autonomous ground vehicles, an image recognition capability is needed for specifically classifying targeted objects (relevant classes) and at the same time identifying as unknown (irrelevant and unseen) objects that do not belong to any known classes, as opposed to falsely classifying them in one of the relevant classes. This paper integrates an unsupervised learning feature extraction framework based on the Instance Discrimination method with an Open-Set Low-Shot (IDLS) classifier for creating the desired new capability. Unlabeled images from the vehicle’s operating environment are used for training the feature extractor while a modest number (less than 40) images for each relevant class and unlabeled irrelevant images are used for training the Open-Set Low-Shot (OSLS) classifier in a manner that enables recognition of images unseen during training as irrelevant. The value and the accuracy of the new IDLS approach are demonstrated through a thorough comparison with alternative unsupervised and fully supervised methods.
RussakovskyODengJSuH, et al. Imagenet large scale visual recognition challenge. Int J Comput Vis2015; 115(3): 211–252.
2.
SanakoyeuABautistaMAOmmerB. Deep unsupervised learning of visual similarities. Pattern Recogn2018; 78: 331–343.
3.
BansalA. Improving task-specific representation via 1M unlabelled images without any extra knowledge. arXiv preprint, 2006https://arxiv.org/abs/2006.13919
4.
CaronMBojanowskiPJoulinA, et al. Deep clustering for unsupervised learning of visual features. In: Proceedings of the European conference on computer vision (ECCV), Munich, 8–14 September 2018, pp. 132–149. New York: Springer.
5.
WuZXiongYYuSX, et al. Unsupervised feature learning via nonparametric instance-level discrimination. arXiv preprint, 2018, https://arxiv.org/pdf/1805.01978.pdf
6.
ZhangJLiWOgunbonaP, et al. Recent advances in transfer learning for cross-data set visual recognition: a problem-oriented perspective. ACM Comput Survey2019; 52(1): 1–38.
7.
XieTLiY. A gradient-based algorithm to deceive deep neural networks. In: International conference on neural information processing, Sydney, NSW, Australia, 12–15 December 2019, pp. 57–65. New York: Springer.
8.
ZhangLTongYJiQ. Active image labeling and its application to facial action labeling. In: European conference on computer vision, Marseille, 12–18 October 2008, pp. 706–719. New York: Springer.
9.
ChenTKornblithSNorouziM, et al. A simple framework for contrastive learning of visual representations. In: Proceedings of the 37th International Conference on Machine Learning, Vienna, Austria, PMLR 119, 2020, pp. 1597–1607.
10.
DodgeSKaramL. Understanding how image quality affects deep neural networks. In: 2016 Eighth international conference on quality of multimedia experience (Qomex), Lisbon, 6–8 June 2016, pp. 1–6. New York: IEEE.
11.
KasapisSZhangGSmerekaJ, et al. Using unlabeled data for increasing low-shot classification accuracy of relevant and open-set irrelevant images. arXiv, 2010, https://arxiv.org/abs/2010.00721
12.
JainLPScheirerWJBoultTE. Multiclass open set recognition using probability of inclusion. In: European conference on computer vision, Zurich, 6–12 September 2014, pp. 393–409. New York: Springer.
13.
ScheirerWJ, et al. Toward open set recognition. IEEE Trans Pattern Anal Mach Intell2012; 35(7): 1757–1772.
14.
ScheirerWJJainLPBoultTE. Probability models for open set recognition. IEEE Trans Pattern Anal Mach Intell2014; 36(11): 2317–2324.
LikasAVlassisNVerbeekJJ. The global k-means clustering algorithm. Pattern Recogn2003; 36(2): 451–461.
18.
GirshickRDonahueJDarrellT, et al. Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, Columbus, OH, 23–28 June 2014, pp. 580–587. New York: IEEE.
19.
HadsellRChopraSLeCunY. Dimensionality reduction by learning an invariant mapping. In: 2006 IEEE computer society conference on computer vision and pattern recognition (CVPR’06), New York, 17–22 June 2006, pp. 1735–1742. New York: IEEE.
20.
DosovitskiyASpringenbergJTRiedmillerM, et al. Discriminative unsupervised feature learning with convolutional neural networks. Adv Neural Inform Process Syst2014; 27: 766–774.
21.
BachmanPDevon HjelmRBuchwalterW. Learning representations by maximizing mutual information across views. arXiv preprint, 2019, https://arxiv.org/abs/1906.00910
22.
ReiteAKangasSSteckZ, et al. Unsupervised feature learning in remote sensing. In: ZelinskiMETahaTMHoweJ, et al. (eds)Applications of Machine Learning. Vol. 11139. International Society for Optics and Photonics, 2019, https://spie.org/Publications/Proceedings/Paper/10.1117/12.2529791?SSO=1
DengJ, et al. Imagenet: a large-scale hierarchical image database. In: 2009 IEEE conference on computer vision and pattern recognition, Miami, FL, 20–25 June 2009, pp. 248–255. New York: IEEE.
27.
ZhouBLapedrizaAKhoslaA, et al. Places: a 10 million image database for scene recognition. IEEE Trans Pattern Anal Mach Intell2017; 40(6): 1452–1464.
28.
ManciniMAkataZRicciE, et al. Towards recognizing unseen categories in unseen domains. In: European conference on computer vision, Glasgow, 23–28 August 2020, pp. 466–483. New York: Springer.
KimHYVlahopoulosN. A multi-level optimization algorithm and a ship design application. In: 12th AIAA aviation technology, integration, and operations (ATIO) Conference and 14th AIAA/ISSMO multidisciplinary analysis and optimization conference, Indianapolis, IN, 17–19 September 2012, p.5555. Reston, VA: ARC.
