Sage Journals: Discover world-class research

Abstract

Often the performance of a classification system is reported in terms of classification accuracy. In an environment with objects unknown to the classification system, classification accuracy may provide unrealistic expectations. In this paper we contrast classification and label accuracy in a challenging classification environment. A statistical-based method is used to identify records not represented in the template library used by the classifier and three different information theory-based methods are used to identify label records likely to be misidentified. These methods are applied to an automatic target recognition (ATR) problem, using features drawn from high-range resolution profiles generated from synthetic aperture radar (SAR) data. An optimization framework is used to select the optimal classification system choices based on the measurement of evaluation. The choices selected by the framework when classification or label accuracy is the optimization focus are contrasted.

Keywords

combat identification synthetic aperture radar entropy label accuracy classification accuracy non-declaration out of library

Get full access to this article

View all access options for this article.

References

Laine T. , Bauer KW A mathematical framework to optimize ATR systems with non-declarations and sensor fusion. Comput Oper Res 2008; 35(6): 1789-1812.

Albrecht TW , Bauer KW Optimization framework for a multiple classifier system with non-registered targets. J Defense Model Simulat 2008; 5(1): 61-81.

Chow CK On optimum recognition error and reject tradeoff. IEEE Trans Inform Theor 1970; IT-16: 41-46.

Fumera G. , Roli F. , and Giorgio G. Reject option with multiple thresholds. Pattern Recogn 2000; 33: 2009-2101.

Devijver PA , Kittler J. Pattern recognition - a statistical approach . London: Prentice-Hall International , 1982.

Haspert KJ ‘Optimum ID Sensor Fusion for Multiple Target Types’, Institute for Defense Analysis (IDA) Document D-2451, Virginia , 2000.

Kullback S. , Leibler RA On information and sufficiency. Ann. Math. Stat. 1951; 22: 78-86.

Williams R. , Gross D. , Palomino A. , Westerkamp J. , and Wardell D. 1D HRR data analysis and ATR assessment. In: Proceedings of the SPIE. 1998, Vol. 3370, pp.588-599.

Williams R. , Westerkamp J. , Gross D. , and Palomino A. Automatic target recognition of time critical moving targets using1D high range resolution (HRR) radar. IEEE Aero Electron Syst Mag 2000; 15(4): 37-43.

10.

Mossing JC , Ross TD , and Bradley J. Evaluation of SAR ATR algorithm performance sensitivity to MSTAR extended operating conditions. In: Proceedings of the SPIE. 1998, Vol. 3370, pp.554-565.

11.

Hughes GF On the mean accuracy of statistical pattern recognizers. IEEE Trans Inform Theor 1968; IT-14(1): 55-63.

12.

Fu KS , Landgrebe DA , and Philips TL Information processing of remotely sensed agricultural data. Proc . IEEE 1969; 57(4): 639-653.

13.

Shahshahani BM , Landgrebe DA The effect of unlabeled samples in reducing the small sample size problem and mitigating the Hughes phenomenon . IEEE Trans Geosci Rem Sens 1994; 32(5): 1087-1095.

14.

Perrone MP , Cooper LN When networks disagree: ensemble methods for hybrid neural networks . In: Neural Networks for Speech and Image Processing. London: Chapman & Hall 1993, pp.126-142.

15.

Liao X. , Runkle P. , and Carin L. Identification of ground targets from sequential HRR radar signatures . In: Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing. 2001, Vol. 5, pp.2897-2900.

16.

Ross TD , Mossing JC The MSTAR evaluation methodology. In: Proceedings of the SPIE . 1999, Vol. 3721, pp.705-713.

17.

Zelnio E. , Ross T. , and Bryant M. A demonstration of the confuser and likelihood modeling benefits for target detection in SAR imagery. In: Proceedings of the SPIE . 2005, Vol. 5808, pp.395-406.

18.

Ramamoorthy L. , Casasent D. Classification and rejection of MSTAR data . In: Proceedings of the SPIE. 2005, Vol. 5437, p.265-276.

19.

Friend M. ‘Combat Identification with Synthetic Aperture Radar, Out-of-Library Identification, and Non-Declarations’, PhD Thesis, Air Force Institute of Technology, Wright-Patterson AFB OH, 2007, AFIT/DS/ENS/07-04.

20.

Boltzmann EL Lectures on gas theory. Am J Phys 1965; 33: 974-975.

21.

Shannon CE A mathematical theory of communication. Bell Syst Tech J 1948; 27: 379-423.

22.

Chang K. , Lu Y. High resolution polarimetric SAR target classification with neural network . In: Proceeding of the 1995 IEEE International Conference on Fuzzy Systems. 1995, Vol. 3, p.1681-1688.

23.

Deco G. , Obradovic D. An information-theoretic approach to neural computing. New York: Springer, 1996.

24.

DeVore MD , O’Sullivan JA , Anand S. , and Schmid NA Probabilistic approach to model extraction from training data. In: Proceedings of the SPIE. 2001, Vol. 4382, pp.358-366.

25.

Wilson JN Landmine discrimination using the Kullback-Leibler distance. In: Proceedings of the SPIE. 2007, Vol. 6553.

26.

Cover TM , Thomas JA Elements of information theory. New York: Wiley-Interscience, 1991.

An Entropy-based Scheme for Automatic Target Recognition

Abstract

Keywords

Get full access to this article

References