Meta-conformity approach to reliable classification

Abstract

The conformity framework has recently been proposed for the task of reliable classification. Given a classifier B, the framework allows to obtain p-values of the classifications assigned to individual instances. However, applying the framework is a difficult problem: we need to construct an instance non-conformity function for the classifier B. To avoid constructing such a function we propose a meta-conformity approach.1

¹

This paper is an extended version of [20].

If a conformity-based classifier M is available, the approach is to train M as a meta classifier that predicts the correctness of each classification of the classifier B. In this way the classification p-values of the classifier B are represented by the classification p-values of the classifier M.

The meta-conformity approach can be used for constructing classifiers with predefined generalization performance. Experiments show that the approach results in classifiers that can outperform existing conformity-based classifiers.