Evaluation of probabilistic consultation systems by simulation

Abstract

This paper reports on an evaluation of alternative inference models applied in probabilistic consultation systems by simulation. In the first section some general remarks about consultation systems are made in order to set the specific system discussed in a wider context. Section 2 gives a description of the different inference models tested. The next section specifies simula tion experiments which aim at an evaluation of the inference algo rithms. Section 4 outlines the actual experiments and in Section 5 the results of the experiments are presented. The main conclusions supported by the experiments are: the rule value model gives more successful predictions than simpler inference models, that a model applying the complete Bayes' formula performs better than one which instead makes use of complementary probabilities in the denominators, and that the estimates of prediction probabilities in models ignoring the statistical dependence among symptoms tend to be highly biased and cannot be expected to be useful reliability indicators.