Observer agreement in diagnosing and grading urothelial papillary neoplasms

Abstract

A Bayesian belief network (BBN), a diagnostic decision support system, enables the processing of our knowledge of histopathology expressed in descriptive linguistic terms, words and concepts. The aim of this study was to evaluate the contribution of a BBN to the improvement of observer agreement and certainty level in grading urothelial papillary neoplasms.

The study was based on 40 cases of non-invasive urothelial papillary tumours subdivided according to the WHO 1973 classification. There were ten urothelial papillomas (UP), ten grade 1 papillary carcinomas (G1), ten grade 2 papillary carcinomas (G2), and ten grade 3 papillary carcinomas (G3). Five consecutive sessions were held with three observers at a time interval of at least two weeks. Three of these sessions (A, B and D) were based on the morphological evaluation of the specimens with a conventional light microscope only and in remaining twosessions (C and E) a BBN was used in addition to the microscope.

Observer agreement was seen in 60% (RMa), 55% (PC) and 65% (MSt) of cases, respectively, in session A where a synthetic approach to decisionmaking was adopted. The level of subjective certainty was “certain” in 20% of cases (a two tier system was adopted: certain vs less certain). A better observer agreement - 70% (RMa), 68% (PC) and 72% (MSt) of cases - was present in session B where an analytical approach based on the evaluation of a series of morphological features was used. The level of certainty was “certain” in 50% of cases. A further increase in the observer agreement - 85% (RMa), 82% (PC) and 86% (MSt) of cases – with high levels of certainty or belief was seen in session C where a BBN was utilised. A drop in the level of agreement - 60% (RMa), 62% (PC) and 65% (MSt) of cases - and a decrease in certainty were seen in session D where the observers were left free to evaluate morphologically the cases without the constrain of either a synthetic or analytical approach. In session E, agreement increased to 83% (RMa), 81% (PC) and 84% (MSt), respectively, when the BBN was used again.

Conventional morphological evaluation of urothelial papillary neoplasms is affected by observer variability and, in many instances, by diagnostic uncertainty. Improvement in observer agreement and certainty level can be achieved with a BBN used in conjunction with a light microscope.