British Thyroid Association 2014 classification ultrasound scoring of thyroid nodules in predicting malignancy: Diagnostic performance and inter-observer agreement

Abstract

Objectives

To assess the inter-observer agreement amongst five observers of differing levels of expertise in applying the British Thyroid Association (2014) guidelines for ultrasound scoring of thyroid nodules (BTA-U score) in the management of thyroid cancer, and to assess the U-score diagnostic performance in predicting malignancy.

Method

A total of 73 consecutive patients were included over a two-year period (July 2012 to July 2014), after referral to a tertiary head and neck oncology centre for ultrasound plus fine needle aspiration and cytology. Our five observers retrospectively and independently reviewed static ultrasound images on PACS and scored the thyroid nodules according to BTA-U classification. The observers were blinded to each other’s scoring, cytology and histology results. Either the Kappa-statistic or intra-class correlation was used to assess the level of inter-observer agreement, plus agreement between the radiological and cytological diagnoses. The diagnostic performance of U-scoring for predicting final histological diagnosis was assessed with sensitivity, specificity, positive and negative predictive values.

Results

A Kappa-value of 0.73 (95% CI: 0.68–0.77) confirmed substantial inter-observer agreement amongst the five observers. All 17 histology confirmed malignant nodules were correctly classified as potentially malignant by all observers. The sensitivity and negative predictive value of BTA-U score in detecting and predicting malignancy were 100%, whereas the specificity and positive predictive values were 34% and 32%, respectively.

Conclusions

There is good inter-observer agreement in using the BTA-U score amongst different observers at differing levels of expertise. Adhering to BTA-U scoring can potentially achieve 100% sensitivity in selecting malignant nodules for sampling.