Abstract
Introduction:
This is the first study of which we are aware to describe the analytical validation (AV) of clinical grade artificial intelligence (AI) algorithms for a commercially available prostate cancer test performed on hematoxylin and eosin stained specimens that is not dependent on a priori established molecules or a priori semantically meaningful morphology.
Methods:
We adapted AV methods used in molecular diagnostics and clinical pathology to two AI biomarkers used in a clinical test for prostate cancer biopsy specimens. The two algorithms included one algorithm with prognostic performance and a second algorithm predictive for treatment benefit from short-term androgen deprivation therapy (ST-ADT). We assessed analytical accuracy, intra-operator reliability, and inter-operator reliability, and biopsy set completeness reliability on two AI algorithms deployed into a clinical laboratory setting. Analytical accuracy was measured using intraclass correlation coefficient (ICC). Reliability studies were assessed using ICC for the prognostic algorithm and percent agreement for the ST-ADT classification algorithm.
Results:
Analytical accuracy ICC was 0.991 and 0.934 for the prognostic and ST-ADT algorithms, respectively. Intra-operator reliability was 0.981 (ICC) and 100% (percent agreement) for the prognostic and ST-ADT algorithms, respectively. Inter-operator reliability was 0.994 (ICC) and 93.3% (percent agreement) for the prognostic and ST-ADT algorithms, respectively. Biopsy-completeness reliability for one versus three prostate biopsy cores was 0.894 (ICC) and 91.67% (percent agreement) for the prognostic and ST-ADT algorithms respectively. For one versus six cores, reliability was 0.857 (ICC) and 95.00% (percent agreement) for the prognostic and ST-ADT algorithms respectively.
Conclusion:
This study describes a novel approach to AV of AI algorithms in prostate cancer and applies this approach to two algorithms translated for use as a clinical grade AI-based laboratory test, supporting analytical validity of the test.
Get full access to this article
View all access options for this article.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
