Hierarchical continuous-time inhomogeneous hidden Markov model for cancer screening with extensive followup data

Abstract

Continuous-time hidden Markov models are an attractive approach for disease modeling because they are explainable and capable of handling both irregularly sampled, skewed and sparse data arising from real-world medical practice, in particular to screening data with extensive followup. Most applications in this context consider time-homogeneous models due to their relative computational simplicity. However, the time homogeneous assumption is too strong to accurately model the natural history of many diseases including cancer. Moreover, cancer risk across the population is not homogeneous either, since exposure to disease risk factors can vary considerably between individuals. This is important when analyzing longitudinal datasets and different birth cohorts. We model the heterogeneity of disease progression and regression using piece-wise constant intensity functions and model the heterogeneity of risks in the population using a latent mixture structure. Different submodels under the mixture structure employ the same types of Markov states reflecting disease progression and allowing both clinical interpretation and model parsimony. We also consider flexible observational models dealing with model over-dispersion in real data. An efficient, scalable Expectation-Maximization algorithm for inference is proposed with the theoretical guaranteed convergence property. We demonstrate our method’s superior performance compared to other state-of-the-art methods using synthetic data and a real-world cervical cancer screening dataset from the Cancer Registry of Norway. Moreover, we present two model-based risk stratification methods that identify the risk levels of individuals.