Optimizing Face Validity and Clinical Relevance of a Mathematical Population Cancer Epidemiology Model Using a Novel Advisory Group Approach

Abstract

Background

Cancer simulation models can answer research and policy questions when prospective evidence is incomplete or not feasible. However, such models require incorporating unmeasureable inputs for which there is often not strong evidence, and model utility is limited if assumptions lack face validity or if the model is not clinically relevant. We systematically incorporated formal advisory input to mitigate these challenges as we developed a microsimulation model of papillary thyroid cancer (PApillary Thyroid CArcinoma Microsimulation model [PATCAM]).

Methods

We used a participatory action research approach incorporating focus group techniques and using principles of bidirectional learning.

Results

We assembled a formal standing advisory group with representation by perspective (medical, patient, and payor), geography, and local practice culture to understand current and historical clinical beliefs and practices about thyroid cancer diagnosis and treatment. The group provided input on critical modeling assumptions and decisions: 1) the role of nodule size in biopsy decisions, 2) trends in provider biopsy behavior, 3) specialty propensity to biopsy, 4) population prevalence of thyroid cancer over time, 5) proportion of malignant tumors showing regression, and 6) cancer epidemiology and diagnostic practices by sex and age. Advisory group questions and concerns about model development will inform future research questions and strategies to communicate and disseminate model results.

Conclusions

We successfully used our advisory group to provide critical inputs on unmeasurable assumptions, increasing the face validity of our model. The use of a standing advisory group improved model transparency and contributed to future research plans and dissemination of model results so they can have maximum impact when guiding clinical decisions and policy.

Highlights

• Unfamiliarity with simulation modeling poses a threat to its acceptability and adoption. The effectiveness of these models is contingent on end-users’ willingness to accept and adopt model results. The effectiveness of the models is further limited if they lack face validity to potential users or do not have clinical relevance.

• Several approaches to overcoming validity challenges have been advanced, such as collaborative modeling, which involves developing multiple models independently using common data sources. However, when only a single model exists, another approach is needed. We used an Advisory Group and “participatory modeling,” which has been used in other settings but has not been previously reported in cancer modeling. We describe the methods used for and results of incorporating a formal advisory group into the development of a cancer microsimulation model.

• The use of a formal, standing advisory group (as opposed to one-off focus groups or interviews) strengthened our model by rigorously vetting modeling assumptions and model inputs with subject matter experts. The formal, ongoing structure promoted transparency. The group education in cancer modeling improved participant ability to provide useful input and may help with dissemination. The advisory group also provided critical feedback about how to effectively communicate model results and informed planned future research questions.