Abstract
We are gratified and honored that, in the letter from Elston et al. 1 concerning our article on the predictive relationship between mitotic index and survival among canine patients with cutaneous mast cell tumors, 2 the authors went to a considerable and thorough effort to validate our findings using 57 dogs from Brazil.
The key area of circumspection in their letter was how to use information about mitotic index (MI) to predict survival. There are a plethora of ways to do this in statistical models, including parametrically (e.g., linear terms, fractional polynomials) and nonparametrically (e.g., categorization, smoothing splines). Through exploratory data analysis, we determined that a model that reasonably described survival in our group of dogs seen at the University of California, Davis (UC Davis) partitioned them into 2 relatively homogeneous (with respect to survival) groups: those with MI ≤ 5, and those with indices > 5. Elston et al., in their letter, performed a validation study and determined that an alternative partition (MI = 0, 1 ≤ MI ≤ 7, MI ≥ 8) comported better with the survival experience of their patients. Figure 1 shows a Kaplan-Meier survival plot of their proposed categorization using the same UC Davis population as in Romansik et al. 2 The figure indicates that the survival probabilities in the 2 lowest MI groups were similar; a log-rank test comparing these 2 groups yielded a P value of .60.
What would explain why the 2 different categorization options provide conflicting results in the 2 populations? One clue is provided in the Elston et al. letter: “When applying a cut-point at MI = 5, our results were very similar (Figs. 1, 2) to those reported by Romansik et al.” This is not quite correct: in the Elston et al. total patient population, the median survival in dogs with an MI > 5 was 8 months, compared with a median survival of 2 months in dogs with an MI > 5 at UC Davis. This underscores the fallibility of classification schemes in clinical veterinary medicine: they may work well in a study population and its corresponding reference population but may not perform nearly as well in a target population of inherently different animals or where measurement standards may not necessarily be completely concordant. The latter can be influenced by how different pathologists measure, count, and ultimately determine MI, and whether such determinations are so precise as to be perfectly replicable by others.
Another clue as to why our results did not completely correspond with those of Elston et al. lies in their statement: “The R 2 value of Cox regression applying the MI stratification of Romansik et al. was 0.05588 and therefore lower than that of the stratification proposed in this study (0.08596).” Regardless of the relatively minor difference between these 2 coefficients of determination, what is remarkable is how low they both are: MI explains less than 10% of the variability in Elston et al.'s Cox regression analysis. This underscores another fallibility of relying heavily on statistical models such as Kaplan-Meier survival function estimation or even Cox regression with sparse data (small study sizes): these models are not sensitive enough to distinguish or accurately quantify other factors that also affect patient outcome. Such studies are then relegated to determining average effects across potentially heterogeneous patient populations even within a community, much less between countries. Thus, patient characteristics such as age, breed, sex, and owner propensity for diagnosis (including biopsy) would be expected to vary, perhaps substantially, between institutions such as ours and Elston et al's.
Therefore, crude analyses that are incognizant of confounding factors such as those noted above, as well as potentially other unmeasurable ones, must be interpreted with caution in light of their limitations. In addition to the limitations discussed above, others can be found on pages 339–340 in Romansik et al. 2 We are heartened and gratified that Elston et al. have at least qualitatively affirmed our finding that MI is predictive of survival in cases of canine mast cell tumor and shared this notable information with Veterinary Pathology. However, no cutoff for a single explanatory variable should be taken too literally in the face of the myriad uncontrolled determinants of health outcomes and measurement accuracy within and between populations.