On MAPE-R as a measure of cross-sectional estimation and forecast accuracy

Abstract

Both demographers and economists evaluate the accuracy of their respective forecasts with measures like mean square error, root mean square error, mean absolute percent error, and mean algebraic percent error. However, demographers tend to approach the issue of forecasting very differently than do economists. Two of the distinctive features of the demographic tradition are the use of the cohort-component method (instead of time-series models) and an emphasis on cross-sectional forecasts (instead of forecasts aggregated over time). From the perspective of this demographic tradition, we examine "MAPE-R" (Mean Absolute Percent Error-Rescaled), a recently developed measure of accuracy designed to overcome shortcomings noted in "MAPE" (Mean Absolute Percent Error), a measure commonly used to evaluate the accuracy of population estimates and forecasts. We show that MAPE-R can be calculated simply, thus overcoming the cumbersome calculation procedure used in its introduction and noted as a feature needing correction. We find this closed form expression for MAPE-R to be a member of the family of power mean-based accuracy measures. This enables it to be placed in relation to other members of this family, which includes HMAPE (Harmonic Mean Absolute Percent Error), GMAPE (Geometric Mean Absolute Percent Error), and MAPE. Given that MAPE-R was designed to be robust in the face of outliers, it is not surprising to find that it is a valid estimator of the median of the distribution(s) generating the absolute percent errors. Simulation studies suggest that MAPE-R is a far more efficient estimator of this median than MEDAPE(Median Absolute Percent Error). Because the Box-Cox transformation on which MAPE-R depends is known to be unstable, we suggest that this represents a line of further research into GMAPE, which, like MAPE-R, is subject neither to the shortcomings observed for MAPE nor to the instability of the Box-Cox transformation. While further lines of research are called for, nothing in our examination of MAPE-R here rules out its use. It also meets the National Research Council's major criteria as a summary measure of accuracy. It is subject to some cautions, but these are no more restrictive than those affecting other accuracy measures, many of which are widely used and have been for some years.