Comments on a Note on Permutation Tests of Significance for Multiple Regression Coefficients by Long,et al.

Abstract

It is noted that a recent paper in the Journal may give a misleading impression of the robustness of classical regression methods. First, the authors show that p-values based on the randomization distributions of regression coefficients may be quite different from p-values based on t-distributions. However, there is no reason why these two types of p-value should be exactly the same, and p-values based on the randomization distributions of t-statistics are usually similar to p-values obtained from t-distributions. These latter p-values are perfectly valid for the purposes of randomization tests. Second, the authors imply that estimating the standard errors of regression coefficients by randomizing the order of the Y values gives better estimates of standard errors than standard theory. In fact these standard errors based on randomization will tend to be too large unless the regression equation accounts for none of the variation in the Y values.