In this essay, the author tries to look forward into the 21st century to divine three things: (i) What skills will researchers in the future need to solve the most pressing problems? (ii) What are some of the most likely candidates to be those problems? and (iii) What are some current areas of research that seem mined out and should not distract us from working on more critical areas? In the course of this discussion, the author identifies 14 different things that fall into one of these areas.
AlmondR. G. (1991). Building blocks for graphical belief models. Journal of Applied Statistics, 18, 63–76.
2.
AlmondR. G. (1995). Graphical belief modeling. New York: Chapman & Hall.
3.
AlmondR. G.SteinbergL. S.MislevyR. J. (2002). Enhancing the design and delivery of assessment systems: A four-process architecture. Journal of Technology, Learning, and Assessment, 1. Retreived from http://www.bc.edu/research/intasc/jtla/journal/v1n5.shtml. Also available as CSE Technical Report 543. Los Angeles: The National Center for Research on Evaluation, Standards, Student Testing (CRESST), Center for Studies in Education, UCLA. Retrieved October 26, 2005, from http://www.cse.ucla.edu/CRESST/Reports/TECH543.pdf.
4.
AlmondR. G.SteinbergL. S.MislevyR. J. (2003). A framework for reusing assessment components. In YanaiH.OkadaA.ShigemasuK.KanoY.MeulmanJ. J. (Eds.), New developments in psychometrics. (pp. 28–288). Tokyo: Springer.
5.
BayesT. (1764). An essay toward solving a problem in the doctrine of chances. Philosophical Transactions of the Royal Society of London for 1763, 53, 370–418.
6.
BenjaminiY.HochbergY. (1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing. Journal of the Royal Statistical Society, Series B, 57, 289–300.
7.
BertinJ. (1983). Semiology of graphics (BergW.WainerH., Trans.). Madison, Wisconsin: University of Wisconsin Press. (Original work published 1973)
8.
BonferroniC. E. (1936). Teoria statistica delle classi e calcolo delle probabilità [Statistical theory of classes and calculating probability]. Pubblicazioni del R Istituto Superiore di Scienze Economiche e Commerciali di Firenze, 8, 3–62.
9.
BourqueM. L. (2005). The history of No Child Left Behind. In PhelpsRichard P. (Ed.), Defending standardized testing (pp. 227–254). Hillsdale, NJ: Lawrence Erlbaum.
10.
BraunH.WainerH. (2007). Value-added assessment. In RaoC. R.SinharayS. (Eds.), Handbook of statistics (volume 27) psychometrics (pp. 867–892). Amsterdam: Elsevier Science.
11.
CronbachL. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16, 297–334.
12.
CronbachL. J.GleserG. C.NandaH.RajaratnamN. (1972). The dependability of behavioral measurements: Theory of generalizability for scores and profiles. New York: John Wiley.
13.
DunnM. C.KadaneJ. B.GarrowJ. R. (2003). Comparing harm done by mobility and class absence: Missing students and missing data. Journal of Educational and Behavioral Statistics, 28, 269–288.
14.
FisherR. A. (1935). The design of experiments. New York: Hafner.
15.
GalsworthyJ. (1924). The White Monkey and a silent wooing (vol. IV of The Forsyte Saga; p. 52). New York: Charles Scribner.
16.
GelmanA.CarlinJ. B.SternH. S.RubinD. B. (2003). Bayesian data analysis. (2nd ed.). New York: Chapman & Hall.
17.
GilksW. R.RichardsonS.SpiegelhalterD. J. (1996). Markov chain Monte Carlo in practice. New York: Chapman & Hall.
18.
GulliksenH. O. (1950). Theory of mental tests. New York: John Wiley. (Reprinted in 1987, Hillsdale, NJ: Lawrence Erlbaum).
19.
HalmosP. R. (1970). How to write mathematics. L’Enseignement mathématique, 16, 122–152.
20.
HilbertD. (1902). Mathematical problems. Bulletin of the American Mathematical Society, 8, 437–479.
21.
HollandP. W. (1986). Statistics and causal inference. Journal of the American Statistical Association, 81, 945–970.
22.
HollandP. W.RubinD. B. (1983). On Lord’s paradox. In WainerH.MessickS. (Eds.), Principals of modern psychological measurement (pp. 3–25) Hillsdale, NJ: Lawrence Erlbaum.
23.
HollandP. W.ThayerD. T. (1988). Differential item performance and the Mantel-Haenszel procedure. In WainerH.BraunH. (Eds.), Test validity (pp. 129–146). Hillsdale, NJ: Lawrence Erlbaum Associates.
24.
HolzingerK. J.SwinefordF. (1937). The bi-factor method. Psychometrika, 2, 41–54.
25.
JöreskogK. G. (1969). A general approach to confirmatory maximum likelihood factor analysis. Psychometrika, 34, 183–202.
26.
LawleyD. N. (1943). On problems connected with item selection and test construction. Proceedings of the Royal Society of Edinburgh, 62-A, Part I, 74–82.
27.
LewinK. (1951). Field theory in social science; selected theoretical papers (p. 169). CartwrightD. (Ed.), New York: Harper & Row.
28.
LinnR. L. (2004). Rethinking the No Child Left Behind accountability system. Paper presented at the Center for Education Policy Forum, Washington, DC. Retrieved October 26, 2005, from http://www.ctredpol.org
29.
LinnR. L. (2005, June 28). Conflicting demands of “No Child Left Behind” and state systems: Mixed messages about school performance. Education Policy Analysis Archives, 13. Retrieved May 15, 2006, from http://epaa.asu.edu/epaa/v13n33/.
30.
LittleR. J. A.RubinD. B. (1987). Statistical analysis with missing data. New York: John Wiley.
31.
LordF. M. (1967). A paradox in the interpretation of group comparisons. Psychological Bulletin, 68, 304–305.
32.
LordF. M.NovickM. R. (1968). Statistical theories of mental test scores. Reading, MA: Addison Wesley.
33.
MeehlP. E.RosenA. (1953). Antecedent probability and the efficiency of psychometric signs, patterns, or cutting scores. Psychological Bulletin, 52, 194–216.
34.
MislevyR. J. (2006). Cognitive psychology and educational assessment. In BrennanR. (Eds.), Educational measurement (4th ed. pp. 257–305). Westport, CT: Praeger.
35.
MislevyR. J.SteinbergL. S.AlmondR. G. (2003). On the structure of educational assessments. Measurement: Interdisciplinary Research and Perspectives, 1, 3–67.
36.
MislevyR. J.SteinbergL. S.BreyerF. J.AlmondR. G.JohnsonL. (2002). Making sense of data from complex assessments. Applied Measurement in Education, 15, 363–378.
37.
NeymanJ. (1923). On the application of probability theory to agricultural experiments (DabrowskaD.SpeedT., Trans.). Statistical Science, 5, 462–472.
38.
NovickM. R.LewisC. (1967). Coefficient alpha and the reliability of composite measurements. Psychometrika, 32, 1–13.
39.
PlayfairW. (1801a). The statistical breviary. London: T. Bensley. (Facsimile reprint edited and annotated by Howard Wainer and Ian Spence, 2006, New York: Cambridge University Press)
40.
PlayfairW. (1801b). The commercial and political atlas, representing, by means of stained copper-plate charts, the progress of the commerce, revenues, expenditure, and debs of England, during the whole of the eighteenth century. (Facsimile reprint edited and annotated by Howard Wainer and Ian Spence, 2006, New York: Cambridge University Press).
41.
RogosaD. (1988, June 28). The Ballad of the Casual Modeler. A song performed at the Annual Meeting of the Psychometric Society by Statboy & the Casuals, Los Angeles, California.
42.
RosenbaumP. R. (2002). Observational studies (2nd ed.). New York: Springer-Verlag.
43.
RubinD. B. (1974). Estimating causal effects of treatments in randomized and non-randomized studies. Journal of Educational Psychology, 66, 688–701.
44.
RubinD. B. (1987). Multiple imputation for nonresponse in surveys. New York: John Wiley.
45.
RubinD. B.StuartE. A.ZanuttoE. L. (2004). A potential outcomes view of value-added assessment in education. Journal of Educational and Behavioral Statistics, 29, 103–116.
46.
SavageL. J. (1954). The foundations of statistics. New York: John Wiley.
47.
SavageS.WainerH. (2008). Until proven guilty: False positives and the war on terror. Chance, 21, 55-58
48.
StrunkWJr.WhiteE. B. (1959). The elements of style. New York: Macmillan.
49.
ThissenD.SteinbergL.WainerH. (1988). Use of item response theory in the study of group differences in trace lines. In WainerH.BraunH. (Eds.), Test validity (pp. 147–169). Hillsdale, NJ: Lawrence Erlbaum.
50.
ThissenD.SteinbergL.WainerH. (1993). Detection of differential item functioning using the parameters of item response models. In HollandP. W.WainerH. (Eds.), Differential item functioning (chap. 4, pp. 67–113). Hillsdale, NJ: Lawrence Erlbaum.
51.
TufteE. R. (1983). 2000). The visual display of quantitative information. Cheshire, CT: Graphics Press.
52.
TufteE. R. (1990). Envisioning information. Cheshire, CT: Graphics Press.
53.
TufteE. R. (1997). Visual explanations. Cheshire, CT: Graphics Press.
54.
TufteE. R. (2006). Beautiful evidence. Cheshire, CT: Graphics Press.
55.
TukeyJ. W. (1953). The problem of multiple comparisons. Mimeographed manuscript. Princeton, NJ: Princeton University. (Reprinted in full in volume VIII of the Collected Works of John W. Tukey by H. Braun, Ed., 1994, New York: Chapman & Hall).
56.
TukeyJ. W. (1977). Exploratory data analysis. Reading, MA: Addison-Wesley.
57.
WainerH. (1993). Measurement problems. Journal of Educational Measurement, 30, 1–21.
58.
WainerH. (1995). Measurement problems. In LaveaultD.ZumboB. D.GessaroliM. E.BossM. W. (Eds.), Modern theories in measurement: Problems and issues (pp. 375–407). Ontario, Canada: University of Ottawa Press.
59.
WainerH. (1997). Visual revelations: Graphical tales of fate and deception from Napoleon Bonaparte to Ross Perot. New York: Copernicus Books. (Reprinted in 2000, Hillsdale, NJ: Lawrence Erlbaum).
60.
WainerH. (2005). Graphic discovery: A trout in the milk and other visual adventures. Princeton, NJ: Princeton University Press.
61.
WainerH. (2009). Picturing the uncertain world: How to understand, communicate and control uncertainity through graphical display. Princeton, New Jersey: Princeton University Press.
62.
WainerH.BradlowE. T.WangX. (2007). Testlet response theory and its applications. New York: Cambridge University Press.
63.
WainerH.BridgemanB.NajarianM.TrapaniC. (2004). How much does extra time on the SAT help?Chance, 17, 19–24.
64.
WangX.BradlowE. T.WainerH.MullerE. (2008). A Bayesian method for studying DIF: A cautionary tale filled with surprises and delights. Journal of Educational and Behavioral Statistics, 33, 363–384.
65.
WilkinsonL. (1999). The grammar of graphics. New York: Springer-Verlag.
66.
WilkinsonL. (2005). The grammer of graphics (2nd ed.). New York: Springer-Verlag.
