Problems associated with the analysis and interpretation of aggregated data in school effects research are discussed. The multilevel character of school effects research is described. Data from the IEA Six-Subjects Survey of Science Achievement in Australia are used to illustrate the differences in between-student and between-school models of the factors influencing student achievement. The conceptual and statistical aspects of selecting the appropriate unit of analysis are discussed. Recommendations are given for collecting and analysing data in future large-scale investigations of schooling.
References
1.
AverchH. A.CarrollS. J.DonaldsonT. S.KieslingH. J.PincusJ.How Effective is Schooling?Santa Monica, California: The Rand Corporation, R-956-PCSF/RC, 1972.
2.
BidwellC.Nations, school districts and schools: are there schooling effects anywhere?Washington, D.C.: Vice-presidential Address, Division G, American Educational Research Association, Washington, D.C., 1975.
3.
BidwellC. E.KasardaJ. D.School district organization and student achievement. American Sociological Review, 1975, 40, 55–70.
4.
BlauP. M.Structural effects. American Sociological Review, 1960, 25, 178–193.
5.
BursteinL.Issues concerning inferences from grouped observations. Paper presented at the Annual Meeting of the American Educational Research Association, Chicago, Illinois, 1974.
6.
BursteinL.Data aggregation in educational research: applications. Technical Report No. 1. Consortium on Methodology for Aggregating Data in Educational Research, 1975. Also, paper presented at the Annual Meeting of the American Educational Research Association, Washington, D.C., 1975a.
7.
BursteinL.The use of data from groups for inferences about individuals in educational research. Unpublished doctoral dissertation, Stanford University. Also, Technical Report No. 7. Consortium on Methodology for Aggregating Data in Educational Research. Milwaukee, Wisc.: Vasquez Associates, Ltd., 1975b.
8.
BursteinL.Assessing differences between grouped and individual level regression coefficients. Paper presented at the Annual Meeting of the American Educational Research Association, San Francisco, 1976.
9.
BursteinL.KnappT. R.The unit of analysis in educational research. Technical Report No. 4. Consortium on Methodology for Aggregating Data in Educational Research. Milwaukee, Wisc.: Vasquez Associates, Ltd., 1975.
10.
ColemanJ. S.CampbellE. Q.HobsonC. J.McPartlandJ.MoodA. M.WeinfeldF. D.YorkR. L.Equality of Educational Opportunity. Washington, D.C.: U.S. Department of Health, Education, and Welfare, U.S. Government Printing Office, 1966.
11.
ComberL. C.KeevesJ. P.Science Education in Nineteen Countries: An Empirical Study. International Studies in Evaluation I. New York: John Wiley and Sons; Stockholm: Almqvist and Wiksell, 1973.
12.
CrainR. L.YorkR. L.Evaluating a successful program: Experimental method and academic bias. School Review, 1976, 84, 233–254.
13.
FarkasG.Specification, residuals and contextual effects. Sociological Methods and Research, 1974, 2, 333–363.
14.
FeigeE. L.WattsH. W.An investigation of the consequences of partial aggregation of micro-economic data. Econometrica, 1972, 40, 343–360.
15.
FlanaganJ. C.Project Talent. Pittsburgh: University of Pittsburgh, 1962.
16.
GarmsW. I.SmithM. C.Educational need and its application to state school finance. Journal of Human Resources, 1970, 5, 304–317.
17.
GlassG. V.StanleyJ. C.Statistical Methods in Education and Psychology. Englewood Cliffs, N. J.: Prentice-Hall, 1970.
18.
GlendeningL.PorterA. C.The effects of correlated units of analysis: violating the assumption of independence. A working paper of the Measurement and Methodology Program, National Institute of Education, November, 1974.
19.
HannanM. T.Aggregation and Disaggregation in Sociology. Lexington, Mass.: Heath, 1971.
20.
HannanM. T.Aggregation gain reconsidered. Paper presented at the Annual Meeting of the American Educational Research Association, San Francisco, 1976.
21.
HannanM. T.BursteinL.Estimation from grouped observations. American Sociological Review, 1974, 39, 374–392.
22.
HannanM. T.FreemanJ. H.MeyerJ. W.Specification of models for organizational effectiveness. (Comment on Bidwell and Kasarda, ASR, February, 1975.)Amer. Soc. Review, 1976, 41, 136–143.
23.
HannanM. T.YoungA. A.NielsenF.Specification bias analysis of the effects of grouping of observations in multiple regression models. Technical Report No. 2. Consortium on Methodology for Aggregating Data in Educational Research, April, 1975. (Also, paper presented at the Annual Meeting of the American Educational Research Association, Washington, D.C., 1975.)
24.
HanushekE. A.The education of negroes and whites. Unpublished Ph.D. dissertation, Massachusetts Institute of Technology, 1968.
25.
HanushekE. A.The Value of Teachers in Teaching. Santa Monica, California: The Rand Corporation, RM-6362-CC/RC, 1970.
26.
HanushekE. A.Education and Race. Lexington, Mass.: D.C. Heath, 1972.
27.
HanushekE. A.JacksonJ. E.KainJ. F.Model specification, use of aggregate data, and the ecological correlation fallacy. Political Methodology, 1974, 1, 89–107.
28.
HauserR. M.Schools and the stratification process. American Journal of Sociology, 1969, 74, 587–611.
29.
HauserR. M.Context and consex: A cautionary tale. American Journal of Sociology, 1970, 75, 645–664.
30.
HauserR. M.Socio-economic Background and Educational Performance. Washington, D.C.: Rose Monograph Series, American Sociological Association, 1971.
HusénT.International Project for the Evaluation of Educational Achievement. Phase I. International Study of Achievement in Mathematics: A Comparison of Twelve CountriesStockholm: Almqvist and Wiksell; New York: John Wiley, 1967.
33.
KeeslingJ. W.WileyD. E.Regression models of hierarchical data. Paper presented at meeting of Psychometric Society, Stanford University, Calif., 1974.
34.
KeevesJ. P.Educational Environment and Student Achievement. Uppsala: Almqvist and Wiksell and Hawthorn, Vic.: ACER, 1972.
35.
KieslingH. J.The Relationship of School Inputs to Public School Performance in New York State. Santa Monica, California: The Rand Corporation, P-4211, 1969.
36.
KieslingH. J.A Study of Cost and Quality of New York School Districts. Washington, D.C.: U.S. Department of Health, Education and Welfare, 1970.
37.
KlitgaardR. E.HallG.Are There Unusually Effective Schools?Santa Monica. California: The Rand Corporation, P-4995, 1973.
38.
LevinH. M.A new model of school effectiveness. In Do Teachers Make a Difference?Washington, D. C.: U.S. Department of Health, Education and Welfare, 1970.
39.
LevineD. U.MaresK. R.MeyerJ. K.StephensonR. S.The Utility and Implications of the 1970 Census Data in Predicting Achievement and Assessing Effects of Concentrated Urban Poverty in Chicago. Kansas City, Miss.: University of Missouri, Center for the Study of Metropolitan Problems in Education, School of Education, 1974.
40.
McDillE. L.RigsbyL. C.Structure and Process in Secondary Schools; The Academic Impact of Educational Climates. Baltimore and London: The Johns Hopkins University Press, 1973.
41.
MarjoribanksK.Cognitive performance: A model for analysis. Australian Journal of Education, 1975, 19, 156–166.
42.
MichelsonS.The association of teacher resourceness with children's characteristics. In How Do Teachers Make a Difference. Washington, D.C.: U.S. Department of Health, Education and Welfare, Office of Education, Bureau of Educational Personnel Development, OE-58042, 120–168, 1970.
43.
NORC (National Opinion Research Center). Southern Schools: An Evaluation of the Effects of the Emergency School Assistance Program and of School Desegregation. Chicago, 1973.
44.
NuthallG. A.Is classroom interaction research worth the effort involved?New Zealand Journal of Educational Studies, 1974, 9, 1–17.
45.
PeakerG. F.The regression analyses of the national survey. Appendix 4 in the Plowden Report. Department of Education and Science. Children and their Primary Schools. A Report of the Central Advisory Council for Education (England). London: H.M.S.O., 1967. Volume 2: Research and Research Surveys, 179–221.
46.
PeakerG. F.The Plowden Children Four Years Later. Slough, England: National Foundation for Educational Research, 1971.
47.
PeakerG. F.An Empirical Study of Education in Twenty-One Countries: A Technical Report. International Studies in Evaluation VIII. Stockholm: Almqvist and Wiksell, 1975.
48.
[The Plowden Report.]Department of Education and Science. Children and their Primary Schools. A report of the Central Advisory Council for Education (England). London: H.M.S.O., 1967.
49.
PoynorH.Selecting units of analysis. In BorichG. (Ed.) Evaluating Educational Programs and Products. Englewood Cliffs, N.J.: Educational Technology Press, 1974.
50.
PurvesA. C.Literature Education in Ten Countries: An Empirical Study. International Studies in Evaluation. II. Sweden: Almqvist and Wiksell, 1973.
51.
RobinsonW. S.Ecological correlations and the behaviour of individuals. American Sociological Review, 1950, 15, 351–357.
52.
RockD. A.BairdL. L.LinnR. L.Interaction between college effects and students' aptitudes. American Educational Research Journal, 1972, 9, 149–161.
53.
ScheuchE. K.Cross-national comparisons using aggregate data: Some substantive and methodological problems. In MerrittR. L.RokkanS. (Eds.) Comparing Nations: The Use of Quantitative Data in Cross-National Research. New Haven: Yale University Press, 1966, 148–156.
54.
SmithM. S.Equality of educational opportunity: The basic findings reconsidered. In MostellerF.MoynihanD. P. (Eds.) On Equality of Educational Opportunity. New York: Random House, 1972, 230–342.
55.
SummersA.WolfeB.Equality of Educational Opportunity Quantified: A Production Function Approach. Philadelphia, Pennsylvania: Federal Reserve Bank of Philadelphia, 1974. (Also, paper presented at the Annual Meeting of the Econometrics Society, New York, 1974.)
56.
SummersA.WolfeB.Which school resources help learning, efficiency and equity?Federal Reserve Bank of Philadelphia Business Review, February, 1975.
57.
ThorndikeR. L.Reading Comprehension Education in Fifteen Countries. International Studies in Evaluation. III. New York: John Wiley and Stockholm: Almqvist and Wiksell, 1973.
58.
WalbergH. J.Social environment as a mediator of classroom learning. Journal of Educational Psychology, 1969, 60, 443–448.
59.
WalbergH. J.RasherS. P.Public school effectiveness and equality: New evidence and its implications. Phi Delta Kappan, 1974, 56, 3–9.
60.
WisemanS.The Manchester Survey. Appendix 9 in the Plowden Report. Department of Education and Science. Children and their Primary Schools. A report of the Central Advisory Council for Education (England). London: H.M.S.O., 1967. Volume 2: Research and Surveys, 349–400.
