Evaluating Compensatory Education: A Case Study

Abstract

This analysis has attempted to assess the impact of a long-range compensatory education program in Newark, New Jersey. Using an array of data from a five-year period and a variety of statistical devices, the authors have found no observable gains in reading level, school attendance, or IQ scores as the result of a five-year analysis of the Newark-Victoria Plan. The experimental subjects who were exposed to the extensive academic programs show little or no gain over their non-plan controls. In certain instances, they actually did more poorly than their controls.

If the analysis had been limited to these data, the plan would necessarily have been judged a failure. However, the use of standardized tests as a sole criterion of improvement is suspect. Therefore, the authors decided to follow the plan children into junior high school to see how they compared with children not exposed to programs as extensive as those of the plan. In this analysis, the major criteria were the actual grades received in courses of study.

In this stage of the evaluation, the plan students markedly outperformed their non-plan controls. Since the teachers were unaware of which students had been exposed to the plan, it is fair to assume that the observed differences are real and are not effects of teacher selection, self-fulfilling prophecies, or the like. The identifiable effects of the Newark-Victoria Plan, to date, appear to lie mainly in the area of academic achievement, but not in those facets measured by standardized tests.

In further analyzing the data, the authors recalled that, in another compensatory education project at the junior high school level, the male children outgained their female counterparts. Combining this observation with the overall finding that the plan children do not perform significantly better in academic areas than their non-plan controls, the authors hypothesized that what the plan children learn are not academic skills, but rather social adjustment skills involving the incorporation of the cues for successful classroom behavior (e.g., deference to teachers, discipline, orderliness, and so on). In pursuing this hypothesis, authors speculated that, in the Cleveland data, the males should show greater improvement than the females in grades and that, further, if the line of reasoning were correct, the lower-ability children would outperform their higher-ability counterparts, when compared to the appropriate control groups.

In re-analyzing the data, the authors found substantial verification of their hypothesis, since the greatest comparative gains were indeed recorded by the male and low-ability groups.

In summary then, four basic suggestions have been presented:

(a) The investment in a fully developed compensatory program such as the one described has little or no impact on reading ability or IQ score.

(b) It is possible to produce academic gains in such a program, even though these gains may not be demonstrated through traditional standardized test procedures. In this program, the gains were apparent in the follow-up analysis of junior high school course grades.

(c) While these gains are desirable and important, they can only be described as modest, since they represent roughly a movement of one-third grade difference between plan and non-plan children (e.g., C+ as opposed to C).

(d) The main thrust of the program may have been in the area of the development and internalization of social adjustment skills. Hence, the largest gains in such programs will be made in the lowest groups (e.g., low-ability male students) rather than the highest groups, as commonly expected.

It is worth noting that these findings indirectly substantiate some of the more controversial findings of the Coleman report. Coleman et al. (1966), indicated that school resources contribute little to academic achievement, as indicated through standardized measures. This analysis has similarly shown that the investment in up-to-date compensatory educational techniques registers little or no gain vis-à-vis reading level and IQ. In addition, Coleman indicated that differences in educational facilities would affect poorer students more than it would better students. This analysis indirectly supports this finding since, in the Newark-Victoria Plan, those who gained most were those normally judged as least likely to succeed academically. Overall, this study suggests answers to certain questions while posing numerous new ones.