The Frequency and Quality of Measures Utilized in Federally-Sponsored Research on Children and Adolescents

Abstract

Every test intended for use in FY ‘75 Federally-funded research on children or youth was placed on a list. The list eventually comprised the titles of 1,570 instruments. Some were mentioned by many principal investigators in their proposals; others were mentioned by only one. Some tests were highly respected instruments; others were not. The question pursued was whether there was a relationship between an instrument’s quality and the frequency with which it was used. To gain a sense of a measure’s quality, we utilized the numerical ratings published by the UCLA Center for the Study of Evaluation. For frequency of use we counted the number of times an instrument was mentioned in 3,538 research proposals on children or youth which are currently being sponsored by the Federal government.

There is a positive relationship between the quality of tests and their frequency of use. But the degree is not equally strong from one test category to another. A preference for the better rated instruments is particularly evident with tests of academic achievement. More equivocal results appear with respect to tests of vocational skills and intelligence, though in certain respects researchers are definitely using the better of those available in these two categories. The anomaly lies in the categories of reading tests and tests of personality, where the higher rated of the tests have no better chance of being utilized than those judged to be of poor quality. All sponsored researchers need not use the same instruments. But the fact that there are particular subject areas such as in reading and personality where the higher rated tests are less likely to be used points to the need for special attention when choosing instruments for future research.