The hypothetical construct of cognitive structure encodes human knowledge internally, and two-stage probes are used to define it operationally to obtain external representations of cognitive structure. First, a proximity matrix is constructed that encodes subjects' responses on a task, and secondly, a scaling method is applied to the proximity matrix to yield a representation of cognitive structure in the form of concept categories or spatial configurations. A new multitechnique-multimethod matrix methodology for construct validation of two-step probes is proposed that modifies and extends Campbell and Fiske's multitrait-multimethod matrix. It is based on the assessment of (i) the reliability and validity of products at each step and (ii) certain important psychometric features of the scaling methods and cognitive structure probes. This methodology was applied to assess the construct validity of the cognitive structure in the domain of classical mechanics. The data-gathering techniques of Free Sort and Tree Construction were fairly reliable and valid, whereas the Word Association technique was not. The combined cognitive structure probes have above moderate reliability and validity, except for those employing multidimensional-scaling. In addition to demonstrating the theoretical value of the multitechnique-multimethod matrix methodology, these results provide empirical evidence for the interpretations given to the hypothetical construct of cognitive structure and support the use of current probes for the derivation of cognitive structure representations.
Get full access to this article
View all access options for this article.
References
1.
AnglinJ. M. (1970) The growth of word meaning. Cambridge, MA: MIT Press.
2.
AusubelD. P.NovakD. J.HanesianH. (1978) Educational psychology: A cognitive view. New York: Holt, Rinehart & Winston
3.
CampbellD. T.FiskeD. W. (1959) Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56, 81–105.
4.
CarverR. P. (1974) Two dimensions of test: Psychometric and edumetric. American Psychologist, 7, 512–518.
5.
ChampagneA. B.GunstoneR. F.KlopferL. E. (1986) Effecting changes in cognitive structure among physics students. In WestL. H. T.PinesA. L. (Eds.), Cognitive structure and conceptual change. New York: Academic Press. Pp. 163–187.
6.
ChampagneA. B.HozR.KlopferL. E. (1987) Construct validation of the cognitive structure of physics concepts. (LRDC Publication Series 19/87, Univer. of Pittsburgh)
7.
CronbachL. J.MeehlP. E. (1955) Construct validity in psychological tests. Psychological Bulletin, 52, 281–302.
8.
DeeseJ. (1965) The structure of associations in language and thought. Baltimore, MD: Johns Hopkins Univer. Press.
9.
FillenbaumS.RapoportA. (1971) Structures in the subjective lexicon. New York: Academic Press.
10.
GarskoffB. E.HoustonJ. P. (1963) Measurement of verbal relatedness: An idiographic approach. Psychological Review, 70, 277–288.
11.
GeeslinW. E. (1973) An exploratory analysis of a mathematics instructional unit. Unpublished doctoral dissertation, Stanford Univer. Dissertation Abstracts International, 1973, 34, 5582A–5583A. (University Microfilms No. 74–6478)
12.
GoodmanL. A.KruskalW. H. (1954) Measures of association for cross-classification. Journal of the American Statistical Association, 49, 732–764.
13.
GorodetskyM.HozR. (1985) Changes in the group cognitive structure of some chemical equilibrium concepts following a university course in general chemistry. Science Education, 69, 185–199.
14.
GreenoG. J. (1983) Response to Phillips. Educational Psychologist, 18, 75–80.
15.
GunstoneR. F. (1980) Structural outcomes of physics instruction. Unpublished doctoral dissertation, Monash Univer., Melbourne.
16.
GunstoneR. F.ChampagneA. B.KlopferL. E. (1981) Instruction for understanding: A case study. Australian Science Teachers Journal, 27, 27–32.
17.
GuttmanA. L. (1968) A general nonmetric technique for finding the smallest coordinate space for a configuration of points. Psychometrika, 33, 469–506.
18.
HambletonR. K.SheehanD. S. (1977) On the evaluation of higher-order science instructional objectives. Science Education, 61, 307–315.
19.
HarasymP.PrechtD. (1971) Wiley's latent partition analysis. Univer. of Alberta, Edmonton.
20.
HausleinP.GoodR. G.CumminsC, (1989) The effects of teaching upon biology content cognitive structure of teachers. Paper presented at the annual meeting of AERA, San Francisco, CA.
21.
HenleyN. M. (1969) A psychological study of the semantics of animal terms. Journal of Verbal Learning and Verbal Behavior, 8, 176–184.
22.
HolmanE. W. (1972) The relation between hierarchical and Euclidean models for psychological distances. Psychometrika, 37, 417–423.
23.
HubertL.BakerF. B. (1976) Data analysis by single-link and complete-link hierarchical-clustering. Journal of Educational Statistics, 1, 87–111.
24.
IsaacP. D.PoorD. D. S. (1974) On the determination of the appropriate dimensionality in data with error. Psychometrika, 39, 91–109.
25.
JohnsonP. E. (1965) Word relatedness and problem-solving skills in high-school physics. Journal of Educational Psychology, 56, 217–224.
26.
JohnsonP. E. (1967) Some psychological aspects of subject-matter structure. Journal of Educational Psychology, 58, 75–83.
27.
JohnsonP. E.CoxD. L.CurranT. E. (1970) Psychological reality of physical concepts. Psychonomic Science, 19, 245–246.
28.
JohnsonS. C. (1967) Hierarchical-clustering schemes. Psychometrika, 32, 241–254.
29.
KozminskyE.HozR. (1988) Use of reference latent categories to evaluate cognitive structure changes following instruction. (Unpublished manuscript, Ben-Gurion Univer., Beer-Sheva)
30.
KruskalJ. B. (1964) Nonmetric multidimensional-scaling: A numerical method. Psychometrika, 29, 115–130.
31.
KruskalJ. B.SeeryT. B.YoungF. W. (1973) KYST: Computer program for multidimensional-scaling and unfolding. Murray Hill, NJ: Bell Labs.
32.
MayerR. E.GreenoJ. G. (1972) Structural differences between learning outcomes produced by different instructional methods. Journal of Educational Psychology, 63, 165–173.
33.
MessickS. (1975) ‘The standard problem’: Meaning and values in measurement and evaluation. American Psychologist, 30, 955–966.
34.
MezzichJ. E. (1982) Comparing cluster analytic methods. In HudsonH. (Ed.), Classifying social data: Proceedings of a National Science Foundation Workshop. San Francisco, CA: Jossey-Bass. Pp. 16–34.
35.
NagyP. (1978) Construct validity of cognitive structures: A comparison of multidimensional methods. Paper presented at the annual conference of the AERA, Toronto, Ontario.
36.
NagyP. (1983) Assessing cognitive structure: A response to Stewart. Science Education, 67, 25–36.
37.
PhillipsD. (1983) On the student's cognitive structure. Educational Psychologist, 18, 59–74.
38.
PreeceP. F. W. (1976a) Mapping cognitive structure: A comparison of methods. Journal of Educational Psychology, 68, 1–8.
39.
PreeceP. F. W. (1976b) Science concepts in semantic space—a multidimensional-scaling study. The Alberta Journal of Educational Research, 12, 281–288.
40.
PreeceP. F. W. (1978a) Exploration of semantic space: Review of research on the organization of scientific concepts in semantic memory. Science Education, 62, 547–562.
41.
PreeceP. F. W. (1978b) Three-year stability of certain word-association matrices. Psychological Reports, 42, 25–26.
42.
RapoportA. (1967) A comparison of two tree-construction methods for obtaining proximity measures among words. Journal of Verbal Learning and Verbal Behavior, 6, 884–890.
43.
RapoportA.FillenbaumS. (1972) An experimental study of semantic structures. In RomneyA. K.ShepardR. N.NerloveS. B. (Eds.), Multidimensional-scaling: Theory and application in the behavioral sciences. Vol. 2. New York: Seminar Press. Pp. 93–131.
44.
RudnitskyA. N. (1976) Content structure, cognitive structure, and their relationship: A methodological investigation. Unpublished doctoral dissertation, Cornell Univer. [Dissertation Abstracts International, 1977, 38, 6251A.] (University Microfilms No. 77–8404)
45.
SchaeferG. (1979) Concept formation in biology: The concept “growth.”European Journal of Science Education, 1, 87–101.
46.
ShavelsonR. J. (1973) Learning from physics instruction. Journal of Research in Science Teaching, 10, 101–111.
47.
ShavelsonR. J. (1974) Methods for examining representations of a subject-matter structure in a student's memory. Journal of Research in Science Teaching, 11, 231–249.
48.
ShavelsonR. J. (1983) On quagmires, philosophical and others: A reply to Phillips. Educational Psychologist, 18, 81–87.
49.
ShavelsonR. J.StantonG. C. (1975) Construct validation: Methodology and application to three measures of cognitive structure. Journal of Educational Measurement, 12, 67–85.
50.
ShepardR. N. (1962a) The analysis of proximities: Multidimensional-scaling with an unknown distance function: I. Psychometrika, 27, 125–140.
51.
ShepardR. N. (1962b) The analysis of proximities: Multidimensional-scaling with an unknown distance function: II. Psychometrika, 27, 219–246.
52.
ShepardR. N. (1969) Some principles and prospects for the spatial representation of behavioral science data. Paper presented at the MSSB Advanced Research Seminar on Measurement and Scaling.
53.
StewartJ. (1979) Content and cognitive structure: Critique of assessment and representation techniques used by science education researchers. Science Education, 63, 395–405.
54.
WaerenY. (1972) Structure in similarity matrices. Scandinavian Journal of Psychology, 13, 5–16.
55.
WileyD. E. (1967) Latent partition analysis. Psychometrika, 32, 183–193.
56.
YackulicA. (1984) Portraying cognitive structures using latent partition analysis. In NagyP. (Ed.), The representation of cognitive structures. Toronto: Ontario Institute for the Study of Education. Pp. 133–139.
