The performance of 30 subjects when solving problems built around logical connectives (Conjunction, Inclusive Disjunction, Exclusive Disjunction, Conditional, and Biconditional) was compared with the results obtained when the same logical connectives were presented using a multiple-choice approach. The processes followed by the subjects in solving 20 problems were evaluated in terms of information theory, making it possible to investigate (a) the processes followed by the subjects and (b) the quality of the final answer Analysis indicated that the problem-solving processes do not necessarily provide the same information as that obtained by the final answers. The knowledge obtained by examining the questions subjects ask is different from the knowledge obtained when examining the answers to multiple-choice items.
Get full access to this article
View all access options for this article.
References
1.
Bei de LibonattiE. I. (1982) Test de Habilidad Diagnóstica–Método evaluativo. Centro Interdisciplinario de Investigaciones en Psicología Matemática y Experimental (CIIPME). Informes de Investigación CIIPME1981–1982. Buenos Aires, Argentina.
2.
Bei de LibonattiE. I. (1983) Estudio sobre la estructura lógica del Test de Habilidad Diagnóstica. Interdisciplinaria, 4, 61–85.
3.
Bei de LibonattiE. I. (1984) Método evaluativo para el Test de Habilidad Diagnóstica. Interdisciplinaria, 5, 1–13.
4.
CerasoJ.ProviteraA. (1971) Sources of error in syllogistic reasoning. Cognitive Psychology, 2, 400–410.
5.
Cortada de KohanN. (1993) Nueva forma de test objetivos elaborados para evitar las respuestas por sugerencias o tanteo al azar “cueing.” In Actas de II Simposio de Epistemología y Metodología de las Ciencias Humanas y Sociales. Mendoza, Argentina, Septiembre 1993.
6.
Del ValJ. (1977) Investigaciones sobre lógica y psicología. Madrid, Alianza Editorial.
7.
Del ValJ.RivièreA. (1975) ‘Si llueve, Elisa lleva sombrero’: una investigación psicológica sobre la tabla de verdad del condicional. Revista de Psicología General y Aplicada, 30, 825–850.
8.
DonnellyM. (1967) A comparison of schema pulling out and the average uncertainty methods. Loyola Psychometric Laboratory, Chicago, IL. Publication No 52.
9.
ElsteinA. (1993) Beyond multiple-choice questions and essays: the need for a new way to assess clinical competence. Academic Medicine, 68, 244–249.
10.
FerlandJ.DorvalJ.LevasseurL. (1987) Measuring higher cognitive levels by multiple choice questions: a myth?Medical Education, 21, 109–103.
11.
FinocchiaroM. (1989) Methodological problems in empirical logic. Communication & Cognition, 23, 313–335.
12.
GrandyR. E. (1979) Inference and if-then. Psychological Review, 86, 152–153.
13.
HancockG.ThiedeK.SaxG.MichaelW. (1993) Reliability of comparably written two-option multiple-choice and true-false test items. Educational and Psychological Measurement, 53, 651–660.
14.
HenleM. (1962) On the relation between logic and thinking. Psychological Review, 69, 366–378.
15.
HenleM. (1984) Episodes in the history of interactionism: on knowing what one is talking about. Revista de Historia de la Psicología, 5, 153–161.
16.
HenleM. (1987) Koffka's principles after fifty years. Journal of the History of the Behavioral Sciences, 23, 14–21.
17.
Johnson-LairdP. (1972) The three-term series problem. Cognition, 1, 57–58.
KampisGyörgy (1993) On understanding how the mind is organized: cognitive maps and the “physics” of mental information processing. In LaszloE.MasulliI.ArtigianiR.CsányiV.(Eds.), The world futures. General Evolution studies: the evolution of cognitive maps, Amsterdam: Gordon & Breach Science. Vol. 5. Pp. 135–150.
20.
KolstadR.KolstadR. (1989) Strategies used to answer MC test items by examinees in top and bottom quartiles. Educational Research Quarterly, 13, 2–5.
21.
LehmanD.NisbettR. (1990) A longitudinal study of the effects of the undergraduate training on reasoning. Developmental Psychology, 26, 952–960.
22.
Merriam-Webster. (1971) Webster's ‘Third new international dictionary’Springfield, MA: Author.
23.
MorganJ.MortonJ. (1984) The distortion of syllogistic reasoning produced by personal convictions. The Journal of Social Psychology, 20, 39–59.
24.
NaessA. (1962) L'emploi de la disjonction chez les adolescents. In BethE.GrizeJ. B.MartinR.MatalonB.NaessA.PiagetJ.(Eds.), Implication formalisation et logique naturelle. Paris: P.U.F. pp. 151–164.
25.
NeelyD. L.SpringstonF. J.McCannS. J. (1994) Does item order affect performance on multiple-choice exams?Teaching of Psychology, 21, 44–45.
26.
PiagetJ. (1952) La logistique axiomatique ou ‘pure,’ la logistique opératoire ou psycholique et les réalités auxquelles elles correspondent. Methodos, 4, 71–84.
27.
PiburnM. (1990) Reasoning about logical propositions and success in science. Journal of Research in Science Teaching, 27, 887–900.
28.
PingliaR. S. (1992) A comparative study of true-false, alternative choice, and multiple-choice item formats. Indian Journal of Psychometry and Education, 23, 49–56.
29.
PollardP. (1982) Human reasoning: some possible effects of availability. Cognition, 12, 65–69.
30.
RimoldiH. J. A. (1955) A technique for the study of problem solving. Educational and Psychological Measurement, 15, 450–461.
31.
RimoldiH. J. A. (1984) Problems of thinking about thinking: language and logical structures in problem processing. In FröehlichW. D.SmithG.DragunsJ. G.HentschelU.(Eds.), Psychological processes in cognition and personality. Hemisphere. Pp. 203–221.
32.
RimoldiH. J. A.DonnellyM. (1967) Experimental considerations concerning category and magnitude scaling. American Journal of Psychology, 80, 602–607.
33.
RimoldiH. J. A.FogliatoH. M.ErdmannJ.DonnellyM. (1964) Problem solving in high school and college students. Cooperative Research Project No. 2199. Office of Education, U.S. Department of Health, Education and Welfare.
34.
RimoldiH. J. A.InsúaA. M.ErdmannJ. (1988) A study of some perceptual and personality correlates of problem solving processes. Interdisciplinaria-Monographs. No. 2. Buenos Aires, Argentina.
35.
RimoldiH. J. A.Richaud DeminziM. C. (1984) Processing information: a method for evaluating some problem solving processes. Interdisciplinaria, 5, 143–166.
Van den BerghH. (1990) On the construct validity of multiple choice items for reading comprehension. Applied Psychological Measurement, 14, 1–12.
38.
VarelaF. (1993) When is a map cognitive? In LaszloE.MasulliI.ArtigianiR.CsányiV.(Eds.), The world futures, general evolution studies: the evolution of cognitive maps. new paradigms for the twenty-first century. Amsterdam: Gordon & Breach Science. Vol. 5. Pp. 9–104.
39.
VeloskyJ. J.RabinowitzJ. J.RobesonM. D. (1988) Cueing in multiple choice questions: a reliable, valid and economical solution. Jefferson Medical College Journal, 27, 371–375.
40.
WaltersK. (1992) Critical thinking, logicism, and the eclipse of imagining. Journal of Creative Behavior, 26, 130–144.
