The current set of three studies further evaluates the validity and application of the Psychological Research Inventory of Concepts (PRIC). In Study 1, we administered the PRIC to a sample of introductory psychology students and online (Mechanical Turk) participants along with measures assessing theoretically related concepts. We found evidence of concurrent validity by demonstrating that PRIC scores relate to education, cognitive effort, and general psychological knowledge. In Study 2, we demonstrated that advanced psychology majors score higher on the PRIC than college graduates who did not major in psychology, suggesting that the PRIC assesses domain-specific knowledge and may be useful in assessment of the undergraduate psychology major. Finally, in Study 3, we demonstrated that scores on the PRIC increase from the start to the end of a research methods course. Together, these studies provide further evidence that the PRIC may be a useful index of learning research methods and statistical knowledge important for the undergraduate psychology degree.
AndersonD. L.FisherK. M.NormanG. J. (2002). Development and evaluation of the conceptual inventory of natural selection. Journal of Research in Science Teaching, 39, 952–978. doi:10.1002/tea.10053
3.
BarronK. E.AppleK. J. (2015). Debating curricular strategies for teaching statistics and research methods: What does the current evidence suggest?Teaching of Psychology, 41, 187–194. doi:10.1177/0098628314537967
4.
CacioppoJ. T.PettyR. E. (1982). The need for cognition. Journal of Personality and Social Psychology, 42, 116–131. doi:10.1037/0022-3514.42.1.116
5.
DuckworthA. L.PetersonC.MatthewsM. D.KellyD. R. (2007). Grit: Perseverance and passion for long-term goals. Journal of Personality and Social Psychology, 9, 1087–1101. doi:10.1037/0022-3514.92.6.1087
6.
EpsteinS.PaciniR.Denes-RajV.HeierH. (1996). Individual differences in intuitive–experiential and analytical–rational thinking styles. Journal of Personality and Social Psychology, 71, 390–405. doi:10.1037/0022-3514.71.2.390
7.
EvansD. L.GrayG. L.KrauseS.MartinJ.MidkiffC.NotarosB. M.…WageK. (2003). Progress on concept inventory assessment tools. Proceedings of the 33 rd ASEE/IEEE Frontiers in Education Conference, pp. TG41–T4G8.
8.
EvansJ. St. B. T. (2008). Dual-processing accounts of reasoning, judgment and social cognition. Annual Review of Psychology, 59, 255–278. doi:10.1146/annurev.psych.59.103006.093629
9.
FariasM.NewheiserA.-K.KahaneG.de ToledoZ. (2013). Scientific faith: Belief in science increases in the face of stress and existential anxiety. Journal of Experimental Social Psychology, 49, 1210–1213. doi:10.1016/j.jesp.2013.05.008
10.
FrederickS. (2005). Cognitive reflection and decision making. Journal of Economic Perspectives, 19, 25–42.
11.
GoodmanJ. K.CryderC. E.CheemaA. (2012). Data collection in a flat world: The strengths and weaknesses of Mechanical Turk samples. Journal of Behavioral Decision Making, 26, 213–224. doi:10.1002/bdm.1753
12.
HallounI. A.HestenesD. (1985a). The initial knowledge state of college physics students. American Journal of Physics, 53, 1043–1055. doi:10.1119/1.14030
13.
HallounI. A.HestenesD. (1985b). Common sense concepts about motion. American Journal of Physics, 53, 1056–1065. doi:10.1119/1.14031
14.
HughesS.LyddyF.LambeS. (2013). Misconceptions about psychological science: A review. Psychology Learning & Teaching, 12, 20–31. doi:10.2304/plat.2013.12.1.20
15.
LawsonT. J. (1999). Assessing psychological critical thinking as a learning outcome for psychology majors. Teaching of Psychology, 26, 207–209.
16.
LawsonT. J.Jordan-FlemingM. K.BodleJ. H. (2015). Measuring psychological critical thinking: An update. Teaching of Psychology, 42, 248–253. doi:10.1177/0098628315587624
17.
LyddyF.HughesS. (2012). Attitudes toward psychology as a science and the persistence of psychological misconceptions in psychology undergraduates. In KarandashevV.McCarthyS. (Eds.), Teaching psychology around the world (Vol. 3). Newcastle upon Tyne, England: Cambridge Scholars.
18.
NorrisP.PaciniR.EpsteinS. (1998). The rational-experiential inventory, short form. Unpublished inventory, University of Massachusetts at Amherst, Amherst, MA.
19.
OppenheimerD. M.MeyvisT.DavidenkoN. (2009). Instructional manipulation checks: Detecting satisficing to increase statistical power. Journal of Experimental Social Psychology, 45, 867–872. doi:10.1016/j.jesp.2009.03.009
20.
PliskeR. M.CaldwellT. L.Calin-JagemanR. J.Taylor-RitzlerT. (2015). Demonstrating the effectiveness of an integrated and intensive research methods and statistics course sequence. Teaching of Psychology, 42, 153–156. doi:10.1177/0098628315573139
21.
SmithD. L.BarkerL. (2008). Using yes-no recognition tests to assess student memory for course content. Teaching of Psychology, 35, 319–326. doi:10.1080/00986280802374468
22.
ToplakM. E.WestR. F.StanovichK. E. (2011). The Cognitive Reflection Test as a predictor of performance on heuristics and biases tasks. Memory & Cognition, 39, 1275–1289. doi:10.3758/s13421-011-0104-1
23.
VeilleuxJ. C.ChapmanK. M. (2017). Development of a research methods and statistics concept inventory. Teaching of Psychology, 44, 203–211.
24.
VittenglJ. R.BosleyC. Y.BresciaS. A.EckardtE. A.NeidigJ. M.ShelverK. S.…SapenoffL. A. (2004). Why are some undergraduates more (and others less) interested in psychological research?Teaching of Psychology, 31, 91–97. doi:10.1207/s15328023top3102
