Students Engaging in Multiple-Source Inquiry Tasks: Capturing Dimensions of Collaborative Online Inquiry and Social Deliberation

Abstract

Multiple-source inquiry that involves collaboration and deliberation is a complex construct; accordingly, valid measurement of these related competencies should require students to demonstrate their collaborative inquiry and social deliberation skills as part of an integrated performance. In this article, we report findings from the first phase of an assessment project designed to develop and evaluate the use of a virtual world platform that simultaneously captures collaboration, online inquiry, and social deliberation in the context of a single scenario-based task. Mixed methods were employed to evaluate preliminary evidence of students’ (ages 14–16) competencies in both face-to-face (n = 5 dyads) and remote (n = 6 dyads) contexts. Results indicated the scenario-based virtual assessment platform successfully captured varying levels of cognitive and social performance (using both product and process data) on several dimensions of collaborative online inquiry and social deliberation. Illustrative examples of dyads demonstrating stronger or weaker collaborative processes and high-level themes that emerged are presented. Implications for classroom settings and large-scale assessment environments are discussed.

Keywords

online inquiry collaboration social deliberation multiple source comprehension assessment

Get full access to this article

View all access options for this article.

References

Barzilai

Zohar

(2012). Epistemic thinking in action: Evaluating and integrating online sources. Cognition and Instruction, 30, 39–85.

Britt

M. A.

Rouet

J. F.

(2012). Learning with multiple documents: Component skills and their acquisition. In Lawson

M. J.

Kirby

J. R.

(Eds.), Enhancing the quality of learning: Dispositions, instruction, and learning processes (pp. 276–314). Cambridge, England: Cambridge University Press.

Chen

Wang

Kirschner

P. A.

Tsai

C. C.

(2018). The role of collaboration, computer use, learning environments, and supporting strategies in CSCL: A meta-analysis. Review of Educational Research, 88, 799–843.

Chung

Leet

Liut

(2013). Investigating F2F peer interaction patterns in collaborative web discovery tasks. Journal of Computer Assisted Learning, 29, 188–206.

Coiro

(2011). Predicting reading comprehension on the Internet: Contributions of offline reading skills, online reading skills, and prior knowledge. Journal of Literacy Research, 43, 352–392.

Coiro

Coscarelli

Maykel

Forzani

(2015). Investigating criteria seventh graders use to evaluate the quality of online information. Journal of Adolescent and Adult Literacy, 58, 546–550.

Coiro

Sparks

J. R.

Kiili

Castek

J. M.

(2018, April 13). NAEP SAIL virtual world for assessing collaborative inquiry and deliberation. In Sparks

(Chair), Developing innovative assessments of 21st century skills: Online inquiry and collaboration. Symposium presented at the annual meeting of the American Educational Research Association, New York, NY.

Coiro

Sparks

Kulikowich

(2018). Assessing online collaborative inquiry and social deliberation as learners navigate multiple sources and perspectives. In Braten

Braasch

McCrudden

(Eds.), Handbook of multiple source use (pp. 485–501). London, England: Routledge.

Deane

Sparks

J. R

. (2019). Scenario-based formative assessment of key practices in the English language arts. In Andrade

Cizek

Bennett

R. E.

(Eds.), Handbook of formative assessment in the disciplines (pp. 68–96). New York, NY: Routledge.

10.

Fearon

J. D

. (1998). Deliberation as discussion. In Elster

(Ed.), Deliberative democracy (pp. 44–68). Cambridge, UK: Cambridge University Press.

11.

Goldman

S. R.

Lawless

Manning

(2013). Research and development of multiple source comprehension assessment. In Britt

M. A.

Goldman

S. R.

Rouet

J. F.

(Eds.), Reading from words to multiple texts (pp. 180–199). New York, NY: Routledge.

12.

Griffin

Care

(Eds.) (2015). Assessment and teaching of 21st century skills: Methods and approach. New York, NY: Springer.

13.

Häkkinen

Mäkitalo-Siegl

(2007). Educational perspectives on scripting CSCL. In Fischer

Kollar

Mandl

Haake

J. M.

(Eds.), Scripting computer-supported collaborative learning: Cognitive, computational, and educational approaches (pp. 263–271). Boston, MA: Springer.

14.

Hao

Liu

Von Davier

Kyllonen

Kitchen

(2016). Collaborative problem solving skills versus collaboration outcomes: Findings from statistical analysis and data mining. Proceedings of the 9th International Conference on Educational Data Mining, Raleigh, NC.

15.

Kreijns

Kirschner

P. A.

Jochems

(2003). Identifying the pitfalls for social collaboration in computer-supported collaborative learning environments: A review of the research. Computers in Human Behavior, 19, 335–353.

16.

Leu

D. J.

Jr. Kinzer

C. K.

Coiro

Cammack

(2004). Toward a theory of new literacies emerging from the Internet and other ICT. In Ruddell

R. B.

Unrau

(Eds.), Theoretical models and processes of reading (5th ed., pp. 1568–1611). Newark, DE: International Reading Association.

17.

Leu

D. J.

Jr. Kinzer

C. K.

Coiro

Castek

Henry

L. A.

(2013). New literacies: A dual-level theory of the changing nature of literacy, instruction, and assessment. In Ruddell

R. B.

Alvermann

(Eds.), Theoretical models and processes of reading (6th ed., pp. 1150–1181). Newark, DE: International Reading Association.

18.

Liu

Hmelo-Silver

C. E.

(2010). Conceptual representation embodied in hypermedia: An approach to promoting knowledge co-construction. In Khine

M. S.

Saleh

I. M.

(Eds.). New science of learning: Cognition, computers and collaboration in education (pp. 341–356). New York, NY: Springer.

19.

Mislevy

R. J.

Steinberg

L. S.

Almond

R. G.

(2003). On the structure of educational assessments. Measurement: Interdisciplinary Research and Perspectives, 1, 3–67.

20.

Murray

(2013). Toward defining, justifying, measuring, and supporting social deliberative skills. Retrieved from http://socialdeliberativeskills.org/documents/AIED2013-MetacogWorkshopMurraySubm.pdf

21.

Murray

Woolf

B. P.

(2013). An exploration of text analysis methods to identify social deliberative skill. In Lane

H. C.

Yacef

Mostow

Pavlik

(Eds.), Artificial intelligence in education. AIED 2013. lecture notes in computer science (Vol. 7926). Berlin, Germany: Springer. Retrieved from https://link.springer.com/chapter/10.1007/978-3-642-39112-5_119

22.

Organisation for Economic Co-operation and Development. (2017). PISA 2015 collaborative problem solving framework. Paris, France: OECD Publishing. Retrieved from www.oecd.org/pisa/pisaproducts/Draft%20PISA%202015%20Collaborative%20Problem%20Solving%20Framework%20.pdf

23.

Rouet

J.-F.

Britt

M. A.

(2011). Relevance processes in multiple document comprehension. In McCrudden

M. T.

Magliano

J. P.

Schraw

(Eds.), Relevance instructions and goal-focusing in text learning (pp. 19–52). Greenwich, CT: Information Age Publishing.

24.

Salomon

Perkins

D. N.

(1998). Individual and social aspects of learning. Review of Research in Education, 23, 1–24.

25.

Schwartz

D. L.

(1995). The emergence of abstract representations in dyad problem solving. The Journal of the Learning Sciences, 4, 321–354.

26.

Schwarz

B. B.

Asterhan

(2010). Argumentation and reasoning. In Littleton

Wood

C. P.

Kleine Staarman

(Eds.), International handbook of psychology in education (pp. 137–176). Bingley, England: Emerald Group.

27.

Strauss

Corbin

(1998). Basics of qualitative research (2nd ed.). Newbury Park, CA: Sage.

28.

Verbeek-Cowart

(2016, October 19). The nature of collaboration in the digital age. Textile Society of America Symposium Proceedings. Retrieved from http://digitalcommons.unl.edu/tsaconf/991

29.

Yen

M. H.

Wang

C. Y.

Chang

W. H.

Chen

Hsu

Y. S.

Liu

T. C.

(2018). Assessing metacognitive components in self-regulated reading of science texts in e-based environments. International Journal of Science and Mathematics Education, 16, 797–816.

30.

Zhang

Quintana

(2012). Scaffolding strategies for supporting middle school students’ online inquiry processes. Computers & Education, 58, 181–196.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.17 MB

0.13 MB

0.21 MB

0.20 MB

0.01 MB