Sage Journals: Discover world-class research

Abstract

Challenges arise when developing a computer-based Intelligent Team Tutoring System (ITTS) that attempts to deliver feedback to teams as effectively as a human tutor. The purpose of this current work is to outline elements of feedback that should be considered when designing feedback for an ITTS. The authors present the results of a study that consisted of 32 participants grouped into 16 teams of two. Each team conducted a surveillance task where they received individual or team feedback. Feedback content was written using either the bald (direct feedback; no need for interpretation) or off-record (general feedback; interpretation needed) etiquette strategy. The results showed that feedback delivered using the bald etiquette strategy positively correlated with improved performance. The results also showed that team level feedback positively correlated with more accurate self-assessment among participants. This suggests that in an ITTS, direct feedback can lead to better performance, and that feedback provided at the team level can help to align self-interpretation of performance with actual task performance.

Get full access to this article

View all access options for this article.

References

Bonner

Gilbert

Winer

Dorneich

MacAllister

Kohl

… Sinatra

A. M.

(2017). Military Team Training Utilizing GIFT. In Interservice/Industry Training, Simulation, and Education Conference (I/ITSEC). Orlando, Florida.

Bonner

Walton

Dorneich

M. C.

Gilbert

S. B.

Sottilare

R. A.

(2015). The Development of a Testbed to Assess an Intelligent Tutoring System for Teams. In Proceedings of the Workshops at the 17th International Conference on Artificial Intelligence in Education (pp. 31-37).

Borwn

Levinson

S. C.

(1987). Universals in language usage: Politeness phenomena. In Goody

E. N.

(Ed.), Questions and politeness: strategies in social interaction (pp. 311–323). Cambridge University Press.

Bradáč

Kostolányová

(2016). Intelligent Tutoring Systems. In Vincenti

Bucciero

Helfert

Glowatz

(Eds.), E-Learning, E-Education, and Online Training (Vol. 180, p. pp 71-78). Cham: Springer International Publishing.

DeShon

R. P.

Kozlowski

S. W. J.

Schmidt

A. M.

Milner

K. R.

Wiechmann

(2004). A Multiple-Goal, Multilevel Model of Feedback Effects on the Regulation of Individual and Team Performance. The Journal of Applied Psychology, 89(6), 1035–56.

Gamper

Knapp

(2002). A Review of Intelligent CALL Systems. Computer Assisted Language Learning, 15(5), 329–342.

Gilbert

S. B.

Slavina

Dorneich

M. C.

Sinatra

A. M.

Bonner

Johnston

… Winer

(2018). Creating a Team Tutor Using GIFT. International Journal of Artificial Intelligence in Education, 28(2), 286–313.

Greenwald

S. W.

Kulik

Kunert

Beck

Fröhlich

Cobb

… Maes

(2017). Technology and Applications for Collaborative Learning in Virtual Reality. In Smith

B. K.

Borge

Mercier

Lim

K. Y.

(Eds.), Making a Difference: Prioritizing Equity and Access in CSCL, CSCL 2017, Volume 2. Philadelphia, PA: International Society of the Learning Sciences.

Hattie

Timperley

(2007). The Power of Feedback. Review of Educational Research, 77(1), 81–112.

10.

Ilgen

D. R.

Fisher

C. D.

Taylor

M. S.

(1979). Consequences of Individual Feedback on Behavior in Organizations. Journal of Applied Psychology, 64(4), 349–371.

11.

Kibble

J. D.

Bellew

Asmar

Barkley

(2016). Team-based learning in large enrollment classes. Advances in Physiology Education, 40(4), 435–442.

12.

Loughry

M. L.

Tosi

H. L.

(2008). Performance Implications of Peer Monitoring. Organization Science, 19(6), 876–890.

13.

Meyer

Miller

Hancock

de Visser

E. J.

Dorneich

(2016). Politeness in Machine-Human and Human-Human Interaction. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 60(1), 279–283.

14.

Phillips

Zimmermann

Bird

(2014). Understanding and improving software build teams. In Proceedings of the 36th International Conference on Software Engineering - ICSE 2014 (pp. 735–744). New York, New York, USA: ACM Press.

15.

Shute

V. J.

Psotka

(1996). Intelligent tutoring systems: Past, Present, and Future. In Jonasse

(Ed.), Handbook of Research on Educational Communications and Technology. Scholastic Publications.

16.

Sottilare

Brawner

Goldberg

Holden

H. K.

(2012). The Generalized Intelligent Framework for Tutoring (GIFT).

17.

von Muhlenen

Rempel

M. I.

Enns

J. T.

(2005). Unique Temporal Change Is the Key to Attentional Capture. Psychological Science, 16(12), 979–986.

18.

Wobbrock

J. O.

Findlater

Gergle

Higgins

J. J.

(2011). The aligned rank transform for nonparametric factorial analyses using only anova procedures. In Proceedings of the 2011 annual conference on Human factors in computing systems - CHI ’11 (p. 143). New York, New York, USA: ACM Press.

19.

Yang

Dorneich

M. C.

(2016). Evaluation of Etiquette Strategies to Adapt Feedback In Affect-Aware Tutoring. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 60(1), 393–397.

20.

Zachary

Cannon-Bowers

Bilazarian

Krecker

Lardieri

Burns

(1999). The Advanced Embedded Training System (AETS): An Intelligent Embedded Tutoring System for Tactical Team Training. International Journal of Artificial Intelligence in Education, 257–277.

Feedback Design Considerations for Intelligent Team Tutoring Systems

Abstract

Get full access to this article

References