Team Coordination Dynamics Measurement in Enroute Care Training: Defining Requirements and Usability Study

Abstract

Prior research indicates a need for objective and reliable measures of team communication and coordination. This need rings true for United States Air Force (USAF) teams, whose instructors heavily employ subjective evaluations within fast-paced training. Instructors must observe a multitude of teams and their communications, leaving room for varied evaluations and missed opportunities for feedback. Recent studies have advanced and tested team coordination measures based on dynamical systems theory, yielding results that illustrate the relationship between team communication flow and performance. This work leverages those measures and applies them in the context of USAF medical training. In this study, we assess the usability of a Team Dynamics Measurement System prototype. Users (n = 4) completed 15 tasks and evaluated system usability, perceived mental effort, system satisfaction, and task difficulty. Results indicate marginally acceptable overall system usability. These results helped identify essential interface modifications for future iterations. Future work and use cases for instructor support are discussed.

Get full access to this article

View all access options for this article.

References

Agency for Healthcare Research and Quality. (2019, June). TeamSTEPPS 2.0 Fundamentals. Agency for Healthcare Research and Quality. https://www.ahrq.gov/teamstepps/instructor/fundamentals/index.html Air Force Medical Service (n.d.). The Vision of Trusted Care. Air Force Medicine Home. https://www.airforcemedicine.af.mil/Priorities/

Albert

Dixon

2003. Is this what you expected? The use of expectation measures in usability testing. Paper presented at the Usability Professionals Association Annual Conference, UPA, Scottsdale, AZ.

Awad

S. S.

Fagan

S. P.

Bellows

Albo

Green-Rashad

De La Garza

Berger

D. H.

(2005). Bridging the communication gap in the operating room with medical team training. The American Journal of Surgery, 190(5), 770-774.

Bangor

Kortum

P.T.

Miller

J.T.

2008. An empirical evaluation of the System Usability Scale. Int. J. Hum.- Comput. Interact. 24, 574–594.

Bekkink

M. O.

Farrell

S. E.

Takayesu

J. K.

(2018). Interprofessional communication in the emergency department: residents’ perceptions and implications for medical education. International Journal of Medical Education, 9, 262.

Brindley

P. G.

Reynolds

S. F.

(2011). Improving verbal communication in critical care medicine. Journal of critical care, 26(2), 155-159.

Brooke

, 1996. SUS: A ‘quick and dirty’ usability scale. In: Jordan

Thomas

Weerdmeester

(Eds.), Usability Evaluation in Industry. Taylor & Francis, London, UK, pp. 189–194.

Cooke

N. J.

Gorman

J. C.

Winner

J. L.

(2007). Team cognition. In Durso

F. T.

Nickerson

R. S.

Dumais

S. T.

Lewandowsky

Perfect

T. J.

(Eds.), Handbook of applied cognition (pp. 239–268). John Wiley & Sons, Inc. https://doi.org/10.1002/9780470713181.ch10

Cover

T. M.

Thomas

J. A.

(2006). Elements of information theory (2nd ed.). Hoboken, NJ: John Wiley.

10.

Finstad

, 2006. The System Usability Scale and non-native English speakers. J. Usability Stud. 1 (4), 185–188.

11.

Flin

O’Connor

Crichton

(2017). Safety at the sharp end: a guide to non-technical skills. CRC Press.

12.

Foronda

MacWilliams

McArthur

(2016). Interprofessional communication in healthcare: An integrative review. Nurse Education in Practice, 19, 36-40.

13.

Gorman

J. C.

Cooke

N. J.

Amazeen

P. G.

Fouse

(2012). Measuring patters in team interaction sequences using a discrete recurrent approach. Human Factors, 54(4), 503-517.

14.

Gorman

J. C.

Demir

Cooke

N. J.

Grimm

D. A.

(2019). Evaluating sociotechnical dynamics in a simulated remotely-piloted aircraft system: a layered dynamics approach. Ergonomics, 62(5), 629-643.

15.

Gorman

J. C.

Grimm

D. A.

Stevens

R. H.

Galloway

Willemsen-Dunlap

A. M.

Halpin

D. J.

(2020). Measuring real-time team cognition during team training. Human factors, 62(5), 825-860.

16.

Gorman

J. C.

Hessler

E. E.

Amazeen

P. G.

Cooke

N. J.

Shope

S. M.

(2012). Dynamical analysis in real time: Detecting perturbations to team communication. Ergonomics, 55(8), 825-839.

17.

Grimm

D. A.

Gorman

J. C.

Stevens

R. H.

Galloway

T. L.

Willemsen-Dunlap

A. M.

Halpin

D. J.

(2017). Demonstration of a method for real-time detection of anomalies in team communication. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 61(1), 282-286.

18.

Havyer

R. D.

Nelson

D. R.

Wingo

M. T.

Comfere

N. I.

Halvorsen

A. J.

McDonald

F. S.

Reed

D. A.

(2016). Addressing the Interprofessional Collaboration Competencies of the Association of American Medical Colleges: a systematic review of assessment instruments in undergraduate medical education. Academic Medicine, 91(6), 865-888.

19.

Interprofessional Education Collaborative Expert Panel. (2011). Core competencies for interprofessional collaborative practice: Report of an expert panel. Washington, D.C.: Interprofessional Education Collaborative.

20.

King

H. B.

Battles

Baker

D. P.

Alonso

Salas

Webster

Toomey

Salisbury

(2008). TeamSTEPPS™: team strategies and tools to enhance performance and patient safety. Advances in Patient Safety: new directions and alternative approaches (Vol. 3: performance and tools).

21.

Lewis

J.R.

1995. IBM computer usability satisfaction questionnaires: psychometric evaluation and instructions for use. Int. J. Hum.-Comput. Interact. 7, 57–78.

22.

Nelson

White

C. F.

Hodges

B. D.

Tassone

(2017). Interprofessional team training at the prelicensure level: a review of the literature. Academic Medicine, 92(5), 709-716.

23.

O’Connor

Keogh

Azuara-Blanco

(2015). Teamwork and communication. In: Flin

Youngson

Yule

(Eds.) Enhancing Surgical Skills: A Primer in Non-Technical Skills (pp. 105-122). London, England: CRC Press.

24.

Roze des Ordons

A. L.

Doig

C. J.

Couillard

Lord

(2017). From communication skills to skillful communication: a longitudinal integrated curriculum for critical care medicine fellows. Academic Medicine, 92(4), 501-505.

25.

Shannon

Weaver

(1949). The mathematical theory of communication. Urbana: University of Illinois Press. doi: 10.1002/j.1538-7305.1948.tb01338.x

26.

Tannenbaum

Salas

(2020). Teams that work: the seven drivers of team effectiveness. Oxford University Press.

27.

Tedesco

D.P.

Tullis

T.S.

2006. A comparison of methods for eliciting post- task subjective ratings in usability testing. Paper presented at the Usability Professionals Association Annual Conference, UPA, Broomfield, CO.

28.

Varpio

Hall

Lingard

Schryer

C. F.

(2008). Interprofessional communication and medical error: a reframing of research questions and approaches. Academic Medicine, 83(10), S76-S81.

29.

Zijlstra

van Doorn

1985. The construction of a scale to measure subjective effort (Tech. Rep.). Delft, Netherlands: Delft University of Technology, Department of Philosophy and Social Sciences.