To review common qualitative and quantitative methods of measuring shared mental models appropriate for use in the healthcare setting.
Background
Shared mental models are the overlap of individuals’ set of knowledge and/or assumptions that act as the basis for understanding and decision making between individuals. Within healthcare, shared mental models facilitate effective teamwork and theorized to influence clinical decision making and performance. With the current rapid growth and expansion of healthcare teams, it is critical that we understand and correctly use shared mental model measurement methods assess optimal team performance. Unfortunately, agreement on the proper measurement of shared mental models within healthcare remains diffuse.
Method
This paper presents methods appropriate to measure shared mental models within healthcare.
Results
Multiple shared mental model measurement methods are discussed with regard to their utility within this setting, ease of use, and difficulties in deploying within the healthcare operational environment. For rigorous analysis of shared mental models, it is recommended that a combination of qualitative and quantitative analyses be employed.
Conclusion
There are multitude of shared mental model measurement methods that can be used in the healthcare domain; although there is no perfect solution for every situation. Researchers can utilize this article to determine the best approach for their needs.
Johnson-LairdPN.A computational analysis of consciousness. Cogn Brain Theory1983;
6: 499–508.
2.
DayEAArthurWJrGettmanD.Knowledge structures and the acquisition of a complex skill. J Appl Psychol2001;
86: 1022.
3.
MathieuJEHeffnerTSGoodwinGFet al.
The influence of shared mental models on team process and performance. J Appl Psychol2000;
85: 273–283.
4.
KlimoskiRMohammedS.Team mental model: Construct or metaphor?J Manage1994;
20: 403–437.
5.
BurtscherMJManserT.Team mental models and their potential to improve teamwork and safety: A review and implications for future research in healthcare. Safety Sci2012;
50: 1344–1354.
6.
MohammedSKlimoskiRRentschJR.The measurement of team mental models: We have no shared schema. Organ Res Methods2000;
3: 123–165.
DeChurchLAMesmer-MagnusJR.Measuring shared team mental models: A meta-analysis. Group Dyn: Theory Res Pract2010;
14: 1–14..
9.
KozlowskiSWKleinKJ. A multilevel approach to theory and research in organizations: Contextual, temporal, and emergent processes. In: Klein KJ and Kozlowski SWJ (eds) Multilevel theory, research, and methods in organizations: Foundations, extensions, and new directions. San Francisco, CA: Jossey-Bass, pp. 3–90.
10.
GeiselmanREFisherRP.Interviewing witnesses and victims. In: Investigative interviewing: Handbook of best practices.
Toronto, ON:
Thomson Reuters Publishers, 2014; 29–53.
11.
WillisGBArtinoARJr. What do our respondents think we’re asking? Using cognitive interviewing to Improve Medical Education surveys. J Grad Med Educ2013; 353–356.
12.
KöhnkenGMilneRMemonAet al.
The cognitive interview: A meta-analysis. Psychol Crime Law1999;
5: 3–27.
13.
ChernyakNErnstingCIcksA.Pre-test of questions on health-related resource use and expenditure, using behaviour coding and cognitive interviewing techniques. BMC Health Serv Res2012;
12: 303.
14.
DiBenedettiDBPriceMAAndrewsEB.Cognitive interviewing in risk minimization survey development: Patient and healthcare professional surveys. Expert Rev Clin Pharmacol2013;
16: 369–373.
15.
DrennanJ.Cognitive interviewing: Verbal data in the design and pretesting of questionnaires. J Adv Nurs2003;
142: 57–63.
16.
IrwinDEVarniJWYeattsKet al.
Cognitive interviewing methodology in the development of a pediatric item bank: A patient reported outcomes measurement information system (PROMIS) study. Health Qual Life Outcomes2009;
7: 3.
17.
BlackNHVernettiBJ.Root-cause analysis: Creating and utilizing a functional database. Professional Safety2015;
60: 60–62.
18.
MoonBMillerA. Exploring macrocognitive healthcare work: Discovering seeds for design guidelines for clinical decision support. In: Proceedings of the 5th international conference on applied human factors and ergonomics AHFE 2014, Kraków, Poland, 19–23 July 2014.
19.
BeattyPCWillisGB.Research synthesis: The practice of cognitive interviewing. Public Opin Quart2007;
1: 287–311.
20.
WillisGBDeMaioTJHarris-KojetinB.Is the bandwagon headed to the methodological promised land? Evaluating the validity of cognitive interviewing techniques. In: SirkenMGHerrmannDJSchechterSet al. (eds) Cognition and survey research.
New York:
John Wiley, 1999, pp. 133–153.
21.
BoltonRNBronkhorstTM.Questionnaire pretesting: Computer-assisted coding of concurrent protocols. In: SchwarzNSudmanS (eds) Answering questions: Methodology for determining cognitive and communicative processes in survey research.
San Francisco, CA:
Jossey-Bass, 1996, pp. 37–64.
22.
WillisGB.Cognitive interviewing: A tool for improving questionnaire design.
London:
Sage Publications, 2004.
23.
GrantDABergE.A behavioral analysis of degree of reinforcement and ease of shifting to new responses in a Weigl-type card-sorting problem. J Exp Psychol1948;
38: 404.
WoodJRWoodLE.Card sorting: Current practices and beyond. J Usab Stud2008;
14: 1–6.
26.
EvansJMRoss BakerG.Shared mental models of integrated care: Aligning multiple stakeholder perspectives. J Health Organ Manag2012;
26: 713–736.
27.
McCombSSimpsonV.The concept of shared mental models in healthcare collaboration. J Adv Nurs2014;
70: 1479–1488.
28.
HysongSJBestRGPughJAet al.
Not of one mind: Mental models of clinical practice guidelines in the Veterans Health Administration. Health Serv Res2005;
40: 829–848.
29.
McCombSALemasterMHennemanEAet al.
An evaluation of shared mental models and mutual trust on general medical units: Implications for collaboration, teamwork, and patient safety. J Patient Saf2017;
13: 237–242.
30.
Nakarada-KordicIWellerJMWebsterCSet al.
Assessing the similarity of mental models of operating room team members and implications for patient safety: A prospective, replicated study. BMC Med Educ2016;
16: 229.
31.
FreerYFergusonLEwingGet al.
Mismatched concepts in a neonatal intensive care unit (NICU): Further issues for computer decision support?J Clin Monitor Comput2002;
17: 441–447.
NovakJD.An alternative to Piagetian psychology for science and mathematics education. Sci Educ1977;
61: 453–477.
34.
NovakJDCañasAJ. The theory underlying concept maps and how to construct and use them. Technical Report IHMC CmapTools, Florida Institute for Human and Machine Cognition, 20008.
35.
DelugachHSEtzkornLHCarpenterSet al.
A knowledge capture approach for directly acquiring team mental models. Int J Hum-Comput Stud2016;
96: 12–21.
GulRBBomanJA.Concept mapping: A strategy for teaching and evaluation in nursing education. Nurse Educ Pract2006;
6: 199–206.
38.
Langan‐FoxJAnglimJWilsonJR.Mental models, team mental models, and performance: Process, development, and future directions. Hum Factors Ergon Manuf Serv Ind2004;
14: 331–352.
39.
LinHMWilliamsonJM.A simple approach for sample size calculation for comparing two concordance correlation coefficients estimated on the same subjects. J Biopharm Stat2015;
25: 1145–1160.
40.
PouchotJKheraniRBBrantRet al.
Determination of the minimal clinically important difference for seven fatigue measures in rheumatoid arthritis. J Clin Epidemiol2008;
61: 705–113.
41.
QianWWangZJLiKW.Medical waste disposal method selection based on a hierarchical decision model with intuitionistic fuzzy relations. Int J Environ Res Public Health2016;
13: 896.
42.
TractenbergREGushtaMMWeinfeldJM.The mastery rubric for evidence-based medicine: Institutional validation via multidimensional scaling. Teach Learn Med2016;
28: 152–165.
43.
LongtinYSaxHLeapeLLet al.
Patient participation: Current knowledge and applicability to patient safety. Mayo Clin Proc2010;
85: 53–62.
44.
CaseiroNTrigoP. Distance ratio: An exploratory application to compare complex networks. In: Proceedings of the European conference on complex systems, 2012, pp. 123–130). Cham: Springer.
45.
SchaffernichtMGroesserSN.A comprehensive method for comparing mental models of dynamic systems. Eur J Oper Res2011;
210: 57–67.
46.
RossSAllenN.Examining the convergent validity of shared mental model measures. Behav Res Methods2012;
44: 1052–1062.
ButtsCT.Revisiting the foundations of network analysis. Science2009;
325: 414–416.
49.
NewmanME.The structure and function of complex networks. SIAM Rev2003;
45: 167–256.
50.
DorogovtsevSNMendesJF.Language as an evolving word web. Proc R Soc Lond B: Biol Sci2001;
268: 2603–2606.
51.
RubinovMSpornsO.Complex network measures of brain connectivity: Uses and interpretations. Neuroimage2010;
52: 1059–1069.
52.
PinkleyRLGelfandMJDuanL.When, where and how: The use of multidimensional scaling methods in the study of negotiation and social conflict. Int Negot2005;
10: 79–96.
53.
GelfandMJTriandisHCChanDK.Individualism versus collectivism or versus authoritarianism?Eur J Soc Psychol1996;
26: 397–410.
54.
ChahalHKumariN.Development of multidimensional scale for healthcare service quality (HCSQ) in Indian context. J Indian Business Res2010;
2: 230–255.
55.
TrochimWKaneM.Concept mapping: An introduction to structured conceptualization in health care. Int J Qual Health Care2005;
17: 187–191.
56.
SegalowitzNSDoucerainMMMeuterRFet al.
Comprehending adverbs of doubt and certainty in health communication: A multidimensional scaling approach. Front Psychol2016;
7: 558.
57.
StoutRJCannon-BowersJASalasEet al.
Planning, shared mental models, and coordinated performance: An empirical link is established. Hum Factors1999;
41: 61–71.
58.
DavisMACurtisMBTschetterJD.Evaluating cognitive training outcomes: Validity and utility of structural knowledge assessment. J Business Psychol2003;
18: 191–206.
59.
LimBCKleinKJ.Team mental models and team performance: A field study of the effects of team mental model similarity and accuracy. J Organ Behav. 2006;
27: 403–418.
Kraiger K and Wenzel LH. Conceptual development and empirical evaluation of measures of shared mental models as indicators of team effectiveness. In: Brannick MT, Salas E and Prince CW (eds) Team performance assessment and measurement: Theory, methods, and applications. 1997.
62.
RyeJARubbaPA.An exploration of the concept map as an interview tool to facilitate the externalization of students’ understandings about global atmospheric change. Journal of Research in Science Teaching: The Official Journal of the National Association for Research in Science Teaching1998;
35(5): 521–546.
63.
GillanDJBreedinSDCookeNJ.Network and multidimensional representations of the declarative knowledge of human-computer interface design experts. Int J Man-Mach Stud1992;
36: 587–615.
64.
McDonaldJEPaapKRMcDonaldDR. Hypertext perspectives: Using Pathfinder to build hypertext systems. In: Schvaneveldt RW (eds) Pathfinder associative networks: Studies in knowledge organization. Norwood, NJ: Ablex Publishing Corp., 1990, pp. 197–211.
65.
VortacOUEdwardsMBManningCA.Sequences of actions for individual and teams of air traffic controllers. Hum-Comput Interact1994; 1
9: 319–343.
66.
BurkeCSSalasEWilsonDKet al.
How to turn a team of experts into an expert medical team: guidance from the aviation and military communities. BMJ Qual Saf2004;
13: i96–104.
67.
HaigKMSuttonSWhittingtonJ.SBAR: A shared mental model for improving communication between clinicians. Jt Comm J Qual Patient Saf2006;
32: 167–175.
68.
Langfield-SmithKWirthA.Measuring differences between cognitive maps. J Oper Res Soc1992;
43: 1135–1150.
69.
MarkíczyLGoldbergJ.A method for eliciting and comparing causal maps. J Manage1995;
21: 305–333.
