Evaluating interdisciplinary research (IDR) teams is essential for improving productivity and outcomes. To contribute to the growing toolkit for evaluating IDR teamwork, we created three scales: IDR Gains, Principal Investigator (PI) Perceptions of Challenges, and Non-PI Perceptions of PIs—designed for use in academic settings. The scales were psychometrically tested with university faculty from various disciplines (n = 345) who were invited to apply for interdisciplinary seed-grants. Confirmatory factor analysis revealed good model fit, and all scales demonstrated high reliability (α ≥ .9) and convergent validity. These instruments offer valuable tools for assessing and enhancing researchers’ experiences with IDR, particularly in colleges/universities.
AjzenI.FishbeinM. (2005). The influence of attitudes on behavior. In AlbarracínD.JohnsonB. T.ZannaM. P. (Eds.), The handbook of attitudes (pp. 173–221). Lawrence Erlbaum Associates.
2.
AntesA. L.McIntoshT. J.Cargill SS.BrutonS.BaldwinK. (2023). Principal investigators’ priorities and perceived barriers and facilitators when making decisions about conducting essential research in the covid-19 pandemic. Science and Engineering Ethics, 29(2), Article 8. https://doi.org/10.1007/s11948-023-00430-8
BagozziR. P. (1981). Evaluating structural equation models with unobservable variables and measurement error: A comment. Journal of Marketing Research, 18(3), 375–381. https://doi.org/10.2307/3150979
5.
BakerD. P.AmodeoA. M.KrokosK. J.SlonimA.HerreraH. (2010). Assessing teamwork attitudes in healthcare: Development of the TeamSTEPPS teamwork attitudes questionnaire. Quality & Safety in Health Care, 19(6), e49. https://doi.org/10.1136/qshc.2009.036129
6.
BarkR. H.KragtM. E.RobsonB. J. (2016). Evaluating an interdisciplinary research project: Lessons learned for organizations, researchers and funders. International Journal of Project Management, 34(8), 1449–1459. https://doi.org/10.1016/j.ijproman.2016.08.004
7.
BellancaL. (2009). Measuring interdisciplinary research: Analysis of co-authorship for research staff at the University of York. Bioscience Horizons: The International Journal of Student Research, 2(2), 99–112. https://doi.org/10.1093/biohorizons/hzp012
8.
BanduraA. (1986). Social foundations of thought and action: A social cognitive theory. Prentice-Hall.
9.
BeyerleinM.GoodsonP.McLeroyK.PattersonM. S.FrenchP.GarrA.LuJ.MengX.SankarS.TaylorZ.WangW. (2020). Evaluation of the President’s Excellence Fund: T3 and X-Grants Initiative round 1. Texas A&M University.
10.
BolducS.KnoxJ.RistrophE. B. (2023). Evaluating team dynamics in interdisciplinary science teams. Higher Education Evaluation and Development, 17(2), 70–81. https://doi.org/10.1108/HEED-10-2021-0069
11.
BrownS. A.SparapaniR.OsinskiK.ZhangJ.BlessingJ.ChengF.HamidA.MohamadiPourM. B.LalJ. C.KothariA. N.CaraballoP.NoseworthyP.JohnsonR. H.HansenK.SunL. Y.CrottyB.ChengY. C.EchefuG.DoshiK.OlsonJ. (2023). Team principles for successful interdisciplinary research teams. American Heart Journal Plus: Cardiology Research and Practice, 32, 100306. https://doi.org/10.1016/j.ahjo.2023.100306
12.
ChenF.CurranP. J.BollenK. A.KirbyJ.PaxtonP. (2008). An empirical evaluation of the use of fixed cutoff points in RMSEA test statistic in structural equation models. Sociological Methods & Research, 36(4), 462–494. https://doi.org/10.1177/0049124108314720
13.
ClancyC. M.TornbergD. N. (2007). TeamSTEPPS: Assuring optimal teamwork in clinical settings. American Journal of Medical Quality, 22(3), 214–217. https://doi.org/10.1177/1062860607300616
14.
ColeM. S.BedeianA. G.BruchH. (2011). Linking leader behavior and leadership consensus to team performance: Integrating direct consensus and dispersion models of group composition. The Leadership Quarterly, 22(2), 383–398. https://doi.org/10.1016/j.leaqua.2011.02.012
15.
DeVellisR. F.CarolynT. T. (2021). Scale development: Theory and applications. Sage.
Falk-KrzesinskiH. J.ContractorN.FioreS. M.HallK. L.KaneC.KeytonJ.KleinJ. T.SpringB.StokolsD.TrochimW. (2011). Mapping a research agenda for the science of team science. Research Evaluation, 20(2), 145–158. https://doi.org/10.1111/j.1547-5069.2007.00161.x
18.
FornellC.LarckerD. F. (1981). Evaluating structural equation models with unobservable variables and measurement error. Journal of Marketing Research, 18(1), 39–50. https://doi.org/10.2307/3151312
19.
FortunatoS.BergstromC. T.BörnerK.EvansJ. A.HelbingD.MilojevićS.PetersenA. M.RadicchiF.SinatraR.UzziB.VespignaniA.WaltmanL.WangD.BarabásiA. L. (2018). Science of science. Science, 359(6379), eaao0185. https://doi.org/10.1126/science.aao0185
GreenleafR. K. (1988). The servant as leader. Robert K. Greenleaf Center.
22.
HallK. L.StokolsD.MoserR. P.TaylorB. K.ThornquistM. D.NebelingL. C.EhretC. C.BarnettM. J.McTiernanA.BergerN. A.GoranM. I.JefferyR. W. (2008). The collaboration readiness of transdisciplinary research teams and centers findings from the National Cancer Institute’s TREC Year-One evaluation study. American Journal of Preventive Medicine, 35(2 Suppl.), S161–172. https://doi.org/10.1016/j.amepre.2008.03.035
23.
HallK. L.VogelA. L.StipelmanB. A.StokolsD.MorganG.GehlertS. (2012). A four-phase model of transdisciplinary team-based research: Goals, team processes, and strategies. Translational Behavioral Medicine, 2(4), 415–430. https://doi.org/10.1007/s13142-012-0167-y
24.
HuL.BentlerP. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1–55. https://doi.org/10.1080/10705519909540118
25.
HuutoniemiK. (2012). Communicating and compromising on disciplinary expertise in the peer review of research proposals. Social Studies of Science, 42(6), 897–921. https://doi.org/10.1177/0306312712458478
26.
HuutoniemiK.KleinJ. T.BruunH.HukkinenJ. (2010). Analyzing interdisciplinarity: Typology and indicators. Research Policy, 39(1), 79–88. https://doi.org/10.1016/j.respol.2009.09.011
27.
HuutoniemiK. I.RafolsI. (2017). Interdisciplinarity in research evaluation. In FrodemanR.KleinJ. T.PachecoR. C. D. S. (Eds.), Oxford handbook of interdisciplinarity (2nd ed., pp. 498–512). Oxford University Press. https://doi.org/10.1093/oxfordhb/9780198733522.013.40
28.
KingH. B.BattlesJ.BakerD. P.AlonsoA.SalasE.WebsterJ.ToomeyL.SalisburyM. (2008). TeamSTEPPS™: Team strategies and tools to enhance performance and patient safety. In HenriksenK.BattlesJ. B.KeyesM. A.GradyM. L. (Eds.), Advances in patient safety: New directions and alternative approaches (Vol. 3: Performance and tools). Agency for Healthcare Research and Quality (US). https://www.ncbi.nlm.nih.gov/books/NBK43686/
29.
KleinJ. T. (2008). Evaluation of interdisciplinary and transdisciplinary research: A literature review. American Journal of Preventive Medicine, 35(2), S116–S123. https://doi.org/10.1016/j.amepre.2008.05.010
30.
LaursenB. K.MotzerN.AndersonK. J. (2022). Pathways for assessing interdisciplinarity: A systematic review. Research Evaluation, 31(3), 326–343. https://doi.org/10.1093/reseval/rvac013
31.
LedfordH. (2015, September16). How to solve the world’s biggest problems. Nature, 525(7569), 308–311. https://doi.org/10.1038/525308a
32.
MahringerC. A.BaesslerF.GerchenM. F.HaackC.JacobK.MayerS. (2023). Benefits and obstacles of interdisciplinary research: Insights from members of the Young Academy at the Heidelberg Academy of Sciences and Humanities. iScience, 26(12), 108508. https://doi.org/10.1016/j.isci.2023.108508
33.
MakrakisV.Kostoulas-MakrakisN. (2016). Interdisciplinary problem-based sustainability education: The case of the CLIMASP-Tempus Minor. In DavimJ. P.Leal FilhoW. (Eds.), Challenges in higher education for sustainability (pp. 15–33). Springer. https://doi.org/10.1007/978-3-319-23705-3_2
34.
MartinC. R.MartinC. J. H. (2017). Minimum sample size requirements for a validation study of the Birth Satisfaction Scale-Revised (BSS-R). Journal of Nursing and Practice, 1(2), 25–30. https://doi.org/10.36959/545/358
35.
MâsseL. C.MoserR. P.StokolsD.TaylorB. K.MarcusS. E.MorganG. D.HallK. L.CroyleR. T.TrochimW. M. (2008). Measuring collaboration and transdisciplinary integration in team science. American Journal of Preventive Medicine, 35(2 Suppl.), S151–S160. https://doi.org/10.1016/j.amepre.2008.05.020
36.
McLeishT.StrangV. (2016). Evaluating interdisciplinary research: The elephant in the peer-reviewers’ room. Palgrave Communications, 2(1), 1–8. https://doi.org/10.1057/palcomms.2016.55
37.
MilojevicS. (2014). Principles of scientific research team formation and evolution. Proceedings of the National Academy of Sciences, 111(11), 3984–3989. https://doi.org/10.1073/pnas.1309723111
38.
MisraS.StokolsD.ChengL. (2016). The transdisciplinary orientation scale: Factor structure and relation to the integrative quality and scope of scientific publications. Journal of Translational Medicine and Epidemiology, 3(2), 1042.
39.
Institute of Medicine and National Research Council. (2003). Large-scale biomedical science: Exploring strategies for future research. The National Academies Press. https://doi.org/10.17226/10718
40.
National Academy of Sciences, National Academy of Engineering, & Institute of Medicine of the National Academies. (2005). Facilitating interdisciplinary research. The National Academies Press. https://doi.org/10.17226/11153
41.
National Research Council. (2015). Enhancing the effectiveness of team science. The National Academies Press. https://doi.org/10.17226/19007.
42.
National Research Council. (2018). Collaborations of consequence: NAKFI’s 15 years igniting innovation at the intersections of disciplines. The National Academies Press. https://doi.org/10.17226/25239
NembhardI. M.EdmondsonA. C. (2006). Making it safe: The effects of leader inclusiveness and professional status on psychological safety and improvement efforts in health care teams. Journal of Organizational Behavior, 27(7), 941–966. https://doi.org/10.1002/job.413
45.
ProchaskaJ. O.DiClementeC. C. (1983). Stages and processes of self-change of smoking: Toward an integrative model of change. Journal of Consulting and Clinical Psychology, 51(3), 390–395. https://doi.org/10.1037/0022-006X.51.3.390
SchumackerR. E.LomaxR. G. (2004). A beginner’s guide to structural equation modeling. Psychology Press.
48.
SkinnerC. S.TiroJ. A.ChampionV. L. (2024). The health belief model. In GlanzK.RimerB. K.ViswanathK. (Eds.), Health behavior: Theory, research, and practice (6th ed., pp. 21–44). Jossey-Bass.
49.
SpaapenJ.DijstelbloemH.WamelinkF. (2007). Evaluating research in context: A method for comprehensive assessment (2nd ed.). Consultative Committee of Sector Councils for Research and Development (COS).
50.
StevensJ.P. (1992). Applied multivariate statistics for the social sciences(2nd ed.). Erlbaum.
TaberK. S. (2018). The use of Cronbach’s alpha when developing and reporting research instruments in science education. Research in Science Education, 48(6), 1273–1296. https://doi.org/10.1007/s11165-016-9602-2
54.
TavakolM.DennickR. (2011). Making sense of Cronbach’s alpha [Editorial]. International Journal of Medical Education, 2, 53–55. https://doi.org/10.5116/ijme.4dfb.8dfd
55.
TaylorC.BrownK.LambB.HarrisJ.SevdalisN.GreenJ. S. A. (2012). Developing and testing TEAM (Team Evaluation and Assessment Measure), a self-assessment tool to improve cancer multidisciplinary teamwork. Annals of Surgical Oncology, 19(13), 4019–4027. https://doi.org/10.1245/s10434-012-2493-1
56.
ThompsonB. (2004). Confirmatory factor analysis decision sequence. In ThompsonB. (Ed.), Exploratory and confirmatory factor analysis: Understanding concepts and applications (pp. 109–132). American Psychological Association. https://doi.org/10.1037/10694-010
57.
VabøA.AlvsvågA.KyvikS.ReymertI. (2016). The establishment of formal research groups in higher education institutions. Nordic Journal of Studies in Educational Policy, 2016(2–3), 33896. https://doi.org/10.3402/nstep.v2.33896
58.
VanneyC. E.MesuradoB.Aguinalde SáenzJ. I. (2024). Measuring qualities needed for interdisciplinary work: The Intellectual Virtues for Interdisciplinary Research Scale (IVIRS). PLoS One, 19(11), e0312938. https://doi.org/10.1371/journal.pone.0312938
59.
WangJ.AhnS.MorganS. E. (2022). Measuring the processes of interdisciplinary team collaboration: Creating valid measures using a many-facet Rasch model approach. Journal of Clinical and Translational Science, 6(1), e134. https://doi.org/10.1017/cts.2022.472
60.
WhiteM. (2022). Sample size in quantitative instrument validation studies: A systematic review of articles published in Scopus, 2021. Heliyon, 8(12), e12223. https://doi.org/10.1016/j.heliyon.2022.e12223
61.
WillisG. B. (2005). Analysis of the cognitive interview in questionnaire design. Oxford University Press.
WolfE. J.HarringtonK. M.ClarkS. L.MillerM. W. (2013). Sample size requirements for structural equation models: An evaluation of power, bias, and solution propriety. Educational and Psychological Measurement, 73(6), 913–934. https://doi.org/10.1177/0013164413495237
64.
WuL.WangD.EvansJ. A. (2019). Large teams develop and small teams disrupt science and technology. Nature, 566(7744), 378–382. https://doi.org/10.1038/s41586-019-0941-9
65.
ZengA.FanY.DiZ.WangY.HavlinS. (2021). Fresh teams are associated with original and multidisciplinary research. Nature Human Behaviour, 5(10), 1314–1322. https://doi.org/10.1038/s41562-021-01084-x
