This study examined how multiple factors, independent of research quality, influence the scientific impact of language assessment articles. A total of 447 papers published between 2018 and 2022 were investigated using a path analysis approach to identify both direct and indirect ways in which extrinsic factors affect citation counts. A conceptual path model was constructed based on existing literature and assessed using a comprehensive bibliometric dataset. The final model showed an excellent fit to the data (χ2 = 65.6, df = 52, p = .097; standardized root mean square residual [SRMR] = .034, root mean square error of approximation [RMSEA] = .024, comparative fit index [CFI] = .980, Tucker–Lewis index [TLI] = .970) and accounted for 57.75% of the variance in citation counts (R² = .5775, p < .001). Among nine examined factors, seven demonstrated significant direct or indirect effects on citation counts: number of self-citations, article age, journal CiteScore, number of countries, number of authors, open access, and research topic. Based on these findings, the study deepens our understanding of citation practices and offers broader implications for citation-based research evaluation in language assessment.
AbramoG.CiceroT.D’AngeloC. A. (2011). Assessing the varying level of impact measurement accuracy as a function of the citation window length. Journal of Informetrics, 5(4), 659–667. https://doi.org/10.1016/j.joi.2011.06.004
2.
AizaW. S. N.ShuibL.IdrisN.NormadhiN. B. A. (2024). Features, techniques and evaluation in predicting articles’ citations: A review from years 2010–2023. Scientometrics, 129(1), 1–29. https://doi.org/10.1007/s11192-023-04845-9
AksnesD. W.LangfeldtL.WoutersP. (2019). Citations, citation indicators, and research quality: An overview of basic concepts and theories. SAGE Open, 9(1). https://doi.org/10.1177/2158244019829575
5.
AksnesD. W.PiroF. N.FossumL. W. (2023). Citation metrics covary with researchers’ assessments of the quality of their works. Quantitative Science Studies, 4(1), 105–126. https://doi.org/10.1162/qss_a_00241
6.
AlthouseB. M.WestJ. D.BergstromC. T.BergstromT. (2009). Differences in impact factor across fields and over time. Journal of the American Society for Information Science and Technology, 60(1), 27–34. https://doi.org/10.1002/asi.20936
7.
Amini FarsaniM.JamaliH. R.BeikmohammadiM.GhorbaniB. D.SoleimaniL. (2021). Methodological orientations, academic citations, and scientific collaboration in applied linguistics: What do research synthesis and bibliometrics indicate?System, 100, 102547. https://doi.org/10.1016/j.system.2021.102547
AryadoustV.TanH. A. H.NgL. Y. (2019). A Scientometric review of Rasch measurement: The rise and progress of a specialty. Frontiers in Psychology, 10, Article 2197. https://doi.org/10.3389/fpsyg.2019.02197
10.
AryadoustV.ZakariaA.LimM. H.ChenC. (2020). An extensive knowledge mapping review of measurement and validity in language assessment and SLA research. Frontiers in Psychology, 11, Article 1941. https://doi.org/10.3389/fpsyg.2020.01941
11.
BaasJ.SchottenM.PlumeA.CôtéG.KarimiR. (2020). Scopus as a curated, high-quality bibliometric data source for academic research in quantitative science studies. Quantitative Science Studies, 1(1), 377–386. https://doi.org/10.1162/qss_a_00019
12.
BlondeelE.EveraertP. (2025). A practical guide to implementing ChatGPT as a secondary coder in qualitative research. International Journal of Accounting Information Systems, 56, 100754. https://doi.org/10.1016/j.accinf.2025.100754
13.
BordonsM.González-AlboB.Moreno-SolanoL. (2023). Improving our understanding of open access: How it relates to funding, internationality of research and scientific leadership. Scientometrics, 128, 4651–4676. https://doi.org/10.1007/s11192-023-04726-1
14.
BornmannL.SchierH.MarxW.DanielH. (2012). What factors determine citation counts of publications in chemistry besides their quality?Journal of Informetrics, 6, 11–18. https://doi.org/10.1016/j.joi.2011.08.004
15.
BrownT. A. (2015). Confirmatory factor analysis for applied research. Guilford publications.
16.
ChenC. (2017). Science mapping: A systematic review of the literature. Journal of Data and Information Science, 2(2), 1–40. https://doi.org/10.1515/jdis-2017-0006
17.
ChongS. W.PlonskyL. (2024). A typology of secondary research in Applied Linguistics. Applied Linguistics Review, 15(4), 1569–1594. https://doi.org/10.1515/applirev-2022-0189
DehghaniL.JahromiR. B.GanjooM. (2011). Citations to highly-cited researchers by their co-authors and their self-citations: How these factors affect highly-cited researchers.’H-Index in Scopus. Webology, 8(2), 91.
20.
DidegahF.ThelwallM. (2013). Which factors help authors produce the highest impact research? Collaboration, journal and document properties. Journal of Informetrics, 7(4), 861–873. https://doi.org/10.1016/j.joi.2013.08.006
21.
DongJ.ZhaoY.BuckinghamL. (2024). Thirty years of writing assessment: A bibliometric analysis of research trends and future directions. Assessing Writing, 61, 100862. https://doi.org/10.1016/j.asw.2024.100862
22.
DongM.GanC.ZhengY.YangR. (2022). Research trends and development patterns in Language Testing over the past three decades: A bibliometric study. Frontiers in Psychology, 13, 801604. https://doi.org/10.3389/fpsyg.2022.801604
23.
DonthuN.KumarS.MukherjeeD.PandeyN.LimW. M. (2021). How to conduct a bibliometric analysis: An overview and guidelines. Journal of Business Research, 133, 285–296. https://doi.org/10.1016/j.jbusres.2021.04.070
FranceschetM.CostantiniA. (2010). The effect of scholar collaboration on impact and quality of academic papers. Journal of Informetrics, 4(4), 540–553. https://doi.org/10.1016/j.joi.2010.06.003
27.
FuL.AliferisC. (2010). Using content-based and bibliometric features for machine learning models to predict citation counts in the biomedical literature. Scientometrics, 85(1), 257–270. https://doi.org/10.1007/s11192-010-0160-5
GroverV.RamanR.StubblefieldA. (2014). What affects citation counts in MIS research articles? An empirical investigation. Communications of the Association for Information Systems, 34(1), 74. https://doi.org/10.17705/1CAIS.03474
31.
HaslamN.BanL.KaufmannL.LoughnanS.PetersK.WhelanJ.WilsonS. (2008). What makes an article influential? Predicting impact in social and personality psychology. Scientometrics, 76(1), 169–185. https://doi.org/10.1007/s11192-007-1892-8
32.
HicksD.WoutersP.WaltmanL.de RijckeS.RafolsI. (2015). Bibliometrics: The Leiden Manifesto for research metrics. Nature, 520(7548), 429–431. https://doi.org/10.1038/520429a
33.
HuL. T.BentlerP. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6(1), 1–55. https://doi.org/10.1080/10705519909540118
34.
HylandK.JiangF. (2018). Changing patterns of self-citation: Cumulative inquiry or self-promotion?Text & Talk, 38(3), 365–387. https://doi.org/10.1515/text-2018-0004
35.
IoannidisJ. P. (2015). A generalized view of self-citation: Direct, co-author, collaborative, and coercive induced self-citation. Journal of Psychosomatic Research, 78(1), 7–11. https://doi.org/10.1016/j.jpsychores.2014.11.008
36.
IoannidisJ. P.ThombsB. D. (2019). A user’s guide to inflated and manipulated impact factors. European Journal of Clinical Investigation, 49(9), Article e13151. https://doi.org/10.1111/eci.13151
37.
IsaacsT.WinkeP. M. (2024). Purposeful turns for more equitable and transparent publishing in language testing and assessment. Language Testing, 41(1), 3–8. https://doi.org/10.1177/02655322231203234
38.
IsaacsT.YanX. (2025). Language Testing behind the scenes: New capabilities, groundbreaking developments, and disruptive forces. Language Testing, 42(1), 3–10. https://doi.org/10.1177/02655322241288490
39.
IsbellD. R.KremmelB. (2020). Test review: Current options in at-home language proficiency tests for making high-stakes decisions. Language Testing, 37(4), 600–619. https://doi.org/10.1177/0265532220943483
40.
KlineR. B. (2023). Principles and practice of structural equation modeling. Guilford publications.
41.
KosmulskiM. (2012). The order in the lists of authors in multi-author papers revisited. Journal of Informetrics, 6(4), 639–644. https://doi.org/10.1016/j.joi.2012.06.006
42.
KoushaK.ThelwallM. (2024). Factors associating with or predicting more cited or higher quality journal articles: An Annual Review of Information Science and Technology (ARIST) paper. Journal of the Association for Information Science and Technology, 75(3), 215–244. https://doi.org/10.1002/asi.24810
43.
KraussA.DanúsL.Sales-PardoM. (2023). Early-career factors largely determine the future impact of prominent researchers: Evidence across eight scientific fields. Scientific Reports, 13(1), 18794. https://doi.org/10.1038/s41598-023-46050-x
44.
KremmelB.IsbellD. R. (2024). Open Science practices in language assessment: Introducing the special issue. Language Testing, 41(4), 697–702. https://doi.org/10.1177/02655322241264092
45.
Langham-PutrowA.BakkerC.RiegelmanA. (2021). Is the open access citation advantage real? A systematic review of the citation of open access and subscription-based articles. PLOS ONE, 16(6), e0253129. https://doi.org/10.1371/journal.pone.0253129
46.
LarivièreV.GingrasY.SugimotoC. R.TsouA. (2015). Team size matters: Collaboration and scientific impact since 1900. Journal of the Association for Information Science and Technology, 66(7), 1323–1332. https://doi.org/10.1002/asi.23266
LiC. H. (2016). Confirmatory factor analysis with ordinal data: Comparing robust maximum likelihood and diagonally weighted least squares. Behavior Research Methods, 48(3), 936–949. https://doi.org/10.3758/s13428-015-0619-7
49.
LiskiewiczT.LiskiewiczG.PaczesnyJ. (2021). Factors affecting the citations of papers in tribology journals. Scientometrics, 126, 3321–3336. https://doi.org/10.1007/s11192-021-03870-w
50.
LiuM.Al-HoorieA. H.HiverP. (2024). Open access in language testing and assessment: The case of two flagship journals. Language Testing, 41(4), 703–728. https://doi.org/10.1177/02655322241261417
51.
LlerasC. (2005). Path analysis. Encyclopedia of Social Measurement, 3(1), 25–30.
52.
MammolaS.PianoE.DorettoA.CaprioE.ChamberlainD. (2022). Measuring the influence of non-scientific features on citations. Scientometrics, 127, 4123–4137. https://doi.org/10.1007/s11192-022-04421-7
53.
McNamaraT.RoeverC. (2006). Language testing: The social dimension. Blackwell Publishing.
54.
MertonR. K. (1968). The Matthew effect in science: The reward and communication systems of science are considered. Science, 159(3810), 56–63. https://doi.org/10.1126/science.159.3810.5
55.
MingersJ.XuF. (2010). The drivers of citations in management science journals. European Journal of Operational Research, 205(2), 422–430. https://doi.org/10.1016/j.ejor.2009.12.008
56.
MoherD.NaudetF.CristeaI. A.MiedemaF.IoannidisJ. P. A.GoodmanS. N. (2018). Assessing scientists for hiring, promotion, and tenure. PLOS Biology, 16(3), Article e2004089. https://doi.org/10.1371/journal.pbio.2004089
57.
MomeniF.DietzeS.MayrP.BiesenbenderK.PetersI. (2023). Which factors are associated with Open Access publishing? A Springer Nature case study. Quantitative Science Studies, 4(2), 353–371. https://doi.org/10.1162/qss_a_00253
58.
MukherjeeD.LimW. M.KumarS.DonthuN. (2022). Guidelines for advancing theory and practice through bibliometric research. Journal of Business Research, 148, 101–115. https://doi.org/10.1016/j.jbusres.2022.04.042
59.
NairL. B.GibbertM. (2016). What makes a ‘good’ title and (how) does it matter for citations? A review and general model of article title attributes in management science. Scientometrics, 107, 1331–1359. https://doi.org/10.1007/s11192-016-1937-y
60.
OnoderaN.YoshikaneF. (2015). Factors affecting citation rates of research articles. Journal of the Association for Information Science and Technology, 66(4), 739–764. https://doi.org/10.1002/asi.23209
61.
PadialA.NaboutJ.SiqueiraT.BiniL.Diniz-FilhoJ. (2010). Weak evidence for determinants of citation frequency in ecological articles. Scientometrics, 85, 1–12. https://doi.org/10.1007/s11192-010-0231-7
62.
PageM. J.McKenzieJ. E.BossuytP. M.BoutronI.HoffmannT. C.MulrowC. D.ShamseerL.TetzlaffJ. M.AklE. A.BrennanS. E.ChouR.GlanvilleJ.GrimshawJ. M.HróbjartssonA.LaluM. M.LiT.LoderE. W.Mayo-WilsonE.McDonaldS.McGuinnessL. A.MoherD. (2021). The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. British Medical Journal, 372, n71. https://doi.org/10.1136/bmj.n71
63.
PlonskyL.SudinaE. (2025). Self-citation practices in applied linguistics: A bibliometric review. Journal of Second Language Studies, 8(2), 264–280. https://doi.org/10.1075/jsls.00052.plo
64.
PuuskaH. M.MuhonenR.LeinoY. (2014). International and domestic co-publishing and their citation impact in different disciplines. Scientometrics, 98, 823–839. https://doi.org/10.1007/s11192-013-1181-7
65.
RatnerB. (2009). The correlation coefficient: Its values range between+ 1/− 1, or do they?Journal of Targeting, Measurement and Analysis for Marketing, 17(2), 139–142. https://doi.org/10.1057/jt.2009.5
66.
SætnanA. R.TøndelG.RasmussenB. (2019). Does counting change what is counted? Potential for paradigm change through performance metrics. Research Evaluation, 28(1), 73–83. https://doi.org/10.1093/reseval/rvy032
67.
ShenH. W.BarabásiA.-L. (2014). Collective credit allocation in science. Proceedings of the National Academy of Sciences of the United States of America, 111, 12325–12330. https://doi.org/10.1073/pnas.1401992111
68.
SjögårdeP.DidegahF. (2022). The association between topic growth and citation impact of research publications. Scientometrics, 127(4), 1903–1921. https://doi.org/10.1007/s11192-022-04293-x
69.
SzomszorM.PendleburyD. A.AdamsJ. (2020). How much is too much? The difference between research influence and self-citation excess. Scientometrics, 123(2), 1119–1147. https://doi.org/10.1007/s11192-020-03417-5
70.
TabachnickB. G.FidellL. S. (2019). Using multivariate statistics (7th ed.). Pearson.
71.
TahamtanI.BornmannL. (2022). The Social Systems Citation Theory (SSCT): A proposal to use the social systems theory for conceptualizing publications and their citations links. Profesional de la información, 31(4), e310411. https://doi.org/10.3145/epi.2022.jul.11
72.
TahamtanI.Safipour AfsharA.AhamdzadehK. (2016). Factors affecting number of citations: A comprehensive review of the literature. Scientometrics, 107(3), 1195–1225. https://doi.org/10.1007/s11192-016-1889-2
73.
Teixeira da SilvaJ. A. (2020). CiteScore: Advances, evolution, applications, and limitations. Publishing Research Quarterly, 36(3), 459–468. https://doi.org/10.1007/s12109-020-09736-y
74.
Teixeira da SilvaJ. A.MemonA. R. (2017). CiteScore: A cite for sore eyes, or a valuable, transparent metric?Scientometrics, 111, 553–556. https://doi.org/10.1007/s11192-017-2250-0
75.
TennantJ. P.WaldnerF.JacquesD. C.MasuzzoP.CollisterL. B.HartgerinkC. H. (2016). The academic, economic and societal impacts of Open Access: An evidence-based review. F1000research, 5. https://doi.org/632.10.12688/f1000research.8460.3
76.
ThelwallM.KoushaK.AbdoliM.StuartE.MakitaM.WilsonP.LevittJ. (2023). Why are coauthored academic articles more cited: Higher quality or larger audience?Journal of the Association for Information Science and Technology, 74(7), 791–810. https://doi.org/10.1002/asi.24755
77.
ThelwallM.WilsonP. (2014). Regression for citation data: An evaluation of different methods. Journal of Informetrics, 8(4), 963–971. https://doi.org/10.1016/j.joi.2014.09.011
78.
TsagariD.CsépesI.(Eds.). (2012). Collaboration in language testing and assessment. Peter Lang.
79.
Villa LarenasS. (2026). Towards inclusive publishing practices in language testing: Voices from the Global South. Language Testing, 43(1), 100–115.
WangX.ZuoJ.LiuF.SunZ. (2023). A scientometric review of research trends in language assessment literacy. Education Sciences, 13(2), 190. https://doi.org/10.3390/educsci13020190
85.
WolfE. J.HarringtonK. M.ClarkS. L.MillerM. W. (2013). Sample size requirements for structural equation models: An evaluation of power, bias, and solution propriety. Educational and Psychological Measurement, 73(6), 913–934. https://doi.org/10.1177/0013164413495237
86.
YangZ.WangP. (2025). Current status and research trend of English language assessment: A bibliometric analysis. Language Testing in Asia, 15(1), 11. https://doi.org/10.1186/s40468-024-00317-w
87.
ZhangX. (2025). Trends in language assessment and testing: A bibliometric study. Studies in Second Language Learning and Teaching, 15(1), 171–198. https://doi.org/10.14746/ssllt.25141
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
