Abstract
Background:
Open access (OA) publications are increasingly common in orthopaedic literature. However, whether OA publications are associated with increased readership or citations among total ankle arthroplasty (TAA) literature is unclear. We hypothesize that compared with non-OA status, OA status is associated with increased social media dissemination, and readership, but not with citation count. This study aimed to analyze social media attention, citations, readership, and cost of TAA OA and non-OA publications.
Methods:
Using a PubMed query search, there were 368 publications from 81 journals, with 25% (91/368) being OA articles and 75% (277/368) non-OA articles from 2016 to 2023. We analyzed the Altmetric Attention Score (AAS), Mendeley readership score, and citations between OA vs non-OA articles. Citations and cost of OA articles were determined using an altered timeline and publisher’s website, respectively. Subgroup analysis was performed among articles published in the top 5 TAA journals (Tables 2 and 3). Negative binomial regression was used while adjusting for days since publication. Significance was considered at P <.05.
Results:
OA publication was associated with a larger mean AAS score (8.7 ± 37.0 vs 4.8 ± 26.3), Mendeley readership (42.4 ± 41.6 vs 34.9 ± 25.7), and Twitter mentions (4.6 ± 7.4 vs 3.3 ± 8.1), but not citations (19.7 ± 24.8 vs 20.3 ± 23.5) (Table 1).
Conclusion:
TAA OA publications and top 5 journals were associated with significantly increased social media attention but not Mendeley readership or citation counts.
This is a visual representation of the abstract.
Introduction
The emergence of the “digital age” of academia has prompted both journals and authors to rethink their approach in engaging readers. According to the Web of Science database, OA publications grew by 14.5% from 1998 to 2018 alone. 16 These increases are also seen in orthopaedic OA journals such as BMC musculoskeletal disorders, which recorded record citations and article downloads from 2011 to 2020. 5 The surge of orthopaedic OA literature has prompted us to assess the benefit, if any, of publishing articles OA vs non-OA. Specifically, we chose to examine the impact of OA vs non-OA on total ankle arthroplasty (TAA), which has become an increasingly common procedure in foot and ankle surgery. 8 Evidently, TAA literature has increased and many advances have been made in the field over the past decade. 2 By understanding the impact of OA publishing on TAA literature, it provides many avenues for publishing authors to potentially gain higher citations and social media attention for the scientific impact they generate.
The benefits of submitting articles to OA journals are unique compared to those of non-OA journals. Consumers often benefit from easier access to submitted research, which increases visibility and impact. Additionally, open-access journals often grant authors the right to retain the copyrights of their work. In contrast, non-OA journals may in some cases be more well known or prestigious while retaining the copyrights to a submitted work. In the traditional model, high-impact non-OA journals charge fees to interested readers. In OA journals, the cost shifts to the author as a “publishing fee,” creating a barrier for some authors to publish in high-impact OA journals. 11 Collectively, each of these factors should be weighed when authors are deciding where to publish.
We hypothesize that in TAA literature, OA publications—compared to non-OA publications—will contribute a greater level of scientific impact with its potential association with increased social media dissemination, cost, and readership, but not citations. Delving into these four pillars of our study will provide a better sense of how OA publications can serve as a better medium for the scientific research community to invest their time. Therefore, this study aimed to assess the relationship, if any, between OA peer-reviewed publications and non-OA peer-reviewed publications in the TAA literature concerning social media attention, Mendeley readership, and citations. Additionally, we hope to analyze the cost associated with publishing articles OA and how that may influence the author’s choice.
Methods and Materials
Data Set
Initially, we performed a search via a PubMed query for peer-reviewed TAA literature ranging from January 1, 2016, to July 18, 2023. Throughout the detailed search, search terms included (Total Ankle Arthroplasty[Title/Abstract]) OR (total ankle arthroplasty[Title/Abstract])) OR (Total Ankle Replacement[Title/Abstract])) OR (total ankle replacement[Title/Abstract])) OR (Total Ankle Reconstruction[Title/Abstract])) OR (total ankle reconstruction[Title/Abstract])) OR (Ankle Arthroplasty[Title/Abstract])) OR (ankle arthroplasty[Title/Abstract])) OR (Ankle Replacement[Title/Abstract])) OR (ankle replacement[Title/Abstract])) OR (Ankle Reconstruction[Title/Abstract])) OR (ankle reconstruction[Title/Abstract])) OR (ankle prosthesis)) OR (Ankle Prosthesis). This search was confirmed for accuracy by a reference librarian. However, categories such as books, book chapters, clinical trial records, news stories, or data sets were not considered. Articles were manually filtered based on relevance and inclusive criteria. For each respective article, the bibliometric data aggregated included the article title, journal/collection title, publication date, OA status, Altmetric Attention Score (AAS), number of Mendeley readers, number of Dimensions citations, Twitter mentions, Facebook mentions, and news mentions.
Open Access (OA) status allows anyone to freely access online publications, data, and journals, and by using the Altmetric Explorer database, the OA status of all corresponding articles can be determined. The Altmetric database is a tool that allows researchers to assess social media dissemination of specific articles; it was necessary to use the database to collect the relevant data for all TAA literature. This database produces an Altmetric Attention Score, which measures the level of attention that each research article output produces based on social media attention volume, the source of attention, and how much the author mentions the article. 1 Through an automated algorithm that calculates the weighted count in various types of sources such as Facebook, Twitter, and news stories, it provides a visual of the reach of the journal to the rest of the research community. Additionally, the Mendeley readership showcases the number of unique individuals who have saved a copy or used the article in some form of their work. Furthermore, to study the references of publications, Altmetric relies on “Dimensions” to extract references from publications, grants, clinical trials, and many patents. In this specific study, there were a total of 368 publications analyzed with 25% (91/368) as OA articles and 75% (277/368) as non-OA articles from 2016 to 2023. Citation data were analyzed using an altered timeline between January 2016 and July 2021. See Appendix A for a full list of included journals.
The top 5 journals in the orthopaedic fields of “Arthroplasty” and “Foot and Ankle” were identified and analyzed in a subgroup analysis by using the h5-index in Google Scholar. 7 The h5-index produces a number of journals that were published in the last five years, as well as providing statistical data on how “h” amount of articles originating from a specific journal were cited “h” times. Subgroup analysis was performed on articles with acceptance to the “top 5 journals” to better compare correlation consistency within the subgroup.
OA publication costs associated with top journals were determined by information accessed on the publisher’s website in August 2023. Average OA costs for the remaining articles were calculated using data from each non–top 5 journal website. On average, publishing costs for all OA journals are $2673, and for top journals $1720.67. Finally, we use measurement metrics to find the cost of each metric for all articles as well as articles in top journals. Cost per metric was then calculated by dividing cost by the specified metric. All associated costs were presented in US dollars.
Data Analyses
Articles were defined based on OA status. After creating OA and non-OA groups, negative binomial regression analyses were performed to detect the significance of OA status on media mentions, citation number, AAS, and Mendeley readers while controlling for days since publication. This was done to factor in the increased probability of older articles receiving higher citations when compared to newer articles simply because of longer social media exposure. Published subgroup analyses of articles published in the top 5 orthopaedic journals were also performed to control for individual journal influence. Descriptive statistics of this subset were calculated using independent t-tests. All statistics were performed using IBM SPSS software 2023 for Macintosh (IBM Corp. Released 2023. IBM SPSS Statistics for Macintosh, Version 29.0. Armonk, NY: IBM Corp). Significance was determined at P <.05.
Results
All Articles
The average AAS for all OA and non-OA articles studied was 5.8, and the average Mendeley readers and average citations were 36.7 and 16.7, respectively. The mean AAS for OA articles was significantly higher compared to the mean AAS for non-OA articles, as well as the SD values (8.7 ± 37.0 vs 4.8 ± 26.3, P = .001). Correspondingly, the mean number of Mendeley readers and SD values for OA articles was higher compared to the mean number of Mendeley readers for non-OA articles but lacked statistical significance (42.4 ± 41.6 vs 34.9 ± 25.7, P = .131). The mean number of citations for OA articles was less than the mean number of citations for non-OA articles; however, this was also not significantly different (19.7 ± 24.8 vs 20.3 ± 23.5, P = .462). The SD values for OA articles were slightly higher than non-OA articles, respectively (24.8 vs 23.5). Our calculations for citations were determined based on a modified range between January 1, 2016, and July 18, 2021. Additionally, the OA articles delineated a greater number of Twitter mentions (4.6 ± 7.4 vs 3.3 ± 8.1, P = .009), Facebook mentions (0.3 ± 0.7 vs 0.1 ± 0.6, P = .039), and news mentions (1.2 ± 8.8 vs 0.5 ± 4.9, P < .001) when compared to the non-OA statistics (Table 1).
Comparison of Citation Rates and Visibility Parameters Between All Articles Analyzed Published With and Without Open Access.
P value calculated using negative binomial regression. Bold values indicate statistical significance.
Citation calculation followed an altered timeline and “n” selection.
Articles in Top Orthopaedic Journals
A total of 212 (57.6%) of all articles studied belonged to the top 5 orthopaedic journals according to the H5-index in Google Scholar. These top 5 journals included Journal of Bone and Joint Surgery (H-Index 290), The Bone and Joint Journal (H-Index 190), Foot & Ankle International (H-Index 120), Journal of Foot and Ankle Surgery (H-Index 74), and Foot & Ankle Specialist (H-Index 19). Based on the published articles throughout the top orthopaedic journals, OA publication was associated with a larger AAS score and corresponding SD values (21.3 ± 81.3 vs 4.0 ± 10.6) and the number of Mendeley readers (38.0 ± 26.3 vs 35.5 ± 24.2), but not the number of Twitter mentions (1.8 ± 2.8 vs 3.6 ± 9.0) or the number of citations (20.8 ± 36.8 vs 22.1 ± 23.2) (Table 2). While analyzing the number of citations and AAS score against the article’s OA status, only the Foot & Ankle International showed a statistically significant difference in AAS whereas Foot & Ankle Specialist showed a statistically significant number of citations (Table 3). Nonetheless, it is important to consider the reduced sample size of the top 5 journal articles retrieved from the Altmetric database when assessing significance.
Comparison of Citation Rates and Visibility Parameters Between Articles Published in Top Orthopaedic Journals With and Without Open Access.
P value calculated using negative binomial regression. Bold values indicate statistical significance.
Citation calculation followed an altered timeline and “n” selection.
Comparison of Top Orthopaedic Journals.
Citation calculation followed an altered timeline and “n” selection.
P value calculated using negative binomial regression. Bold values indicate statistical significance.
Cost of Open Access Publication
The mean cost of OA publication for all articles was $2673.00, as mentioned above. For all articles, the average cost per citation and AAS point were $174.02 and $309.01 respectively. The mean cost for OA publication in top journals was $1720.67 with costs ranging from $862 to $2300, using data provided on each journal’s website. The average cost per citation and AAS point in top journals were $46.77 ($47.14 to $191.18) and $21.15 ($55.36 to $1720.67), respectively.
Discussion
As the scope of OA publishing continues to proliferate with its available options, the path to publishing literature will become more difficult because of the existence of additional publishing costs. With the benefit of providing free access for readers to tune into the scientific literature, the tradeoff is that the publication fees fall on the responsibility of the author(s) or the institutions affiliated with the literature. In most cases, authors may be willing to pay publication fees if it provides higher readership and accessibility, but other factors like number of citations and social media attention are vital to recognize as well. The purpose of this TAA literature study was to understand the relationship between OA publications and increased dissemination when compared to non-OA articles. In our investigation that revolved around peer-reviewed TAA literature—including OA and non-OA privileges—results showed that although OA publication was generally associated with increased social media attention, it was not associated with increased readership or citations. In fact, a similar trend of OA articles producing greater social media attention was observed in the top orthopaedic journals as well. With these findings, we encourage researchers to weigh the different options for article interaction when publishing OA literature related to TAA.
OA publications have found a surprising niche among publishers in the 21st century. Social media has become a critical tool in maximizing interaction, and it is no surprise that publishers are trending to OA with powerful platforms like Twitter and Facebook at their disposal. 9 In addition, removing the paywall for consumers may help those who are interested but not willing to pay to view the article. In our study, we observed a pronounced increase in social media attention when comparing OA and non-OA articles. This could be due to more engagement with those outside of academia via social media. Significant correlations were made between OA and Twitter (tweets, retweets) and Facebook mentions when compared to non-OA. This trend can be seen in studies investigating articles outside of orthopaedics.4,9 In the context of both above-mentioned studies, it should be noted that neither was controlled for days since publication. This inherently leads to articles published at an earlier time frame, garnering more social media dissemination as opposed to literature published later. To our knowledge, there has been no orthopaedic literature investigating the direct relation between OA TAA literature and increased social media attention. Thus, publishing authors should consider the benefit of increased social media dissemination when choosing to publish OA TAA literature.
Conflicting data exist on the correlation between social media exposure and number of citations. Although our study noted a pronounced difference in social media attention between OA and non-OA, we did not see the same relationship when viewing citations. This could support the idea that OA publications may lead to scientists sharing more articles on social media, but the academic discourse may not be reflected in citation counts. Throughout various literature topics, there is a suggestion that the largest indicator for the number of citations is the time since publishing the article.12,14 Although this correlation has been minimally studied in orthopaedic literature, some studies emphasize the lack of correlation of citations with social media exposure in orthopaedic literature,18,19 whereas others suggest a strong relationship.13,17 These articles may be limited in scope, however, because of their subspecialty nature. The wide variation of citations among subspecialties should be taken into account when assessing the success of social media exposure. In the context of the above literature, it is clear that more research is needed on the relationship between citations and social media exposure in orthopaedic literature. Although OA articles did depict a larger social media presence, this did not translate into higher citation rates. Many factors, including the scope of the study, the popularity of the posting author, the timing of the social media post, and the impact of the study may have affected our findings. Our study concluded that no significant relationship exists between OA articles that received increased social media attention and a number of citations. Despite that, more research should be conducted on orthopaedic literature to clarify any misconceptions.
Traditionally, large academic institutions are the highest producers of academic literature. These institutions, along with their staff, likely have a subscription to both OA and non-OA literature. Out of the total non-OA articles studied, 40 articles were from academic centers whereas 160 articles of the OA articles were from academic centers. Academic centers were determined based on a review of author sources for each article. Although faculty subscription rates have not been studied in orthopaedic literature, it is known that research output and involvement positively correlate with faculty ranking and status. 3 This presents an interesting perspective in viewing the relationship between academic centers and publication resources. Although we only briefly discussed the costs associated with top academic journals, it can be assumed that these costs would be more difficult to overcome for those with fewer resources. These are likely authors who are not attached to major academic centers and, thus, do not have robust research centers and financial resources. It is difficult to account for the myriad of reasons an author may choose OA over non-OA, but we suspect that this could play a vital role when comparing. In the context of our study, this idea may explain the pronounced increase of social media attention attached to OA articles. With the primary consumers of orthopaedic literature being those with knowledge of the field and in academia, it points to the limited social media dissemination attached to non-OA articles.12,15 Published articles may not be accessible to those who are not directly affiliated with an academic institution or who cannot afford non-OA fees. 12 To confirm this speculation, more research should be conducted. However, it does provide more context to the pronounced difference in social media dissemination between OA and non-OA articles.
In terms of cost analysis, we calculated the cost per citation ($174.02) and the cost per social media attention ($309.01) for all articles. We conducted the same analysis for top orthopaedic journals and found the average cost per citation ($46.77) and the average cost per social media attention ($21.15). This was calculated using the data described in our Results section. The costs associated with these 2 metrics varied widely among the journals analyzed. This has not been reported but may be helpful to authors wishing to gain a broader understanding of the benefits of OA publishing.
Within our study, there may have been potential limitations that are important to address as readers assess the quantitative data. One of the main limitations was the database itself: the Altmetric Explorer. The platform only allows searches through PubMed databases. Although this may be a limitation, a past large-scale study that chose about 50 000 articles from another database, by the name of Cochrane, delineated that PubMed covers 82.8% of the included studies from the original database. 6 The Altmetric Explorer platform may also have missed certain data points like mentions and retweets. In Tables 1 and 2, there are only a few rows present that describe data from sources such as Twitter, Facebook, and news mentions because of the lack of statistical significance in the other mediums. Additionally, the database search limited TAA articles to only publications from 2016 to 2023 because prior publications before 2016 may not be the best candidates for this study. Also, articles were not stratified based on the year of publication. This may have introduced a bias when reporting our conclusion because of articles published longer possibly accruing more social media attention. We included negative binomial regression to adjust for days since publication as well as an altered timeline for citation calculation. Still, authors should keep this possible limitation in mind when viewing our results.
Moreover, Mendeley readership played a key role in our study; however, the correlation between readership numbers and actual reads may not be completely accurate. Despite this limitation, a recent survey shed light on how most Mendeley readers have read their respective articles or had the pure intention to read it, which may reduce this limitation. 10 The database restricted our knowledge of the quality of the study and the relationship between OA publication, our study, and its applicability to other social media mediums and metrics. Our findings when comparing citations and readership in OA to non-OA may have been confounded by outside factors. In many cases, authors of breakthrough articles prefer to submit to exclusive non-OA journals. This is due in part to the prestige associated with non-OA journals. It is worth considering that the disparity of citations and readership could be partly due to OA journals receiving less impactful work. Moreover, many non-OA journals have “cascade” OA counterparts that may receive less impactful work if articles are deemed unfit for non-OA publication. Inherently, more impactful work for the orthopaedic community will likely receive higher citations than less impactful work. This leads to a higher chance of non-OA articles receiving more citations and/or readership. Our study found no significance in citations between the 2 groups; it should be noted that several factors may contribute to our findings that could serve as confounders as mentioned above. Despite these factors, our study provides publishing authors valuable insight when comparing OA and non-OA journals. Finally, when stratifying articles included in top journals, we used the “h-index” from Google Scholar as a guiding metric. These metrics were calculated based on how many articles met a predetermined citation threshold. Another common manner of gauging journal dissemination is the impact factor. The impact factor is determined as the mean number of citations each article within a journal receives. By not including impact factors of journals in our study, the top 5 journals may be skewed to those with higher h-indexes but not necessarily a higher impact factor. Furthermore, the h-index values of the top 5 included journals vary widely. Therefore, it is difficult to make a comprehensive comparative assessment of the top 5 articles, mainly because of the wide variation in journal impact. We understand the limitations this may bring to our study when generalizing our data. Despite this, the h-index remains reliable in terms of gauging journal impact and guiding authors on where to publish. Our study provides valuable data and insight to authors seeking to maximize their research outreach and dissemination.
Conclusion
OA publications have increased in prevalence tremendously in orthopaedic literature. Despite the increased accessibility of OA publications to readers, it is not known if it translates to higher social media attention, citations, or readership among peer-reviewed publications in TAA. This study compared OA TAA literature to non-OA TAA literature in hopes of establishing trends of dissemination associated with OA status. Our results showed OA publications were associated with greater AAS (social media attention), but not Mendeley readership or number of citations. In the subgroup analysis of the top 5 orthopaedic journals, similar trends were highlighted with results showcasing the same link of greater social media attention, but no impact on citations or readership. On further analysis across top orthopaedic journals, only 1 journal showed a significant difference in AAS score, and 1 journal showed a significant difference in citation count, which is the minimum error interval. In hopes of understanding the impact of dissemination and the scale of benefits compared to the consequences of publishing an OA article—including career advancement, research impact, costs, author recognition, and resource allocation—this study emphasizes how authors of TAA literature may wish to consider their objectives before publishing an article in OA or non-OA format.
Supplemental Material
sj-pdf-1-fao-10.1177_24730114241247817 – Supplemental material for Open Access Publication in Total Ankle Arthroplasty Literature Is Associated With Increased Social Media Attention, but Not Increased Citations
Supplemental material, sj-pdf-1-fao-10.1177_24730114241247817 for Open Access Publication in Total Ankle Arthroplasty Literature Is Associated With Increased Social Media Attention, but Not Increased Citations by Trayce Gray, Andrew B. Harris, Rahi Patel, Julius Oni and Amiethab Aiyer in Foot & Ankle Orthopaedics
Footnotes
Appendix A
Ethical Approval
Ethical approval was not sought for the present study because this is a review of published articles.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article. Disclosure forms for all authors are available online.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
