Sage Journals: Discover world-class research

Abstract

Objectives

Proficiency in medical writing and publishing is essential for medical researchers. Workshops can play a valuable role in addressing these issues. However, there is a lack of systematic summaries of evidence on the evaluation of their impacts. So, in this systematic review, we aimed to evaluate all articles published on the impact of such workshops worldwide.

Methods

We searched Ovid EMBASE, Ovid Medline, ISI Web of Science, ERIC database, and grey literature with no language, time period, or geographical location limitations. Randomized controlled trials, cohort studies, before-after studies, surveys, and program evaluation and development studies were included. We performed a meta-analysis on data related to knowledge increase after the workshops and descriptively reported the evaluation of other articles that did not have sufficient data for a meta-analysis. All analyses were performed using Stata software, version 15.0.

Results

Of 23 040 reports, 222 articles underwent full-text review, leading to 45 articles reporting the impacts of workshops. Overall, the reports on the impact of such workshops were incomplete or lacked the necessary precision to draw acceptable conclusions. The workshops were sporadic, and researchers used their own method of assessment. Meta-analyses of the impact on the knowledge showed that workshops could nonsignificantly increase the mean or percentage of participants’ knowledge.

Conclusion

In the absence of systematic academic courses on medical writing/publishing, workshops are conducted worldwide; however, reports on educational activities during such workshops, the methods of presentations, and their curricula are incomplete and vary. Their impact is not evaluated using standardized methods, and no valid and reliable measurement tools have been employed for these assessments.

Keywords

Medical writing publishing systematic review meta-analysis workshop impact

Introduction

Proficiency in medical writing and familiarity with medical publishing standards is essential when it comes to publishing findings of medical research.¹ The act of publication represents the final stage of research, and without it, all the prior stages of conceptualization, planning, data collection, analysis, and inference would be significantly futile.²

Having undertaken credible research, success in publishing involves a number of challenges: Knowing how to structure one's paper based on the recommendations of the international organizations of the field,³ such as the International Committee of Medical Journal Editors; awareness of checklists for elements to be included in types of published articles^4,5; knowledge of ethics of issues such as authorship^6,7; considering the target audience; choosing the target journal; using optimal English; and knowing how to deal with editors and peer reviewers.⁸ Typically, postgraduate research-focused medical education tends to overlook these subjects. As a result, workshops could play a valuable role in addressing this gap.⁹

Reports of sporadic workshops on aspects of medical writing/publishing suggest that they are mainly based on the prior experiences of the presenters.⁹ On the other hand, the impact of such workshops on increasing the number of publications or on increasing the confidence and/or knowledge of participants in a longitudinal way is not clear. Although some articles have demonstrated the usefulness of such workshops on the actual practice of participants,^10–15 there is no universal consensus on whether such workshops can significantly change the research outputs of the participants. A systematic review of comparative studies of formalized, a priori-developed training programs in writing for scholarly publication has demonstrated that few such evaluations exist, highlighting a knowledge gap regarding their effectiveness.¹⁶

The important field of optimal approaches to medical publishing—also called medical journalology—thus lacks systematic summaries of evidence on the evaluation of such workshops and their short- and long-term impacts. So, in this systematic review, we aimed to evaluate all articles published on the impact and effectiveness of medical writing/publishing workshops worldwide. We did not delve into workshops pertaining to nonmedical research or publishing in other disciplines because different fields of research or scientific publishing have their own unique specifications and issues. This report is part of a larger project with different research questions on various aspects of medical writing/publishing workshops, which have a shared search strategy and database review.

Methods

This systematic review is reported based on the PRISMA guideline.¹⁷

Types of Studies

Randomized controlled trials, cohort studies comparing workshop participants to nonparticipants, before-after studies of workshop participants, surveys of the experience of workshop participants after the course, and program evaluation and program development studies were included.

Types of Participants

We included all the abovementioned studies in which the participants were medical students, graduate students, faculty members, or any other adult participants who were willing to learn medical writing, related ethical issues, and/or publication standards and nuances.

Types of Outcome Measures

Impact on knowledge and confidence in medical writing/publishing, from questionnaires completed by workshop participants,

Impact on the number of manuscripts publications in peer-reviewed journals, from self-reports by participants or searching of the literature,

Impact on the number of abstract submissions to related conferences, from self-reports by participants or searching of the literature,

Impact on confidence in dealing with peer reviewers and journal editors, from participant self-report,

Impact on feedback received from peer reviewers and editors, from participant self-report,

Impact on skills in medical writing/publishing, from evaluation instruments or participants’ self-report.

Search Methods for Identification of Studies

Electronic searches

We searched Ovid EMBASE (from 1974 to October 2022) using the search terms and combinations in Appendix 1, and Ovid Medline from 1946 to October 2022 (Appendix 2). A librarian was involved in different stages of selecting the keywords and searching the related databases. Database-specific keywords were used for the search, Mesh terms were used to search Medline, and Emtree terms were used to search the Embase. We also searched the ISI Web of Science database (from the inception to October 2022) using the related keywords (Appendix 3). Another librarian familiar with searching the ERIC database was involved in searching this database, as a specialized database for medical education, using the related keywords. We did not limit the search to any specific language, time period, geographical location, or number of centers for performing the study.

Searching Other Resources

We also evaluated the reference lists of the eligible articles for any further published articles. To assess the grey literature, Google Scholar was also searched using the related keywords, and the first 100 hits of the Google Scholar search were reviewed for relevant articles.

We made a snowballing approach to experts in the field, including the editors of the journals and also researchers of the medical writing/publishing area, by directly emailing whom we knew as well as by writing a related announcement in the related forums of World Association of Medical Editors, and Eastern Mediterranean Association of Medical Editors asking for introducing any related articles published in journals which are not indexed in the abovementioned databases or any on-going workshops with unpublished data. PRISMA flow diagram shows the number of articles retrieved from different databases (Figure 1). Also, Appendix 4 shows the PRISMA checklist.

Figure 1.

PRISMA flow diagram for the impact of medical writing/publishing workshops.

Data Collection and Analysis

Selection of studies

Multiple independent reviewers screened the titles and abstracts of the retrieved studies through the Covidence software. However, each abstract and title were screened by only 2 independent reviewers. The full texts of the studies that passed the screening were also independently assessed by 2 reviewers, considering the eligibility criteria. Any disagreement between the reviewers was resolved by a third reviewer with a thorough discussion with the previous 2 reviewers.

Data Extraction and Management

A Google sheet was prepared for extracting data from the included articles, and 2 reviewers independently extracted data from each article for the listed outcomes after piloting the process for one study. Differences between the reviewers regarding the data were resolved by a third reviewer with a thorough discussion with the previous 2 reviewers.

The gathered data were about the characteristics of the studies, such as the design, the aim of the study, the number of participants in each workshop, inclusion and exclusion criteria, method of recruitment of participants, type of possible intervention, any comparison with the intervention, and possible outcomes.

Dealing With Missing Data

We attempted to contact the authors of studies with missing data. We did not use strategies of imputation, deletion, or modeling to address missing data.

Certainty of Evidence

We could not use GRADE to check the certainty of evidence.

Data Synthesis

We planned to perform a meta-analysis if the data permit; otherwise, a descriptive evaluation of articles would be presented. For the meta-analyses, the presence of heterogeneity was examined using the χ² test and graphical methods (forest plot). Funnel plots and Begg and Egger tests were used to assess publication bias. Also, the Metatarium command was used to estimate the effect size of the relation in the presence of missing studies due to publication bias. All analyses were performed using Stata statistical software (version 15.0, Stata Corp), and the significance level was considered to be less than .05. This study was not registered in PROSPERO.

Results

Our search in different databases led to 27 737 articles. We also retrieved 189 articles from other sources. After removing the duplicates, we could locate 23 040 unique reports. After screening the titles and abstracts, 222 articles underwent full-text review, which finally led to 45 articles reporting the impacts of workshops. Very few articles sufficiently reported their educational approaches. Some reported education through seminars or hands-on workshops, and some others reported tutor training approaches or educational lectures without further clarification.

Of the 45 articles that reported the impact, 21 reported increases, though measured differently, in the number of publications after the workshops.^14,18–37 Eleven articles reported an increase in knowledge after the workshops.^{15,30,38–46} Seven articles reported increases in the confidence of the participants after the workshops.^{15,21,23,30,34,38,40} Six articles reported an increase in the satisfaction rate of the participants.^{14,30,40,47–49} And 6 articles reported about self-rated skills.^{30,38,39,44,46,50} Some of the mentioned articles had overlaps in reporting different categories as indicators of the impact of such workshops. Table 1 shows the number of studies that assessed different outcomes, methods of assessment, and the outcomes assessed.

Table 1.

The Number of Studies that Assessed Different Outcomes, Methods of Assessment, and the Outcomes Assessed.

OUTCOMES: INCREASE IN	NO. OF STUDIES	AUTHORS	METHOD OF ASSESSMENT	RESULTS
Number of Publications	21	Henninger & Nolan¹⁸	Participants from both long-term and short-term intervention groups reported through a questionnaire whether they were successful in publishing.	At the 1-year follow up, 5 nurses had submitted, were accepted or published an article. 3/10 nurses in the long-term group were successful, while 2/8 in the short-term were successful.
		Pololi et al¹⁹	Participants reported the number of manuscripts submitted and accepted for publication.	The initial cohort of participants had submitted or had accepted for publication 16 manuscripts, and the second cohort, 11.
		Ried et al²⁰	Study subjects reported through a survey whether they submitted and published manuscripts.	19 individuals in the participant group published their research in a peer-reviewed journal; 5 nonparticipants published their paper. 7 participants submitted for publication; 6 nonparticipants submitted. Time trend analysis on publication rates was borderline significant (P < .06), suggesting that a greater percentage of articles was published by earlier cohorts (2002: 50% and 2003: 31%) than recent cohorts (2004: 23%, 2005: 12% and 2006: 21%).
		Steinert et al¹⁴	Used one-year follow-up questionnaire to determine how many participants submitted/published/presented work.	At the one-year follow-up, 9 participants submitted a total of 14 educational manuscripts, 11 being accepted for publication. 10 participants presented a total of 38 abstracts. Overall, 16 participants either submitted a paper to publish or an abstract to present.
		Cameron et al²¹	Administered questionnaire 6 months after workshop to probe number of participants with publications.	56.8% of all participants agreed that the course helped them to publish an article. In response to, “How many manuscripts have you had accepted for publication since the workshop?” 3 participants said 2 or more, 19 participants said one, and 23 participants said none.
		Rickard et al²²	Participants reported publication outcomes through a questionnaire administered 2 years after the workshop.	2 years after the group meetings, 7 participants published at least one article. Submissions increased from 9 (before workshop) to 33 articles after. Participants attending more than 10 meetings submitted an average of 5.4 manuscripts in the following 2 years; those attending less than 10 meetings submitted an average of 1.2 manuscripts.
		Sridhar et al²³	Collected data on participant publications	From May 2005 to May 2007, 10/98 participants published 11 case reports between 16 and 41 months after completing the workshop.
		Richardson & Carrick-Sen²⁴	Collected participant data on writing outcomes.	One year after the workshop, 12 participants submitted a total of 11 papers (2 individuals wrote a paper together). 4 submissions were published.
		Rubens²⁵	Collected data on participant outcomes following the workshop.	2 participants attending the 2009 Abstract Writing workshop had abstracts accepted at a 2010 conference. They, as well as 2 other participants had abstracts accepted at a 2012 conference. 1 researcher published in PLoS One, and had abstracts accepted at 2 conferences.
		Guillerm et al²⁶	Participants self-reported publication outcomes through a questionnaire; researchers verified statements by searching on MEDLINE.	25/76 participants published an article before the course—significantly fewer than after (n = 38, P = .03). A total of 75 papers were published before the course. After the course, 10 OR (operational research course) fellows published a total of 147 papers. 28 non-OR fellows published a total of 110 papers.
		Jernigan et al²⁷	Followed writing/publication outcomes among participants.	1 participant submitted a paper for publication.
		Islam et al²⁸	Followed writing/publication outcomes among participants.	2 manuscripts, as worked on by 3 students, were published.
		Kulage & Larson²⁹	Participants self-reported publication outcomes through a questionnaire.	4 manuscripts, as worked on during the workshop, were published.
		Duncanson et al³⁰	Study subjects reported publication outcomes (manuscript completion, submission and publication). 2 researchers conducted Google Scholar search to verify statements.	26/50 participants submitted manuscripts; 21 papers were published. Comparatively, 15% of nonparticipants published (χ²(2, N = 112) = 10.02, P = .0015).
		Gardner et al³¹	Researchers followed participant publishing records as they finished their PhD. They conducted a PubMed search for each student.	All 6 graduate students published a first-author, peer-reviewed article. The group, on average, published 2.33 ± 1.36 first-author publications and 1.16 ± 1.16 s-author publications. The students contributed to an average of 2.83 ± 2.92 publications.
		Arrazola et al³²	Followed participants’ manuscript outcomes via email, by requesting progress updates.	As of April 2020, 24/39 participants submitted their manuscripts for publication. Of the 24 submitted, 17 were accepted, 4 were rejected, 2 were under review, and 1 was withdrawn.
		Dhakal & Tornwall³³	Participants self-reported publication outcomes through questionnaire.	Regardless of tenure-track status, faculty published more both before and after than nonfaculty participants. Tenure-track faculty published more than non tenure-track. The 9 DNP (doctor of nursing practice) participants published 3 articles before the course and 9 articles after. The 7 participants with a master's degree in a nursing discipline had not published before the course; 2 published after.
		Harvey et al³⁴	Researchers followed publication outcomes among participants.	2 participants submitted to their chosen journal during the study. 1 participant submitted for publication as the current study was being written. A total of 2 manuscripts have been published.
		Suma et al³⁵	Students updated workshop organizers on the status of their papers.	78/100 manuscripts drafted during the workshop were submitted in the subsequent month. At the 3-month follow-up, 21 were accepted, 12 were published and 45 were in the review stage.
		Tsujimoto et al³⁶	Researchers searched PubMed for systematic reviews associated with hospitals that conducted the workshop, comparing numbers in 3 years before and after.	There was a 3-fold increase in the number of publications from before versus after the workshop.
		Chandratre et al³⁷	Not detailed.	In 2020-2021, students completed 3 case reports, presented at a conference, and 1 letter to the editor. In 2021-2022, students were involved in at least 1 project. More than 15 case reports were accepted. Scholarly productivity was 5 times greater than the previous year.
Knowledge	11	Wajekar et al¹⁵	Faculty individual, with experience of being reviewer for at least 2 peer-reviewed anaesthesia journals, evaluated case report writings by participants. They used a checklist with a 3-point scale (1 = clearly below expectations, 3 = performs above expectations).	Scientific writing scores increased from an average of 28.3 preworkshop to 36.3 post workshop. Analytical writing scores increased from an average of 11.3 preworkshop to 13.2 post workshop.
		Duncanson et al³⁰	Study subjects retrospectively rated their knowledge levels through a questionnaire. They specifically used a 5-point Likert scale.	Writing knowledge increased from an average of 2.0 preprogram to 3.5 postprogram (P = .0003). Publishing knowledge increased from an average of 1.4 preprogram to 3.4 postprogram (P = .0003).
		Sheikh & Levy³⁸	Participants rated improvements through a questionnaire, using a 6-point Likert scale.	Participants perceived improvements in their ability to write a research protocol (mean scores 3.20 vs 3.90, P < .01).
		Jawaid et al³⁹	Participants answered yes/no questions about knowledge in a questionnaire.	Response of 8 questions regarding knowledge about different aspects of medical writing showed that there was a highly significant difference (P < .005).
		Sabouni et al⁴⁰	Researchers administered a pre- and postcourse test, made of the same 13 questions.	Participants improved significantly in 30% of the questions. There was no significant change in 62% of the questions. Participants showed a significant decline in 1 of the 13 questions.
		Goyal et al⁴¹	Q 1-5 on assessment tested participant knowledge, and was administered both before and after the workshop.	Percentage of respondents failing to correctly name journal indexing agencies significantly reduced from 48.2% to 12.5%. Percentage unable to correctly name referencing style also reduced from 46.4% to 12.5%. There were significant improvements in the correct responses to 4 of 5 questions, except for the question assessing that the discussion section should contain comparisons of present study with preexisting literature.
		Trigotra et al⁴²	Students answered a semistructured questionnaire both pre- and postworkshop.	96% rated their knowledge ≥3 in posttest as compared to 18% in pretest.
		Goyal et al⁴³	Participants filled a questionnaire, with 7 questions assessing knowledge.	A comparison of the responses to the pre- and postquestionnaires indicated a significant improvement in participants’ knowledge.
		Agarwal et al⁴⁴	Mentees self-reported their understanding through a questionnaire.	Mentees reported a significant improvement (P < .001) in their understanding of plagiarism writing, the similarity index in antiplagiarism software, plagiarism as a misrepresentation of another's work, plagiarism as a criminal offence and the ability to recognize and avoid unintentional plagiarism.
		de Oliveira et al⁴⁵	Participants answered a multiple-choice survey about the content each day.	Overall correct answers were significantly higher (P < .01) in the postsession questionnaires, every day. Normalized learning gains by day were 0.36, 0.53, and 0.28 on the first, second, and third day, respectively.
		Fernandez et al⁴⁶	Students rated knowledge through a questionnaire.	Overall increases in scores for knowledge (mean 1.6; 95% CI 1.6-1.7) after the course were significant (P < .001).
Confidence	7	Wajekar et al¹⁵	Students assessed their confidence levels through a questionnaire consisting of 10 questions. They used a 5-point Likert scale (1 = extremely confident, 5 = not at all confident).	Using a reverse Likert scale as 1 = extremely confident; scientific writing confidence scores improved, from an average of 7.0 preworkshop to 5.0 post workshop. Analytical writing confidence scores improved, from an average of 12.0 preworkshop to 8.3 post workshop. Overall writing skills improved, from an average of 4.0 preworkshop to 2.3 post workshop.
		Cameron et al²¹	Administered self-assessments before and after workshop, probing participant confidence related to writing. Used a 5-point Likert scale (1 = very confident, 5 = very unconfident).	There were increases in confidence levels on these measures of 1.12 to 1.44 points.
		Sridhar et al²³	Participants rated their competence in writing a case report through a questionnaire, on a 5-point Likert scale (1 = low, 5 = high).	Perceived competence significantly increased from an average of 2.5 to 3.5 (+0.99; 95% CI, 0.88-1.12; P < .001). The percentage of participants self-reporting a rating >3 increased from 12.0% (pre) to 50.0% (post), P < .001.
		Duncanson et al³⁰	Participants retrospectively reported confidence in first bootcamp versus most recent. They specifically used a 5-point Likert scale (1 = no confidence; 5 = very confident) to rate their confidence.	Writing confidence increased from an average of 1.9 preprogram to 3.4 postprogram (P = .0003). Publishing confidence increased from an average of 1.1 preprogram to 3.2 postprogram (P = .0004).
		Harvey et al³⁴	Participants rated their confidence through a questionnaire using a 5-point Likert scale (1 = not at all, 5 = completely).	Participants rated their confidence in “ability to write manuscript for publication” at an average of 3.8.
		Sheikh & Levy³⁸	Participants rated their confidence through a questionnaire, on a 6-point Likert scale.	16 participants reported an improvement in their confidence in performing research. Mean preworkshop scores were 3.47, with an average improvement of 0.53 (P < .01)
		Sabouni et al⁴⁰	Participants rated their confidence through a questionnaire, on a 7-point Likert scale (1 = strongly disagree; 7 = strongly agree)	Prior to the course, 55% of students felt confident enough to produce a research paper. After completion of the course, this increased to 83%.
Satisfaction	6	Steinert et al¹⁴	Participants reported their satisfaction through a questionnaire, on a 5-point Likert scale.	In the immediate postworkshop evaluation, participants rated the workshop at an average of 4.4/5.
		Duncanson et al³⁰	Participants answered a series of questions through a questionnaire about their perceptions of the workshop structure and content.	All questionnaire respondents would recommend the bootcamp to others; 43 individuals rated it as a positive experience. Program features that were highly rated include: rotation of writing “partners” (n = 42, 96%), receiving writing feedback (n = 41, 93%), weekly workload (n = 41, 93%) and reviewing others’ work (n = 40, 90%). 11 participants mentioned that the time commitment was insufficient or they could not commit 40 h over 6 weeks.
		Sabouni et al⁴⁰	Participants rated their personal satisfaction with the course through a questionnaire, on a 3-point Likert scale.	The average satisfaction rate was 8.14/10.
		Bydder et al⁴⁷	Participants reported opinions through a questionnaire.	All participants found the workshop useful (3 felt strongly). All participants would recommend it to others (4 felt strongly) and felt that workshops should be available in the future (3 felt strongly).
		Phadtare et al⁴⁸	Participants reported satisfaction through a questionnaire, on a 5-point Likert scale (1 = Strongly disagree; 5 = Strongly agree).	The online group was more satisfied (4.3 ± 0.73, ranging from 3 to 5) than the traditional group (3.09 ± 1.11, ranging from 1 to 5) (P = .001).
		Cameron et al⁴⁹	Participants completed a course evaluation measuring their satisfaction, on a 10-point Likert scale.	Researchers assessed the overall response as enthusiastic. Average scores ranged from 9.1 to 9.7.
Skills	6	Duncanson et al³⁰	Participants retrospectively reported experience in the first bootcamp versus most recent. They specifically used a 5-point Likert scale (1 = no experience; 5 = very experienced) to rate their experience.	Writing experience increased from an average of 1.9 preprogram to 3.5 postprogram (P = .0003). Publishing experience increased from an average of 0.8 preprogram to 3.6 post-program (P = .0003).
		Sheikh & Levy³⁸	Participants reported improvements in skills through a questionnaire, on a 6-point Likert scale.	28 participants believed themselves to have improved in at least one of the writing categories asked about.
		Jawaid et al³⁹	Participants rated their skills through a questionnaire, using a 5-point Likert scale (1 = no skills, 5 = expert).	All 6 questions requiring participants to rate skills showed significant improvements between before and after the workshop (P < .0001).
		Agarwal et al⁴⁴	Mentees self-reported their skills through a questionnaire.	For scientific writing, mentees reported a significant improvement (P < .001) in writing ability, grammar and punctuation, use of appropriate terminology, ability to identify appropriate articles and choose the correct references.
		Fernandez et al⁴⁶	Students rated skills through a questionnaire.	Overall increases in scores for skills (mean 1.4; 95% CI 1.3 to 1.4) after the course were significant (P < .001).
		Geithner & Pollastro⁵⁰	Students rated their skills through a questionnaire, on a 5-point Likert scale.	Students demonstrated significant improvements in skills. The greatest improvements were in their self-rated ability to critically read scientific literature (+1.3 difference from precourse to postcourse) and ability to identify critical content of different components of a scientific paper/laboratory report (+1.4 difference from precourse to postcourse).

In 8 articles, the researchers assessed the impact of the workshops or training sessions by evaluating the assignments done by the participants or by subjective assessment of the manuscripts written by the participants.^{28,35,41,44,45,47,48,51} Each workshop had unique measures to evaluate the impact of their assessments. For example, manuscript grading by the instructors and 2 additional faculty members²⁸ or assessing each written project by 14 reviewers with 3 different evaluation methods.⁴⁴ This consisted of grading with a structured rubric, a feedback form, and free-form comments on the written assessment.⁴⁴

Some studies measured success through the quality of one final written assessment.^47,48 However, the grading of this assessment varied from a single instructor determining the grade of each assessment using the Six-Subgroup Quality Scale to ensure all manuscripts were marked consistently⁴⁷ to focusing on standardized parameters to determine quality.⁴⁸

Some authors used multiple-choice and free-form questions before and after training sessions to evaluate the knowledge learned.^41,45

A few studies used structured checklists or scoring systems to retrieve consistent marking from graders.^35,44,51 Only one study used single-blind academics to grade final assessments.⁵¹

Weekly assessments were used in one study to assess progress throughout the workshop. Participants of this study reported the lowest average final writing assessment grade, though, because many of the final manuscripts required major revisions.⁴⁴

Some studies focused on 1 to 3 hands-on exercises, where participants wrote their own research papers or writing assignments with guided feedback.^28,35,41,51 On average, participants received a grade of 70.8% or higher on their written assessments in 4 studies.^28,35,48,51 Participants were able to generate publications using this method in 2 of these studies.^28,35

Studies that evaluated participants’ knowledge before the workshop or training sessions found that participants were uneducated on reference styles, manuscript drafting, academic writing, and publishing, along with the required structure and content of articles.^41,48 Self-reported and assessed evaluations found that participants’ skills and knowledge had increased by the end of the session.^41,44,45,47 Multiple-choice questionnaires and SQSS were the most common measures for evaluating skills gained.^41,45,47 Only one of these studies reported skills self-assessed by participants.⁴¹ Skills that improved included publishing and writing guideline knowledge, critical thinking, and writing quality and ability. In one study, authors reported using standardized forms where mentors reported their observed soft skills achieved over the course of the workshop.⁴⁴

Mentorship was used as a tool to train participants in some articles.^28,35,44,47 Other studies used lectures and/or group discussions to train participants.^41,45,48,51 While most of the articles evaluated the impact of in-person learning, only 2 articles evaluated the impact of online workshops. Both of these received positive feedback from participants regarding the online structure of the workshops.^45,47

A majority of these studies found that participants were satisfied with the feedback, content, and/or structure of the courses.^{28,35,41,45,47,48} Suggestions for improvements included the use of more examples, more practice with hands-on exercises, and longer teaching periods.^41,48

Effect of Medical Writing Workshops on the Increase in Knowledge

The only category that provided sufficient articles with rich quantitative data to conduct a meta-analysis was the one reporting on knowledge increase. Of the 11 articles in this category, 3 articles did not report sufficient data to be included in the meta-analysis.^42,43,46 We reached out to the authors for the relevant data, but unfortunately, we did not receive any response. The remaining 8 studies included 4 studies, which reported the mean changes in knowledge after the educational intervention and 4 other studies reported the percentage of changes in knowledge. So, 2 separate meta-analyses were done.

SD was not reported in 2 studies.^30,38 We once more sent emails to the authors requesting the related SD, but we did not receive any reply. So, the SD was calculated using the rule of 4 sigma by dividing the range of scores by 4. The SD of the mean difference was estimated using the formula: SD²= [(SD before²+ SD after²) − (2 × R × SD before × SD after)] where the correlation coefficient (R) was considered as 0.7.⁵²

Effect of Medical Writing Workshops or Educational Sessions on the Mean Changes of Knowledge

The results of the meta-analysis of the 4 studies showed a mean increase of 1.13 (95% CI: −0.66 to 2.92; P = .22) based on a 5-point scale from 1 to 5.^15,30,38,44 In other words, medical writing workshops or educational sessions could not increase the knowledge of participants from a statistical point of view. Figure 2 shows the related forest plot.

Figure 2.

Forest plot of the effect of medical writing workshops or educational sessions on the mean changes in knowledge.

Figure 3 shows the funnel plot demonstrating no publication bias. Both Begg test (P = .68) and Egger test (P = .99) confirmed the lack of publication bias.

Figure 3.

Funnel plot for evaluation of publication bias in the effect of medical writing workshops or educational sessions on the mean changes of knowledge.

Effect of Medical Writing Workshops or Educational Sessions on the Percent Changes in Knowledge

Meta-analysis of the remaining 4 studies, which reported an increase in the percent of knowledge after the workshops, showed a percent change of 15.2% (95% CI: −32% to 62%; P = .52); however, the percent of change was not statistically significant.^39–41,45 Figure 4 shows the related forest plot. Similarly, no publication bias was found using the Egger test (P = .09); however, Begg test showed publication bias (P = .03). Figure 5 shows the related funnel plot. For more evaluation of the publication bias, we used the Metatarium command of the STATA software to estimate the adjusted percent change in the knowledge level by including possible missing studies, which could be the cause of publication bias, and include them in the final calculations (Figure 6). By including possible missing studies using the mentioned command, the final adjusted percent change in the knowledge level was 13% (95% CI: −30% to 56%; P = .56). Checking for the certainty of evidence using GRADE was not possible because of the “very, very low certainty.”

Figure 4.

Forest plot of the effect of medical writing workshops on the percent of change in knowledge.

Figure 5.

Funnel plot for evaluation of publication bias in the effect of medical writing workshops on the percent of change in knowledge.

Figure 6.

The results of adding possible missing studies using the metatarium command of STATA software.

Discussion

Overall, the reports on the impact of medical writing/publishing workshops or training sessions in the articles were incomplete or lacked the necessary precision to draw consistent conclusions. The workshops were sporadic, and researchers used their own method of assessment. Some studies reported grading the participants’ works by the instructors and assessed each written project, while some other studies reported evaluating the quality of final written tasks as an indicator of the impact of their workshops. Researchers-made multiple-choice questions and forms were used in different articles. Overall, the authors believed the participants were satisfied with the content and administration of the workshops and considered this to be a positive impact of their workshops.

The only impact for which some articles provided analyzable data was an increase in knowledge. The findings of our meta-analyses indicated that medical writing/publishing workshops could increase the mean or percentage of participants’ knowledge, but the observed changes were not statistically significant. Considering the lack of a systematic course for teaching medical writing and publishing and the relatively low resources used by workshops compared with a full course, any increase in the knowledge of participants after the workshops is important and worth investing in. Statistical significance for a group of participants may not be representative of statistical significance in increasing knowledge at the individual level. Statistical nonsignificance that is reported here is about the mean or percentage of knowledge increase among a group of participants; however, at the individual level, at least some people may experience knowledge increase, which is important.

Lack of systematic training for medical writing and publishing standards has led to the fact that many clinicians will not be taught how to write for publication at any stage of their career⁵³ or may lead to different publication ethics misconduct due to lack of knowledge,^54,55 rather than fraud.⁵⁶ This formal training gap leads to the use of different alternative methods, such as mentoring programs to train applicants. Pololi and colleagues reported the effect of a Collaborative Mentoring Program on the writing and publishing abilities of 18 assistant professors who could finally publish one scholarly publication after the program¹⁹; however, the long-term impact of such programs, the quality of the manuscripts, the quality of the intervention, and the quality of the journals in which manuscripts were published are not clear.¹⁶

Workshops, an integral component of nonsystematic training programs, play a pivotal role in teaching medical writing and publishing but might not cover all the preferences of peer reviewers. They can give advice that is likely to help with common issues but won't address every peer reviewer's unique preferences. Our investigation revealed that the impact of such workshops on the sustainability of learning and the enhancement of participants’ knowledge and confidence remained unverified. This finding aligns with the results reported by Galipeau and colleagues, who conducted a systematic review to investigate whether training in writing for scholarly publication, journal editing, or manuscript peer review improves the quality of health research reporting. Their analysis also indicated that the included studies were generally small and inconclusive regarding the effects of such training for authors. Furthermore, these studies were deemed of questionable validity and susceptible to misinterpretation. Similar to their findings, ours underscores the existing gaps in creating an evidence-based understanding of how to improve the scientific quality of research output for authors.¹⁶

Strengths and Limitations

The primary strength of our study is its uniqueness, being the first systematic exploration of articles focusing specifically on the impact of medical writing/publishing workshops to date. Additionally, it represents the first meta-analysis in this field, contributing to the establishment of evidence-based literature that can guide the organization of such workshops in the future. Our study's strength is further underscored by the exhaustive search conducted across various databases, utilizing a comprehensive list of database-specific keywords. The inclusion of grey literature and a snowballing approach to identify all relevant articles adds significance to our findings.

However, an unavoidable limitation of this study is the incomplete reporting of findings in individual studies and the lack of information on how workshop presenters delivered their educational materials. This limitation impedes our ability to offer a robust recommendation regarding the optimal approach for conducting such workshops.

Another limitation of this study was that we could not check for the certainty of evidence. Using the GRADE approach to certainty of evidence, observational studies begin as low-quality evidence and can be rated down to very low because of a number of issues, including the risk of bias. The available studies were mostly single arm (ie, no concurrent control group not receiving the intervention). Any such studies would be rated down to very low simply because of the study design, without any further limitations because of the risk of bias in the conduct of the studies. Since GRADE does not have a category for “very, very low certainty” there was no need to address additional limitations in the conduct of the studies. Despite these limitations, our study contributes valuable insights to the existing body of knowledge on medical writing and publishing workshops.

Conclusions

In the absence of academic courses on medical writing/publishing, workshops are conducted worldwide; however, reports on their educational activities, presentation methods, and curricula are often incomplete and vary. Their impact is mostly not evaluated using standardized methods, and no valid and reliable tools have been employed for these assessments.

The definition of impact varies in different workshops, ranging from changes in knowledge to confidence in writing skills, participant satisfaction, and subjective measurements. Although workshops could lead to an increase in participants’ knowledge, statistically significant differences are often not observed.

There is a need for a standardized measurement tool for the evaluation of such workshops. Academic teaching courses with defined curricula are recommended for graduate students and medical researchers who need to publish their studies, although they may not be suitable for practitioners who do not have the flexibility in their role to leave bedside care roles in order to participate in such courses. This approach can address the current gaps and enhance the effectiveness of medical writing/publishing education.

Supplemental Material

sj-pdf-1-mde-10.1177_23821205241269378 - Supplemental material for Impact of Performing Medical Writing/Publishing Workshops: A Systematic Survey and Meta-Analysis

Supplemental material, sj-pdf-1-mde-10.1177_23821205241269378 for Impact of Performing Medical Writing/Publishing Workshops: A Systematic Survey and Meta-Analysis by Behrooz Astaneh, Ream Abdullah, Vala Astaneh, Sana Gupta, Hadi Raeisi Shahraki, Aminreza Asadollahifar and Gordon Guaytt in Journal of Medical Education and Curricular Development

Supplemental Material

sj-pdf-2-mde-10.1177_23821205241269378 - Supplemental material for Impact of Performing Medical Writing/Publishing Workshops: A Systematic Survey and Meta-Analysis

Supplemental material, sj-pdf-2-mde-10.1177_23821205241269378 for Impact of Performing Medical Writing/Publishing Workshops: A Systematic Survey and Meta-Analysis by Behrooz Astaneh, Ream Abdullah, Vala Astaneh, Sana Gupta, Hadi Raeisi Shahraki, Aminreza Asadollahifar and Gordon Guaytt in Journal of Medical Education and Curricular Development

Supplemental Material

sj-pdf-3-mde-10.1177_23821205241269378 - Supplemental material for Impact of Performing Medical Writing/Publishing Workshops: A Systematic Survey and Meta-Analysis

Supplemental material, sj-pdf-3-mde-10.1177_23821205241269378 for Impact of Performing Medical Writing/Publishing Workshops: A Systematic Survey and Meta-Analysis by Behrooz Astaneh, Ream Abdullah, Vala Astaneh, Sana Gupta, Hadi Raeisi Shahraki, Aminreza Asadollahifar and Gordon Guaytt in Journal of Medical Education and Curricular Development

Supplemental Material

sj-pdf-4-mde-10.1177_23821205241269378 - Supplemental material for Impact of Performing Medical Writing/Publishing Workshops: A Systematic Survey and Meta-Analysis

Supplemental material, sj-pdf-4-mde-10.1177_23821205241269378 for Impact of Performing Medical Writing/Publishing Workshops: A Systematic Survey and Meta-Analysis by Behrooz Astaneh, Ream Abdullah, Vala Astaneh, Sana Gupta, Hadi Raeisi Shahraki, Aminreza Asadollahifar and Gordon Guaytt in Journal of Medical Education and Curricular Development

Footnotes

Acknowledgments

The authors highly appreciate the sincere collaboration of Neera Bhatnagar and Hannah McIntosh for their professional help and insight in searching different databases. The authors would also like to thank Eric McMullen, Ya Gao, Kyle Tong, Luis Enrique Colunga Lozano, and John Basmaji for their help during the screening process.

Authors Contributions

BA: Conceptualization, Data curation, Methodology, Project administration, Validation, Writing – original draft, Writing – review & editing; RA: Data curation, Project administration, Writing – review & editing; VA: Data curation, Project administration, Writing – review & editing; SG: Data curation, Project administration, Writing – review & editing; HRS: Formal analysis, Writing – review & editing; AA: Data curation, Project administration, Writing – review & editing; GG: Conceptualization, Data curation, Methodology, Project administration, Supervision, Validation, Writing – review & editing; All authors read and approved the final version and accept the accountability.

Authors’ Note

Availability of data and material: All relevant data are within the manuscript and its Supplemental Materials.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Behrooz Astaneh

Supplemental Material

Supplemental material for this article is available online.

References

Cahn

Benjamin

. Academic writing workshop for medical school faculty. MedEdPORTAL. 2012;8:9289. doi:10.15766/mep_2374-8265.9289

Cabanas

Bridgeman

Hermes-DeSantis

. Publish or perish: success with publication in pharmacy residency training. Curr Pharm Teach Learn. 2018;10(12):1647-1651. doi:10.1016/j.cptl.2018.08.017

Astaneh

Masoumi

. Professional medical writing and ethical issues: a developing country’s perspective. Eur Sci Ed. 2011;37(3):85.

Sarveravan

Astaneh

Shokrpour

. Adherence to the CONSORT statement in the reporting of randomized controlled trials on pharmacological interventions published in Iranian medical journals. Iran J Med Sci. 2017;42(6):532-543.

Shaghaghian

Astaneh

. Adherence to the strengthening the reporting of observational studies in epidemiology statement in observational studies published in Iranian medical journals. Iran J Public Health. 2020;49(8):1520-1529. doi:10.18502/ijph.v49i8.3896

Astaneh

Schwartz

Guyatt

. Biomedical authorship: common misconducts and possible scenarios for disputes. J Acad Ethics. 2021;19(4):455-464. doi:10.1007/s10805-021-09435-z

Barbour

Astaneh

Irfan

. Challenges in publication ethics. Ann R Coll Surg Engl. 2016;98(04):241-243. doi:10.1308/rcsann.2016.0104

Coverdale

Roberts

Balon

, et al. Writing for academia: getting your research into print: AMEE guide No. 74. Med Teach. 2013;35(2):e926-e934. doi:10.3109/0142159x.2012.742494

Rathore

Mansoor

. How to conduct a workshop on medical writing: tips, advice and experience sharing. J Pak Med Assoc. 2015;65(6):665-668.

10.

Cargill

Smernik

. Embedding publication skills in science research training: a writing group programme based on applied linguistics frameworks and facilitated by a scientist. High Educ Res Dev. 2016;35(2):229-241. doi:10.1080/07294360.2015.1087382

11.

Johnson

Symes

PhD

, et al. Mentoring disadvantaged nursing students through technical writing workshops. Nurse Educ. 2007;32(4):168-172. doi:10.1097/01.NNE.0000281087.43088.f5

12.

Oyibo

. Developing a beginner's guide to writing a clinical case report: a pilot evaluation by junior doctors. Cureus. 2019;11(12):e6370. doi:10.7759/cureus.6370

13.

Rathore

Fatima

Farooq

, et al. Combating scientific misconduct: the role of focused workshops in changing attitudes towards plagiarism. Cureus. 2018;10(5):e2698. doi:10.7759/cureus.2698

14.

Steinert

McLeod

Liben

, et al. Writing for publication in medical education: the benefits of a faculty development workshop and peer writing group. Med Teach. 2008;30(8):e280-e285. doi:10.1080/01421590802337120

15.

Wajekar

Salgaonkar

Chincholi

, et al. Impact of basic medical writing workshop on case report writing by postgraduate anaesthesia trainees: a pilot study. Indian J Anaesth. 2018;62(7):502-508. doi:10.4103/ija.IJA_98_18

16.

Galipeau

Moher

Campbell

, et al. A systematic review highlights a knowledge gap regarding the effectiveness of health-related training programs in journalology. J Clin Epidemiol. 2015;68(3):257-265. doi:10.1016/j.jclinepi.2014.09.024

17.

Moher

Liberati

Tetzlaff

, et al. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Ann Intern Med. 2009;151(4):264-269. doi:10.7326/0003-4819-151-4-200908180-00135

18.

Henninger

Nolan

. A comparative evaluation of two educational strategies to promote publication by nurses. J Contin Educ Nurs. 1998;29(2):79-84. doi:10.3928/0022-0124-19980301-10

19.

Pololi

Knight

Dunn

. Facilitating scholarly writing in academic medicine. J Gen Intern Med. 2004;19(1):64-68. doi:10.1111/j.1525-1497.2004.21143.x

20.

Ried

Montgomery

Stocks

, et al. General practice research training: impact of the Australian Registrar Research Workshop on research skills, confidence, interest and involvement of participants, 2002-2006. Fam Pract. 2008;25(2):119-126. doi:10.1093/fampra/cmn010

21.

Cameron

Deming

Notzon

, et al. Scientific writing training for academic physicians of diverse language backgrounds. Acad Med. 2009;84(4):505-510. doi:10.1097/ACM.0b013e31819a7e6d

22.

Rickard

McGrail

Jones

, et al. Supporting academic publication: evaluation of a writing course combined with writers’ support group. Nurse Educ Today. 2009;29(5):516-521. doi:10.1016/j.nedt.2008.11.005

23.

Sridhar

ARM

Willett

Castiglioni

, et al. Scholarship opportunities for trainees and clinician educators: learning outcomes from a case report writing workshop. J Gen Intern Med. 2009;24(3):398-401. doi:10.1007/s11606-008-0873-9

24.

Richardson

Carrick-Sen

. Writing for publication made easy for nurses: an evaluation. Br J Nurs. 2011;20(12):756-759. doi:10.12968/bjon.2011.20.12.756

25.

Rubens

. Science writing workshops for AIDS vaccine researchers in Africa. Med Writing. 2013;22(2):108-111. doi:10.1179/2047480613Z.000000000117

26.

Guillerm

Berger

Bissell

, et al.

What happens after participants complete a union-MSF structured operational research training course?

Public Health Action. 2014;4(2):89-95. doi:10.5588/pha.13.0014

27.

Jernigan

VBB

Brokenleg

Burkhart

, et al. The implementation of a participatory manuscript development process with Native American Tribal awardees as part of the CDC Communities Putting Prevention to Work initiative: challenges and opportunities. Prev Med. 2014;67(S1):S51-S57. doi:10.1016/j.ypmed.2014.01.027

28.

Islam

Gunaseelan

Khan

. A research elective course on dietary supplements to engage doctor of pharmacy students in primary literature evaluation and scholarly activity. J Pharm Pract. 2015;28(6):577-584. doi:10.1177/0897190013516510

29.

Kulage

Larson

. Implementation and outcomes of a faculty-based, peer review manuscript writing workshop. J Prof Nurs. 2016;32(4):262-270. doi:10.1016/j.profnurs.2016.01.008

30.

Duncanson

Webster

Schmidt

. Impact of a remotely delivered, writing for publication program on publication outcomes of novice researchers. Rural Remote Health. 2018;18(2):4468. doi:10.22605/RRH4468

31.

Gardner

Salto

Riggs

, et al. Supporting the writing productivity of biomedical graduate students: an integrated, structured writing intervention. CBE Life Sci Educ. 2018;17(3):ar45. doi:10.1187/cbe.16-12-0350

32.

Arrazola

Polster

Etkind

, et al. Lessons learned from an intensive writing training course for applied epidemiologists. Public Health Rep. 2020;135(4):428-434. doi:10.1177/0033354920932659

33.

Dhakal

Tornwall

. The scholarship circle: an introduction to writing for publication for nursing faculty. J Med Libr Assoc. 2020;108(1):98-105. doi:10.5195/jmla.2020.685

34.

Harvey

Barker

Tynan

. Writing a manuscript for publication: an action research study with allied health practitioners. Focus Health Prof Educ. 2020;21(2):1-16.

35.

Suma

Hathur

Kulkarni

, et al. Research fair: a three pronged innovative effort for fruitful publications out of post graduate dissertations. Int J Health Allied Sci. 2020;9(3):277-283. doi:10.4103/ijhas.IJHAS_161_20

36.

Tsujimoto

Kataoka

Sato

, et al. A model six-month workshop for developing systematic review protocols at teaching hospitals: action research and scholarly productivity. BMC Med Educ. 2021;21(1):98. doi:10.1186/s12909-021-02538-6

37.

Chandratre

Marfowaa

Abdel-Reheem

, et al. Promoting mentorship and scholarship among underrepresented minority medical students. WMJ. 2022;121(3):171.

38.

Sheikh

Levy

. Writing research protocols: an innovative approach. Asthma Gen Pract. 1999;7(3):39-42. doi:10.1038/pcrj.1999.22

39.

Jawaid

Masood

, et al. An analysis of interactive hands-on workshops on medical writing. J Pak Med Assoc. 2011;61(1):66-70.

40.

Sabouni

Chaar

Bdaiwi

, et al. An online academic writing and publishing skills course: help Syrians find their voice. Avicenna J Med. 2017;7(3):103-109. doi:10.4103/ajm.AJM_204_16

41.

Goyal

Misra

Rajadhyaksha

, et al. Effectiveness of a 1-day workshop on scientific writing conducted by the Indian journal of rheumatology. Indian J Rheumatol. 2018;13(2):117-120. doi: 10.4103/injr.injr_36_18

42.

Trigotra

Jaiswal

Mittal

, et al. Impact of a publication ethics orientation program on the knowledge and attitude of postgraduate students of health sciences. J Clin Diagn Res. 2019;13(2):LC04-LC09. doi:10.7860/JCDR/2019/40008.12548

43.

Goyal

Dua

Kedia

, et al. Usefulness of a workshop on scientific writing and publication in improving the baseline knowledge deficit among postgraduates. J R Coll Physicians Edinb. 2020;50(3):316-321. doi:10.4997/JRCPE.2020.323

44.

Agarwal

Leisegang

Panner Selvam

, et al. An online educational model in andrology for student training in the art of scientific writing in the COVID-19 pandemic. Andrologia. 2021;53(3):e13961. doi:10.1111/and.13961

45.

de Oliveira

LMA

Cordeiro-Spinetti

Neves

FPG

, et al. Going online in pandemic time: a DivulgaMicro workshop experience. J Microbiol Biol Educ. 2021;22(1):22.21.79. doi:10.1128/jmbe.v22i1.2493

46.

Fernandez

Garcia

Seres

, et al. Students’ satisfaction and perceived impact on knowledge, attitudes and skills after a 2-day course in scientific writing: a prospective longitudinal study in Spain. BMJ Open. 2018;8(1):e018657. doi:10.1136/bmjopen-2017-018657

47.

Bydder

Packer

Semmens

. The value of a scientific writing training workshop for radiologists and radiation oncologists. Australas Radiol. 2006;50(1):29-32. doi:10.1111/j.1440-1673.2005.01543.x

48.

Phadtare

Bahmani

Shah

, et al. Scientific writing: a randomized controlled trial comparing standard and online instruction. BMC Med Educ. 2009;9(27). doi:10.1186/1472-6920-9-27

49.

Cameron

Chang

Pagel

. Scientific English: a program for addressing linguistic barriers of international research trainees in the United States. J Cancer Educ. 2011;26(1):72-78. doi:10.1007/s13187-010-0143-5

50.

Geithner

Pollastro

. Doing peer review and receiving feedback: impact on scientific literacy and writing skills. Adv Physiol Educ. 2016;40(1):38-46. doi:10.1152/advan.00071.2015

51.

Salamonson

Koch

Weaver

, et al. Embedded academic writing support for nursing students with English as a second language. J Adv Nurs. 2010;66(2):413-421. doi:10.1111/j.1365-2648.2009.05158.x

52.

Kelley

. Statistical models for meta-analysis: a brief tutorial. World J Methodol. 2012;2(4):27-32. doi:10.5662/wjm.v2.i4.27

53.

Murray

Newton

. Facilitating writing for publication. Physiotherapy. 2008;94(1):29-34. doi:10.1016/j.physio.2007.06.004

54.

Aniket

. Managing research misconduct: is anyone getting it right? Br Med J. 2011;343(d8212). doi:10.1136/bmj.d8212

55.

Elizabeth

. Coping with scientific misconduct. Br Med J. 2011;343(d6586). doi:10.1136/bmj.d6586

56.

Marušić

. Author misconduct: editors as educators of research integrity. Med Educ. 2005;39(1):7-8. doi:10.1111/j.1365-2929.2004.02050.x

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

1.66 MB

1.69 MB

2.42 MB

0.08 MB