Sage Journals: Discover world-class research

Abstract

Using the technical and professional communication service course as the site for research, and student learning outcomes (SLOs) as the specific focus, we gathered, coded, and analyzed 503 SLOs from 93 institutions. Our results show the top outcomes are rhetoric, genre, writing, design, and collaboration. We discuss these outcomes and then we offer programmatic implications drawn from the data that encourage technical and professional communication program administrators and faculty to use common SLOs, to improve outcome development, and to reconsider the purpose of the service course for students.

Keywords

student learning outcomes service course

Service courses in technical and professional communication (TPC) often do not receive the same sort of attention as other curricular areas. Most likely, this is because the course “always exists in an institutional and curricular environment that responds to other courses, programs, and units and the experiences of students and teachers is formed and shaped, to a greater or lesser extent by the expectations and goals of others” (Russell, 2007, pp. 261–262). This relegation to something seen as an ancillary interest leaves it to receive less programmatic effort, which Read and Michaud (2018a) go as far to say that the course suffers from a “systemic underinvestment” (p. 19).

While the service course may not always get its full due, scholars have, on occasion, taken a closer look at the course. For example, Kimball (2017) argued for greater incorporation of technical communication within college writing instruction and highlighted the need for service courses “serve students in roles well beyond their professional identities” by helping them develop their communication skills (p. 348). Schreiber et al. (2018) edited a special journal issue where the articles focused on the course's sustainability (Arduser, 2018; Carnegie, 2018); an examination of assignments to replicate the demands of the workplace (Francis, 2018); and a framework for contextual analysis to better position the course within institutional settings (St.Amant, 2018). More recently, Bay (2022) and Chen (2021) described their approaches to updating courses to include issues of justice and equity, while Doan (2021) discussed how instructors are teaching the common assignment of resumes and cover letters.

We enter into these conversations and follow the definition of service course as “introductory courses for nonmajors delivered primarily as a service to other departments and programs on campus” (Meloncon and England, 2011, p. 398). We focus on service as a benefit to others, particularly the way these courses serve students. In fact, the goal of the service course is to prepare students to “adap[t] emergent knowledge to specific workplace or community-based contexts” (Scott, 2008, p. 382) by focusing on “real-audience needs, problem solving, and learning to communicate information that has real cultural, legal, and ethical obligations” (Melonçon, 2018a, p. 208). More directly, however, we provide an answer to specific calls for “research on the outcomes of the service course” because TPC has “little understanding of outcomes at the field level” (Melonçon, 2018a, p. 223) and there is potential for a “a shared set of standardized outcomes’‘ that could align curricula and increase instructor confidence (Read & Michaud, 2018a, p. 244). We wanted to focus on student learning outcomes (SLOs) because they should describe for students the goals of the learning experience (see e.g., Allan, 1996; Carriveau, 2016; Clegg et al., 2021; Driscoll & Wood, 2007).

Using the service course as the site for research, and SLOs as the specific focus, we asked: If we collected and examined current SLOs from diverse institutions across the US, would a set of common field-wide outcomes emerge? To answer this question, we begin by discussing the outcomes literature in TPC. Next, we describe how we gathered, coded, and analyzed 503 SLOs from 93 institutions. Our results show the most common outcomes in the data set are rhetoric, genre, writing, design, and collaboration. Drawing from the data and existing research, we describe these outcomes and discuss what was surprising or missing from the data. In the final section, we discuss programmatic implications that call for TPC program administrators (TPC PAs) and faculty to improve outcome development and to reconsider the purpose of the service course.

Like what Clegg and his collaborators (2021) did at the TPC undergraduate degree program level, our intent with this research is to provide a data-driven, field-wide snapshot of SLOs in the service course at a diverse range of institutions rather than attempt to create a prescriptive “outcomes statement.” We hope these findings can serve as a baseline for TPC PAs and faculty to assist them with creating SLOs for their own service courses, while transparently communicating those goals to students to create a cohesive learning experience.

What TPC Knows About Student Learning Outcomes

Allen (2004) argued for the need to embed assessment of learning outcomes in programmatic work, to “monitor the ongoing needs” (p. 95) of the program rather than including it as an afterthought. TPC did take Allen's advice to heart and to date the field has a robust body of work that addresses assessment and SLOs (see e.g., Carter et al., 2003; Coppola et al., 2016; Henschel & Melonçon, 2014; St.Amant & Nahrwold, 2007). Much of this published research is conceptual in nature, that is, describing the concept of how or why to use outcomes for assessment. An example of conceptual articulation of outcomes for assessment is Say (2015), who advocated for outcome use in graduate program assessment, but did not specifically examine SLOs at any institutions. Outside of these studies with emphasis on program assessment, the field has few examples of using SLOs for assessment at the course level. The most notable example is Johnson and Elliot (2010), who provided a detailed, statistical model for using SLOs to assess a single course within their undergraduate major.

Even though these studies on SLOs as part of assessment provide TPC PAs important examples, they do not address using SLOs, writing SLOs, or using them to build student centered courses. Borrowing on the voluminous work on course outcomes from composition studies (see e.g., Behm et al., 2012; Gallagher, 2019; Harrington et al., 2005), Maid (2005) used the Council of Writing Program Administration's (WPA) Outcomes Statement for first-year writing to guide the creation of outcomes and an assessment plan for a TPC degree program. Maid (2005) suggested that with minor changes, these “all encompassing” (p.140) outcomes fit the needs of the program. Later, though, Maid and D’Angelo (2012) reconsidered Maid's previous view on the use of WPA outcomes statement and concluded that the outcomes from the WPA Outcomes Statement did not speak to what TPC needed as a whole. Also prompted by the WPA outcomes statement, Ilyasova and Bridgeford (2014) established a descriptive outcomes statement specifically for TPC. The authors use their own TPC programs to examine how to implement these outcomes. They explained that the aim was to “make explicit these shared practices, what our field values, and what we expect students to learn in our programs” (Ilyasova & Bridgeford, 2014, p. 59), echoing earlier calls like those of Allen (2004).

Outcomes in Single Institution Research

Specific to the service course, TPC has several key examples of the implementation of use of SLOs. Using a business communication course, Lucas and Rawlins (2015) developed a competency framework at a single institution that they said could be adapted to other programs and institutions. Similar to outcomes, these competencies were used to explicitly “connec[t] both learning and grading with the development of practical skills” (Lucas & Rawlins, 2015, p. 190). In another example from a single institution, Newmark and Bartolotta (2021) ask if it's possible to “reformulate a curriculum…so that the founding premises of that curriculum have an eye towards students’ eventual workplace realities” (Newmark & Bartolotta, 2021, p.148). They work to answer this question through the creation of nine SLO's.

Another example of discussing outcomes in a course similar to the service course is Kenzie and McCall (2018). In looking at a healthcare writing course at a large R1 institution, Kenzie and McCall echo Ilyasova and Bridgeford's (2014) program outcomes statement by modifying it for a course and aligning outcomes to proposed assignments. The authors explained that by using programmatic outcomes they were able to structure the course learning outcomes, which are then echoed in the assignment sequence. Kenzie and McCall also focus their work and insights on their own institutional setting.

Outcomes in Field Wide Research

The work noted to this point focused on single institutional settings. However, TPC scholarship has recently argued for more field-wide research (see e.g., Melonçon, Rosselot-Merritt & St.Amant, 2020 and St.Amant & Graham, 2019) because “programs need data from other programs to help them contextualize their own data” (Schreiber & Melonçon, 2019, p. 257). Using a study design focused on gathering information from across institutions, Barker (2012) conducted a pilot study that examined a handful of TPC programs’ student learning outcomes (PSLOs). Notably, Clegg and his collaborators (2021) offered TPC the most developed examination of the use of program SLOs (PSLO's), which are SLOS that guide each course in a degree program. They identified common PSLOs amongst TPC undergraduate degree programs in the US, and they argued that a greater understanding of field-wide program SLOs can assist TPC PAs in revising and developing programs (Clegg et al., 2021, p. 27). Ultimately, the authors suggested “using common PSLO's, eliminating embedded outcomes, and considering PSLO's beyond assessment” (Clegg et al., 2021, p. 19) as ways of building and sustaining TPC programs. Thus, Clegg et al. offered TPC the first field-wide look at PSLOs. Currently, TPC lacks a field-wide view of SLOs in the most ubiquitous course the field teaches—the service course. In the next section, we describe how we gathered and coded a snapshot of field wide SLO data.

Methodology

Across the nation, 311 institutions house TPC degree programs, all of which offer some iteration of the service course with enrollments from two to 100 sections per term (Melonçon, 2022), and 79% of community colleges offer the service course (Bivens et al., 2020). In addition to these figures, there are literally hundreds of uncounted service courses offered at institutions that do not offer TPC degree programs. Since TPC does not have complete data of the total number of institutions where the service course is offered, we were unable to create a quantitative sampling plan that would allow for a high confidence interval. Thus, we started with a purposive sample, which means “subjects are selected based on study purpose with the expectation that each participant will provide unique and rich information of value to the study” (Etikan, 2016, p. 4). In this case, the characteristics were faculty who teach the service course with the expectation faculty would provide information about their specific course. Then from within the broader purposive sample, we invoked a convenience sample, which follows the standard definition of participants being drawn from a source conveniently accessible to the researcher. This dual approach meant we contacted faculty and administrators in our networks that teach the service course or direct a program focused on the TPC service course. The findings of a study based on convenience and purposive sampling can only be generalized to the (sub)population from which the sample is drawn and not to the entire population. The findings can certainly, however, establish a historical baseline that could be used for future research, while also offering information for TPC PAs to use immediately in their programmatic work.

We chose to gather syllabi since it is the record from specific courses where SLOs are most likely to be routinely recorded and available to students. We gathered syllabi from 93 institutions (see Table 1) that resulted in 503 outcomes from 76 institutions with 18% of institutions (n = 17) providing materials that contained no SLOs. The institutional review boards (IRBs) of [redacted] reviewed this project, and based on the terms of those IRBs, information is shared in aggregate, and institutions are only identified by their Carnegie Classification. (Refer to https://carnegieclassifications.iu.edu/classification_descriptions/basic.php for more details.) The SLO data is available for download [URL redacted].

Table 1.
Type of institution and number of syllabi in the data set.

Type of institution # of syllabi

R1 27

R2 13

R3 8

D/PU 1

M1/ML 24

M2/MM 5

M3/MS 1

BAC 3

Associates 11

Description of Coding Process

We use “code” to mean “a word or short phrase that symbolically assigns a summative…attribute for a portion of language-based or visual data” (Saldana, 2009, p. 3), and we understand the process of coding as systematic (Geisler, 2018) and iterative (Saldana, 2009). We started with the codebook created by Clegg et al. (2021), whose work examined TPC undergraduate degree program outcomes. Even though we are looking at a single course (that is often not part of a degree program), we still felt that many of the codes would apply and, if necessary, could be adapted as our coding process unfolded. Ultimately, the only modification to Clegg et al.'s (2021) code book was one of absence; we had no outcomes coded as theory. Thus, our final codebook included 16 codes. See Appendix A for codes and descriptions.

We approached the coding process by implementing Smagorinsky's (2008) collaborative coding, which he described as a process wherein “we reach agreement on each code through collaborative discussion rather than independent corroboration” (p. 401). Our research team is composed of three people at varying stages of their careers, which means each of us approached course level outcomes quite differently based on our knowledge and experiences. Collaborative coding offers “a means through which levels of expertise may emerge through the process of discussion in relation to data” (Smagorinsky, 2008, p. 402). Thus, this process allowed space for our differing expertise and knowledge, and it also allowed for discussions of differences until we arrived at a consensus, which ensured that all our expertise was recognized and integrated.

We used a two-stage coding process and then a consistency coding check that affords TPC the opportunity to replicate research and to offer findings that have an increased level of trustworthiness through transparency that is key to qualitative research studies.

Coding Process

In the first round of coding, one of the authors coded the entire dataset. The other two authors then coded the outcomes marking only the outcomes where a difference in interpretation of the outcome occurred. This first round of coding resulted in an 82% agreement between the three researchers. In the second round of coding, the authors met to reconcile the codes where there was no agreement. For the outcomes in which there was disagreement (18%, n = 100), we followed the concept of consensus as described by Smagorinsky (2008) and refined by Clegg et al. (2021). We met and discussed the differences until we settled on what the final code would be.

At this stage of the coding process, we again followed Clegg et al. (2021) by paying close attention to outcomes as a contextualized set of knowledges, skills, and abilities. Even though Clegg et al. (2021) examined program outcomes, thinking holistically also worked well within the context of a single course's outcomes. For example, a ML institution had seven outcomes. Initial round coding resulted in three disagreements. During consensus discussions, we considered that rather than two writing outcomes one of them also had an emphasis on editing. The second writing outcome was changed to editing. Because like Clegg et al., “we assumed a school wouldn’t want every outcome to be coded the same, so we discussed whether that outcome was appropriately coded or if another appropriate code fits the parameters, given the context of the institution's other outcomes” (p. 22). Once we completed the second round of coding, we performed consistency coding.

Consistency Coding

We also used Clegg et al.'s (2021) consistency coding, which they defined as “the internal process of ensuring that we applied the same code for like items” (p. 4). This step is used to check for inconsistencies in the application of a code. For example, all outcomes coded as “writing” were reviewed for consistency of use for that code. Consistency coding was done by one member of the research team. In this step of coding, only 5 outcomes were flagged for consistency issues. Following the same process of collaborative coding described above, the authors discussed and changed 2 codes (leaving 3 unchanged). This step of consistency coding underscores Saldana's (2009) suggestion that coding is an iterative process.

Limitations

SLO's are constantly in the process of being updated, due to shifting programs, assignments, and leadership, and there is no way to completely know the impact of local institutional pressures. For example, we had no way of knowing if faculty (or programs) are restricted or encouraged to limit to a certain number of SLOs. Terminology could also have been an influencing factor because we included outcomes specifically labeled as such in sample syllabi, for example the term “outcomes” could have also included the terms “objectives” or “goals”. While this may have been due to programmatic or university level idiosyncrasies, the inconsistency and differences in terminology led us to only include the information under outcomes which may have affected the findings.

Even though the data suggests some commonalities that can lead to effective practice, it is important to note that the data is not generalizable. This is an historical dataset, meaning information will have variations depending on the time it was gathered. The strength of an historical dataset lies in its ability to provide a baseline for future research on SLOs in the service course by enabling comparison over time. Additionally, future research should develop a more systematic sampling plan that could offer results that could be more generalizable and more so, to determine if any differences exist between types of institutions (e.g., those with TPC degree programs and those without).

Findings of Service Course SLOs

In this section, we report the findings from the field wide SLOs. Figure 1 shows the relationship between the institutions and the number of SLOs that institution includes for their service course.

Figure 1.
Relationship between percentage of institutions and the number of SLOs in a service course.

Figure 1 shows the relationship between percentage of institutions and the number of SLOs in a service course, detailing the number of institutions that fall within a specific range of outcomes. The most common range was 4–6 outcomes with 29% of schools (n = 27) falling into this category, while the lowest range was 1–3 for 12% (n = 11). There were 18% of schools (n = 17) that had no outcomes, and 14% (n = 13) that had over 10 outcomes. The most common range of 4–6 SLOs aligns with educational research that suggests 4–7 SLOs are best practices (Kuh, Jankowski, Ikenberry, & Kinzie, 2014).

Most Common Unique Outcomes

We coded 503 outcomes, and we present them in Table 2 from two different perspectives:
Total outcomes

Unique outcomes

Table 2.
All codes (n = 16) and counts of unique outcomes (n = 419) and total outcomes (n = 503).

Code # unique # total

Rhetoric 51 69

Genre 47 61

Writing 45 59

Design 42 54

Collaboration 34 37

Communication 27 29

Critical thinking 27 28

Research 27 28

Professionalization 24 27

Technology 20 26

Editing 18 20

Ethics 17 18

Practice 14 16

Project management 12 16

Culture 8 9

Other 6 6

Total outcomes is self explanatory and accounts for each outcome in the data set (n = 503). Unique outcomes eliminate any duplicate outcome at the same school (n = 419). For example, an R1 institution with 14 outcomes includes 2 outcomes each for rhetoric, writing, technology, and genre. In summarizing the unique outcomes, we then only included one outcome from each of those four coding categories.

The top five outcomes are rhetoric, genre, writing, design, and collaboration. The order of the top five outcomes did not shift in the total outcomes and the unique outcomes. This suggests that not only did our sample find these categories important, but it also speaks to the importance of these concepts to the whole of TPC. To find further patterns within our data, and because the top five did not change, we examined the codes to show the percentage of institutions with the inclusion of the most common outcomes.

Figure 2 shows the relationship between the top five outcomes (rhetoric, genre, writing, design, and collaboration) and the percentage of institutions who have that number of SLOs in the service course.

Figure 2.
Relationship between institutions and the incorporation of the five most common outcomes.

Of the 93 institutions, 24% percent (n = 22) of institutions included three of the common outcomes, making this the most prevalent. While most included at least three outcomes, it is important to note that 18% (n = 17) of them had no outcomes listed on their syllabi. This is similar to the number of institutions, also 18% (n = 17) who included four common outcomes. Additionally, only 9% (n = 8) of institutions included all five of the most common SLOs. Finally, the remaining categories use 1–2 outcomes, with 14% (n = 13) including one and 17% (n = 16) including two.

Explanation of Common SLOs

In this section, we offer insights from the data into what representative outcomes looked like in the data. These examples from diverse institutions can be used as a guide to assist TPC PAs and faculty in developing their own SLOs.

Rhetoric

The rhetoric code included any outcomes focused on traditional rhetorical elements such as purpose, audience, context, and the rhetorical situation. The code was designed to identify TPC's emphasis on rhetorical awareness.
Analyze rhetorical aspects of audience, purpose, and context to communicate technical information effectively in written, oral, and visual media. (R2)

Appropriately adapt tone, style, and content depending on audience, purpose and genre (R1)

Students will demonstrate an awareness of the rhetorical situation by identifying and analyzing various types of audiences and purposes faced in the technical/professional environment (ML)
Finding this SLO as one of the most common is not surprising considering TPC's emphasis on rhetoric as a key theory (Dubinsky, 2018), and Maid and D'Angelo's (2012) reflection on whether rhetoric was the uber outcome.

Genre

The genre code included outcomes focused on specific forms of documents. The code was applied to categorizations of genre in general as well as specific genres, such as reports, instructions, or correspondence.
Identify different types of technical documents and their elements (R3)

Be able to produce a variety of documents commonly used in the workplace (R1)

To write various types of technical reports (A)
The first two SLOs from the data illustrate the broad emphasis on genre, while the third one illustrates a specific type of genre (technical report).

Genre as a common SLO aligns with previous research on the service course that found genres were key to teaching the course (Melonçon, 2018a; Read & Michaud, 2018a). Previous research also highlighted the emphasis within TPC curricula on genre awareness (Henze, 2018; Meloncon & Henschel, 2013) and in being able to write common types of workplace genres (Blythe et al., 2014).

Writing

The writing code included outcomes that explicitly foregrounded the practice of writing or the writing process to include composition of texts and elements of style.
Develop an effective, clear writing style (ML)

Students will understand and know how to follow the stages of the writing process (prewriting/writing/rewriting) and apply them to technical and workplace writing tasks (A)

Draft and revise writing to ensure concision, clarity, cohesion, and coherence (R1)
Because research within TPC on writing relies on the style and development of the writing itself (Jones, 2018), it was again unsurprising that writing emerged as a top outcome. The first and third SLO's from the data focused on style, while the second looked at the writing process itself. While these SLO's are somewhat similar to the above outcome of rhetoric, with regards to style, these outcomes focused more on the production of the writing.

Design

The design code included a wide range of interpretations and applications related to the integration of document design principles or the creation of visualizations. In understanding the differences among document design and visual literacy, this code was meant to capture TPC's emphasis on visual-related skills. The following outcomes are representative of the outcomes coded as design:
An awareness of how text organization and the overall design of a document contributes to its effectiveness (R1)

Demonstrate that you understand how to integrate written content, graphics, and basic design principles to create usable, persuasive, and reader-friendly documents (R2)

Construct graphics that clearly communicate information visually (ML)
Scholars in TPC have argued for some time of the importance and necessity of visual communication (see e.g., Brumberger & Northcut, 2013; Wolfe, 2015; Brumberger, 2018). A top SLO focused on aspects of visual communication aligns with larger trends within the field, as well as the necessity for this skill in the workplace (Brumberger, 2007).

Collaboration

The collaboration code included outcomes that focus on working with people. The code was applied to statements emphasizing collaborative work, such as team projects and discussions.
Collaborate with peers to produce documents (ML)

Develop interpersonal & team interaction skills (R1)

To be productive in collaborative groups in a professional setting (A)
There is also a connection between common SLOs and the language used in the workplace on what comprises value. For example, according to a recent report, 62% of employers reported collaboration as a key skill and ability for college graduates (Finley, 2021). Collaboration has also been the focus of TPC research for some time for these same reasons (see e.g., Brewer & Holmes, 2016; England & Brewer, 2018; Wong et al., 2016).

The data suggest that rhetoric, writing, genre, design, and collaboration are key outcomes for students to takeaway from the service course. These outcomes do align with TPC scholarship and what the field has determined to be key skills and abilities (see e.g., Henschel & Melonçon, 2014; Stanton, 2017) that position students to be successful in the workplace and as an informed citizen. This type of field wide data gives TPC PAs and faculty a needed data point to contextualize their local data.

What is Surprising from the SLOs Data

As a collaborative authorship team composed of a PhD candidate, an early career faculty member (two years on the job) and a mid-career faculty member (14 years on the job), the following are what we all agreed were surprising findings from the field-wide data.

The number of institutions that include the common unique outcomes is less than we would have expected. The common knowledges, skills, and abilities to be a successful workplace communicator and an active and engaged citizen writer have remained consistent through the years. We then expected to see more institutions have more of the common SLOs. For example, understanding purpose and audience (rhetoric) is key to writing effectively. However, not every school even included rhetoric as an SLO. Admittedly, TPC PAs and faculty may be omitting this SLO because it is obvious and necessary, but it begs the question of what this assumption does for student's understanding of what they should be getting out of the course.

We also had concerns about the writing and editing outcomes. When the writing outcome is read alongside those labeled as editing, too many SLOs focused on academic writing or proofreading rather than more complex issues that may be found in writing situations in the workplace. While recent research has discussed errors in workplace writing (Gubala et al., 2020), one of the implications of that research asked how and why to integrate issues of grammar into the TPC service course. As a field, TPC needs to engage in conversations about the correctness of writing and its place and role within the service course. As Gubala and her collaborators point out (2020), there is tension between linguistic justice in TPC classrooms and expectations in workplaces that the field needs to grapple with. Thus, the writing outcome needs to be expanded to include more specificity, as well as more nuance on what aspects of writing will be the focus.

Finally, we were surprised about the lack of direct emphasis on ethics and/or issues of justice. With the scholarly emphasis on social justice and its role specific to teaching and program design (e.g., Walton & Agboka, 2021; Hitt, 2018; Mallette & Hawks, 2020; Medina & Walker, 2018; Melonçon, 2018b), the field needs to consider if an implicit focus on ethics or justice needs to be made explicit in SLOs. That is, few schools included an SLO specific to ethics and none specifically invoked “justice.” However, some programs made it clear in other locations within the syllabus that the program had a foundation in or commitment to understanding the role of writing in issues of ethics and justice in the workplace. TPC has to think forward to ensure this focus can be attained by attending to the many stakeholders of these courses (St.Amant, 2018) and more so, to demonstrate in concrete ways how issues of ethics, justice, and inclusion can work and be challenged in workplaces and in the world.

What the SLO Data may Mean Moving Forward

The findings section answered our primary research question: If we collected and examined current SLOs from diverse institutions across the US, would a set of common field-wide outcomes emerge? Some common outcomes were identified, but after our initial work with the data, we found ourselves considering what we should do with the data. In this section, we answer the “so what?” question in two distinct ways. First, we encourage faculty to improve SLO development, specifically in writing clearer and more student friendly outcomes. Then, we move to a larger pedagogical and values driven concern of how the SLO data raise a number of questions and concerns abut the purpose of the service course. This second consideration of what the field wide data may mean is more speculative and preliminary and as much data driven research does, posits additional questions for TPC to consider.

Improve Outcome Development and use

While we agree with Clegg and his collaborators (2021) that outcomes should be student facing, we have to question whether students fully understand the outcomes as they are written. Many of the SLOs in our data set were embedded outcomes, which is defined as “single outcome statements that include more than one outcome” (Clegg et al., 2021, p. 4). Embedded outcomes conflate skills, causing confusion for students about which tasks contribute to specific skills. With embedded outcomes, students are unable to parse one skill from another, which leaves the student with an unclear understanding of what they are being asked to do. Embedded outcomes present an opportunity for increasing outcome clarity. When reading an outcome, students should clearly understand the single skill that outcome emphasizes. Following are some examples of embedded outcomes:
To use appropriate hardware and software to produce reports, instructions, and proposals in print or electronic form (ML)

Reveal the organization of their communications by using forecasting and transitional statements, headings, and effective page/document design. Observe appropriate generic conventions and formats for technical documents. (R1)
In the example from the ML above, the outcome seems to indicate a proficiency in technological literacy (hardware and software) and in specific types of genres (reports, instructions, and proposals). Better outcomes would split these apart and then move to make the technological literacy outcome more specific, while also making clear the types of genres that would be expected and assessed in the course. The R1 example has similar but different problems. It is conflating writing and style, design, and genres all in a single outcome. By splitting these single outcomes apart into three outcomes and then attaching them (with more specific outcomes) to assignments, students would be able to see the course expectations and start to gain an understanding of the knowledge and practice of writing more clearly. Making explicit the connections between the learning outcomes, assignments, and the assessment mechanisms for those assignments ensures that faculty clearly communicate specific learning expectations for students (see e.g., Carriveau, 2016).

Strong verbs that direct students toward learning are also necessary (Ilhan & Gezer, 2017). Outcomes without strong verbs means students may not understand what the outcome really means. For example, an ML institution included this outcome: Develop the most common types of documents produced in professional settings. The word “develop” does not identify for students a concrete deliverable. In addition to failing to provide clear understanding for students, the outcome does not identify characteristics that could be commented on in formative or summative assessment tied to outcomes. A word like “create” or “design” communicates to students a concrete set of skills that produces a specific deliverable, skills that can be applied in the creation of the document. The skills that contribute to “creating” a document can be commented on in formative or summative assessment with direct ties to an outcome. Use of strong verbs can build from understanding, application, and creation, allowing students and instructors to share a common language in generating and assessing student work.

As another example, following are the SLOs from another ML institution
Understand the components of technical writing and technical communication

Identify the steps in the writing process

Perform the steps in the writing process

Create audience profiles

Write documents that demonstrate readability and coherence

Understand basic design principles that apply to page design and to document design

Understand basic principles of the use of graphics

Understand the elements of usability

Create documents with usability needs addressed

Understand the elements of reporting

Understand the elements of writing a proposal

Write effective descriptions and instructions

Create electronic communication appropriate to the workplace

Understand and use correct and varied sentence structure
The ML example above illustrates the need for better outcome development. The SLO verbs, such as understand, are not strong and descriptive for students. For example, what does “understand the basic principles of the use of graphics” mean? Is this an outcome on learning theories behind graphics, the way to create graphics, or choosing appropriate graphic types depending on the goals of the project? A better outcome would include a different and descriptive verb based on the level of proficiency expected as it relates to what is truly meant by understanding graphics. By using strong verbs, writing SLOs that are concrete and accessible, and using language that is transparent to students, these SLOs can be made more useful to both students and instructors.

Beyond improving outcome development, TPC PAs and faculty need to use SLOs to develop and to maintain the service course. Moving beyond assessment (Clegg et al., 2021; Sonnenberg et al., 2022), SLOs can help guide faculty and students in knowing what the course does, which aligns with Allen (2004) who argued that TPC must be ready to “identify our own standards of performance in a technical communication course, the kinds of experiences we want students to have in our programs, and the outcomes we hope to encourage” (p. 95). Naming what we expect for students to learn—the experiences and outcomes—is particularly important for the service course. Since the majority of service courses are taught by contingent faculty, who do not have a specialization in TPC (Read & Michaud, 2018a), these instructors can use the SLOs to better understand the rationale of the writing course as it relates to their own disciplinary knowledge. Using the SLOs as a guide, students can better articulate what they know and what they can do with writing and communication skills.

Reconsider the Purpose of the Service Course

The most common outcomes in the data set for the service course mirrored the most common outcomes for TPC undergraduate degree programs (see Clegg et al., 2021). This alignment led us to wonder how a single course could achieve the same outcomes as a full degree program. The short answer is that it could not. While this similarity does suggest key concepts that TPC values for student learning, it also raised the question about the purpose of the service course. Namely, what can TPC PAs and faculty reasonably expect the course to do for students or, in other words, what is the purpose of the service course? While not a new topic, this question takes on a new dimension when viewed through the findings about SLOs.

Considering the many stakeholders involved, should the field focus more exclusively on key components that can be achieved (or the very least advanced) within the timeframe of an academic term? We cannot disagree with Carter and his collaborators (2003) when they wrote “programmatic thinking encourages us to view the technical writing service course not only as a program in itself but also, and perhaps more importantly, as a part of each student's entire educational program…” (p. 102). But, at the same time, one single course cannot stand in for all advanced writing practice for students across disciplines. For example, following are the SLOs from an Associate level institution:
Locate, analyze, and synthesize various professional materials

Summarize important points.

Generate oral and written critical responses that reflect higher-level thinking skills.

Display openness to other viewpoints.

Work productively in a team.

Contribute constructively to group discussion.

Demonstrate use of computer technologies in written, oral, and visual communication.

Select appropriate medium and document type for presenting information

Demonstrate use of computer technologies in written, oral, and visual communication
While these outcomes may articulate generally appropriate abilities to facilitate in the classroom, it would not be possible to fully develop all these skills in a single course. Rather than attempt to do it all, outcomes should focus on core skills that reflect a programmatic vision and can be developed in specific assignments, fostering students’ skills that transfer concretely to the workplace and their civic lives. The limitations of a single course should encourage TPC PAs and faculty to consider thoughtfully what can actually be accomplished in a single term.

More so, TPC's research on assessment so often focuses on full achievement, or competency, of outcomes that the field has not adequately considered the idea of a continuum of student learning. For example, full degree program creation and assessment often has faculty considering what SLO may be introduced in a course and then the same SLO would be developed in separate courses, and then assessed at full competency in a capstone course. But how do TPC PAs and faculty move to assess an SLO in the service course, knowing this may be one of the few—and often the last—opportunities students have to merge their disciplinary knowledge with writing knowledge and practice? In other words, how can we incorporate common outcomes from the field-wide data in a way that promotes measuring and assessing SLOs by degree of competency, which is ultimately a more just and equitable approach?

SLOs, particularly in the service course, afford students with varying levels of abilities the opportunity to do well in the course and allow instructors a greater ability to meet students where they are. By assessing improvement in learning, rather than matching to some ideal deliverable that simply does not exist, outcomes become a way to “see the development of students’ writing through their academic careers” as well as to “gain a better understanding of the broad development of students’ writing skills and knowledge” (Carter et al., 2003, p. 110). Reevaluating our understanding of competency and proficiency not only serves to help students, but it also assists with the goal of focusing on core skills that can be achieved within the course itself. Specifically, SLO's should guide the course and be tied directly to assignments. Newmark and Bartolotta (2021), for example, sought to create a “through line” (2021, p.148) throughout their curriculum to promote consistency and help center the direction of a course. In doing so, they were able to thread outcomes throughout assignments and courses, rather than briefly connecting them. While this idea of levels of advancement on a continuum is something many experienced faculty do, TPC needs to do research in what this may look like and how successful it is for student learning and experiences in service courses.

Conclusion

It was not our intention to create a prescriptive TPC outcomes statement for the service course. Rather, we wanted to provide data to show a field wide snapshot of SLOs to provide information on current practices, which is something TPC has not had. In examining outcomes from 93 institutions, we coded 503 outcomes (along with 17 institutions that did not have outcomes listed on their syllabi). This led to the most common outcomes of rhetoric, genre, writing, design, and collaboration. While the service course may be in service to other majors, it is a key component to students’ educational experiences. SLO data also pointed to the need for the field to improve outcome development, to use SLOs for service course development, and to consider in more nuanced ways what the purpose of the service course is.

Understanding the service course through its SLOs can help TPC PAs continuously improve the course for the multiple stakeholders it serves. We echo previous scholars (see e.g., Knievel, 2007; Melonçon, 2018a; Schreiber et al., 2018) who have all said in various ways that the future of TPC is tied to the future of the service course. That future is better faced with field-wide data that can be implemented within localized situations to better serve the students we teach.

Type of institution	# of syllabi
R1	27
R2	13
R3	8
D/PU	1
M1/ML	24
M2/MM	5
M3/MS	1
BAC	3
Associates	11

Code	# unique	# total
Rhetoric	51	69
Genre	47	61
Writing	45	59
Design	42	54
Collaboration	34	37
Communication	27	29
Critical thinking	27	28
Research	27	28
Professionalization	24	27
Technology	20	26
Editing	18	20
Ethics	17	18
Practice	14	16
Project management	12	16
Culture	8	9
Other	6	6

Footnotes

Acknowledgements

We would also like to thank all the faculty who so graciously and generously shared their syllabi with us.

The authors would like to acknowledge and to send a big thank you to Samantha McGilvray for her work on this project in getting so much data organized, as well as Katie Walkup and Spencer Bennington, who provided assistance and different points in this project's evolution.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Lisa Melonçon

Appendix A

Following are the codes we used in coding the service course SLOs.

Collaboration: Used for outcomes that explicitly focus on working with other people in any capacity.

Communication: Used for both oral and written communication. Originally, we used Oral and Writing as separate codes; however, upon IRR review, it was felt that it was better to combine these into one unified code due to the prevalence of both being mentioned in individual outcomes statements.

Critical Thinking: Anything having to do with explicit mention of problem solving, organization of complex information, or combining information and skills together.

Culture: Used for outcomes in which culture (North American or international culture, cultural critique and communication, and company culture) was mentioned. Also used to designate issues related to power, power dynamics, and issues related to theoretical methodologies.

Design: A vast outcome category that incorporated elements such as visual and textual design outcomes of a product. Design incorporating audio-visual elements was later folded under this umbrella code in order to fit SLOs that mentioned emerging technologies.

Ethics: Focused on anything explicitly mentioning ethics within an outcome. Most of the PSLOs that fall under this designation only focused on ethics within a field or ethical writing.

Genres: Used for codes that specifically mentioned genres in general or the use and coverage of specific genres. For example: correspondence, proposals, or electronic environments.

Other: Used for codes that did not quite fit into any of our established codes or were outliers specific to an individual code. For example, if information architecture was used in an outcome, it would best fit in the Other category of coding.

Practice: Focused on any reference to production of documents, especially those outcomes that focused on usability and user experience.

Project Management: Concerned with the management of projects, documents, and deliverables.

Professionalization: Unlike Practice, we used Professionalization as a code for outcomes that referred expectations within a given career field or job preparation. For example: Use of disciplinary knowledge in the workplace, professional standards of etiquette.

Research: Used for outcomes explicitly mentioning research practices, methodologies, and ideology. For example: identifying credible and useful sources, gather and assessing data, use of primary and second research.

Rhetoric: Defined as outcomes focused on traditional rhetorical references to purpose and audience.

Technology: Referred to outcomes specifying technological literacy, use of technology, and/or use of software.

Writing: Used for objectives that specifically labelled writing as the primary objective within an outcome. For example: composing texts, describe technical information in readable prose.

References

Allan

(1996). Learning outcomes in higher education. Studies in Higher Education, 21(1), 93–108. https://doi.org/10.1080/03075079612331381487

Allen

(2004). The impact of student learning outcomes assessment on technical and professional communication programs. Technical Communication Quarterly, 13(1), 93–108.

Arduser

(2018). Specialized technical writing service courses as a program sustainability tool? Programmatic Perspectives, 10(1), 12–43.

Barker

(2012). Program assessment: The role of outcomes. Programmatic Perspectives, 4(2), 183–208.

Bay

(2022). Fostering diversity, equity, and inclusion in the technical and professional communication service course. IEEE Transactions on Professional Communication, 65(1), 213–225. https://doi.org/10.1109/TPC.2021.3137708

Behm

Glau

G. R.

Holdstein

D. H.

Roen

White

(Eds.). (2012). The WPA outcomes statement-A decade later. Parlor Press.

Bivens

K. M.

Elliott

T. J.

Wiberg

G. K. H.

(2020). Locating technical and professional communication at two-year institutions. Programmatic Perspectives, 11(2), 6–66.

Blythe

Lauer

Curran

P. G.

(2014). Professional and technical communication in a web 2.0 world. Technical Communication Quarterly, 23(4), 265–287. https://doi.org/10.1080/10572252.2014.941766

Brewer

E. C.

Holmes

(2016). Better communication = better teams: A communication exercise to improve team performance. IEEE Transactions on Professional Communication, 59(3), 288–298. https://doi.org/10.1109/TPC.2016.2590018

10.

Brumberger

(2007). Visual communication in the workplace: A survey of practice. Technical Communication Quarterly, 16(4), 369–395.

11.

Brumberger

(2018). Designing teaching to teach design. In Bridgeford

(Ed.), Teaching professional and technical communication: A practicum in a book (pp. 105–121). Utah State University Press.

12.

Brumberger

E. R.

Northcut

(Eds.). (2013). Design texts: Teaching visual communication (pp. 1–11). Baywood.

13.

Carnegie

T. A. M.

(2018). Sustaining service learning in the service course. Programmatic Perspectives, 10(1), 110–141.

14.

Carriveau

R. S.

(2016). Connecting the dots: Developing student learning outcomes and outcomes-based assessment. Stylus Publishing.

15.

Carter

Anson

C. M.

Miller

C. R.

(2003). Assessing technical writing in institutional contexts: using outcomes-based assessment for programmatic thinking. Technical Communication Quarterly, 12(1), 101–114.

16.

Chen

(2021). Trial and error: Designing an introductory course to technical communication. In Klein

M. J.

(Ed.), Effective teaching of technical communication theory, practice, and application (pp. 147–165). The WAC Clearinghouse; University Press of Colorado.

17.

Clegg

Lauer

Phelps

Melonçon

(2021). Programmatic outcomes in undergraduate technical and professional communication programs. Technical Communication Quarterly, 30(1), 19–33. https://doi.org/10.1080/10572252.2020.1774662

18.

Coppola

N. W.

Elliot

Newsham

Klobucar

(2016). Programmatic research in technical communication: An interpretive framework for writing program assessment. Programmatic Perspectives, 8(2), 5–45.

19.

Doan

(2021). Teaching workplace genre ecologies and pedagogical goals through résumés and cover letters. Business and Professional Communication Quarterly, 84(4), 294–317. https://doi.org/10.1177/23294906211031810

20.

Driscoll

Wood

(2007). Developing outcomes-based assessment for learner-centered education: A faculty Introduction. Stylus.

21.

Dubinsky

J. M.

(2018). Rhetorical analysis: A foundational skill for PTC teachers. In Bridgeford

(Ed.), Teaching professional and technical communication: A practicum in a book (pp. 18–41). Utah State University Press.

22.

England

P. S.

Brewer

P. E.

(2018). What do instructors need to know about teaching collaboration? In Bridgeford

(Ed.), Teaching professional and technical communication: A practicum in a book (pp. 159–175). Utah State University Press.

23.

Etikan

(2016). Comparison of convenience sampling and purposive sampling. American Journal of Theoretical and Applied Statistics, 5(1), 1–4. https://doi.org/10.11648/j.ajtas.20160501.11

24.

Finley

(2021). How college contributes to workforce success: Employer views on what matters most. A. o. A. C. a. Universities. https://www.aacu.org/sites/default/files/files/research/AACUEmployerReport2021.pdf

25.

Francis

A. M.

(2018). A survey of assignment requirements in service technical and professional communication classes. Programmatic Perspectives, 10(1), 44–76.

26.

Gallagher

C. W.

(2019). Articulation, not standardization. College Composition and Communication, 70(3), 492–498.

27.

Geisler

(2018). Coding for complexity: The interplay among methodological commitments, tools, and workflow in writing research. Written Communication, 35(2), 215–249.

28.

Gubala

Larson

Melonçon

(2020). Do writing errors bother professionals? An analysis of the most bothersome errors and how the writer's ethos is affected. Journal of Business and Technical Communication, 34(3), 250–286.

29.

Harrington

Rhodes

Fischer

R. O.

Malenczyk

(Eds.). (2005). The outcomes books: Debates and consensus after the WPA outcomes statement. Utah State University Press.

30.

Henschel

Melonçon

(2014). Of horsemen and layered literacies: Assessment instruments for aligning technical and professional communication undergraduate curricula with professional expectations. Programmatic Perspectives, 6(1), 3–26.

31.

Henze

(2018). Teaching genre in professional and technical communication. In Bridgeford

(Ed.), Teaching professional and technical communication: A practicum in a book (pp. 72–88). Utah State University Press.

32.

Hitt

(2018). Foregrounding accessibility through (inclusive) universal design in professional communication curricula. Business and Professional Communication Quarterly, 81(1), 52–65. https://doi.org/10.1177/2329490617739884

33.

Ilhan

Gezer

(2017). A comparison of the reliability of the SOLO- and revised bloom's taxonomy-based classifications in the analysis of the cognitive levels of assessment questions. Pegem Eğitim ve Öğretim Dergisi, 7(4), 637–662.

34.

Ilyasova

K. A.

Bridgeford

(2014). Establishing an outcomes statement for technical communication. In Bridgeford

Kitalong

K. S.

Williamson

(Eds.), Sharing our intellectual traces: Narrative reflections from administrators of professional, technical, and scientific communication programs (pp. 53–80). Baywood.

35.

Johnson

C. S.

Elliot

(2010). Undergraduate technical writing assessment: A model. Programmatic Perspectives, 2(2), 110–151.

36.

Jones

(2018). Teaching students about style in technical communication. In Bridgeford

(Ed.), Teaching professional and technical communication: A practicum in a book (pp. 42–57). Utah State University Press.

37.

Kenzie

McCall

(2018). Teaching writing for the health professions: Disciplinary intersections and pedagogical practice. Technical Communication Quarterly, 27(1), 64–79. https://doi.org/10.1080/10572252.2017.1402573

38.

Kimball

(2017). The golden age of technical communication. Journal of Technical Writing and Communication, 47(3), 330–358.

39.

Knievel

(2007, Oct. 11–13). Growing the Service Course: Anticipating Problems, Promise in the Technical Communication ‘Mini-Program’ [Conference presentation] (pp. 89–90) Council of Programs in Technical and Scientific Communication Program, Greenville, SC. https://cptsc.org/wp-content/uploads/2018/01/2007.pdf

40.

Kuh

G. D.

Jankowski

Ikenberry

S. O.

Kinzie

J. L.

(2014). Knowing what students know and can do: the current state of student learning outcomes assessment in US colleges and universities. National Institute for Learning Outcomes Assessment.

41.

Lucas

Rawlins

J. D.

(2015). The competency pivot: introducing a revised approach to the business communication curriculum. Business and Professional Communication Quarterly, 78(2), 167–193.

42.

Maid

(2005). Using the outcomes statement for technical communication. In Harrington

Rhodes

Fischer

R. O.

Malenczyk

(Eds.), The outcomes book: Debates and consensus after the WPA outcomes statement (pp. 139–149). Utah State University Press.

43.

Maid

D’Angelo

(2012). Is rhetorical knowledge the Uber outcome. In Behm

Glau

G. R.

Holdstein

D. H.

Roen

White

(Eds.), The WPA outcomes statement-A decade later (pp. 257–270). Parlor Press.

44.

Mallette

Hawks

(2020). Building student agency through contract grading in technical communication. The Journal of Writing Assessment, 13(2), n.p. http://journalofwritingassessment.org/article.php?article=158

45.

Medina

Walker

(2018). Validating the consequences of a social justice pedagogy: Explicit values in course-based grading contracts. In Eble

Haas

(Eds.), Key theoretical frameworks (pp. 46–67). Utah State University Press. 10.7330/9781607327585.c002

46.

Melonçon, L. (2022) TechComm Programmatic Central. Unpublished data.

47.

Melonçon

(2018a). Critical postscript: On the future of the service course in technical and professional communication. Programmatic Perspectives, 10(1), 202–230.

48.

Melonçon

(2018b). Orienting access in our business and professional communication classrooms. Business and Professional Communication Quarterly, 81(1), 34–51. https://doi.org/10.1177/2329490617739885

49.

Meloncon

England

(2011). The current Status of contingent faculty in technical and professional communication. College English, 73(4), 396–408.

50.

Meloncon

Henschel

(2013). Current state of US undergraduate degree programs in technical and professional communication. Technical Communication, 60(1), 45–64.

51.

Melonçon

Rosselot-Merritt

St.Amant

(2020). A field-wide metasynthesis of pedagogical research in technical and professional communication. Journal of Technical Writing and Communication, 50(1), 91–118.

52.

Newmark

Bartolotta

(2021). Creating the “through-line” by engaging industry certification standards in SLO redesign for a core curriculum technical writing course. In Klein

M. J.

(Ed.), Effective teaching of technical communication theory, practice, and application (pp. 147–165). The WAC Clearinghouse; University Press of Colorado.

53.

Read

Michaud

(2018a). Hidden in plain sight: Findings from a survey on the multi-major professional writing course. Technical Communication Quarterly, 27(3), 227–248. https://doi.org/10.1080/10572252.2018.1479590

54.

Read

Michaud

(2018b). Who teaches technical and professional communication service courses?: Survey results and case studies from a national study of instructors from all carnegie institutional types. Programmatic Perspectives, 10(1), 77–109. https://cptsc.org/wp-content/uploads/2018/06/vol10.1.pdf.

55.

Russell

D. R.

(2007). Rethinking the articulation between business and technical communication and writing in the disciplines: Useful avenues for teaching and research. Journal of Business and Technical Communication, 21(3), 248–277.

56.

Saldana

(2009). The coding manual for qualitative researchers. Sage.

57.

Say

B. H.

(2015). Developing learning outcomes in professional writing and technical communication programs: Obstacles, benefits, and potential for graduate program improvement. Programmatic Perspectives, 7(2), 25–49.

58.

Schreiber

Carrion

Lauer

(2018). Revisiting the service course to map out the future of the field [special issue]. Programmatic Perspectives, 10(1), 252–278.

59.

Schreiber

Melonçon

(2019). Creating a continuous improvement model for sustaining programs in technical and professional communication. Journal of Technical Writing and Communication, 49(3), 252–278.

60.

Scott

J. B.

(2008). The practice of usability: teaching user engagement through service-learning. Technical Communication Quarterly, 17(4), 381. https://doi.org/10.1080/10572250802324929

61.

Smagorinsky

(2008). The method section as conceptual epicenter in constructing social science research reports. Written Communication, 25(3), 389–411. https://doi.org/10.1177/0741088308317815

62.

Sonnenberg, M., Gubala, C., Burry, J., Griffith, J., Zarlengo, T., & Melonçon, L. (2022) Implementing a continuous improvement model for assignment evaluation at the technical and professional communication program level. Journal of Technical Writing and Communication, 3. https://doi.org/10.1177/00472816221124605

63.

St.Amant

(2018). Contextualizing the technical communication service course: A research approach to reviewing, re-thinking, and revising the service course. Programmatic Perspectives, 10(1), 142–171.

64.

St.Amant

Graham

S. S.

(2019). Research that resonates: A perspective on durable and portable approaches to scholarship in technical communication and rhetoric of science. Technical Communication Quarterly, 28(2), 99–111. https://doi.org/10.1080/10572252.2019.1591118

65.

St.Amant

Nahrwold

(2007). Acknowledging complexity: Rethinking program review and assessment in technical communication. Technical Communication, 54(4), 409–411.

66.

Stanton

(2017). So technical/professional writing (TPC) programs offer what students need for their start in the workplace? A comparison of requirements in program curricula and job ads in industry. Technical Communication, 64, 226–236.

67.

Walton

Agboka

(Eds.). (2021). Equipping technical communicators for social justice work: Theories, methodologies, and pedagogies. Utah State University Press.

68.

Wolfe

(2015). Teaching students to focus on the data in data visualization. Journal of Business and Technical Communication, 29(3), 344–359. https://doi.org/10.1177/1050651915573944

69.

Wong

L. H. M.

C. X. J.

Davison

R. M.

Zhu

Zhang

(2016). Web 2.0 and communication processes at work: Evidence from China. IEEE Transactions on Professional Communication, 59(3), 230–244. https://doi.org/10.1109/TPC.2016.2594580

A Field Wide Snapshot of Student Learning Outcomes in the Technical and Professional Communication Service Course

Abstract

Keywords

What TPC Knows About Student Learning Outcomes

Outcomes in Single Institution Research

Outcomes in Field Wide Research

Methodology

Description of Coding Process

Coding Process

Consistency Coding

Limitations

Findings of Service Course SLOs

Most Common Unique Outcomes

Explanation of Common SLOs

Rhetoric

Genre

Writing

Design

Collaboration

What is Surprising from the SLOs Data

What the SLO Data may Mean Moving Forward

Improve Outcome Development and use

Reconsider the Purpose of the Service Course

Conclusion

Footnotes

Acknowledgements

Declaration of Conflicting Interests

Funding

ORCID iD

Appendix A

References