Writing Empirical Articles: Transparency,Reproducibility,Clarity,and Memorability

Abstract

This article provides recommendations for writing empirical journal articles that enable transparency, reproducibility, clarity, and memorability. Recommendations for transparency include preregistering methods, hypotheses, and analyses; submitting registered reports; distinguishing confirmation from exploration; and showing your warts. Recommendations for reproducibility include documenting methods and results fully and cohesively, by taking advantage of open-science tools, and citing sources responsibly. Recommendations for clarity include writing short paragraphs, composed of short sentences; writing comprehensive abstracts; and seeking feedback from a naive audience. Recommendations for memorability include writing narratively; embracing the hourglass shape of empirical articles; beginning articles with a hook; and synthesizing, rather than Mad Libbing, previous literature.

They began three and a half centuries ago (Wells, 1998). Since then, they have been written and read; cited, abstracted, and extracted; paywalled and unpaywalled; preprinted and reprinted. They arose as correspondences between pairs of scientists (Kronick, 1984), then morphed into publicly disseminated conference presentations (Schaffner, 1994). By the 20th century, they had grown into the format we use today (Mack, 2015). They are empirical journal articles, and their raison d’être was and continues to be communicating science.

Many of us baby boomers honed our empirical-article writing skills by following Bem’s (1987) how-to guide. We applied Bem’s recommendations to our own articles, and we assigned his chapter to our students and postdocs. The 2004 reprint of Bem’s chapter retains a high recommendation from the American Psychological Association (2010) in its “Guide for New Authors”; it appears in scores of graduate and undergraduate course syllabi (Gernsbacher, 2017a); and its advice is offered by numerous universities’ writing centers (e.g., Harvard College, 2008; Purdue Online Writing Lab, 2012; University of Connecticut, n.d.; University of Minnesota, n.d.; University of Washington, 2010).

However, psychological scientists have recently confronted their questionable research practices (John, Loewenstein, & Prelec, 2012), many of which arise during the writing (or revising) process (Sacco, Bruton, & Brown, 2018). Questionable research practices include

• failing to report all the studies conducted, conditions manipulated, participants tested, data collected, or other “researcher degrees of freedom” (Simmons, Nelson, & Simonsohn, 2011, p. 1359);

• fishing through statistical analyses to report only those meeting a certain level of statistical significance, which is a practice known as p-hacking (Simonsohn, Nelson, & Simmons, 2014);

• reporting an unpredicted result as though it had been hypothesized all along, which is a practice known as hypothesizing after the results are known (often referred to as HARKing; Kerr, 1998); and

• promising that the reported results bear implications beyond the populations sampled or materials and tasks administered (Simons, Shoda, & Lindsay, 2017).

Unfortunately, some of these questionable reporting practices seem to be sanctioned in Bem’s how-to guide (Devlin, 2017; Vazire, 2014). For example, Bem’s chapter seems to encourage authors to p-hack their data. Authors are advised to

examine [your data] from every angle. Analyze the sexes separately. Make up new composite indexes. If a datum suggests a new hypothesis, try to find additional evidence for it elsewhere in the data. If you see dim traces of interesting patterns, try to reorganize the data to bring them into bolder relief. If there are participants you don’t like, or trials, observers, or interviewers who gave you anomalous results, drop them (temporarily). Go on a fishing expedition for something — anything — interesting. (Bem, 1987, p. 172; Bem, 2004, pp. 186–187)

Bem’s chapter has also been interpreted as encouraging authors to hypothesize after the results are known (Wagenmakers, Wetzels, Borsboom, & van der Maas, 2011). After acknowledging “there are two possible articles you can write: (a) the article you planned to write when you designed your study or (b) the article that makes the most sense now that you have seen the results,” Bem noted the two potential articles “are rarely the same” and directed authors to write the latter article by “recentering your article around the new findings and subordinating or even ignoring your original hypotheses” (Bem, 1987, pp. 171–173; Bem, 2004, pp. 186–187).

This article provides recommendations for writing empirical journal articles that communicate research processes and products transparently with enough detail to allow replication and reproducibility. ¹ Like Bem’s chapter, this article also provides recommendations for writing empirical articles that are clear and memorable.

Disclosures

Open materials for this article, which are available at https://osf.io/q3pna/, include a list of publicly available course syllabi that mention Bem’s (1987, 2004) “Writing the Empirical Journal Article” chapter and a tally of word and sentence counts, along with citation counts, for Clark and Clark (1939, 1940, 1947), Harlow (1958), Miller (1956), and Tolman (1948).

Recommendations for Transparency

Researchers write empirical journal articles to report and record why they conducted their studies, how they conducted their studies, and what they observed in their studies. The value of these archival records depends on how transparently researchers write their reports. Writing transparently, means, as the vernacular connotes, writing frankly.

Preregister your study

The best way to write transparent empirical articles is through preregistration (Chambers et al., 2013). Preregistering a study involves specifying the study’s motivation, hypothesis, method, materials, sample, and analysis plan—basically everything but the results and discussion of those results—before the study is conducted. Preregistration is a “time-stamped research plan that you can point to after conducting a study to prove to yourself and others that you really are testing a predicted relationship” (Mellor, as cited in Graf, 2017, para. 3). Indeed, most of our common statistical tests rest on the assumption that we have preregistered, or at the least previously specified, our predictions (Wagenmakers, Wetzels, Borsboom, van der Maas, & Kievit, 2012).

For more than 20 years, medical journals have required preregistration for researchers conducting clinical trials (Maxwell, Kelley, & Rausch, 2008). More recently, sites such as Open Science Framework and AsPredicted.org allow all types of researchers to document their preregistration, and preregistration is considered a best practice by psychologists of many stripes: cognitive (de Groot, 2014), clinical (Tackett et al., 2017), comparative (Stevens, 2017), developmental (Donnellan, Lucas, Fraley, & Roisman, 2013), social (van ’t Veerab & Giner-Sorolla, 2016), personality (Asendorpf et al., 2013), relationship (Campbell, Loving, & Lebelc, 2014), neuroscience (Button et al., 2013), and neuroimaging (Poldrack et al., 2017).

The benefits of preregistration are plentiful, both to our sciences and to ourselves. As Mellor noted (cited in Graf, 2017, para. 8), “Every step that goes into a preregistration: writing the hypotheses, defining the variables, and creating statistical tests, are steps that we all have to take at some point. Making them before data collection can improve the researcher’s study design.” Misconceptions about preregistration are also plentiful. For instance, some researchers mistakenly believe that if a study is preregistered, unpredicted analyses cannot be reported; they can, but they need to be identified as exploratory (see, e.g., Neuroskeptic, 2013). Other researchers worry that purely exploratory research cannot be preregistered; it can, but it needs to be identified as exploratory (see, e.g., McIntosh, 2017). Preregistration manifests transparency and is, therefore, one of the most important steps in conducting and reporting research transparently.

Submit a registered report

A further step in writing transparent articles is to submit a registered report. Registered reports are journal articles for which both the authors’ preregistrations and their subsequent manuscripts undergo peer review. (Preregistration outside of submission as a registered-report journal article does not require peer review, only documentation.)

Registered reports epitomize how most of us were trained to do research. For our dissertations and masters’ theses, even our senior theses, we submitted our work to review at two stages: after we designed the study (e.g., at our dissertation proposal meeting) and after we collected and analyzed the data and interpreted our results (e.g., at our final defense). The same two-stage review occurs with registered-report journal articles (Nosek & Lakens, 2014). More and more journals are providing authors with the option to publish registered reports (for a list, see Center for Open Science, n.d.). The beauty of registered reports is that, as with our dissertations, our success depends not on the shimmer of our results but on the soundness of our ideas and the competence of our execution.

Distinguish confirmation from exploration

Writing transparently means distinguishing confirmation from exploration. To be sure, exploration is a valid and important mode of scientific inquiry. The exploratory analyses Bem wrote about (“examine [your data] from every angle”) are vital for discovery—and should not be discouraged. However, it is also vital to distinguish exploratory from confirmatory analyses. For example, clarify whether “additional exploratory analysis was conducted” (Brockhaus, 1980, p. 517), “data were derived from an exploratory questionnaire” (Scogin & Bienias, 1988, p. 335), or “results . . . should be interpreted cautiously because of their exploratory nature” (Martin & Stelmaczonek, 1988, p. 387). Entire research projects may be exploratory (McIntosh, 2017), but they must be identified as such (e.g., “Prediction of Improvement in Group Therapy: An Exploratory Study,” Yalom, Houts, Zimerberg, & Rand, 1967; and “Personality and Probabilistic Thinking: An Exploratory Study,” Wright & Phillips, 1979).

Show your warts

Scientific reporting demands showing your work (Vazire, 2017); transparent scientific reporting demands showing your warts. If participants were excluded, explain why and how many: for example, “Two of these subjects were excluded because of their inability to comply with the imagery instructions at least 75% of the time” (Sadalla, Burroughs, & Staplin, 1980, p. 521). Similarly, if data were lost, explain why and how many: for example, “Ratings for two subjects were lost to equipment error” (Vrana, Spence, & Lang, 1988, p. 488) or “Because of experimenter error, processing times were not available for 11 subjects” (McDaniel & Einstein, 1986, p. 56).

If one or more pilot studies were conducted, state that. If experiments were conducted in an order different from the reported order, state that. If participants participated in more than one study, state that. If measures were recalculated, stimuli were refashioned, procedures were reconfigured, variables were dropped, items were modified—if anything transgressed the prespecified plan and approach—state that.

Writing transparently also requires acknowledging when results are unpredicted: for example, “An unexpected result of Experiment 1 was the lack of an age . . . effect . . . due to different presentation rates” (Kliegl, Smith, & Bakes, 1989, p. 251) or “Unexpectedly, the female preponderance in depressive symptoms is strongly demonstrated in every age group in this high school sample” (Allgood-Merten, Lewinsohn, & Hops, 1990, p. 59). Concede when hypotheses lack support: for example, “we were unable to demonstrate that free care benefited people with a high income” (Brook et al., 1983, p. 1431) or “we cannot reject the null hypothesis with any confidence” (Tannenbaum & Smith, 1964, p. 407).

Consider placing a Statement of Transparency in either your manuscript or your supplementary materials: for example, “Statement of Transparency: The data used in the present study were initially collected as part of a larger exploratory study” (Werner & Milyavskaya, 2017, p. 3) or “As described in the Statement of Transparency in our online supplemental materials, we also collected additional variables and conducted further analyses that we treat as exploratory” (Gehlbach et al., 2016, p. 344). Consider ending your manuscript with a Constraints on Generality statement (Simons et al., 2017), which “defines the scope of the conclusions that are justified by your data” and “clarifies which aspects of your sample of participants, materials, and procedures should be preserved in a direct replication” (p. 1125; see Simons et al., 2017, for examples).

Recommendations for Reproducibility

The soul of science is that its results are reproducible. Reproducible results are repeatable, reliable, and replicable. But reproducing a result, or simply trying to reproduce it, requires knowing in detail how that previous result was obtained. Therefore, writing for reproducibility means providing enough detail so readers will know how each result was obtained.

Document your research fully

Many researchers appreciate that empirical studies need to be reported accurately and completely—in fact, fully enough to allow other researchers to reproduce them—but they encounter a barrier: Many journals enforce word limits; some even limit the number of tables and figures that can accompany each article or the number of sources that can be cited. Journals’ limits can stymie authors’ efforts to write for reproducibility. After using the maximum number of words allowed for methods and results, turn to open-science tools. Repositories, such as Open Science Framework (osf.io), PubMed Central (ncbi.nlm.nih.gov/pmc/), and Mendeley Data (mendeley.com/datasets), allow researchers to make their materials and data publicly available, which is a best practice quickly becoming mandatory (Lindsay, 2017; Munafò et al., 2017; Nosek et al., 2015). These repositories also allow researchers to make detailed documentation of their methods and results publicly available.

For example, I recently analyzed 5 million books, 25 million abstracts, and 150 million journal articles to examine scholars’ use of person-first (e.g., person with a disability) versus identity-first (e.g., disabled person) language (Gernsbacher, 2017b). Because the journal that published my article limited me to 2,000 words, eight citations, and zero tables or figures, I created and posted on Open Science Framework an accompanying technical report (Gernsbacher, 2016), which served as my open notebook. For the current article, I also created a technical report (Gernsbacher, 2017a) to document the course syllabi that assign Bem’s chapter (mentioned earlier) and the word counts that illustrate classic articles’ concision (mentioned later).

By taking advantage of open-science repositories, authors can document

• why they qualify for the 21-Word Solution, which is a statement authors can place in their Method section to verify they have “report[ed] how [they] determined [their] sample size, all data exclusions (if any), all manipulations, and all measures in the study” (Simmons, Nelson, & Simonsohn, 2012, p. 4);

• how they fulfilled the Preferred Reporting Items for Systematic Reviews and Meta-Analyses checklist (PRISMA, 2015); and

• that they have met other methodological or statistical criteria (e.g., they have provided their data, materials, and code; Lindsay, 2017).

An accompanying technical report can serve as a publicly accessible lab notebook, which also comes in handy for selfish reasons (Markowetz, 2015; McKiernan et al., 2016). A tidy, publicly accessible lab notebook can be, like tidy computer documentation, “a love letter you write to your future self” (Conway, 2005, p. 143).

Document your research cohesively

Documentation should also be cohesive. For instance, rather than posting a slew of separate supplementary files, consider combining all the supporting text, summary data, and supplementary tables and figures into one composite file. More helpfully, annotate the composite file with a table of contents or a set of in-file bookmarks. A well-indexed composite file can reduce the frustration readers (and reviewers) incur when required to open multiple supplementary files (often generically named Supp. Fig.1, Supp. Fig. 2, etc.). Posting a well-indexed composite file on an open-science platform can also ensure that valuable information is available outside of journals’ paywalls, with guaranteed access beyond the life of an individual researcher’s or journal’s Web site (e.g., Open Science Framework guarantees their repository for 50 years).

Cite sources responsibly

As Simkin and Roychowdhury (2003) advised in the title of their study demonstrating high rates of erroneous citations, “read before you cite.” Avoid “drive by citations” (Perrin, 2009), which reference a study so generically as to appear pro forma. Ensure that a specific connection exists between your claim and the source you cite to support that claim. Is the citation the original statement of the idea, a comprehensive review, an example of a similar study, or a counterclaim? If so, make that connection clear, rather than simply grabbing and citing the first article that pops up in a Google Scholar search.

Interrogate a reference before citing it, rather than citing it simply because other articles do. For example, I tallied hundreds of articles that mistakenly cited Rizzolatti et al. (1996) as providing empirical evidence for mirror neurons in humans—despite neither Rizzolatti et al.’s data nor their text supporting that claim (Gallese, Gernsbacher, Heyes, Hickok, & Iacoboni, 2011).

Try to include a linked DOI (digital object identifier) for every reference you cite. Clicking on a linked DOI takes your readers directly to the original source, without having to search for it by its title, authors, journal, or the like. ² Moreover, a DOI, like an ISBN, provides a permanent link to a published work; therefore, DOIs obviate link rot and guarantee greater longevity than standard URLs, even journal publishers’ URLs.

Recommendations for Clarity

Empirical articles are becoming more difficult to read, as an analysis of nearly three-quarter million articles in more than 100 high-impact journals recently demonstrated (Plavén-Sigray, Matheson, Schiffler, & Thompson, 2017). Sentences in empirical articles have grown longer, and vocabulary has grown more abstruse. Therefore, the primary recommendation for achieving clarity in empirical articles is simple: Write concisely using plain language (Box 1 provides additional suggestions and resources for clear writing).

Box 1.

Additional Recommendations for Clear Writing

Use Precise Terms. Concision requires precision. Rather than writing that a dependent variable is related to, influenced by, or affected by the independent variable, state the exact relation between the two variables or the precise effect one variable has on another. Did manipulating the independent variable increase, decrease, improve, worsen, augment, diminish, negate, strengthen, weaken, delay, or accelerate the dependent variable? Most important, use precise terms in your title. Follow the example of Parker, Garry, Engle, Harper, and Clifasefi (2008), who titled their article “Psychotropic Placebos Reduce the Misinformation Effect by Increasing Monitoring at Test” rather than “The Effects of Psychotropic Placebos on Memory.”

Omit Needless Words. Numerous wordy expressions can be replaced by one word. For example, replace due to the fact that, for the reason that, or owing to the fact that with because; replace for the purpose of with for; have the capability of with can; in the event that with if; during the course of with during; fewer in number with fewer; in order to with to; and whether or not with whether. And replace the well-worn and wordy expression that appears in numerous acknowledgements, we wish to thank, with simply we thank.

Build Parallel Structures. Parallel structure aids comprehension (Fraizer, Taft, Roeper, Clifton, & Ehrlich, 1984), whereas disjointed structure (e.g., Time flies like an arrow; fruit flies like a banana) impedes comprehension (Gernsbacher, 1997). Simons (2012) demonstrated how to build parallel structure with the example sentence Active reconstruction of a past experience differs from passively hearing a story about it. That sentence lacks parallel structure because the first half uses a noun phrase (Active reconstruction), whereas the second half uses a gerundive nominal (passively hearing). But the sentence can easily be made parallel: Actively reconstructing a past experience differs from passively hearing a story about it.

Listen to Your Writing. Try reading aloud what you have written (or use text-to-speech software). Listening to your writing is a great way to catch errors and get a feel for whether your writing is too stilted (and your sentences are too long).

Read About Writing. Read about how to write clearly in Pinker’s (2015) book, Zinsser’s (2016) book, Wagenmakers’s (2009) article, Simons’s (2012) guide, and Gernsbacher’s (2013) graduate-level open-access course. Try testing the clarity of your writing with online readability indices (e.g., https://readable.io/text/, https://wordcounttools.com/).

Write short sentences

Every writing guide, from Strunk and White’s (1959) venerable Elements of Style to the prestigious journal Nature’s (2014) guide, admonishes writers to use shorter, rather than longer, sentences. Shorter sentences are not only easier to understand, but also better at conveying complex information (Flesch, 1948). The trick to writing short sentences is to restrict each sentence to one and only one idea. Resist the temptation to embed multiple clauses or parentheticals, which challenge comprehension. Instead, break long, rambling sentences into crisp, more concise ones. For example, write the previous three short sentences rather than the following long sentence: The trick to writing short sentences is to restrict each sentence to one and only one idea by breaking long, rambling sentences into crisp, more concise ones while resisting the temptation to embed multiple clauses or parentheticals, which challenge comprehension.

How short is short enough? The Oxford Guide to Plain English (Cutts, 2013) recommends averaging no more than 15 to 20 words per sentence. Such short, crisp sentences have been the mainstay of many great psychological scientists, including Kenneth and Mamie Clark. Their 1939, 1940, and 1947 articles reporting young Black children’s racial identification and self-esteem have garnered more than 2,500 citations. These articles figured persuasively in Brown v. Board of Education (1954). And these articles’ sentences averaged 16 words.

Write short paragraphs

Combine short sentences into short paragraphs. Aim for around five sentences per paragraph. Harlow’s “The Nature of Love” (1958), Tolman’s “Cognitive Maps in Rats and Men” (1948), and Miller’s “The Magical Number Seven, Plus or Minus Two” (1956), which have been cited more than 2,000, 5,000, and 25,000 times, respectively, average five sentences per paragraph.

The prototypical five-sentence paragraph comprises a topic sentence, three supporting sentences, and a conclusion sentence. For example, a paragraph in Parker, Garry, Engle, Harper, and Clifasefi’s (2008, p. 410) article begins with the following topic sentence: “One of the puzzles of human behaviour is how taking a substance that does nothing can cause something.” The paragraph continues with three (in this case, conjoined) supporting sentences: “Phoney painkillers can lessen our pain or make it worse; phoney alcohol can lead us to do things we might otherwise resist, and phoney feedback can even cause us to shed body fat.” The paragraph then concludes with the sentence “Perhaps Kirsch (2004, p. 341) said it best: ‘Placebos are amazing.’”

Write comprehensive abstracts

Compiling a technical report and placing it on an open-source platform can circumvent a journal’s word limit for a manuscript. However, a journal’s word limit for an abstract is more difficult to circumvent. That limit is firm, and an abstract can often be the sole content that is read, particularly if the rest of the article lies behind a paywall. Therefore, authors need to make the most of their 150 or 250 words so that an abstract can inform on its own (Mensh & Kording, 2017). A clear abstract states the study’s primary hypothesis; its major methodology, including its sample size and sampled population; its main findings, along with their summary statistics; and its key implications. A clear abstract is explicit, concrete, and comprehensive, which was advice offered by Bem (1987, 2004).

Seek naive feedback

One of the best ways to ensure that a message is clear is to assess its clarity according to a naive audience (Traxler & Gernsbacher, 1995). Indeed, the more naive the audience, the more informative the feedback (Traxler & Gernsbacher, 1992, 1993). Unfortunately, some researchers seek feedback on their manuscripts from only their coauthors or fellow lab members. But coauthors and fellow lab members are hardly naive. Better feedback can be obtained from readers who are unfamiliar with the research—and unfamiliar with even the research area. If those readers say the writing is unclear (or a figure or table is confusing), it is, by definition, unclear (or confusing); it is best to revise for clarity.

Recommendations for Memorability

Most researchers want their articles not only to be read but also to be remembered. The goal in writing a memorable article is not necessarily to pen a flashy article; rather, the goal is to compose an article that enables readers to remember what they have read days or months later, as well as paragraphs or pages later (Gernsbacher, 1990).

Write narratively

The primary tool for increasing memorability is writing narratively (Bruner, 1991). An empirical article should tell a story, not in the sense of a tall tale but in the spirit of a coherent and logical narrative. Even authors who bristle at the notion of scholarly articles as stories must surely recognize that empirical articles resemble Aristotelian narratives: Introduction sections begin with a premise (the previous literature) that leads to an inciting incident (. . . however, . . .) and conclude with a therefore (the methods used to combat the inciting incident). Thus, Introduction sections and Method sections are empirical articles’ Act One, their setups. Results sections are empirical articles’ Act Two, their confrontations. And Discussion sections are empirical articles’ Act Three, their resolutions.

Writing Act One (introduction and methods) prior to collecting data, as we would do if submitting a registered report, helps us adhere to Feynman’s (1974) warning not to fool ourselves (e.g., not to misremember what we did vs. did not predict and, consequently, which analyses are vs. are not confirmatory). Writing all sections narratively, as setup, confrontation, and then resolution, should increase their short- and long-term memorability. Similarly, writing methods and results as sequences of events should increase their memorability. For methods, Bem (1987, 2004) recommended leading readers through the procedure as if they were research participants, which is an excellent idea. For results, readers can be led through the analytic pipeline in the sequence in which it occurred.

Embrace the hourglass

Bem advised that an article should be written “in the shape of an hourglass. It begins with broad general statements, progressively narrows down to the specifics of your study, and then broadens out again to more general considerations” (Bem, 1987, p. 175; Bem, 2004, p. 189). That advice should also not be jettisoned (Devlin, 2017). Call it the hourglass or call it the “broad-narrow-broad” structure (Mensh & Kording, 2017, p. 4), the notion is that well-written empirical articles begin broadly (theories and questions), narrow to specifics (methods and results), and end broadly (implications). Authors who embrace the hourglass shape aid their readers, particularly readers who skim (Weinstein, 2016).

Begin with a hook

Journal editors advise that articles “should offer a clear, direct, and compelling story that first hooks the reader” (Ragins, 2012, p. 497). For example, Oyserman et al. (2017) began their article with the following hook, which led directly to a statement articulating what their article was about (illustrated here in italics):

Will you be going to that networking lunch? Will you be tempted by a donut at 4 pm? Will you be doing homework at 9 pm? If, like many people, your responses are based on your gut sense of who you are—shy or outgoing, a treat lover or a dieter, studious or a procrastinator—you made three assumptions about identity: that motivation and behavior are identity based, that identities are chronically on the mind, and that identities are stable. (p. 139)

Newman et al. (2014) began their article with the following hook:

In its classic piece, “Clinton Deploys Vowels to Bosnia,” the satirical newspaper The Onion quoted Trszg Grzdnjkln, 44. “I have six children and none of them has a name that is understandable to me or to anyone else. Mr. Clinton, please send my poor, wretched family just one ‘E.’ Please.” The Onion was onto something when it suggested that people with hard to pronounce names suffer while their more pronounceable counterparts benefit. (p. 1, italics added)

Jakimik and Glenberg (1990) began their article with the following hook:

You’re zipping through an article in your favorite journal when your reading stops with a thud. The author has just laid out two alternative hypotheses and then referred to one of them as “the former approach.” But now you are confused about which was first, which was second. You curse the author and your own lack of concentration, reread the set-up rehearsing the order of the two hypotheses, and finally figure out which alternative the author was referring to. We have experienced this problem, too, and we do not think that it is simply a matter of lack of concentration. The subject of this article is the reason for difficulty with referring devices such as “the former approach.” (p. 582, italics added)

Synthesize previous literature (rather than Mad Lib it)

In the game of Mad Libs, one player generates a list of words from specified categories, for instance, a proper name, an activity, and a number. Then, the other player fills a template sentence with that list of generated terms. In a similar way, some authors review the literature by Mad Libbing terms into sentence templates, for example, [author’s name] investigated [research topic] with [number] of participants and found a statistically significant effect of [variable] on [variable].” A more memorable, albeit more difficult, way to review the literature is to synthesize it, as Aronson (1969) illustrated in his synthesis of previous studies on cognitive dissonance:

The research [on cognitive dissonance] has been as diverse as it has been plentiful; its range extends from maze running in rats (Lawrence and Festinger, 1962) to the development of values in children (Aronson and Carlsmith, 1963); from the hunger of college sophomores (Brehm et al., 1964) to the proselytizing behavior of religious zealots (Festinger et al., 1956). The proliferation of research testing and extending dissonance theory results from the generality and simplicity of the theory. (p. 1)

Notice that Aronson wrote a coherent narrative in which phenomena, not researchers, are the topics. That is what is meant by synthesizing, not Mad Libbing, previous literature.

Even technical literature can be synthesized rather than Mad Libbed, as Guillem et al. (2011) demon-strated:

Cortical acetylcholine (ACh) release from the basal forebrain is essential for proper sensory processing and cognition (1-3) and tunes neuronal and synaptic activity in the underlying cortical networks (4,5). Loss of cholinergic function during aging and Alzheimer’s disease results in cognitive decline, notably a loss of memory and the ability to sustain attention (6,7). Interfering with the cholinergic system strongly affects cognition (3,8-13). Rapid changes in prefrontal cortical ACh levels at the scale of seconds are correlated with attending and detecting cues (14,15). Various types of nicotinic ACh receptor (nAChR) subunits are expressed in the prefrontal cortex (PFC) (16-18). . . . However, the causal relation between nAChR β2 subunits (henceforth β2-nAChRs) expressed in the medial PFC (mPFC) and attention performance has not yet been demonstrated. (p. 888)

Guillem et al. began with a premise (“Cortical acetylcholine (ACh) release from the basal forebrain is essential”), which they then supported with the literature. They further developed their premise (“Loss of cholinergic function during aging and Alzheimer’s disease results in cognitive decline,” “Interfering with the cholinergic system strongly affects cognition,” and “Rapid changes in prefrontal cortical ACh levels . . . are correlated with attending and detecting cues”), and they concluded with their “However.” They synthesized the literature to tell a story.

Conclusion

Writing clearly and memorably need not be orthogonal to writing transparently and enabling reproducibility. For example, in their seminal article on false memories for words presented in lists, Roediger and McDermott (1995) documented their experimental procedure fully enough to allow replication, including most recently a preregistered replication (Zwaan et al., 2017); they provided their research materials openly (in an appendix); they told their story in short paragraphs (average length of 5.1 sentences) and short sentences (average length of 18 words); they embraced an hourglass shape (e.g., their discussion began by relating their study to prior work, continued by contrasting experiments that measured false recall vs. false recognition, extended to discussing phenomenological experience, and broadened to articulating implications); and they transparently acknowledged parallel efforts by another research team (“While working on this article, we learned that Don Read was conducting similar research, which is described briefly in Lindsay & Read, 1994,” p. 804). A well-written empirical article that enables reproducibility and transparency can also be clear and memorable.

Barring extraordinary disruption, empirical journal articles are likely to survive at least a couple more decades. Authors will continue to write empirical articles to communicate why they did their studies, how they did their studies, what they observed, and what those observations mean. And readers will continue to read empirical articles to receive this communication. The most successful articles will continue to embody Grice’s (1975) maxims for communication: They will be informative, truthful, relevant, clear, and memorable.

Supplemental Material

GernsbacherAMPPSOpenPracticesDisclosure – Supplemental material for Writing Empirical Articles: Transparency, Reproducibility, Clarity, and Memorability

Supplemental material, GernsbacherAMPPSOpenPracticesDisclosure for Writing Empirical Articles: Transparency, Reproducibility, Clarity, and Memorability by Morton Ann Gernsbacher in Advances in Methods and Practices in Psychological Science