Abstract
Objective
To quantify the use of question marks in titles of published studies.
Design and setting
Literature review.
Participants
All Pubmed publications between 1 January 2013 and 31 December 2013 with an available abstract. Papers were classified as being clinical when the search terms clin*, med* or patient* were found anywhere in the paper’s title, abstract or the journal’s name. Other papers were considered controls. As a verification, clinical journals were compared to non-clinical journals in two different approaches. Also, 50 highest impact journals were explored for publisher group dependent differences.
Main outcome measure
Total number of question marks in titles.
Results
A total of 368,362 papers were classified as clinical and 596,889 as controls. Clinical papers had question marks in 3.9% (95% confidence interval 3.8–4.0%) of titles and other papers in 2.3% (confidence interval 2.3–2.3%; p < 0.001). These findings could be verified for clinical journals compared to non-clinical journals. Different percentages between four publisher groups were found (p < 0.01).
Conclusion
We found more question marks in titles of clinical papers than in other papers. This could suggest that clinicians often have a question-driven approach to research and scientists in more fundamental research a hypothesis-driven approach. An alternative explanation is that clinicians like catchy titles. Publishing groups might have pro- and anti-question mark policies.
Introduction
‘To be or not to be’ was once posed as the question. 1 Researchers argued other questions to be the question ever since. This can be found in titles of papers like: ‘LP (lumbar puncture) or not LP, that is the question …’, ‘Renal replacement therapy: to treat, or not to treat, that is the question’ and ‘To PPT (patterns of physical transformation) or not to PPT? That is the question’!2–4 These researchers apparently face questions that seem more urgent than life.
Titles play an important role in framing data queries, searching references or scanning journal outlines. 5 An article’s impact can be influenced by the title. 6 Article title types relate to the number of downloads or citations: papers with questions in their titles are downloaded more often but cited less often when compared to other papers. 7 Question marks are, however, increasingly used in titles. 8
A keystone tradition in fundamental science is to start from a hypothesis and then test whether the hypothesis can be rejected, resulting in hypothesis-driven research. 9 Clinicians, often confronted with practical questions from daily practice towards which diagnostic or treatment option is best, might take a question-driven approach in their research. For instance, the first paper in the Pubmed database containing a question mark in the title was published in 1853 and discusses whether hypospadias is a bar to marriage, which reflects a practical question. 10
We compared the number of question marks used in titles of published clinical articles to less-clinical studies.
Methods
Data selection
We performed a literature search in the Pubmed database. We included papers that had an abstract available, which are predominantly original research and review papers and exclude letters to editors and other small reports. We restricted this search to publication dates in 2013 due to the large amount of titles and limited computing space. We divided papers into clinical papers and ‘less-clinical’ papers using the search terms (hasabstract[text] AND (‘2013/01/01’[PDAT]: ‘2013/12/31’[PDAT])) AND (clin* OR med* OR patient*) for clinical papers and the search terms (hasabstract[text] AND (‘2013/01/01’[PDAT]: ‘2013/12/31’[PDAT])) NOT (clin* OR med* OR patient*) for less-clinical papers.
Next, we verified our findings by comparing clinical versus non-clinical journals. We reviewed papers with available abstracts from a selection of 50 journals, which had highest impact factors in 2012 (ISI Web of Science) and had more than one abstract available on Pubmed. We extended our search from 1 to 10 years, so between 2004 and 2013, to increase the number of titles per individual journal. We used the journal’s title as search term ([journal]) within the advanced search builder in Pubmed together with search terms (hasabstract[text] AND (‘2013/01/01’[PDAT]: ‘2013/12/31’[PDAT])). We counted the number of question marks in paper titles per journal. We compared the journals which we considered clinical journals, i.e. journal containing the words medical, diseases, clinical, cancer and Lancet, to all other journals for the percentage of question marks.
We then verified our findings by comparing specialist clinical journals to specialist non-clinical journals. We chose to use all neurological versus neuroscience journals (2004–2013) for this comparison. We considered clinical neurological journals, all the journals that contained ‘neurology’ in their title but excluded all journals containing any other specification (e.g. behavioural neurology), and we considered non-clinical neuroscience journals, all journals containing ‘neuroscience’ in their title without other specifications.
We also analysed the relation between number of question marks and the journal’s impact factors and differences between different publishing groups in the 50 journals with high impact factors. We restricted the last analysis to the publishing groups Nature, Nature reviews, Lancet and Annual reviews.
Statistical analyses
We compared the number of question marks in titles of clinical and less-clinical papers using the chi-square test. We verified the findings for clinical versus non-clinical journals using one-sided t-tests.
For the 50 highest impact factor journals, we correlated the impact factor to the percentage of papers with question marks in their title using Pearson’s correlation. We compared the number of question marks in titles from different publishing groups using a one-way analysis of variance with post hoc Tukey’s honestly significant difference test.
Statistical analyses were performed using R (www.r-project.org) and Excel. A p value below 0.05 was considered to be significant.
Results
A total of 368,362 papers were classified as clinical and 596,889 as less-clinical controls; 28,110 papers (2.9%) contained question marks in their title. 3.9% of the clinical papers had question marks (95% confidence interval [CI]: 3.8–4.0%) and 2.3% of the less-clinical papers had question marks ([CI 2.3–2.3%], p < 0.001; Figure 1).
Left: the percentage of clinical and less-clinical papers using question marks. The clinical papers contain more question marks in their titles than the less-clinical papers (p < 0.001). Right: the percentages of titles with question marks per journal for four different journal groups (in order from highest to lowest percentage of question marks): (1) the Nature group including Nat Biotechnol, Nat Immunol, Nat Cell Biol, Nat Med, Nature, Nat Nanotechnol, Nat Mater, Nat Methods, Nat Genet, Nat Chem, Nat Photonics and Nat Physics; (2) the Nature Reviews group including Nat Rev Microbiol, Nat Rev Drug Discov, Nat Rev Cancer, Nat Rev Neurosci, Nat Rev Immunol, Nat Rev Mol Cell Bio and Nat Rev Genet; (3) the Annual Reviews group including Ann Rev Neurosc, Ann Rev Cell Dev Biol, Ann Rev Genet, Ann Rev Physiol, Ann Rev Biochem, Ann Rev Pathol-mech, Ann Rev Plant Biol, Ann Rev Immunol, Ann Rev Astron Atr and Ann Rev Pharmacol; (4) the Lancet group including Lancet Infect Dis, Lancet Oncol, Lancet and Lancet Neurol. The percentages of titles with question marks per journal are lower for the Nature group compared to the Nature Reviews group (p < 0.001) and the Lancet group (p < 0.001) and is lower in the Annual Reviews group compared to the Nature Reviews group (p < 0.001) and the Lancet Group (p = 0.008).
A similar difference was found between 13 clinical high-impact journal (3.3% question-marks) and 37 other high-impact journals (2.0%; p = 0.03) and between 26 neurology journals (3.8% question marks) and 16 neuroscience journals (2.6%; p = 0.04). We found no correlation between the impact factor of journals and the percentage of papers with question marks in their title (r = 0.07). A striking finding was that the New England Journal of Medicine contains just one paper with a question mark out of the 2270 papers published. 11 Differences between publisher groups were found for the averaged percentages of papers with question marks in their title per journal (analysis of variance F value 23.7; p < 0.001; Figure 1).
Discussion
Clinical papers contain more question marks in their titles compared to less-clinical papers. This suggests that clinical researchers tend to use more question marks than other researchers, which fits with our hypothesis that clinicians have a more question-driven approach to research compared to scientists in fundamental research. A difference was found between publisher groups, suggesting the existence of an editorial policy bias.
This study does not actually prove that clinicians have a question-driven approach to research. We tried to clarify research approach by looking through abstracts and introductions for the words ‘hypothesis’ and ‘question’. What we found supported our thoughts. We felt, however, that it was impossible to quantify this. Often a question was hypothesised or a hypothesis was questioned. A similar ambiguity can be found in our own study: we posed a hypothesis and answered a question. In research, the terms ‘hypothesis’ and ‘question’ are used intertwined, while they imply a different scientific reasoning.
The essential difference between hypothesis-driven research and question-driven research is that a hypothesis can only be refuted and thus only indirectly proven, whereas a question can be answered. A hypothesis is useful for original experiments with unknown outcome. 12 Current research is, however, often inductive in nature, as we do not want to discard previous experiences. 13 In inductive reasoning, either a model is based on sufficient existing data or a question is formulated. It has been stated before that clinical trials should not be framed with a hypothesis aimed at falsification but rather should be explicitly inductive because the work of physicians is explicitly inductive. 14
An alternative explanation for our findings is that clinicians might prefer to use catchy titles to fulfil the societal, social-media and TV-series-based personality of a quick-witted person. 15 Some editor groups seem to be less fond of the use of question marks in titles than others. This might be explained by the knowledge that question-mark titles are cited less, 7 that they disapprove catchy titles or that they disapprove the suggestion of inductive rather than hypothesis-driven research.
It might be time for the clinical scientific society to accept that a great quantity of medical research is inductive instead of hypothesis driven. We should abandon the quest for formulating hypotheses, while we are actually dealing with models and questions. It is important for investigators to question themselves whether they are truly dealing with a hypothesis or rather a model or question during the design and analysis of their scientific studies.
Finally, even science sometimes falls short giving answers to the bigger questions in life. In contrast to Hamlet’s soliloquy, however, research papers should aim to answer questions that are posed in titles of publications.