Sage Journals: Discover world-class research

Abstract

Aim:

This study examines use and impacts of systematic reviews produced by the Campbell Collaboration’s Social Welfare Coordinating Group (SWCG) on practice, policy, and research.

Methods:

A mixed-method research design was used to examine impacts of 52 systematic reviews published by the SWCG. We conducted author surveys and retrieved multiple sources of bibliometric data.

Results:

Campbell SWCG reviews were downloaded 136,356 times and cited 3,184 times. Most reviews did not receive significant attention in alternative outlets (i.e., social media). Review authors provided evidence that reviews were used directly to make changes in policy or practice or inform future research.

Discussion:

Assessing the use and impacts of research is challenging. While downloads and citations provide evidence that these reviews receive attention, it was more difficult to determine the extent to which the reviews were used to impact practice or policy. More work is needed to better track and assess impacts of Campbell reviews.

Keywords

systematic review research impact mixed methods methodological article

Get full access to this article

View all access options for this article.

References

Altmetric (2016). How is the Altmetric attention score calculated? Retrieved from https://help.altmetric.com/support/solutions/articles/6000060969-how-is-the-altmetric-score-calculated-

Armstrong

Pettman

Burford

Doyle

Waters

(2012). Tracking and understanding the utility of Cochrane reviews for public health decision-making. Journal of Public Health, 34, 309–313.

Bastian

Glasziou

Chalmers

(2010). Seventy-five trials and eleven systematic reviews a day: How will we ever keep up? PLoS Med, 7, e1000326. doi:10.1371/journal.pmed.1000326

Bero

L. A.

Jadad

A. R.

(1997). How consumers and policymakers can use systematic reviews for decision making. Annals of Internal Medicine, 127, 37–42.

Bunn

Trivedi

Alderson

Hamilton

Martin

Iliffe

(2014). The impact of Cochrane Systematic Reviews: A mixed method evaluation of outputs from Cochrane Review Groups supported by the UK National Institute for Health Research. Systematic Reviews, 3, 125.

Cohen

Schroeder

Newson

King

Rychetnik

Milat

A. J.

… Chapman

. (2015). Does health intervention research have real world policy and practice impacts: Testing a new impact assessment tool. Health Research Policy and Systems, 13, 3. doi:10.1186/1478-4505-13-3

Dinsmore

Allen

Dolby

(2014). Alternative perspectives on impact: The potential of ALMs and altmetrics to inform funders about research impact. PLoS Biology, 12, e1002003. doi:10.1371/journal.pbio.1002003

Hanney

S. R.

Watt

Jones

T. H.

Metcalf

(2013). Conducting retrospective impact analysis to inform a medical research charity’s funding strategies: The case of Asthma UK. Allergy, Asthma & Clinical Immunology, 9, 17. doi:10.1186/1710-1492-9-17

MacRoberts

M. H.

MacRoberts

B. R.

(2010). Problems of citation analysis: A study of uncited and seldom-cited influences. Journal of the American Society for Information Science and Technology, 61, 1–12.

10.

Moed

H. F.

Halevi

(2015). Multidimensional assessment of scholarly research impact. Journal of the Association for Information Science and Technology, 66, 1988–2002.

11.

Mulrow

C. D.

(1994). Rationale for systematic reviews. British Medical Journal, 309, 597–599.

12.

Nutley

S. M.

Walter

Davies

H. T. O.

(2007). Using evidence: How research can inform public services. Bristol, UK: The Policy Press.

13.

Penfield

Baker

M. J.

Scoble

Wykes

M. C.

(2014). Assessment, evaluations, and definitions of research impact: A review. Research Evaluation, 23, 21–32.

14.

Petrosino

(2013). Reflections on the genesis of the Campbell Collaboration. The Experimental Criminologist, 8, 9–12.

15.

Piwowar

(2013). Introduction altmetrics: What, why and where? Bulletin of the American Society for Information Science and Technology, 39, 8–9. doi:10.1002/bult.2013.1720390404

16.

Shen

Clarke

Wang

Zhong

. (2014). Production and citation of Cochrane systematic reviews: A bibliometrics analysis. Journal of Evidence-Based Medicine. Advance online publication. doi:10.1111/jebm.12101

17.

Soydan

(2008). Applying randomized controlled trials and systematic reviews in social work research. Research on Social Work Practice, 18, 311–318.

18.

Tricco

A. C.

Cardoso

Thomas

S. M.

Motiwala

Sullivan

Kealey

M. R.

… Shepperd

. (2016). Barriers and facilitators to uptake of systematic reviews by policy makers and health care managers: a scoping review. Implementation Science, 11, 4. doi:10.1186/s13012-016-0370-1

19.

Veletsianos

(2012). Higher education scholars’ participation and practices on twitter. Journal of Computer Assisted Learning, 28, 336–349.

20.

Weller

. (2015). Social media and altmetrics: An overview of current alternative approaches to measuring scholarly impact. In Welpe

I. M.

Wollersheim

Ringelhan

Osterloh

(Eds.), Incentives and performance (pp. 261–276). Switzerland: Springer International Publishing.