Emerging international standards for measurement and evaluation of public relations: A critical analysis

Abstract

After 30 years of modest progress in measurement and evaluation of public relations since Jim Grunig uttered his cri de coeur about lack of evaluation, a flurry of activity has occurred in the past few years. A new momentum started with the Barcelona Declaration of Measurement Principles in 2010. In 2011, a Coalition for Public Relations Research Standards was formed by three leading international PR and research organizations. In 2012, the group expanded to 11 professional associations which worked in collaboration with advertising and media organizations and companies representing employer perspectives to publish a number of definitions and standards for measurement and evaluation in 2012 and 2013. Concurrently, there have been renewed debates about measurement concepts such as Return on Investment (ROI). As the industry reaches the 20^th anniversary of the International Public Relations Association ‘Gold Paper on Evaluation’ published in 1994, it appears that progress is at last being made. This paper welcomes and commends initiatives taken, but presents a critical analysis that reveals continuing gaps and problematic issues to address in the latest efforts to measure the value of PR and a substantial gap between theory and practice.

Keywords

Measurement evaluation standards PR research ROI metrics

Get full access to this article

View all access options for this article.

References

AMEC [Association for Measurement and Evaluation of Communication] (2012) Glossary: Plain speaking. London, UK: AMEC. Available at: http://amecorg.com/2012/06/glossary_plain_speaking/#E (accessed 16 December 2013)

Berg

(2007) Qualitative Research Methods for the Social Sciences, 6th edn. Boston, MA: Allyn & Bacon.

Blanchard

(2011) Social Media ROI: Managing and Measuring Social Media Efforts in Your Organization. Indianapolis, IN: Que; Boston, MA: Pearson.

Bowen

(2009) What communication professionals tell us regarding dominant coalition access and gaining membership. Journal of Applied Communication Research 37(4): 418–443.

Broom

Dozier

(1990) Using Research in Public Relations: Applications to Program Management. Englewood Cliffs, NJ: Prentice-Hall.

Cheng

de Gregario

(2008) Does (linking with) practice make perfect? A survey of public relations scholars’ perspectives. Journal of Public Relations Research 20(4): 377–402.

Cutlip

Center

Broom

(1985) Effective Public Relations, 6^th edn. Englewood Cliffs, NJ: Prentice Hall.

Daniels

(2012a) Valid Metrics Frameworks. Presentation to the 4^th European Summit on Measurement, Dublin, Ireland, June. Available at: http://amecorg.com/downloads/dublin2012/Valid-Metrics-Framework-Mike-Daniels.pdf (accessed 16 December 2013)

Daniels

(2012b) ROI in PR measurement: Developing a practical strategy. First Asia PR Pacific Summit on Measurement, Hong Kong, March. Available at: http://amecorg.com/images/public/downloads/apac/Workshop_B.pdf (accessed 16 December 2013)

10.

Digital Analytics Association (2013) Social media standards definitions: Reach and impressions. Wakefield, MA. Available at: http://www.smmstandards.com/wp-content/uploads/2013/03/SMM-Standard-Definitions_DAA_v4.pdf (accessed 16 December 2013)

11.

Edwards

Hodges

(eds) (2011) Public Relations, Society and Culture: Theoretical and Empirical Explorations. Abingdon, UK: Routledge.

12.

Ehling

(1992) Estimating the value of public relations and communication to an organisation. In: Grunig

(ed) Excellence in Public Relations and Communication Management. Hillsdale, NJ: Lawrence Erlbaum Associates, pp. 617–638.

13.

Eisenmann

Geddes

Paine

. (2012) Proposed interim standards for metrics in traditional media analysis. Gainesville, FL: Institute for Public Relations. Available at: http://www.instituteforpr.org/topics/proposed-interim-standards-for-metrics-in-traditional-media-analysis (accessed 16 December 2013)

14.

Erickson

(2008) Plugged In: The Generation Y Guide to Thriving at Work. Boston, MA: Harvard Business School.

15.

European Commission (2013) Social entrepreneurship: The social business initiative. Brussels, Belgium: European Commission. Available at: http://ec.europa.eu/internal_market/social_business/index_en.htm (accessed 16 December 2013)

16.

Facebook (2013) What’s the difference between impressions and reach. Page post metrics. Facebook: Redwood, CA. Available at: http://www.facebook.com/help/274400362581037/ (accessed 16 December 2013)

17.

Flamholtz

(1985) Human Resource Accounting: Advances in Concepts, Methods and Applications. San Francisco, CA: Jossey-Bass.

18.

Grunig

(1983) Basic research provides knowledge that makes evaluation possible. Public Relations Quarterly 28(3): 28–32.

19.

Grunig

Dozier

(2002) Excellent Organisations and Effective Organisations: A Study of Communication Management in Three Countries. Mahwah, NJ: Lawrence Erlbaum Associates.

20.

Grunig

Dozier

(2006) The excellence theory. In: Botan

Hazelton

(eds) Public Relations Theory II. Mahwah, NJ: Lawrence Erlbaum Associates, 21–62.

21.

Institute for Public Relations (2010) Barcelona declaration of measurement principles. Gainesville, FL: IPR. Available at: http://www.instituteforpr.org/topics/barcelona-declaration-of-measurement-principles/ (accessed 16 December 2013)

22.

IAB [Interactive Advertising Bureau] (2004a) Ad impression measurement guidelines. New York, NY: IAB. Available at: http://www.iab.net/guidelines/508676/guidelines/campaign_measurement_audit (accessed 16 December 2013)

23.

IAB [Interactive Advertising Bureau] (2004b) Interactive audience measurement and advertising campaign reporting and audit guidelines, Version 6.0b, Global Version. New York, NY: IAB. Available at: http://www.iab.net/media/file/Global_meas_guidelines.pdf (accessed 16 December 2013)

24.

IAB [Interactive Advertising Bureau] (2009) Audience reach measurement guidelines. New York, NY: IAB. Available at: http://www.iab.net/guidelines/508676/guidelines/audiencemeasurement (accessed 16 December 2013)

25.

IIRC (International Integrated Reporting Council) (2013) International Integrated Reporting Framework. London, UK. Available at http://www.theiirc.org/international-ir-framework (accessed 31 January 2013)

26.

International Public Relations Association (1994) Public Relations evaluation: Professional accountability, Gold Paper No. 11. Geneva, Switzerland: IPRA.

27.

Investopedia (2013) Definition of return on investment – ROI. Available at: http://www.investopedia.com/terms/r/returnoninvestment.asp (accessed 16 December 2013)

28.

Lee

Yoon

(2010) Return on investment (ROI) of international public relations: A country-level analysis. Public Relations Review 36(1): 15–20.

29.

Likely

(2012) Principles for the use of return on investment (ROI), Benefit-cost ration (BCR) and cost-effectiveness analysis (CEA) financial metrics in a public relations/communication (PR/C) department. Paper presented to the 15th International Public Relations Research Conference, 8–10 March, Miami, FL.

30.

Likely

Watson

(2013) Measuring the edifice: Public relations measurement and evaluation practices over the course of 40 years. In: Sriramesh

Zerfass

Kim

(eds) Current Trends and Emerging Topics in Public Relations and Communication Management. New York: Routledge, pp. 143–162.

31.

Likely

Rockland

Weiner

(2006) Perspectives on the ROI of media relations publicity efforts. Gainesville, FL: Institute for Public Relations. Available at: http://www.instituteforpr.org/research_single/perspectives_on_the_roi/ (accessed 16 December 2013)

32.

Lindenmann

(1993) An ‘effectiveness yardstick’ to measure public relations success. PR Quarterly 38(1): 7–9.

33.

Lindenmann

(1997/2003) Guidelines and standards for measuring and evaluating PR effectiveness. Gainesville, FL: Institute for Public Relations. Available at: http://www.instituteforpr.org/topics/effectiveness-programs-activities (accessed 16 December 2013)

34.

Macey

Schneider

(2008) The meaning of employee engagement. Industrial and Organizational Psychology 1: 3–30.

35.

Macnamara

(1992) Evaluation: The Achilles heel of the public relations profession. International Public Relations Review 15(4): 17–31.

36.

Macnamara

(1999) Research in public relations: A review of the use of evaluation and formative research. Asia Pacific Public Relations Review (1)2: 107–133.

37.

Macnamara

(2002) Research and evaluation. In: Tymson

Lazar

(eds) The New Australian and New Zealand Public Relations Manual, 21^st century edn. Sydney: Tymson Communications, pp. 100–134.

38.

Macnamara

(2005) Jim Macnamara’s Public Relations Handbook, 5^th edn. Sydney, NSW: Archipelago Press.

39.

Macnamara

(2012) Public Relations Theories, Practices, Critiques. Sydney, NSW: Pearson.

40.

Macnamara

(2013) The toe bone to the head bone’ logic model: An approach to connect PR and corporate communication to organisation and business outcomes. London, UK: Association for Measurement and Evaluation of Communication. Available at: http://amecorg.com/2013/05/toe-bone-to-the-head-bone-logic-model/ (accessed 16 December 2013)

41.

Marklein

Paine

(2012) The march to standards. Presentation to the 4^th European Summit on Measurement, Dublin, Ireland, June. Available at: http://amecorg.com/downloads/dublin2012/The-March-to-Social-Standards-Tim-Marklein-and-Katie-Paine.pdf (accessed 16 December 2013)

42.

Marx

(1845) Theses on Feuerbach. In: Lough

(trans), Marx/Engels Selected Works, Vol. 1, Part xi. Moscow: Progress Publishers, 13–15. Available at:http://www.marxists.org/archive/marx/works/1845/theses/theses.htm (accessed 16 December 2013)

43.

Meng

Berger

(2012) Measuring return on investment (ROI) organizations’ internal communication effort. Journal of Communication Management 16(4): 332–354.

44.

Meyer

Smith

(2000) HRM practices and organisational commitment: A test of a mediation model. Canadian Journal of Administrative Services 17: 319–331.

45.

Michaelson

Stacks

(2011) Standardization in public relations measurement and evaluation. Public Relations Journal 2: 1–22.

46.

Neuendorf

(2002) The Content Analysis Guidebook. Thousand Oaks, CA: Sage Publications.

47.

Nicholls

Mackenzie

Somers

(2007) Measuring Real Value: A DIY Guide to social Return on Investment. London, UK: New Economics Foundation.

48.

Noble

Watson

(1999) Applying a unified public relations evaluation model in a European context. Paper presented to the Transnational Communication in Europe: Practice and Research Congress, Berlin, Germany.

49.

Okay

(2008) The place of theory in public relations practice. In: Hansen-Horn

Neff

(eds) Public Relations: From Theory to Practice. Boston, MA: Pearson, pp. 301–342.

50.

Paarlberg

(2013) Proposed social media standards: Engagement and conversation. #SMMStandards blog post, 18 March. Available at: http://www.smmstandards.com/2013/03/proposed-social-media-standards-engagement-and-conversation/#more-111 (accessed 16 December 2013)

51.

Paine

(2007) Measuring Public Relationships: The Data-driven Communicator’s Guide to Success, Paarlberg

(ed). Berlin, NH: KD Paine & Partners.

52.

Paine

(2011) The four big reasons why we need social media standards. The Measurement Standard, 25 May. Available at: http://kdpaine.blogs.com/themeasurementstandard/2011/05/4-big-reasons-why-we-need-social-media-standards.html (accessed 16 December 2013)

53.

Paine

(2012) Standards for social media measurement: A progress report. Presentation to the Federation Internationale des Bureaux d’Extraits de Presse (FIBEP) conference, Krakow, Poland, October. Available at: http://ebookbrowse.com/standards-for-social-media-measurement-a-progress-report-katie-paine-pdf-d434658158 (accessed 16 December 2013)

54.

Powell

Groves

Dimos

(2011) ROI of Social Media: How To Improve the Return on Your Social Marketing Investment. Singapore: John Wiley & Sons.

55.

Ragan/NASDAQ OMX Corporate Solutions (2013) PR Measurement. Chicago, IL. Available at: http://bit.ly/PRMeasureWPPR (accessed 16 December 2013)

56.

Rhoades

Eisenberger

Armeli

(2001) Affective commitment to the organisation: The contribution of perceived organisational support. Journal of Applied Psychology 86: 825–836.

57.

Scholten

Nicholls

Olsen

. (2006) Social Return on Investment: A Guide to SROI Analysis. Amstelveen, the Netherlands: Lenthe Publishers.

58.

Schultz

(2002) Measuring return on brand communication. International Journal of Medical Marketing 2(4): 349–358.

59.

Schultz

(2004) IMC: The Next Generation. New York, NY: McGraw-Hill.

60.

Sheldrake

(2011) The Business of Influence: Reframing Marketing and PR for the Digital Age. London, UK: Wiley.

61.

SMMStandards (2012a) Sources and Methods Transparency Table. Available at: http://www.smmstandards.com/category/content-sourcing-methods/ (accessed 16 December 2013)

62.

SMMStandards (2012b) Social Media Standard Definitions for Reach and Impressions. Available at: http://www.smmstandards.com/2013/03/proposed-social-media-standard-definitions-for-reach-and-impressions-from-the-digital-analytics-association/ (accessed 16 December 2013)

63.

Social E-valuator (2013) Impact and value. Utrecht, the Netherlands: De Alchemist. Available at: http://www.socialevaluator.eu/WhatIsSROI.aspx (accessed 16 December 2013)

64.

Solis

(2010) ROI: How to measure return on investment in social media. @Brian Solis, 22 February. Available at: http://www.briansolis.com/2010/02/roi-how-to-measure-return-on-investment-in-social-media/ (accessed 16 December 2013)

65.

SROI Network (2012) A Guide to Social Return on Investment 2012. Haddington, East Lothian, Scotland: SROI Network. Available at: http://www.thesroinetwork.org/publications/doc_details/241-a-guide-to-social-return-on-investment-2012 (accessed 16 December 2013)

66.

Stacks

(ed) (2006) Dictionary of public relations measurement and research. Gainesville, FL: Institute for Public Relations. Available at: http://www.instituteforpr.org/iprwp/wp-content/uploads/PRMR_Dictionary.pdf (accessed 16 December 2013)

67.

Stacks

(2011) Primer of Public Relations Research, 2^nd edn. New York, NY: The Guildford Press.

68.

Stacks

Bowen

(eds) (2013) Dictionary of public relations measurement and research. Gainesville FL: Institute for Public Relations. Available at: http://www.instituteforpr.org/topics/dictionary-of-public-relations-measurement-and-research/ (accessed 16 December 2013)

69.

Stacks

Michaelson

(2010) A Practitioner’s Guide to Public Relations Research, Measurement, and Evaluation. New York, NY: Business Experts Press.

70.

Watson

(1997) Measuring the success rate: Evaluating the PR process and PR programs. In: Kitchen

(ed) Public Relations Principles and Practices. Boston, MA: International Thomson Business Press, pp. 283–299.

71.

Watson

(2008) Public relations research priorities: A Delphi study. Journal of Communication Management 12(2): 104–123.

72.

Watson

(2012) The evolution of public relations measurement and evaluation. Public Relations Review 38(3): 390–398.

73.

Watson

(2013) ROE may be more accurate than ROI. FiftyoneFiftyZero, 22 March. Available at: http://fiftyonezeroone.blogspot.com.au/2013/03/roe-may-be-more-accurate-than-roi.html (accessed 16 December 2013)

74.

Watson

Noble

(2007) Evaluating Public Relations: A Best Practice Guide to Public Relations Planning, Research and Evaluation, 2^nd edn. London and Philadelphia, PA: Kogan Page.

75.

Watson

Zerfass

(2011) Return on investment in public relations: A critique of concepts used by practitioners from communication and management sciences perspectives. PRism 8(1): 1–14. Available at: http://www.prismjournal.org/vol8_1.html (accessed 16 December 2013)

76.

Watson

Zerfass

(2012) ROI and PR evaluation: Avoiding ‘smoke and mirrors’. International Public Relations Research Conference, Miami, FL, March. Available at: http://www.instituteforpr.org/iprwp/wp-content/uploads/Watson-Zerfass-ROI-IPRRC-Miami-20121.pdf (accessed 16 December 2013)

77.

Weiner

(2006) Unleashing the Power of PR: A Contrarians Guide to Marketing and Communication. San Francisco, CA: John Wiley & Sons.

78.

Weiner

(2013) Public relations research and measurement: Are CPMs a way forward? Research Conversations, 26 February. Gainesville, FL: Institute for Public Relations. Available at: http://www.instituteforpr.org/2013/02/public-relations-research-and-measurement-are-cpms-a-way-forward/ (accessed 16 December 2013)

79.

Wright

Hinson

(2012) Examining how social and emerging media have been used in public relations between 2006 and 2012: A longitudinal analysis. Public Relations Journal 6(4): 1–40.

80.

Wright

Gaunt

Leggetter

, et al. (2009) Global survey of communications measurement 2009 – final report. London, UK: Association for Measurement and Evaluation of Communication. Available at: http://amecorg.com/wp-content/uploads/2011/08/Global-Survey-Communications_Measurement-20091.pdf (accessed 16 December 2013)

81.

Zerfass

(2010) Assuring rationality and transparency in corporate communications. Theoretical foundations and empirical findings on communication controlling and communication performance management. In: Dodd

Yamamura

(eds) Ethical Issues for Public Relations Practice in a Multicultural World, 13th International Public Relations Research Conference. Gainesville, FL: Institute for Public Relations, pp. 947–966.

82.

Zerfass

Verčič

Verhoeven

. (2012) European Communication Monitor 2012: Challenges and Competencies for Strategic Communication. Brussels: Helios Media, EACD/EUPRERA.