Surveying the Upper Echelons: An Update to Cycyota and Harrison (2006) on Top Manager Response Rates and Recommendations for the Future

Abstract

Nearly 2 decades ago, Cycyota and Harrison (2006) documented a concerning trend of declining executive survey response rates and projected a continued decrease in the future. Their seminal work has significantly influenced the methodologies of upper echelons survey research. Our study examines the manner in which Cycyota and Harrison’s paper has impacted the existing upper echelons literature and replicates their study by analyzing peer-reviewed studies published post-2006. We reveal that executive response rates have largely stabilized since Cycyota and Harrison’s initial findings. Furthermore, we expand upon their research by identifying specific geographical contexts and contact methodologies associated with higher (and lower) response rates. Finally, we lend insight into the evolving landscape of executive survey research and offer practical implications for future methodological endeavors in the upper echelons.

Keywords

survey types of research design replication design significance testing data sampling and modeling

Get full access to this article

View all access options for this article.

References

Aguinis

Ramani

R. S.

Alabduljader

(2018). What you see is what you get. Enhancing methodological transparency in management research. Academy of Management Annals, 12(1), 83–110. https://doi.org/10.5465/annals.2016.0011

Aguinis

Ramani

R. S.

Villamor

(2019). The first 20 years of organizational research methods: Trajectory, impact, and predictions for the future. Organizational Research Methods, 22(2), 463–489. https://doi.org/10.1177/1094428118786564

Anseel

Lievens

Schollaert

Choragwicka

(2010). Response rates in organizational science, 1995–2008: A meta-analytic review and guidelines for survey researchers. Journal of Business and Psychology, 25(3), 335–349. https://doi.org/10.1007/s10869-010-9157-6

Bandiera

Prat

Hansen

Sadun

(2020). CEO behavior and firm performance. Journal of Political Economy, 128(4), 1325–1369. https://doi.org/10.1086/705331

Bartholomew

Smith

A. D.

(2006). Improving survey response rates from chief executive officers in small firms: The importance of social networks. Entrepreneurship Theory and Practice, 30(1), 83–96. https://doi.org/10.1111/j.1540-6520.2006.00111.x

Bartunek

J. M.

Rynes

S. L.

Ireland

R. D.

(2006). What makes management research interesting, and why does it matter? Academy of Management Journal, 49(1), 9–15. https://doi.org/10.5465/amj.2006.20785494

Baruch

Holtom

B. C.

(2008). Survey response rate levels and trends in organizational research. Human Relations, 61(8), 1139–1160. https://doi.org/10.1177/0018726708094863

Bednar

M. K.

Westphal

J. D.

(2006). Surveying the corporate elite: Theoretical and practical guidance on improving response rates and response quality in top management survey questionnaires. In Ketchen

Bergh

(Eds.), Research methodology in strategy and management (Vol. 3, pp. 37–56). Emerald Group Publishing Limited.

Bergh

D. D.

Sharp

B. M.

Aguinis

(2017). Is there a credibility crisis in strategic management research? Evidence on the reproducibility of study findings. Strategic Organization, 15(3), 423–436. https://doi.org/10.1177/1476127017701076

10.

Bolinger

Brookman

Thistle

(2023). The relative importance of CEOS and non-CEOS over firm performance. Journal of Management and Organization, 29(1), 1–13. https://doi.org/10.1017/jmo.2019.47

11.

Borgholthaus

C. J.

White

J. V.

Harms

P. D.

(2023). CEO dark personality: A critical review, bibliometric analysis, and research agenda. Personality and Individual Differences, 201(1), 111951. https://doi.org/10.1016/j.paid.2022.111951

12.

Cialdini

R. B.

(2009). Influence: Science and practice. Pearson Education.

13.

Cohen

(1992). A power primer. Psychological Bulletin, 112(1), 155–159. https://doi.org/10.1037/0033-2909.112.1.155

14.

Couper

M. P.

(2000). Web surveys: A review of issues and approaches. The Public Opinion Quarterly, 64(4), 464–494. https://doi.org/10.1086/318641

15.

Crawford

S. D.

Couper

M. P.

Lamias

M. J.

(2001). Web surveys: Perceptions of burden. Social Science Computer Review, 19(2), 146–162. https://doi.org/10.1177/089443930101900202

16.

Credé

Harms

Niehorster

Gaye-Valentine

(2012). An evaluation of the consequences of using short measures of the Big Five personality traits. Journal of Personality and Social Psychology, 102(4), 874–888. https://doi.org/10.1037/a0027403

17.

Cycyota

C. S.

Harrison

D. A.

(2006). What (not) to expect when surveying executives: A meta-analysis of top manager response rates and techniques over time. Organizational Research Methods, 9(2), 133–160. https://doi.org/10.1177/1094428105280770

18.

DeSimone

J. A.

DeSimone

A. J.

Harms

Wood

(2018). The differential impacts of two forms of insufficient effort responding. Applied Psychology, 67(2), 309–338. https://doi.org/10.1111/apps.12117

19.

DeSimone

J. A.

Harms

P. D.

DeSimone

A. J.

(2015). Best practice recommendations for data screening. Journal of Organizational Behavior, 36(2), 171–181. https://doi.org/10.1002/job.1962

20.

Dillman

D. A.

(1978). Mail and telephone surveys: The total design method. Wiley.

21.

Dillman

D. A.

(1991). The design and administration of mail surveys. Annual Review of Sociology, 17(1), 225–249. https://doi.org/10.1146/annurev.so.17.080191.001301

22.

Dillman

D. A.

(2000). Mail and internet surveys: The tailored design method. Wiley.

23.

Erikson

E. H.

(1982). The life cycle completed. WW Norton and Company.

24.

Falconer

D. J.

Hodgett

R. A.

(1999). Why executives don’t respond to your survey. In Proceedings of the 10th Australasian Conference on Information Systems.

25.

Fulton

B. R.

(2018). Organizations and survey research: Implementing response enhancing strategies and conducting nonresponse analyses. Sociological Methods and Research, 47(2), 240–276. https://doi.org/10.1177/0049124115626169

26.

Hambrick

D. C.

(2007). Upper echelons theory: An update. Academy of Management Review, 32(2), 334–343. https://doi.org/10.5465/amr.2007.24345254

27.

Hambrick

D. C.

Mason

P. A.

(1984). Upper echelons: The organization as a reflection of its top managers. Academy of Management Review, 9(2), 193–206. https://doi.org/10.2307/258434

28.

Harrell

F. E.

(2001). Regression modeling strategies: With applications to linear models, logistic regression, and survival analysis. Springer.

29.

Hmieleski

K. M.

Sheppard

L. D.

(2019). The Yin and Yang of entrepreneurship: Gender differences in the importance of communal and agentic characteristics for entrepreneurs’ subjective well-being and performance. Journal of Business Venturing, 34(4), 709–730. https://doi.org/10.1016/j.jbusvent.2018.06.006

30.

Holmes

R. M.

Jr Hitt

M. A.

Perrewe

P. L.

Palmer

J. C.

Molina-Sieiro

(2021). Building cross-disciplinary bridges in leadership: Integrating top executive personality and leadership theory and research. The Leadership Quarterly, 32(1), 101490. https://doi.org/10.1016/j.leaqua.2020.101490

31.

Holtom

Baruch

Aguinis

Ballinger

G. A.

(2022). Survey response rates: Trends and a validity assessment framework. Human Relations, 75(8), 1560–1584. https://doi.org/10.1177/00187267211070769

32.

Knudsen

H. K.

Ducharme

L. J.

Roman

P. M.

(2009). Turnover intention and emotional exhaustion “at the top”: Adapting the job demands-resources model to leaders of addiction treatment organizations. Journal of Occupational Health Psychology, 14(1), 84–95. https://doi.org/10.1037/a0013822

33.

Lance

C. E.

Baranik

L. E.

Lau

A. R.

Scharlau

E. A.

(2009). If it ain’t trait it must be method: (Mis)application of the multitrait–multimethod design in organizational research. In Lance

C. E.

Vandenberg

R. J.

(Eds.), Statistical and methodological myths and urban legends: Received doctrine, verity, and fable in the organizational and social sciences (pp. 337–360). Routledge.

34.

Lin

Zheng

Liu

Wright

(2019). Forgotten or not? Home country embeddedness and returnee entrepreneurship. Journal of World Business, 54(1), 1–13. https://doi.org/10.1016/j.jwb.2018.08.003

35.

Seidl

McNulty

(2021). Challenges and practices of interviewing business elites. Strategic Organization, 19(1), 81–96. https://doi.org/10.1177/1476127020980969

36.

Maes

Sels

(2014). SMEs’ radical product innovation: The role of internally and externally oriented knowledge capabilities. Journal of Small Business Management, 52(1), 141–163. https://doi.org/10.1111/jsbm.12037

37.

McDonald

M. L.

Westphal

J. D.

(2011). My brother’s keeper? CEO identification with the corporate elite, social support among CEOs, and leader effectiveness. Academy of Management Journal, 54(4), 661–693. https://doi.org/10.5465/amj.2011.64869104

38.

McDonald

M. L.

Westphal

J. D.

(2013). Access denied: Low mentoring of women and minority first-time directors and its negative effects on appointments to additional boards. Academy of Management Journal, 56(4), 1169–1198. https://doi.org/10.5465/amj.2011.0230

39.

Murphy

K. R.

Russell

C. J.

(2017). Mend it or end it: Redirecting the search for interactions in the organizational sciences. Organizational Research Methods, 20(4), 549–573. https://doi.org/10.1177/1094428115625322

40.

Nachum

(2010). When is foreignness an asset or a liability? Explaining the performance differential between foreign and local firms. Journal of Management, 36(3), 714–739. https://doi.org/10.1177/0149206309338522

41.

Neely

B. H.

Lovelace

J. B.

Cowen

A. P.

Hiller

N. J.

(2020). Metacritiques of upper echelons theory: Verdicts and recommendations for future research. Journal of Management, 46(6), 1029–1062. https://doi.org/10.1177/0149206320908640

42.

Neville

Byron

Post

Ward

(2019). Board independence and corporate misconduct: A cross-national meta-analysis. Journal of Management, 45(6), 2538–2569. https://doi.org/10.1177/0149206318801999

43.

Pfeffer

(1992). Managing with power: Politics and influence in organizations. Harvard Business School Press.

44.

Porter

Nohria

(2018). How CEOs manage time. Harvard Business Review, 96, 42–51.

45.

Quigley

T. J.

Hambrick

D. C.

(2015). Has the “CEO effect” increased in recent decades? A new explanation for the great rise in America’s attention to corporate leaders. Strategic Management Journal, 36(6), 821–830. https://doi.org/10.1002/smj.2258

46.

Rosenthal

(1994). Science and ethics in conducting, analyzing, and reporting psychological research. Psychological Science, 5(3), 127–134. https://doi.org/10.1111/j.1467-9280.1994.tb00646.x

47.

Rovelli

(2020). “I am stuck in meetings”: Understanding the relation of CEO time management with TMT size and gender diversity. European Management Journal, 38(5), 777–790. https://doi.org/10.1016/j.emj.2020.02.010

48.

Samiee

Athanassiou

(1998). International strategy research: Cross-cultural methodology implications. Journal of Business Research, 43(2), 79–96. https://doi.org/10.1016/S0148-2963(97)00184-7

49.

Saylors

Trafimow

(2021). Why the increasing use of complex causal models is a problem: On the danger sophisticated theoretical narratives pose to truth. Organizational Research Methods, 24(3), 616–629. https://doi.org/10.1177/1094428119893452

50.

Schwens

Zapkau

F. B.

Bierwerth

Isidor

Knight

Kabst

(2018). International entrepreneurship: A meta–analysis on the internationalization and performance relationship. Entrepreneurship Theory and Practice, 42(5), 734–768. https://doi.org/10.1177/1042258718795346

51.

Seshadri

Broekemier

G. M.

(2022). Small business executives’ online survey response intentions: The effects of incentives and survey length. Small Business Institute Journal, 18(2), 1–9. https://doi.org/10.53703/001c.32575

52.

Spector

P. E.

(1994). Using self-report questionnaires in OB research: A comment on the use of a controversial method. Journal of Organizational Behavior, 15(5), 385–392. https://doi.org/10.1002/job.4030150503

53.

Wales

W. J.

Gupta

V. K.

Mousa

F.-T.

(2013). Empirical research on entrepreneurial orientation: An assessment and suggestions for future research. International Small Business Journal, 31(4), 357–383. https://doi.org/10.1177/0266242611418261

54.

Walgrave

Joly

J. K.

(2018). Surveying individual political elites: A comparative three-country study. Quality and Quantity, 52(5), 2221–2237. https://doi.org/10.1007/s11135-017-0658-5

55.

White

J. V.

Borgholthaus

C. J.

(2022). Who’s in charge here? A bibliometric analysis of upper echelons research. Journal of Business Research, 139(1), 1012–1025. https://doi.org/10.1016/j.jbusres.2021.10.028 .

56.

WorldBank. (2023). World development indicators: Individuals using the internet as a percentage of population .

57.

Zolotoy

O’Sullivan

Martin

G. P.

Veeraraghavan

(2019). The role of affect in shaping the behavioral consequences of CEO option incentives. Journal of Management, 45(7), 2920–112951. https://doi.org/10.1177/0149206318771179