Archival Data in Micro-Organizational Research: A Toolkit for Moving to a Broader Set of Topics

Abstract

Compared to macro-organizational researchers, micro-organizational researchers have generally eschewed archival sources of data as a means of advancing knowledge. The goal of this paper is to discuss emerging opportunities to use archival research for the purposes of advancing and testing theory in micro-organizational research. We discuss eight specific strengths common to archival micro-organizational research and how they differ from other traditional methods. We further discuss limitations of archival research, as well as strategies for mitigating these limitations. Taken together, we provide a toolkit to encourage micro-organizational researchers to capitalize on archival data.

Keywords

Big Data archival research organizational behavior management secondary data research methods

Get full access to this article

View all access options for this article.

References

Acemoglu

Johnson

Robinson

J. A.

2001. The colonial origins of comparative development: An empirical investigation. The American Economic Review, 91: 1369-1401.

Agarwal

Xie

Vovsha

Rambow

Passonneau

2011. Sentiment analysis of Twitter data. In Proceedings of the Workshop on Languages in Social Media: 30-38. Stroudsburg, PA: Association for Computational Linguistics.

Antonakis

Bastardoz

Liu

Schriesheim

C. A.

2014. What makes articles highly cited? The Leadership Quarterly, 25: 152-179.

Antonakis

Bendahan

Jacquart

Lalive

2010. On making causal claims: A review and recommendations. The Leadership Quarterly, 21: 1086-1120.

Barnes

C. M.

Drake

C. L.

in press. Prioritizing sleep health: Public health policy recommendations. Perspectives on Psychological Science.

Barnes

C. M.

Reb

Ang

2012. More than just the mean: Moving to a dynamic view of performance-based compensation. Journal of Applied Psychology, 97: 711-718.

Barnes

C. M.

Schaubroeck

J. M.

Huth

Ghumman

2011. Lack of sleep and unethical behavior. Organizational Behavior and Human Decision Processes, 115: 169-180.

Barnes

C. M.

Wagner

D. T.

2009. Changing to daylight saving time cuts into sleep and increases workplace injuries. Journal of Applied Psychology, 94: 1305-1317.

Barnes

C. M.

Wagner

D. T.

Ghumman

2012. Borrowing from sleep to pay work and family: Expanding time-based conflict to the broader non-work domain. Personnel Psychology, 65: 789-819.

10.

Becker

W. J.

Cropanzano

2011. Dynamic aspects of voluntary turnover: An integrated approach to curvilinearity in the performance–turnover relationship. Journal of Applied Psychology, 96: 233-246.

11.

Bellizzi

J. A.

Hasty

R. W.

2002. Supervising unethical sales force behavior: Do men and women managers discipline men and women subordinates uniformly? Journal of Business Ethics, 40: 155-166.

12.

Bernerth

J. B.

Taylor

S. G.

Walker

J. H.

Whitman

D. S.

2012. An empirical investigation of dispositional antecedents and performance-related outcomes of credit scores. Journal of Applied Psychology, 97: 469-478.

13.

Bing

M. N.

Stewart

S. M.

Davison

H. K.

Green

P. D.

McIntyre

M. D.

2007. Integrative typology of personality assessment for aggression: Implications for predicting counterproductive workplace behavior. Journal of Applied Psychology, 92: 722-744.

14.

Bligh

M. C.

Kohles

J. C.

Meindl

J. R.

2004. Charisma under crisis: Presidential leadership, rhetoric, and media responses before and after the September 11th terrorist attacks. The Leadership Quarterly, 2: 211-239.

15.

Bollen

K. A.

Brand

J. E.

2010. A general panel model with random and fixed effects: A structural equations approach. Social Forces, 89: 1-34.

16.

Box

G. E. P.

Tiao

G. C.

1973. Bayesian inference in statistical analysis. Reading, MA: Addison-Wesley.

17.

Briner

R. B.

Walshe

N. D.

2013. The causes and consequences of a scientific literature we cannot trust: An evidence-based practice perspective. Industrial and Organizational Psychology, 6: 269-272.

18.

Cambria

Schuller

Xia

Havasi

2013. New avenues in opinion mining and sentiment analysis. IEEE Intelligent Systems, 28(2): 15-21.

19.

Cameron

Chaudhuri

Erkal

Gangadharan

2009. Propensities to engage in and punish corrupt behavior? Experimental evidence from Australia, India, Indonesia and Singapore. Journal of Public Economics, 93: 843-851.

20.

Chan

1998. Functional relations among constructs in the same content domain at different levels of analysis: A typology of composition models. Journal of Applied Psychology, 83: 234-246.

21.

Chatman

J. A.

Flynn

F. J.

2005. Full-cycle organizational psychology research. Organization Science, 16: 434-447.

22.

Chen

Leung

Chen

2009. Bringing national culture to the table: Making a difference through cross-cultural differences and perspectives. The Academy of Management Annals, 3: 217-249.

23.

Cohen

West

S. G.

Aiken

L. S.

2003. Applied multiple regression/correlation analysis for the behavioral sciences (3rd ed.). Mahwah, NJ: Erlbaum.

24.

Conlon

D. E.

Van Dyne

Milner

K. Y.

2004. The effects of physical and social context on evaluations of captive, intensive service relationships. Academy of Management Journal, 47: 433-445.

25.

Dai

Milkman

K. L.

Hofmann

D. A.

Staats

B. R.

2015. The impact of time at work and time off from work on rule compliance: The case of hand hygiene in health care. Journal of Applied Psychology, 100: 846-862.

26.

Detert

J. R.

Treviño

L. K.

Burris

E. R.

Andiappan

2007. Managerial modes of influence and counterproductivity in organizations: A longitudinal business-unit-level investigation. Journal of Applied Psychology, 92: 993-1005.

27.

Dierdorff

E. C.

Ellington

J. K.

2008. It’s the nature of the work: Examining behavior-based sources of work-family conflict across occupations. Journal of Applied Psychology, 93: 883-892.

28.

Dilchert

Ones

D. S.

Davis

R. D.

Rostow

J. D.

2007. Cognitive ability predicts objectively measured counterproductive work behaviors. Journal of Applied Psychology, 92: 616-627.

29.

Duriau

V. J.

Reger

R. K.

Pfarrer

M. D.

2007. A content analysis of the content analysis literature in organization studies: Research themes, data sources, and methodological refinements. Organizational Research Methods, 10: 5-34.

30.

Feldman

2013. Techniques and applications for sentiment analysis. Communications of the ACM, 56(4): 82-89.

31.

Fioretti

2013. Agent-based simulation models in organization science. Organizational Research Methods, 16: 227-242.

32.

Giebels

Taylor

P. J.

2009. Interaction patterns in crisis negotiations: Persuasive arguments and cultural differences. Journal of Applied Psychology, 94: 5-19.

33.

Gino

Shu

L. L.

Bazerman

M. H.

2010. Nameless + harmless = blameless: When seemingly irrelevant factors influence judgment of (un)ethical behavior. Organizational Behavior and Human Decision Processes, 111: 102-115.

34.

Halaby

C. N.

2004. Panel models in sociological research: Theory into practice. Annual Review of Sociology, 30: 507-544.

35.

Hamilton

B. H.

Nickerson

J. A.

2003. Correcting for endogeneity in strategic management research. Strategic Organization, 1: 51-78.

36.

Highhouse

Zickar

M. J.

Yankelevich

2010. Would you work if you won the lottery? Tracking changes in the American work ethic. Journal of Applied Psychology, 95: 349-357.

37.

Hinds

P. J.

Liu

Lyon

J. B.

2012. Putting the global in global work: An intercultural lens on the process of cross-national collaboration. The Academy of Management Annals, 5: 1-54.

38.

Hirst

Van Knippenberg

Zhou

Quintane

Zhu

2015. Heard it through the grapevine: Indirect networks and employee creativity. Journal of Applied Psychology, 100: 567-574.

39.

Hitt

M. A.

Boyd

B. K.

2004. The state of strategic management research and a vision of the future. Research Methodology in Strategic Management, 1: 1-31.

40.

Hollenbeck

J. R.

DeRue

D. S.

Mannor

M. J.

2006. Statistical power and parameter stability when subjects are few and tests are many: Comment on Peterson, Smith, Martorana, and Owens (2003). Journal of Applied Psychology, 91: 1-5.

41.

Hunter

J. E.

Schmidt

F. L.

2004. Methods of meta-analysis: Correcting error and bias in research findings (2nd ed.). Newbury Park, CA: Sage.

42.

Ilgen

D. R.

Hollenbeck

J. R.

Johnson

M. D.

Jundt

D. K.

2005. Teams in organizations: From input-process-output models to IMOI models. Annual Review of Psychology, 56: 517-543.

43.

Ioannidis

J. P.

2014. How to make more published research true. PLoS Medicine, 11(10): e1001747. http://journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.1001747

44.

Iversen

G. R.

1984. Bayesian statistical inference. Beverly Hills, CA: Sage.

45.

Johnson

R. C.

Allen

T. D.

2013. Examining the links between employed mothers’ work characteristics, physical activity, and child health. Journal of Applied Psychology, 98: 148-157.

46.

Judge

T. A.

Hurst

Simon

L. S.

2009. Does it pay to be smart, attractive, or confident (or all three)? Relationships among general mental ability, physical attractiveness, core self-evaluations, and income. Journal of Applied Psychology, 94: 742-755.

47.

Judge

T. A.

Ilies

Dimotakis

2010. Are health and happiness the product of wisdom? The relationship of general mental ability to educational and occupational attainment, health, and well-being. Journal of Applied Psychology, 95: 454-468.

48.

Judge

T. A.

Kammeyer-Mueller

J. D.

2012. On the value of aiming high: The causes and consequences of ambition. Journal of Applied Psychology, 97: 758-775.

49.

Keller

R. T.

2006. Transformational leadership, initiating structure, and substitutes for leadership: A longitudinal study of research and development project team performance. Journal of Applied Psychology, 91: 202-210.

50.

Kepes

McDaniel

M. A.

2013. How trustworthy is the scientific literature in industrial and organizational psychology? Industrial and Organizational Psychology, 6: 252-268.

51.

Kouloumpis

Wilson

Moore

2011. Twitter sentiment analysis: The good the bad and the OMG! In Proceedings of the Fifth International AAAI Conference on Weblogs and Social Media: 538-541. Palo Alto, CA: AAAI Press.

52.

Kozlowski

S. W. J.

Klein

K. J.

2000. A multilevel approach to theory and research in organizations: Contextual, temporal, and emergent processes. In Klein

K. J.

Kozlowski

S. W. J.

(Eds.), Multilevel theory, research and methods in organizations: Foundations, extensions, and new directions: 3-90. San Francisco: Jossey-Bass.

53.

Kuncel

N. R.

Klieger

D. M.

2007. Application patterns when applicants know the odds: Implications for selection research and practice. Journal of Applied Psychology, 92: 586-593.

54.

Larrick

R. P.

Timmerman

Carton

Abrevaya

2011. Temper, temperature, and temptation: Heat-related retaliation in baseball. Psychological Science, 22: 423-428.

55.

Leavitt

Mitchell

Peterson

2010. Theory pruning: Strategies for reducing our dense theoretical landscape. Organizational Research Methods, 13: 644-667.

56.

LeBreton

J. M.

Senter

J. L.

2008. Answers to twenty questions about interrater reliability and interrater agreement. Organizational Research Methods, 11: 815-852.

57.

Lee

J. J.

Gino

Staats

B. R.

2014. Rainmakers: Why bad weather means good productivity. Journal of Applied Psychology, 99: 504-513.

58.

Lyness

K. S.

Heilman

M. E.

2006. When fit is fundamental: Performance evaluations and promotions of upper-level female and male managers. Journal of Applied Psychology, 91: 777-785.

59.

Martell

R. F.

Lane

D. M.

Emrich

1996. Male-female differences: A computer simulation. American Psychologist, 51: 157-158.

60.

Minbashian

Luppino

2014. Short-term and long-term within-person variability in performance: An integrative model. Journal of Applied Psychology, 99: 898-914.

61.

Mirsky

2014. Workplace injuries may rise right after daylight saving time. Scientific American, March 9. http://www.scientificamerican.com/podcast/episode/time-change-injuries/. Accessed August 17, 2015.

62.

Mitchell

T. R.

James

L. R.

2001. Building better theory: Time and the specification of when things happen. Academy of Management Review, 26: 530-547.

63.

Murphy

K. R.

Myors

1998. Statistical power analysis: A simple and general model for traditional and modern hypothesis tests. Mahwah, NJ: Erlbaum.

64.

Newman

D. A.

Jacobs

R. R.

Bartram

2007. Choosing the best method for local validity estimation: Relative accuracy of meta-analysis versus a local study versus Bayes-analysis. Journal of Applied Psychology, 92: 1394-1413.

65.

Newman

D. A.

Sin

2009. How do missing data bias estimates of within-group agreement? Sensitivity of SDWG, CVWG, rWG(J), rWG(J)*, and ICC to systematic nonresponse. Organizational Research Methods, 12: 113-147.

66.

K. Y.

Van Dyne

2001. Individualism-collectivism as a boundary condition for effectiveness of minority influence in decision making. Organizational Behavior and Human Decision Processes, 84: 198-225.

67.

Nosek

B. A.

Spies

J. R.

Motyl

2012. Scientific utopia: II. Restructuring incentives and practices to promote truth over publishability. Perspectives on Psychological Science, 7: 615-631.

68.

Nunnally

J. C.

Bernstein

1994. Psychometric theory. New York: McGraw Hill.

69.

Pang

Lee

2008. Opinion mining and sentiment analysis. Foundations and Trends in Information Retrieval, 2(1-2): 1-135.

70.

Popper

1959. The logic of scientific discovery. London: Routledge.

71.

Pritchard

Jones

Roth

Stuebing

Ekeberg

1988. Effects of group feedback, goal setting, and incentives on organizational productivity. Journal of Applied Psychology, 73: 337-358.

72.

Probst

T. M.

Brubaker

T. L.

Barsotti

2008. Organizational injury rate underreporting: The moderating effect of organizational safety climate. Journal of Applied Psychology, 93: 1147-1154.

73.

Roberts

B. W.

Caspi

Moffitt

T. E.

2003. Work experiences and personality development in young adulthood. Journal of Personality and Social Psychology, 84: 582-593.

74.

Rosenthal

Rosnow

R. L.

1991. Essentials of behavioral research: Methods and data analysis. Boston: McGraw Hill.

75.

Sakaluk

J. K.

Williams

A. J.

Biernat

2014. Analytic review as a solution to the misreporting of statistical results in psychological science. Perspectives on Psychological Science, 9: 652-660.

76.

Scandura

T. A.

Williams

E. A.

2000. Research methodology in organizational studies: Current practices and implications for future research. Academy of Management Journal, 43: 1248-1264.

77.

Schwab

D. P.

1980. Construct validity in organizational behavior. Research in Organizational Behavior, 2: 3-43.

78.

Shadish

W. R.

Cook

T. D.

Campbell

D. T.

2001. Experimental and quasi-experimental designs for generalized causal inference. New York: Wadsworth.

79.

Shen

Kiger

T. B.

Davies

S. E.

Rasch

R. L.

Simon

K. M.

Ones

2011. Samples in applied psychology: Over a decade of research in review. Journal of Applied Psychology, 96: 1055-1064.

80.

Silberzahn

Simonsohn

Uhlmann

E. L.

2014. Matched names analysis reveals no evidence of name meaning effects: A collaborative commentary on Silberzahn and Uhlmann (2013). Psychological Science, 25: 1504-1505.

81.

Silberzahn

Uhlmann

E. L.

2013. It pays to be Herr Kaiser: Germans with noble-sounding surnames more often work as managers. Psychological Science, 24: 2437-2444.

82.

Simmons

Nelson

Simonsohn

2011. False-positive psychology: Undisclosed flexibility in data collection and analysis allow presenting anything as significant. Psychological Science, 22: 1359-1366.

83.

Simonsohn

2013. Just post it: The lesson from two cases of fabricated data detected by statistics alone. Psychological Science, 24: 1875-1888.

84.

Sims

C. S.

Drasgow

Fitzgerald

L. F.

2005. The effects of sexual harassment on turnover in the military: Time-dependent modeling. Journal of Applied Psychology, 90: 1141-1152.

85.

SINTEF. 2013. Big Data, for better or worse: 90% of world’s data generated over last two years. ScienceDaily, May 22. http://www.sciencedaily.com/releases/2013/05/130522085217.htm. Accessed August 13, 2014.

86.

Smith-Jentsch

K. A.

Mathieu

J. E.

Kraiger

2005. Investigating linear and interactive effects of shared mental models on safety and efficiency in a field setting. Journal of Applied Psychology, 90: 523-535.

87.

Spector

P. E.

Allen

T. D.

Poelmans

S. A. Y.

Lapierre

L. M.

Cooper

C. L.

O’Driscoll

Sanchez

J. I.

Abarca

Alexandrova

Beham

Brough

Ferreiro

Fraile

C. Q.

Moreno-Veláquez

Pagon

Pitariu

Salamatov

Shima

Suarez Simoni

Ling Siu

Widerszal-Bazyl

2007. Cross-national differences in relationships of work demands, job satisfaction and turnover intentions with work-family conflict. Personnel Psychology, 60: 805-835.

88.

Spector

P. E.

Cooper

C. L.

Poelmans

Allen

T. D.

O’Driscoll

Sanchez

J. I.

Siu

O. L.

Dewe

Hart

de Moraes

L. F. R.

Ostrognay

G. M.

Sparks

Wong

2004. A cross-national comparative study of work/family stressors, working hours, and well-being: China and Latin America vs. the Anglo world. Personnel Psychology, 57: 119-142.

89.

Spector

P. E.

Cooper

C. L.

Sanchez

J. I.

O’Driscoll

Sparks

Bernin

Büssing

Dewe

Hart

Miller

Renault

Moraes

Ostrognay

G. M.

Pagon

Pitariu

Poelmans

Radhakrishnan

Russinova

Salamatov

Salgado

Shima

Siu

O. L.

Stora

J. B.

Teichmann

Theorell

Vlerick

Westman

Widerszal-Bazyl

Wong

2002. A 24 nation/territory study of work locus of control in relation to well-being at work: How generalizable are western findings? Academy of Management Journal, 45: 453-466.

90.

Timmerman

T. A.

2007. It was a thought pitch: Personal, situational, and target influences on hit-by-pitch events across time. Journal of Applied Psychology, 92: 876-884.

91.

van Mierlo

Vermunt

J. K.

Rutte

C. G.

2009. Composing group-level constructs from individual-level survey data. Organizational Research Methods, 12: 368-392.

92.

Varnum

M. E.

Kitayama

2010. What’s in a name? Popular names are less common on frontiers. Psychological Science, 22: 176-183.

93.

Wagner

D. T.

Barnes

C. M.

Lim

V. K. G.

Ferris

D. L.

2012. Lost sleep and cyberloafing: Evidence from the laboratory and a daylight saving time quasi-experiment. Journal of Applied Psychology, 97: 1068-1076.

94.

Wang

Zhan

Liu

Shultz

K. S.

2008. Antecedents of bridge employment: A longitudinal investigation. Journal of Applied Psychology, 93: 818-830.

95.

Weller

Holtom

B. C.

Matiaske

Mellewigt

2009. Level and time effects of recruitment sources on early voluntary turnover. Journal of Applied Psychology, 94: 1146-1162.

96.

Wimbush

J. C.

Dalton

D. R.

1997. Base rate for employee theft: Convergence of multiple methods. Journal of Applied Psychology, 82: 756-763.