This article argues that social and personality psychology theory and findings have been unnecessarily hampered by our reliance on a set of methodological lenses that sacrifice external validity and generalizability. We further urge researchers to renew their use of both classic and evolving alternative methodologies. In particular, we discuss and consider pros and cons of field experiments, big data, archival research, and observations. We propose that it is up to us, as researchers, to ensure that we maximize the breadth of our theory and quality of data by expanding and triangulating the ways in which we collect data.
AdangO. M. (2016). A method for direct systematic observation of collective violence and public order policing. Sociological Methods & Research. doi:10.1177/0049124116661578
2.
AinsworthM. D. S.BleharM. C.WatersE.WallS. (1978). Patterns of attachment: A psychological study of the strange situation. Hillsdale, NJ: Erlbaum.
3.
American Psychological Association. (2004). Demonstrating the power of social situations via a simulated prison experiment. Psychological science: Research in action. Retrieved fromhttp://www.apa.org/research/action/prison.aspx
4.
ArnettJ. J. (2008). The neglected 95%: Why American psychology needs to become less American. American Psychologist, 63, 602.
5.
BarsadeS. G.RamarajanL.WestenD. (2009). Implicit affect in organizations. Research in Organizational Behavior, 29, 135–162.
6.
BonatoP. (2003). Wearable sensors/systems and their impact on biomedical engineering. IEEE Engineering in Medicine and Biology Magazine, 22, 18–20.
7.
ChaffinD.HeidlR.HollenbeckJ. R.HoweM.YuA.VoorheesC.CalantoneR. (2017). The promise and perils of wearable sensors in organizational research. Organizational Research Methods, 20, 3–31. doi: 10.1177/1094428115617004
8.
CrosnoeR.ElderG. H.Jr. (2002). Successful adaptation in the later years: A life course approach to aging. Social Psychology Quarterly, 65, 309–328.
DruckmanJ. N.KamC. D. (2009). Students as experimental participants: A defense of the ‘narrow data base.’doi:10.2139/ssrn.1498843
11.
DufauS.DuñabeitiaJ. A.Moret-TatayC.McGonigalA.PeetersD.AlarioF. X.…KtoriM. (2011). Smart phone, smart science: How the use of smartphones can revolutionize research in cognitive science. PloS One, 6, e24974.
12.
EkmanP. (2009). Telling lies: Clues to deceit in the marketplace, politics, and marriage (Rev. ed.). New York, NY: W.W. Norton & Company.
13.
FestingerL.RieckenH. W.SchachterS. (1956). When prophecy fails: A social and psychological study of a modern group that predicted the destruction of the world. Minneapolis: University of Minnesota Press.
14.
FowlesD. C. (1980). The three arousal model: Implications of Gray’s two-factor learning theory for heart rate, electrodermal activity, and psychopathy. Psychophysiology, 17, 87–104.
15.
GarciaD.SikströmS. (2014). The dark side of Facebook: Semantic representations of status updates predict the Dark Triad of personality. Personality and Individual Differences, 67, 92–96.
GhaziasgarM.De La CruzN.BagulaA. B.ConnanJ. (2016). Visual data mining: A great opportunity for criminal investigation. In IsafiadeO. E.BagulaA. B. (Eds.), Data mining trends and applications in criminal science and investigations (pp. 112–141). Hershey, PA: IGI Global.
GottmanJ. M.LevensonR. W. (1992). Marital processes predictive of later dissolution: Behavior, physiology, and health. Journal of Personality and Social Psychology, 63, 221.
20.
HaneyC.BanksW. C.ZimbardoP. G. (1973). A study of prisoners and guards in a simulated prison. Naval Research Reviews, 9, 1–17.
21.
HeblM. R.FosterJ. B.MannixL. M.DovidioJ. F. (2002). Formal and interpersonal discrimination: A field study of bias toward homosexual applicants. Personality and Social Psychology Bulletin, 28, 815–825.
22.
HeblM. R.KingE. B.GlickP.SingletaryS. L.KazamaS. (2007). Hostile and benevolent reactions toward pregnant women: Complementary interpersonal punishments and rewards that maintain traditional roles. Journal of Applied Psychology, 92, 1499.
23.
HenrichJ.HeineS. J.NorenzayanA. (2010). The weirdest people in the world?Behavioral and Brain Sciences, 33, 61–83.
24.
HernandezI.NewmanD. A.JeonG. (2015). Twitter analysis: Methods for data management and a word count dictionary to measure city-level job satisfaction. In TonidandelS.KingE.CortinaJ. (Eds.), Big data at work: The data science revolution and organizational psychology (pp. 64–114). New York, NY: Routledge.
25.
HongL. K.DuffR. W. (2002). Modulated participant-observation: Managing the dilemma of distance in field research. Field Methods, 14, 190–196.
26.
KimT.ChangA.HollandL.PentlandA. S. (2008, 10). Meeting mediator: Enhancing group collaboration using sociometric feedback. In Proceedings of the 2008 ACM conference on computer supported cooperative work (pp. 457–466). Chicago, IL: ACM.
27.
KimT. J.ChuM.BrdiczkaO.BegoleJ. (2009, 4). Predicting shoppers’ interest from social interactions using sociometric sensors. In Extended abstracts of the 2009 ACM conference on computer supported cooperative work (pp. 4513–4518). Chicago, IL: ACM.
28.
KingE. B.AhmadA. S. (2010). An experimental field study of interpersonal discrimination toward Muslim job applicants. Personnel Psychology, 63, 881–906.
29.
KingE. B.DawsonJ. F.WestM. A.GilraneV. L.PeddieC. I.BastinL. (2011). Why organizational and community diversity matter: Representativeness and the emergence of incivility and organizational performance. Academy of Management Journal, 54, 1103–1118.
30.
KingE. B.HeblM. R.Botsford MorganW.AhmadA. (2013). Experimental field research on sensitive organizational topics. Organizational Research Methods, 16, 501–521.
31.
KingE. B.ShapiroJ. R.HeblM. R.SingletaryS. L.TurnerS. (2006). The stigma of obesity in customer service: A mechanism for remediation and bottom-line consequences of interpersonal discrimination. Journal of Applied Psychology, 91, 579.
32.
KirschbaumC.PirkeK. M.HellhammerD. H. (1993). The ‘Trier Social Stress Test’—A tool for investigating psychobiological stress responses in a laboratory setting. Neuropsychobiology, 28, 76–81.
33.
LachmanS. J. (1965). History and methods of physiological psychology: A brief overview. Detroit, MI: Hamilton Press.
34.
LevensonR. W.GottmanJ. M. (1983). Marital interaction: Physiological linkage and affective exchange. Journal of Personality and Social Psychology, 45, 587.
35.
LewinK. (1939). Field theory and experiment in social psychology: Concepts and methods. American Journal of Sociology, 44, 868–896.
36.
ManerJ. K. (2016). Into the wild: Field research can increase both replicability and real-world impact. Journal of Experimental Social Psychology, 66, 100–106.
37.
ManovichL. (2011). Trending: The promises and the challenges of big social data. Debates in the Digital Humanities, 2, 460–475.
38.
MartinL. R.FriedmanH. S.SchwartzJ. E. (2007). Personality and mortality risk across the life span: The importance of conscientiousness as a biopsychosocial attribute. Health Psychology, 26, 428.
39.
MaxwellS. E. (2004). The persistence of underpowered studies in psychological research: Causes, consequences, and remedies. Psychological Methods, 9, 147.
40.
MulderL. J. M. (1992). Measurement and analysis methods of heart rate and respiration for use in applied environments. Biological Psychology, 34, 205–236.
41.
Olguín OlguínD. (2007). Sociometric badges: Wearable technology for measuring human behavior (Doctoral dissertation). Massachusetts Institute of Technology, Cambridge.
42.
OwayjanM.KashourA.Al HaddadN.FadelM.Al SoukiG. (2012,12). The design and development of a lie detection system using facial micro-expressions. In 2012 2nd international conference on advances in computational tools for engineering applications (ACTEA) (pp. 33–38). IEEE.
43.
PaluckE. L. (2011). Peer pressure against prejudice: A high school field experiment examining social network change. Journal of Experimental Social Psychology, 47, 350–358.
44.
PaluckE. L. (2016). How to overcome prejudice. Science, 352, 147–147.
45.
PaluckE. L.CialdiniR. B. (2014). Field research methods. In ReisH. T.JuddC. M. (Eds.), Handbook of Research Methods in Social and Personality Psychology (pp. 81–98). Cambridge, U.K.: Cambridge University Press.
46.
PaluckE. L.ShepherdH. (2012). The salience of social referents: A field experiment on collective norms and harassment behavior in a school social network. Journal of Personality and Social Psychology, 103, 899.
47.
PaluckE. L.ShepherdH.AronowP. M. (2016). Changing climates of conflict: A social network experiment in 56 schools. Proceedings of the National Academy of Sciences, 113, 566–571.
48.
ParkS.JayaramanS. (2003). Enhancing the quality of life through wearable technology. IEEE Engineering in Medicine and Biology Magazine, 22, 41–48.
49.
PashlerH.WagenmakersE. J. (2012). Editors’ introduction to the special section on replicability in psychological science a crisis of confidence?Perspectives on Psychological Science, 7, 528–530.
50.
PatelS.ParkH.BonatoP.ChanL.RodgersM. (2012). A review of wearable sensors and systems with application in rehabilitation. Journal of Neuroengineering and Rehabilitation, 9, 1.
51.
PetersonR. A. (2001). On the use of college students in social science research: Insights from a second-order meta-analysis. Journal of Consumer Research, 28, 450–461.
52.
RunkelP. J.McGrathJ. E. (1972). Research on human behavior: A systematic guide to method. New York, NY: Holt, Rinehart and Winston.
53.
SearsD. O. (1986). College sophomores in the laboratory: Influences of a narrow data base on social psychology’s view of human nature. Journal of Personality and Social Psychology, 51, 515–530.
54.
ShreveM.GodavarthyS.GoldgofD.SarkarS. (2011, 3). Macro-and micro-expression spotting in long videos using spatio-temporal strain. In 2011 IEEE international conference on automatic face & gesture recognition and workshops (FG 2011) (pp. 51–56). IEEE.
55.
ShultzK. S.HoffmanC. C.Reiter-PalmonR. (2005). Using archival data for I-O research: Advantages, pitfalls, sources, and examples. The Industrial-Organizational Psychology, 42, 31–37.
56.
TonidandelS.KingE.CortinaJ. (2015). Big data at work: The data science revolution and organizational psychology. New York, NY: Routledge.
57.
TuckerJ. S.FriedmanH. S.SchwartzJ. E.CriquiM. H.Tomlinson-KeaseyC.WingardD. L.MartinL. R. (1997). Parental divorce: Effects on individual behavior and longevity. Journal of Personality and Social Psychology, 73, 381.
58.
VallurupalliS.PaydakH.AgarwalS. K.AgrawalM.Assad-KottnerC. (2013). Wearable technology to improve education and patient outcomes in a cardiology fellowship program-a feasibility study. Health and Technology, 3, 267–270.
59.
ZhouH.FishbachA. (2016). The pitfall of experimenting on the web: How unattended selective attrition leads to surprising (yet false) research conclusions. Journal of Personality and Social Psychology, 111, 493–504.
60.
ZimbardoP. G. (1973). On the ethics of intervention in human psychological research: With special reference to the Stanford prison experiment. Cognition, 2, 243–256.
