Sociologists increasingly face choices among competing algorithms that represent reasonable approaches to the same task, with little guidance in choosing among them. We develop a strategy that uses simulated data to identify the conditions under which different methods perform well and applies what is learned from the simulations to predict which method will perform best on never-before-seen empirical data sets. We apply this strategy to a class of methods that group respondents to attitude surveys according to whether they share construals of a given domain. This allows us to identify the relative strengths and weaknesses of the methods we consider, including relational class analysis, correlational class analysis, and eight other such variants. Results support the “no free lunch” view that researchers should abandon the quest for one best algorithm in favor of matching algorithms to kinds of data for which each is most appropriate and provide direction on how to do so.
AnderssonMatthew A.HarknessSarah K.. . 2017. “When Do Biological Attributions of mental Illness Reduce Stigma? Using Qualitative Comparative Analysis to Contextualize Attributions.” Society and Mental Health8:175–94.
2.
BaldassarriDeliaGoldbergAmir. . 2014. “Neither Ideologues Nor Agnostics: Alternative Voters’ Belief System in an Age of Partisan Politics.” American Journal of Sociology120(1):45–95.
3.
BendittLauren. 2015. “Race, Gender, and Public-Sector Work: Prioritizing Occupational Values as a Labor Market Privilege.” Research in Social Stratification and Mobility42:73–86.
4.
BlondelVincent D.GuillaumeJean-LoupLambiotteRenaudLefebvreEtienne. 2011. “Fast Unfolding of Communities in Large Networks.” Journal of Statistical Mechanics: Theory and Experiment2008(10):P10008.
5.
BoutylineAndrei. 2017. “Improving the Measurement of Shared Cultural Schemas with Correlational Class Analysis: Theory and Method.” Sociological Science4(15):353–93.
BreigerRonald L.1974. “The Duality of Persons and Groups.” Social Forces53(2):181–90.
8.
CalińskiTadeuszHarabaszJerzy. 1974. “A Dendrite Method for Cluster Analysis.” Communications in Statistics-Theory and Methods3(1):1–27.
9.
DaenekindtStijn. 2017. “On the Structure of Dispositions: Transposability of and Oppositions Between Aesthetic Dispositions.” Poetics63:43–52.
10.
DaenekindtStijnde KosterWillemvan der WaalJeroen. 2017. “How People Organise Cultural Attitudes: Cultural Belief Systems and the Populist Radical Right.” West European Politics40(4):791–811. DOI: 10.1080/01402382.2016.1271970.
11.
DavisJ. T. (2019). “Funding God’s Policies, Defending Whiteness: Christian Nationalism and Whites’ Attitudes Towards Racially-Coded Government Spending.” Ethnic and Racial Studies42(12):2123–42.
12.
DiMaggioPaulEvansJohnBrysonBethany. 1996. “Have Americans’ Social Attitudes Become More Polarized.” American Journal of Sociology102(3):690–755.
13.
DiMaggioPaulGoldbergAmir. . 2018. “Searching for Homo Economicus: Variation in Americans’ Construals of and Attitudes toward Markets.” European Journal of Sociology/Archives Européennes de Sociologie59:151–89.
14.
DiMaggioPaulSotoudehRaminaGoldbergAmirShepherdHana. 2018. “Culture Out of attitudes: Relationality, Population Heterogeneity and Attitudes Toward Science and Religion in the US.” Poetics68:31–51.
15.
FanMingyuGuNannanQiaoHeongZhangBo. 2010. “Intrinsic Dimension Estimation of Data by Principal Component Analysis.”arXiv preprint arXiv:1002.2050.
16.
GoldbergAmir. 2011. “Mapping Shared Understandings Using Relational Class Analysis: The Case of the Cultural Omnivore Reexamined.” American Journal of Sociology116(5):1397–436.
17.
GoldbergAmirSteinSarah. 2016. “RCA: Relational Class Analysis.” R package, version 2.0. https://rdrr.io/cran/RCA/.
18.
GoodfellowIanBengioYoshuaCourvilleAaron. 2016. Deep Learning. Cambridge: MIT Press.
19.
GrimmerJustinKingGary. 2010. “General Purpose Computer-Assisted Clustering and Conceptualization.” Proceedings of the National Academy of Sciences108(7):2643–50.
20.
GrimmerJustinStewartBrandon M.. 2013. “Text as data: The Promise and Pitfalls of Automatic Content Analysis Methods for Political Texts.” Political Analysis21(3):267–97.
LangJ. B.AgrestiA.. . 1994. “Simultaneously Modeling Joint and Marginal Distributions of Multivariate Categorical Responses.” Journal of the American Statistical Association89(426):625–32.
23.
LewisGregory B.FrankSue A.. 2002. “Who Wants to Work for the Government?” Public Administration Review62(4):395–404.
24.
LynnFreda B.EllerbachGeorge. 2017. “A Position With a View: Educational Status and the Construction of the Occupational Hierarchy.” American Sociological Review82:32–58.
25.
MartinJohn LeviWileyJames A.. . 2000. “Algebraic Representations of Beliefs and Attitudes II: Microbelief Models for Dichotomous Belief Data.” Sociological Methodology30(1):123–64.
ReifMatthiasShafaitFaisalDengelAndreas. 2012. “Meta-Learning for Evolutionary Parameter Optimization of Classifiers.” Machine Learning87:357–80. https://doi.org/10.1007/s10994-012-5286-7
28.
RookDane. 2014. “Relational Distance in Doxastic Finance: Local Risk, Revisited.” (April 8, 2014). Available at SSRN: https://ssrn.com/abstract=2422276or https://dx.doi.org/10.2139/ssrn.2422276
29.
TumerKagainGhoshJoydeep. 1996. “Error Correlation and Error Reduction in Ensemble Classifiers.” Connection Science8(3-4):385–404.
30.
VanschorenJoaquin. 2019. “Meta-learning.” Pp. 35–61 in Automated Machine Learning, edited by HutterF.VanschorenJ.KotthoffL.. New York: Springer International Publishing.
31.
WangChongBleiDavid M.. 2011. “Collaborative Topic Modeling for Recommending Scientific Articles.” Pp. 448–56 in Proceedings of the 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. https://doi.org/10.1145/2020408.2020480
32.
WileyJames A.MartinJohn L.. 1999. “Algebraic Representations of Beliefs and Attitudes: Partial Order Models for item Responses.” Sociological Methodology29(1):113–46.
33.
WuAngela Xiao. 2014. “Ideological Polarization Over a China-as-Superpower Mind-set: An Exploratory Charting of Belief Systems Among Chinese Internet Users, 2008-2011.” International Journal of Communication8:2650–79.
34.
WuCaryWilkesRima. 2018. “Finding Critical Trustrs: A Response Pattern Model of Political Trust.” International Journal of Comparative Sociology59:110–38.
35.
YangZ.AlgesheimerR.TessoneC. J.. 2016. “A Comparative Analysis of Community Detection Algorithms on Artificial Networks.” Scientific Reports6(1):1–18.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
