We examine how six search engines filter and rank information in relation to the queries on the U.S. 2020 presidential primary elections under the default—that is nonpersonalized—conditions. For that, we utilize an algorithmic auditing methodology that uses virtual agents to conduct large-scale analysis of algorithmic information curation in a controlled environment. Specifically, we look at the text search results for “us elections,” “donald trump,” “joe biden,” “bernie sanders” queries on Google, Baidu, Bing, DuckDuckGo, Yahoo, and Yandex, during the 2020 primaries. Our findings indicate substantial differences in the search results between search engines and multiple discrepancies within the results generated for different agents using the same search engine. It highlights that whether users see certain information is decided by chance due to the inherent randomization of search results. We also find that some search engines prioritize different categories of information sources with respect to specific candidates. These observations demonstrate that algorithmic curation of political information can create information inequalities between the search engine users even under nonpersonalized conditions. Such inequalities are particularly troubling considering that search results are highly trusted by the public and can shift the opinions of undecided voters as demonstrated by previous research.
BatesD.MaechlerM.BolkerB.WalkerS.ChristensenR. H. B.SingmannH.DaiB.ScheiplF.GrothendieckG.GreenP.FoxJ. (2020). Lme4: Linear mixed-effects models using “eigen” and S4 (1.1-23) [Computer software]. https://cran.r-project.org/package=lme4
3.
CardosoB.MagalhãesJ. (2011). Google, Bing and a new perspective on ranking similarity. In Proceedings of the 20th ACM International Conference on information and knowledge management (pp. 1933–1936). https://doi.org/10.1145/2063576.2063858
4.
CourtoisC.SlechtenL.CoenenL. (2018). Challenging Google search filter bubbles in social and political information: Disconforming evidence from a digital methods case study. Telematics and Informatics, 35(7), 2006–2015. https://doi.org/10.1016/j.tele.2018.07.004
5.
DiakopoulosN.TrielliD.StarkJ.MussendenS. (2018). I vote for—How search informs our choice of candidate. In TambiniD.MooreM. (Eds.), Digital dominance: The power of Google, Amazon, Facebook, and Apple (p. 22). Oxford University Press.
6.
DuttonW. H.ReisdorfB. C.DuboisE.BlankG. (2017). Search and politics: The uses and impacts of search in Britain, France, Germany, Italy, Poland, Spain, and the United States [Quello Center Working Paper No. 2944191]. https://ora.ox.ac.uk/objects/uuid:2cec8e9b-cce1-4339-9916-84715a62066c
EpsteinR.RobertsonR. E. (2015). The search engine manipulation effect (SEME) and its possible impact on the outcomes of elections. Proceedings of the National Academy of Sciences, 112(33), E4512–E4521. https://doi.org/10.1073/pnas.1419828112
9.
FeuzM.FullerM.StalderF. (2011). Personal web searching in the age of semantic capitalism: Diagnosing the mechanisms of personalisation. First Monday. https://doi.org/10.5210/fm.v16i2.3344
HaimM.ArendtF.ScherrS. (2017). Abyss or shelter? On the relevance of web search engines’ search results when people Google for suicide. Health Communication, 32(2), 253–258. https://doi.org/10.1080/10410236.2015.1113484
HannakA.SapiezynskiP.Molavi KakhkiA.KrishnamurthyB.LazerD.MisloveA.WilsonC. (2013). Measuring personalization of web search. Proceedings of the 22nd International Conference on World Wide Web, 527–538. https://doi.org/10.1145/2488388.2488435
14.
HelbergerN.KarppinenK.D’AcuntoL. (2018). Exposure diversity as a design principle for recommender systems. Information, Communication & Society, 21(2), 191–207. https://doi.org/10.1080/1369118X.2016.1271900
15.
HinmanL. M. (2008). Searching ethics: The role of search engines in the construction and distribution of knowledge. In SpinkA.ZimmerM. (Eds.), Web search: Multidisciplinary perspectives (pp. 67–76). Springer. https://doi.org/10.1007/978-3-540-75829-7_5
16.
KayM.MatuszekC.MunsonS. A. (2015). Unequal representation and gender stereotypes in image search results for occupations. In Proceedings of the 33rd Annual ACM Conference on human factors in computing systems (pp. 3819–3828). https://doi.org/10.1145/2702123.2702520
17.
Kliman-SilverC.HannakA.LazerD.WilsonC.MisloveA. (2015). Location, location, location: The impact of geolocation on web search personalization. In Proceedings of the 2015 internet measurement conference (pp. 121–127). https://doi.org/10.1145/2815675.2815714
18.
KrollJ.HueyJ.BarocasS.FeltenE.ReidenbergJ.RobinsonD.YuH. (2017). Accountable algorithms. University of Pennsylvania Law Review, 165(3), 633.
19.
KulshresthaJ.EslamiM.MessiasJ.ZafarM. B.GhoshS.GummadiK. P.KarahaliosK. (2019). Search bias quantification: Investigating political bias in social media and web search. Information Retrieval Journal, 22(1), 188–227. https://doi.org/10.1007/s10791-018-9341-2
20.
KuznetsovaA.BrockhoffP. B.ChristensenR. H. B.JensenS. P. (2020). Lmertest: Tests in linear mixed effects models (3.1-2) [Computer software]. https://cran.r-project.org/package=lmertest
21.
LaidlawE. B. (2010). A framework for identifying Internet information gatekeepers. International Review of Law, Computers & Technology, 24(3), 263–276. https://doi.org/10.1080/13600869.2010.522334
22.
MakhortykhM.UrmanA.UlloaR. (2020). How search engines disseminate information about COVID-19 and why they should do better. Harvard Kennedy School Misinformation Review, 1 (COVID-19 and misinformation). https://doi.org/10.37016/mr-2020-017
23.
MittelstadtB. (2016). Automation, algorithms, and politics\textbar auditing for transparency in content personalization systems. International Journal of Communication, 10(0), 12.
24.
NobleS. U. (2018). Algorithms of oppression: How search engines reinforce racism. New York University Press.
25.
PanB.HembrookeH.JoachimsT.LorigoL.GayG.GrankaL. (2007). In Google we trust: Users’ decisions on rank, position, and relevance. Journal of Computer-Mediated Communication, 12(3), 801–823. https://doi.org/10.1111/j.1083-6101.2007.00351.x
RobertsonR. E.JiangS.JosephK.FriedlandL.LazerD.WilsonC. (2018). Auditing partisan audience bias within Google search. Proceedings of the ACM on Human-Computer Interaction, 2(CSCW), 1–148. https://doi.org/10.1145/3274417
29.
RobertsonR. E.LazerD.WilsonC. (2018). Auditing the personalization and composition of politically-related search engine results pages. In Proceedings of the 2018 World Wide Web conference (pp. 955–965). https://doi.org/10.1145/3178876.3186143
30.
SandvigC.HamiltonK.KarahaliosK.LangbortC. (2014). Auditing algorithms: Research methods for detecting discrimination on internet platforms. Data and Discrimination: Converting Critical Concerns into Productive Inquiry, 22, 4349–4357.
31.
SchultheißS.SünklerS.LewandowskiD. (2018). We still trust in Google, but less than 10 years ago: An eye-tracking study. Information research. An International Electronic Journal, 23(3). https://eric.ed.gov/?id=EJ1196314
32.
SchulzW.HeldT.LaudienA. (2005). Search engines as gatekeepers of public communication: Analysis of the German framework applicable to internet search engines including media law and anti trust law. German Law Journal, 6(10), 1419–1431. https://doi.org/10.1017/S2071832200014401
33.
SEMrush. (n.d.). SEMrush Sensor – Google’s rank and algorithm tracking tool. Semrush. Retrieved March 26, 2021, fromhttps://www.semrush.com/sensor/
34.
SinghV. K.ChaykoM.InamdarR.FloegelD. (2020). Female librarians and male computer programmers? Gender bias in occupational images on digital media platforms. Journal of the Association for Information Science and Technology, asi.24335. https://doi.org/10.1002/asi.24335
SteinerM.MaginM.StarkB.GeißS. (2020). Seek and you shall find? A content analysis on the diversity of five search engines’ results on political queries. Information, Communication & Society, 0(0), 1–25. https://doi.org/10.1080/1369118X.2020.1776367
37.
SweeneyM. (2013). Not just a pretty (inter)face: A critical analysis of Microsoft’s “Ms. Dewey”[University of Illinois at Urbana-Champaign]. http://hdl.handle.net/2142/46617
38.
TrevisanF.HoskinsA.OatesS.MahloulyD. (2018). The Google voter: Search engines and elections in the new media ecology. Information, Communication & Society, 21(1), 111–128. https://doi.org/10.1080/1369118X.2016.1261171
39.
TrielliD.DiakopoulosN. (2019). Search as news curator: The role of Google in shaping attention to news information. In Proceedings of the 2019 CHI Conference on human factors in computing systems (pp. 1–15). https://doi.org/10.1145/3290605.3300683
40.
UnkelJ.HaimM. (2019). Googling politics: Parties, sources, and issue ownerships on Google in the 2017 German federal election campaign. Social Science Computer Review. https://doi.org/10.1177/0894439319881634
41.
Van AelstP.StrömbäckJ.AalbergT.EsserF.de VreeseC.MatthesJ.HopmannD.SalgadoS.HubéN.StępińskaA.PapathanassopoulosS.BerganzaR.LegnanteG.ReinemannC.SheaferT.StanyerJ. (2017). Political communication in a high-choice media environment: A challenge for democracy?Annals of the International Communication Association, 41(1), 3–27. https://doi.org/10.1080/23808985.2017.1288551
WebberW.MoffatA.ZobelJ. (2010). A similarity measure for indefinite rankings. ACM Transactions on Information Systems, 28(4), 1–38. https://doi.org/10.1145/1852102.1852106
44.
WhiteR. W.HorvitzE. (2015). Belief dynamics and biases in web search. ACM Transactions on Information Systems, 33(4), 18:1–18:46. https://doi.org/10.1145/2746229
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
