When AI Gets It Wrong: A Black Female Faculty Member Confronts Bias Within a Higher Education Institution

Abstract

This case examines Dr. Jacqueline Rivers, a Black woman faculty member at Midfield University, who encountered racially biased, aggressive, and unfriendly outputs from a generative artificial intelligence (AI) tool, LexiAI, when she was using it for lesson planning and research. The narrative explores the intersections of race, technology, and higher education leadership, prompting reflections on digital equity, faculty well-being, and institutional responsibility. Amid the 2025 federal rollback of Diversity, Equity, and Inclusion (DEI) initiatives under the Trump administration, this case challenges educational leaders to address AI biases that may harm underrepresented faculty and exacerbate existing inequities. The discussion invites considerations of leadership ethics, AI governance, and culturally responsive practices.

Keywords

generative AI antiracism social justice AI regulations

Get full access to this article

View all access options for this article.

References

Briscoe

McIntosh

(2022). A fractured school community: Decolonizing leaders needed for responsive leadership. Journal of Cases in Educational Leadership, 26(1), 16–30. https://doi.org/10.1177/15554589221128051

Buolamwini

Gebru

(2018). Gender shades: Intersectional accuracy disparities in commercial gender classification. Proceedings of Machine Learning Research, 81, 1–15.

Ibadullaeva

(2025). Beyond screens: An adaptive leadership approach to addressing digital distortions and student well-being. Journal of Cases in Educational Leadership, 26(1), 50–65.

Khalifa

Gooden

M. A.

Davis

J. E.

(2016). Culturally responsive school leadership: A synthesis of the literature. Review of Educational Research, 86(4), 1272–1311. https://doi.org/10.3102/0034654316630383

UNESCO. (2021). Recommendation on the ethics of artificial intelligence. https://unesdoc.unesco.org/ark:/48223/pf0000381137

Zhang

Aslan

A. B.

(2021). AI technologies for education: Recent research & future directions. Computers and Education: Artificial Intelligence, 2, 100025. https://doi.org/10.1016/j.caeai.2021.100025

Abtan

(2021). Overcoming racial equity fatigue. https://Ssir.org/Articles/Entry/Overcoming_racial_equity_fatigue#

Akbarighatar

Pappas

Vassilakopoulou

(2023). A sociotechnical perspective for responsible AI maturity models: Findings from a mixed-method literature review. International Journal of Information Management Data Insights, 3(2), 100193. https://doi.org/10.1016/j.jjimei.2023.100193

Bass

B. M.

(1985). Leadership and performance beyond expectations. Free Press.

10.

Bender

Gebru

McMillan-Major

Shmitchell

(2021). On the dangers of stochastic parrots: Can language models be too big? In FAccT ’21: Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency (pp. 610–623). Association for Computing Machinery. https://doi.org/10.1145/3442188.3445922

11.

Benjamin

(2019). Race after technology: Abolitionist tools for the new Jim Code. Polity Press.

12.

Briscoe

McIntosh

(2022). A fractured school community: Decolonizing leaders needed for responsive leadership. Journal of Cases in Educational Leadership, 26(1), 16–30. https://doi.org/10.1177/15554589221128051

13.

Buolamwini

Gebru

(2018). Gender shades: Intersectional accuracy disparities in commercial gender classification. Proceedings of Machine Learning Research, 81, 1–15.

14.

Chance

N. L.

(2021). A phenomenological inquiry into the influence of crucible experiences on the leadership development of Black women in higher education senior leadership. Educational Management Administration & Leadership, 49(4), 174114322110194. https://doi.org/10.1177/17411432211019417

15.

Chin

M. H.

Afsar-Manesh

Bierman

A. S.

Chang

Colón-Rodríguez

C. J.

Dullabh

Duran

D. G.

Fair

Hernandez-Boussard

Hightower

Jain

Jordan

W. B.

Konya

Moore

R. H.

Moore

T. T.

Rodriguez

Shaheen

Snyder

L. P.

Srinivasan

. . .Ohno-Machado

(2023). Guiding principles to address the impact of algorithm bias on racial and ethnic disparities in health and health care. JAMA Network Open, 6(12), Article e2345050. https://doi.org/10.1001/jamanetworkopen.2023.45050

16.

Diem

Iverson

D. E.

Welton

A. D.

Walters

S. W. F.

(2024). A path toward racial justice in education: Anti-racist policy decision making in school districts. Educational Policy, 38(7), 1526–1562. https://doi.org/10.1177/08959048241271329

17.

Eubanks

(2018). Automating inequality: How high-tech tools profile, police, and punish the poor. St. Martin’s Press.

18.

Fuller

Hollingworth

B. P.

(2019). Exploring intersectionality and the employment of school leaders. Journal of Educational Administration, 57(2), 134–151. https://doi.org/10.1108/jea-07-2018-0133

19.

Gooden

M. A.

Dantley

M. E.

(2012). Centering race in a framework for leadership preparation. Journal of Research on Leadership Education, 7(2), 237–253. https://doi.org/10.1177/1942775112455266

20.

Gooden

M. A.

Davis

B. W.

Spikes

Hall

D. L.

Lee

(2018). Leaders changing how they act by changing how they think: Applying principles of an anti-racist principal preparation program. Teachers College Record, 120(14), 1–26.

21.

Hafez

Wazir

Karimi

(2023). AI as complex sociotechnical systems: Problems, approaches and reflections. https://www.womeninai.at/wp-content/uploads/2023/11/WhitePaper_AI_as_SociotechnicalSystems_final.pdf

22.

Heifetz

Grashow

Linsky

(2009). The practice of adaptive leadership: Tools and tactics for changing your organization and the world. Harvard Business Press.

23.

Ibadullaeva

(2025). Beyond screens: An adaptive leadership approach to addressing digital distortions and student well-being. Journal of Cases in Educational Leadership, 28(2), 119–134. https://doi.org/10.1177/15554589251320850

24.

Khalifa

Gooden

M. A.

Davis

J. E.

(2016). Culturally responsive school leadership: A synthesis of the literature. Review of Educational Research, 86(4), 1272–1311. https://doi.org/10.3102/0034654316630383

25.

Knaus

C. B.

Rogers-Ard

(2023). Critical race teacher leadership: Leading antiracist systems change. Urban Education, 60(8), 2281–2307. https://doi.org/10.1177/00420859231192083

26.

Kudina

van de Poel

(2024). A sociotechnical system perspective on AI. Minds and Machines, 34(3), 21. https://doi.org/10.1007/s11023-024-09680-2

27.

Lopez

A. E.

Jean-Marie

(2021). Challenging anti-Black racism in everyday teaching, learning, and leading: From theory to practice. Journal of School Leadership, 31(1–2), 50–65. https://doi.org/10.1177/1052684621993115

28.

Moore

(2025, January 24). Trump’s executive orders rolling back DEI and accessibility efforts, explained. ACLU. American Civil Liberties Union. https://www.aclu.org/news/racial-justice/trumps-executive-orders-rolling-back-dei-and-accessibility-efforts-explained

29.

NAFSA. (2025, June). Executive actions under the Trump-Vance administration touching on DEI and gender. https://www.nafsa.org/regulatory-information/executive-actions-under-trump-vance-administration-touching-dei-and-gender

30.

National Institute of Standards and Technology. (2023). Artificial intelligence risk management framework (AI RMF 1.0) (NIST AI 100-1). https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.100-1.pdf

31.

Ostrager

A.-E.

Jordan

J. M.

High

T. R.

(2025, February 10). President Trump acts to roll back DEI initiatives. Harvard Law School Forum on Corporate Governance. https://corpgov.law.harvard.edu/2025/02/10/president-trump-acts-to-roll-back-dei-initiatives/

32.

Raji

I. D.

Smart

White

R. N.

Mitchell

Gebru

Hutchinson

Smith-Loud

Theron

Barnes

(2020). Closing the AI accountability gap: Defining an end-to-end framework for internal algorithmic auditing. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency (FAT ‘20)* (pp. 33–44). Association for Computing Machinery. https://doi.org/10.1145/3351095.3372873

33.

Richard

M. S.

Cosner

(2024). Centering equity within principal preparation and development: An integrative review of the literature. Education Sciences, 14(9), 1–59. https://doi.org/10.3390/educsci14090944

34.

Shah

Aoudeh

Cuglievan-Mindreau

Flessa

(2022). Tempering applied critical leadership: The im/possibilities of leading for racial justice in school districts. Educational Administration Quarterly, 59(1), 179–217. https://doi.org/10.1177/0013161x221137877

35.

Stone

(2024, February 21). Diversity fatigue: Addressing burnout in DEI efforts. Diversio. https://diversio.com/diversity-fatigue/

36.

Tulsiani

(2024, July 12). Strategies to mitigate bias in AI algorithms. elearning industry. https://elearningindustry.com/strategies-to-mitigate-bias-in-ai-algorithms

37.

UNESCO. (2022). Recommendation on the ethics of artificial intelligence. https://unesdoc.unesco.org/ark:/48223/pf0000381137

38.

UNESCO. (2023). Guidance for generative AI in education and research. https://unesdoc.unesco.org/ark:/48223/pf0000386693

39.

Weiner

Burton

L. J.

Borders

(2019). Microaggressions in administrator preparation programs: How Black female participants experienced them. International Journal of Qualitative Studies in Education, 32(9), 1150–1170. https://doi.org/10.1080/09518398.2019.1635285

40.

The White House. (2025a, January 20). Ending radical and wasteful government DEI programs and preferencing. https://www.whitehouse.gov/presidential-actions/2025/01/ending-radical-and-wasteful-government-dei-programs-and-preferencing/

41.

The White House. (2025b, January 21). Ending illegal discrimination and restoring merit-based opportunity. https://www.whitehouse.gov/presidential-actions/2025/01/ending-illegal-discrimination-and-restoring-merit-based-opportunity/