Abstract
Background
Large language models (LLMs) harbor systematic biases that are particularly consequential in workplace and HR contexts, where their outputs increasingly influence hiring, job design, and organizational decisions. Yet existing bias-evaluation approaches remain methodologically fragmented, limiting practitioners’ ability to assess deployment risks.
Objective
This study introduces BiasLab, a multilingual dual-framing framework to quantify and compare directional output-level bias in LLMs, demonstrated across six workplace and HR-relevant topics.
Methods
BiasLab combines mirrored affirmative and reverse prompt pairs, randomized wrapper perturbations, fixed-choice response constraints, and polarity-aligned scoring. Ten LLMs were evaluated across six topics, gender in leadership, employment gap candidates, age in hiring, remote versus office work, four-day versus five-day work weeks, and AI-assisted versus human-only hiring, spanning 12 languages and 30 iterations per framing direction, yielding 43,200 responses.
Results
All ten models showed consistent directional preferences across every topic: favoring female managers, gap candidates as equally capable, older workers, remote work, the four-day week, and AI-assisted hiring. A recurring asymmetric pattern emerged in which models rejected disfavored claims more strongly than they endorsed their opposites, a distinction invisible to single-frame designs.
Conclusions
BiasLab provides a standardized, reproducible instrument for measuring directional preferences across models. Whether a preference constitutes bias in a fairness sense is topic-dependent: for protected attributes such as gender and age it maps onto equal-employment standards, whereas elsewhere it is better described as systematic preference. These findings have direct implications for HR decision-making, and the framework lets organizations compare and vet models before adopting them for hiring.
Keywords
Get full access to this article
View all access options for this article.