A Leaderboard to Benchmark Ethical Biases in LLMs

Marcos Gomez-Vazquez, Sergio Morales, German Castignani, Robert Clarisó, Aaron Conrardy, Louis Deladiennee, Samuel Renault and Jordi Cabot
[pdf]

AIMMES 2024 | Workshop on AI bias: Measurements, Mitigation, Explanation Strategies
Amsterdam, Netherlands
https://ceur-ws.org/Vol-3744/

Abstract

This paper introduces a public leaderboard that comprehensively assesses and benchmarks Large Language Models (LLMs) according to a set of ethical biases and test metrics. The initiative aims to raise awareness about the status of the latest advances in development of ethical AI, and foster its alignment to recent regulations in order to guardrail its societal impacts.

Keywords

Large Language Models, Leaderboard, Ethics, Biases, Testing