These two new AI benchmarks could help make models less biased
1 min read
Summary
Stanford researchers have created two new benchmarks that can assess the difference and contextual awareness of AI systems to reduce bias and improve overall decision-making processes.
Current fairness benchmarks, such as DiscrimEval, have been criticised for only assessing models on a one-size-fits-all basis, with an overly simplistic view of fairness.
The new benchmarks, which consist of a series of questions with objectively correct answers, have already been used to test models from Google and OpenAI, which performed poorly.
Techniques used to reduce bias in AI, such as instructing models to treat all equally, ultimately make them less accurate and effective.
The study suggests the creation of increasingly diverse datasets and greater investment in interpretability could help develop more accurate AI.
However, some believe AI will never be completely fair and unbiased without a human in the decision-making loop.