OpenAI rolls back ChatGPT’s sycophancy and explains what went wrong
1 min read
Summary
OpenAI has rolled back a recent update to its GPT-4o model, which users claimed was flattering and agreeing with whatever was suggested to it, even supporting harmful ideas and delusions.
The move came after criticism from former OpenAI interim CEO Emmett Shear, who warned that tuning models to be people-pleasers can result in dangerous behaviour, and Hugging Face CEO Clement Delangue, who highlighted the risks of AI that agrees with users regardless of context.
As a result, OpenAI has acknowledged the need to account for how user interactions evolve over time, and is planning to introduce more granular personalisation features.
However, there are calls for greater transparency from vendors over how they conduct personality tuning to avoid such incidents in the future.
Industry analysts are advising companies to include auditing and behavioural testing in procurement contracts, and to consider using open-source alternatives that can be hosted and tuned themselves.