To Make Language Models Work Better, Researchers Sidestep Language
1 min read
Summary
Large language models (LLMs) like OpenAI’s GPT do not think in language as humans do but in mathematical “latent space” as they are based on deep neural networks that transform sequences of numbers into each other.
Two recent research papers have shown that LLMs can be trained to think and reason independently of language, with the model thinking in numbers and then being given a “secret key” to return to language.
This allows LLMs to perform increasingly complex reasoning tasks far more efficiently than normal, while also being more economical with their use of computer resources as they no longer have to keep converting their internal mathematical formats into text.
The next step is to develop systems that allow LLMs to determine when they have finished thinking in numbers and to return to language themselves, without needing a prompt to do so.
