Sakana introduces new AI architecture, ‘Continuous Thought Machines’ to make models reason with less guidance — like human brains
1 min read
Summary
Japanese AI start-up Sakana has unveiled a new AI model architecture called Continuous Thought Machines (CTM), which it claims will bring about a new era of AI language models that can handle a wider range of cognitive tasks more like the way humans reason through unfamiliar problems.
Rather than using fixed, parallel layers that process inputs all at once like transformer models, CTM’s time-based architecture allows models to adjust their reasoning depth and duration dynamically and progressives, depending on the complexity of each new task.
The company has made the CTM architecture open source on GitHub and includes training scripts, pretrained checkpoints, plotting utilities and analysis tools to allow researchers and engineers to experiment and further the model’s development.
Although the firm claims CTM is not designed to chase leaderboard-topping benchmark scores, it has indicated initial tests have shown it performs competitively on the ImageNet-1K benchmark, with 72.47% top-1 and 89.89% top-5 accuracy, and shows strong sequential and adaptive behaviour on maze-solving and image classification tasks.