DeepSeek-V3 now runs at 20 tokens per second on Mac Studio, and that’s a nightmare for OpenAI
1 min read
Summary
Chinese startup DeepSeek has launched a new large language model (LLM) that is causing ripples in the AI industry for its capabilities and how it is being deployed.
The 641GB model, DeepSeek-V3-0324, is available on huggingface.com under an MIT open-source license.
This means the model is freely available for commercial use, which is rare for an LLM of this capability.
What is also notable is that it can run on consumer-grade hardware, specifically Apple’s Mac Studio with M3 Ultra chip, as well as other high-end consumer hardware.
This means it can be run locally, rather than in the more conventional data centre, which is a major departure from current AI requirements.
DeepSeek’s open-source strategy is now challenging the dominant US closed garden model and is transforming China’s AI ecosystem.