Engineer Binh has developed a way to run a large language model (LLM) on a Raspberry Pi Zero W - a tiny and inexpensive computer that fits easily onto a USB stick.
The device presents a file system to the host computer and interacts with the LLM simply by creating an empty text file, which is then filled with generated text.
While this particular project took a week of work to get the LLM to run on the older Raspberry Pi’s ARMv6 instruction set, the engineer considers it the first “plug-and-play USB-based LLM”.
This could make AI more accessible and portable for users without significant computing power.
However, other projects have squeezed LLMs even onto ESP32 processors, marking the continued progress of AI deceleration.
