slvDev Runs 28.9M-Parameter LLM at 9.88 Tokens/Second on $10 ESP32-S3 Microcontroller
A developer known as slvDev has demonstrated a 28.9-million-parameter language model generating text at 9.88 tokens per second on an ESP32-S3. Running a language model on a sub-$10 microcontroller changes the cost floor for local AI.