Qwen llm on Raspberry Pi 5 (16 GB)

TL;DR

Yes, this 30B Qwen3 runs on a Raspberry Pi. On a Pi 5 (16GB), Q3_K_S-2.70bpw [KQ-2] hits 8.03 TPS at 2.70 BPW and maintains 94.18% of BF16 quality. It genuinely feels real-time. More broadly, the same pattern shows up everywhere else: ByteShape models give you a better TPS/quality tradeoff than the alternatives (here we look at Unsloth and MagicQuant).

For those interested, https://old.reddit.com/r/LocalLLaMA/comments/1q5m2n6/a_30b_qwen_model_walks_into_a_raspberry_pi_and/