WebLLM in Production: Cold Starts, Service-Worker Blocking, and the Missing 14B WebGPU Build
We pulled WebLLM into the lab to see whether browser inference can replace a server GPU. What we found: the paper's 71–80% native-throughput result, an estimated multi-gigabyte first-run download, a documented service-worker UI-freeze report we could not reproduce, and a 14B model that still has no WebGPU build.


