Back to articles

Tag archive

#webgpu

W
Nov 7, 2026

WebLLM in Production: Cold Starts, Service-Worker Blocking, and the Missing 14B WebGPU Build

We pulled WebLLM into the lab to see whether browser inference can replace a server GPU. What we found: the paper's 71–80% native-throughput result, an estimated multi-gigabyte first-run download, a documented service-worker UI-freeze report we could not reproduce, and a 14B model that still has no WebGPU build.

Nov 7, 202611 min read0 reactions0 comments