Back to articles

Tag archive

#ec2

Six Serving Engines on One EC2 G6: Gemma 4 E2B on an NVIDIA L4
Nov 6, 2026

Six Serving Engines on One EC2 G6: Gemma 4 E2B on an NVIDIA L4

llama.cpp, vLLM, SGLang, mistral.rs, a plain PyTorch engine and LiteRT-LM serve the same 4-bit Gemma 4 E2B on one g6.4xlarge, measured with the same prompts, the same 1-to-64-request sweep and the same GSM8K and BFCL questions. Answers are level on every engine; vLLM with Google's drafter is the fastest at every load, up to 4,516 output tokens per second.

Nov 6, 202613 min read4 reactions0 comments