#inference

Khi chạy LLM inference hay Whisper offline trên laptop, rào cản lớn nhất không phải CPU mà chính là...

LLM inference isn't compute-bound. It's memory-bandwidth-bound, and most infra teams are optimizing the wrong metric entirely.

Unlock MiMo v2.5's hybrid SWA efficiency techniques and optimize ML inference speed by up to 70% without sacrificing accuracy

Bài viết này là bản tóm tắt kỹ thuật. Canonical URL:...

Chinese AI startup DeepSeek is developing its own chip aimed at running trained models rather than training them, and is simultaneously raising its first-ever outside capital -- about $7 billion at a $52-59 billion valuation.

Google published a GKE Inference Gateway benchmark that looks, at first, like normal cloud provider performance marketing. You know the genre. A chart goes

Everyone optimizes models for FLOPs, but when an LLM generates text, the bottleneck is usually memory, not math. The reason is a structure called the KV cache. Understanding it explains your token costs, your context limits, and why 2026's big wins are memory tricks.

Etched booked $1B in orders for an inference-only AI chip at a $5B valuation. Here's what a specialised chip means for cost, and for a Sri Lankan builder's bill.

Diffusion text models generate in parallel blocks rather than left to right; Google's open DiffusionGemma and Inception's Mercury 2 are now in a head-to-head over speed.

Closed frontier models are raising prices and tightening access just as Chinese open-weight models slash theirs, a structural reversal with big consequences for who builds with AI.

A small model guesses ahead and a big model checks the work in parallel - and this week two efforts pushing that idea, DeepSeek's DSpark and JetSpec, lit up the front page while the community argued over whether it's truly 'lossless.'

A free local-AI tool now runs Google's Gemma model far faster on Apple computers using a trick where a small model drafts words and the big one checks them in bulk.