Ashraful’s Blog

Back to articles

Tag archive

#localllminference

#speculativedecoding #localllminference #draftmodel #autoregressivedecoding

Speculative decoding for local LLM inference: how a small draft model accelerates a large one without changing outputs

If you've been running models locally, you already know the drill. Every token an LLM produces...

Jul 7, 20268 min read0 reactions0 comments

#ggufquantization #localllminference #q4kmvsq5kmvsq80 #consumergpullm

Choosing a quantization format for local LLM inference: GGUF Q4_K_M vs Q5_K_M vs Q8_0 on consumer GPUs

If you are running LLMs locally, you quickly realize that VRAM is the only currency that matters....

Jun 19, 20263 min read0 reactions0 comments

#gemma4 #codexcli #localllminference #llmengineering

Running Gemma 4 Locally via Codex CLI: What Actually Works in Practice

Straight Answer Running Gemma 4 locally via Codex CLI enables execution in an isolated environment;...

Apr 21, 20262 min read0 reactions0 comments