
datasette-llm 0.1a11:通过 OpenAI 兼容接口接入本地模型
更新要点:支持 OpenAI 兼容接口 datasette-llm 是 Datasette 的大语言模型配置中间层插件,负责为其他 Datasette...
Tag archive

更新要点:支持 OpenAI 兼容接口 datasette-llm 是 Datasette 的大语言模型配置中间层插件,负责为其他 Datasette...
Project Maya (MIT, github.com/mw00/project-maya) is a free project whose pitch is the same one this...

Klepai is our free desktop chat for the model servers you host: Ollama and any OpenAI-compatible...
A Mixture of Experts puts a hot set of experts in memory and a cold set on the drive. Deciding that split is an engine job. How Strata keeps the hot set resident, and why that split is what a speed number actually measures.
An MoE answers like a small model on an ordinary machine because each token lights only a few experts. The rest sit on the drive and only load when a later token routes to them. Here is what has to be true for that to hold.
Quantisation is a per-tensor choice about how wide each weight is written. Set it deliberately and a larger model runs on the hardware you own, with the accuracy cost measured, not asserted.

llama.cpp and ggml hit 1.0.0 on 13 Nov 2026 with semver guarantees and FP8 KV caches. On a Windows call the audio engine still sets the floor for any local STT or translation overlay.
A 552B MoE answers from a single gaming PC because the experts are tiered across VRAM, pinned RAM and NVMe and only the hot set is read. What has to be true for that to work, and what it costs.

Strata keeps resident only the experts that are hot, and streams weights through ordinary DDR instead of expensive VRAM. What that buys a small organisation, and what it asks of the box.
Running Mano-P on the Cheapest Mac It Survives On Our docs recommend an M4 Mac with 32GB...
Kindi (AGPL-3.0 with a commercial-license path, github.com/ahtx/kindi) is Amir Husain's inference...
Speech Studio v0.5.0 ships six local TTS engines plus STT on Windows. What the first-sound numbers mean for real-time call overlays.