
Hybrid Inference Routing: Control Over Black-Box AI
TL;DR: Microsoft and GitHub announced Project HydraFusion on October 8, introducing automated...
Tag archive

TL;DR: Microsoft and GitHub announced Project HydraFusion on October 8, introducing automated...

Proof of concept of a 2-of-3 committee on CPU (Laya, VegaML, and Arc Decide) on an Intel Core i7-8550U, with an ONNX INT8 NLI cross-encoder (DeBERTa-v3-small) as an alternative still to be measured.
A project called Deltafin runs the full uncompressed Kimi K3 model — 2.8 trillion parameters, with 1.45 TB of expert weights — on a single MacBook Pro by streaming experts from four external SSDs on demand, sustaining almost exactly one token per sec
slotstream, a single Swift binary released as a Show HN on September 1, 2026, runs the 104 GB Qwen3.8-Flash-Next mixture-of-experts model on Macs with a fraction of that memory by keeping a small trunk resident and reading expert weights off the SSD
Conventional memory contract prices rose roughly 93% to 98% quarter over quarter in early 2026 and are forecast to climb another 58% to 63%, as suppliers divert capacity to AI servers -- repricing the exact component local AI depends on.
Support for Qwen3.8-Flash-Next landed in llama.cpp on August 27, adding a sparse-attention graph, vision, three quantizer fixes and machinery to stream a 97.7 GiB n-gram table that never has to sit on the GPU.
The M5 Ultra Mac Studio configures to 512GB of unified memory at 1.2TB/s, enough to load almost any open-weight model in existence, but its memory bandwidth still sets a hard ceiling on how fast those models can generate text.
TielCoder, a 4-bit re-quantization of the Ornith-1.5 mixture-of-experts model, fits in 22.4 GB and fixed 12 of 25 recent real-world coding issues in its maintainer's tests, the same count as Claude Opus 4.6 at medium effort.
A specialist fork of llama.cpp ships hand-written kernels for AMD's decade-old GFX906 architecture, making cheap used MI50 and Radeon VII cards usable for local inference, and upstream maintainers are now discussing porting the work back.
The importance matrix used to compress Qwen3.8-27B contains no entries for block 64, the model's multi-token-prediction head, because that head never activates during the standard calibration run, which is where the most aggressive one-bit builds sta
The most widely used local AI inference engine published its first semantic version tag on August 17, 2026, after years of releasing by build number, while a pending contribution adds a speculative-decoding mode that adjusts its own draft depth on th
A source-verified 2026 matrix for choosing open source AI developer tools by job: coding, workflows, local inference, interfaces, and models.