#continuallearning

Originally published on andrew.ooo — visit the original for any updates, code snippets that aged...

Most language models are trained once and frozen forever. mini-AGI is a small byte-level model that keeps learning continuously on a single 8GB consumer GPU, using a mixture-of-experts architecture that pages experts to disk and a simple learning-rate trick to avoid catastrophic forgetting.

Discover how Mini-AGI’s lightweight continual learning model can reshape AI-agent workflows and reduce GPU costs for production builders.

A new preprint reports that combining replay, function and weight anchors with merged LoRA raises final retention in a narrow sequential Q&A test from 1.2% to 34.9%, while general abilities still deteriorate.

Macaron-V1 introduces a new framework for experiential intelligence, utilizing a Mixture-of-LoRA...

Mind Lab released open weights for Macaron-V1, a continual-learning system that never touches its base model and instead composes small specialist adapters on top, picking exactly one per user turn.

New data arrives every month. Retraining from the original base model every time is wasteful. What continual fine-tuning gets right, and its real risk

What Changed Remote sensing change detection (RSCD) is a critical task in various...

Turing-winner Richard Sutton launched Oak Lab with a north-star goal of a trillion-parameter agent that learns and plans in real time on about 20 watts, betting on continual experiential learning over the static pre-train-then-freeze paradigm behind

When you continually pre-train an LLM into a 'world model', it quietly forgets general knowledge — and a little data mixing buys most of it back. But how much it forgets depends heavily on how you specialize it, and full fine-tuning pays three costs a single benchmark undersells.

MIT, Tencent, and Huawei independently published continual learning papers in 2026. Their convergence reveals AI's real bottleneck.

Dario Amodei says continual learning will be solved this year. Here is what AI agent memory actually means for builders shipping agents right now. Three patterns, real tradeoffs, practical guidance.