A
Aug 2, 2026Adam vs AdamW: When Weight Decay Actually Matters
The 0.2% Accuracy Drop That Cost Us 3 Days Swapping Adam for AdamW in a ResNet-50 training...
Aug 2, 20261 min read0 reactions0 comments
Back to articles
Tag archive
#adamw
A
Jul 8, 2026A giant benchmark tested 24 optimizers - and AdamW's edge held up
OmniOpt ran a controlled bake-off of more than two dozen modern training optimizers across model sizes from 60M to 1B parameters, and its main lesson is deflating: no challenger cleanly dethrones AdamW, because an optimizer's advantage depends heavil
Jul 8, 20263 min read0 reactions0 comments
U
Nov 3, 2025Understanding the Design of Optimizers with me
Ok, it's midnight during Halloween, and I'm talking about Optimizers. Such a thrill lol. And the goal...
Nov 3, 20258 min read1 reactions2 comments