Tag archive
Training a 14-skill policy library for MicroDuck, a bipedal robot duck (15× XL330 servos, 50Hz), with mjlab (MuJoCo Warp) + PPO on a single GPU — serial training queues, OOM downgrade-and-retry, NaN-safe patches, and a different reward-design philosophy per skill. All skill rollout GIFs included.
从 DQN、PPO、SAC 到 RLHF，系统梳理强化学习的核心算法原理及其在 LLM 对齐中的关键应用。
The γ=0.99 Default Is Destroying Your Long-Horizon Training Your PPO agent trained...
RLHF Burned $50K Before We Admitted SFT Would've Worked Reinforcement Learning from Human...
The Bug That Only Shows Up After 100K Steps Your PPO agent trains perfectly for 100,000...
The DPO Hype Promised to Kill RLHF — It Didn't Everyone said Direct Preference...
SAC Uses 40% More VRAM Than PPO on the Same Task I expected PPO to be the memory hog. It...
CleanRL Beats Stable Baselines3 by 2.3x — But There's a Catch I spent a week training PPO...
The 500K Step Wall Most RL tutorials show you CartPole with dense rewards every step. Then...
The Bug That Killed My Agent at Step 523,000 Your PPO agent trains beautifully for 500,000...
The Counterintuitive Truth About Sample Efficiency SAC should destroy PPO on sample...
Why A2C Often Trains Faster Than PPO (Until It Doesn't) Most RL tutorials pick PPO as the...