#eval

Specific Labs dropped Real-SWE, an enterprise-code SWE benchmark, and the leaderboard is a great...

Our NL→SQL benchmark scored a frontier model as junk on 5 hard questions. It never hallucinated — a 5s prod timeout aborted it mid-answer and the handler mislabeled the abort as a parse failure.

Why production-grade agent evals are less about adding metrics and more about defining product claims, trustworthy oracles, and what actually deserves to block a release.

Claude API eval framework: golden test set, LLM-as-judge, regression detection. 50-prompt suite costs $0.30/run, catches 90% of regressions.

If you build or evaluate scoped agents: any talk about the agent in your test context makes it defend...

為了回答「Openclaw vs Hermes 到底哪個 Agent 比較聰明？」這個問題，我把兩隻 Agent 拉進同一場評測： 大腦都換成 MiniMax 2.7（變項只剩 Agent...

Hallucinations are not one bug — they are four. Each fails a different check; here's the harness.

in 4 lines Manifest V3 is now required for eval() in Chrome extensions, so you need to...

I pick a more or less random snippet from Twitter and analyze it in depth in this series. It shall help me, and you understand the trickery behind it. This episode features insights about the feature you shouldn't use: eval.

This week I had the pleasure to get to know more about the infamous eval() function, as it was my fir...

Lazy evaluation --- load fast, go fast, use only when needed.