
LLM como juez y MCP: cómo se ayudan y sus límites
Dónde poner un LLM como juez con MCP: antes y después de llamar a una tool, al evaluar tus tools y como servidor MCP. Con el caso de Nortia, datos y límites.
Tag archive

Dónde poner un LLM como juez con MCP: antes y después de llamar a una tool, al evaluar tus tools y como servidor MCP. Con el caso de Nortia, datos y límites.
I walk through Brex's approach of building AI agent permission policy from observed traffic instead of writing rules first. CrabTrap, its open-source
Benchmark rule-based guardrails against LLM judges for latency, accuracy, and cost. Learn how to architect hybrid safety pipelines for agents in 2026.
LLM-as-a-judge reliability depends on more than an 80% agreement score. Here
Free models can judge paid models at near-zero cost, but only after you audit the judge itself....
A new study rewrote research papers to change only their rhetoric while preserving every scientific claim, and found AI reviewers shifted their overall scores by up to nine tenths of a point, with the effect strongest near the accept-reject boundary.
A study of 2,500 machine learning papers using an automated checker found that the average number of objective mistakes in a NeurIPS paper rose from 3.8 in 2021 to 5.9 in 2025, a 55 percent increase over four years.
A new benchmark finds that vision-language models used to grade whether a computer-use agent finished its task systematically accept failed runs as successes, and that judgment quality varies more across operating systems than across judges.
LLM evals in 2026 span offline benchmarks, online traces, guardrails, and judge calibration, and most teams conflate them. I share the 12 failure modes I