Tag archive
We benchmarked DSPy GEPA against manually engineered prompts, measuring held-out accuracy, optimization token cost, latency, and transfer to a newer model.
What role do feedback loops play in agentic AI systems: which loop design wins, what degrades accuracy, and when the loop costs more than it returns.
Six stages, real C# code, and the one number that convinced me this wasn’t busywork Here’s...
Your .NET Agent Is in Production, Your Engineering Discipline Isn’t. So I Went and Built the...
Evaluating LLM applications is the rigorous engineering discipline of quantitatively measuring,...
Originally published on andrew.ooo — visit the original for any updates, code snippets that aged...
A free AI model is stable enough to use when the same prompt, repeated ten times, produces a clear...
GPT-6 Astra scored 62.7% and 99.9% on ARC-AGI-3 with the same weights. The harness made the difference, and that should change how you evaluate your o
A 20-episode lab test: giving an AI agent a live view of its own token budget raised cost 29-72% and cut accuracy on one of two models.
Evaluating an AI agent involves looking beyond individual outputs and understanding how effectively...
A five-agent SDLC pipeline sounds out of reach on a Sri Lankan budget. The two ideas underneath it — spec enrichment and Cohen's kappa — cost almost nothing.
Off-the-shelf LLMs are unreliable resume screeners. We built ReadyShortlist with a rubric, evidence quotes, and a 380-case eval set—hitting 84% precision. Here’s the full post-mortem.