Tag archive
How to design automatic fallback across LLM providers: provider abstraction, health checks, prompt portability, cost-aware routing and the failure cas
How to stream tokens from an LLM to a browser with FastAPI and SSE: the endpoint shape, the client, error handling mid-stream, and the pitfalls with p
How to keep an AI product's bill predictable: per-request context caps, embedding and response caches, routing simple requests to small models, fallba
What an 'agent' is in production terms — a loop with tools — and the patterns that make one safe to ship: least-privilege tools, retrieved text as dat
How to test software whose core component is non-deterministic: test the deterministic parts hard, hold golden prompts with expected properties, run a