#tensorrt

A practical walkthrough of doing INT8 post-training quantization the right way on RTX 5090 + TensorRT 11. 1,500 stratified calibration samples + NVIDIA ModelOpt, 56 seconds of work, 71k NPS — 1.8× the previous TRT 10 + auto-FP16 baseline on the same model. Covers calibration set design, the ModelOpt invocation, and what TRT 11's strongly-typed model is actually buying you on Blackwell.

TensorRT's trt.Dims.__iter__ causes a hard SIGSEGV/SIGABRT crash when used inside a PyGObject-loaded GStreamer plugin, but works fine in plain Python. This article explains the three required conditions and the safe fix.

512MiB ≠ 512MB — the silent trtexec bug This is a short one. But it cost me a full day,...

On a single NVIDIA H100 SXM5, TensorRT 10.2 delivers 2,400 tokens/sec for Llama 4 8B inference – 3.1x...

RAG (Retrieval-Augmented Generation) systems are bottlenecked by inference latency: in 2024...

In 2024, 72% of AI engineering teams report wasting 40+ hours per quarter debugging inference...

In March 2024, a Fortune 500 team discovered that their TensorRT-optimized inference pipeline was...

TensorRT vs Mistral 2: A Head-to-Head Performance Comparison As AI workloads shift toward...

\n If your LLM serving stack is stuck at 120 tokens/sec per A100, you’re leaving 50% of your...

TensorRT Deep Dive OpenVINO: Avoid Deployment for Developers For developers working on...

In 2024, we benchmarked 127 production-grade CV and LLM models across 4 GPU architectures and 2 Intel...

Sécurité IA en temps réel : percées en adversarial learning et défenses autonomes