
You fine-tuned at full precision. You serve in FP8. Those were two different models.
transformers v5.22.0 made FP8 models trainable with LoRA. The interesting part is the train/serve gap it closes - and why a small quality gap is worse than a large one.


