Back to articles

Tag archive

#networkarchitecture

S
Aug 29, 2026

Scheduling every job on a GPU that can only hold one model

I run this homelab on two small boxes. One never turns off and just serves DNS and background services; the other has a GPU with a memory pool that can hold exactly one loaded model at a time, nothing more. This post covers the queue and gateway that constraint forced me to build, and the network shape wrapped around both boxes.

Aug 29, 20265 min read0 reactions0 comments