48bd0096ff
Measured failure: a diffusion job took 46 s instead of 3 s, squeezed into 1.6 GB. The LLM yielded correctly, an inference reloaded it two seconds later, and plan_release then refused to touch it because it was "busy" -- so ComfyUI crawled while Ollama held 13 GB for the whole run. "Never interrupt busy work" looks like the safe rule and is not. Preempting a lower-priority tenant is safe precisely because releasing is asynchronous: an Ollama unload queues behind its running request and applies when that finishes, so nothing is killed mid-flight. That is what makes fast handoff possible at all, and refusing to do it defeats the purpose of the service. A tenant may now be asked for memory if it is idle, whatever its rank, or if it is busy and ranks strictly below the demander. Equal or higher priority is never interrupted, so peers cannot fight. Idle tenants are still preferred over preempting busy ones. Verified against the real contention: with Ollama at 12.38 GB and 98% utilisation, a diffusion job released it within three seconds and completed in 18 s rather than 46 s. Tests updated to the corrected rule, and the fixture's priorities aligned with what actually ships -- it still had the LLM outranking diffusion from before that was swapped. Tests: 246. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
18 KiB
18 KiB