Add barrier-confirmed yielding, measured residency, persistence and closed-loop tuning
Nine changes, in rough order of how much they affect real behaviour: 1. VRAM yield is now a barrier. Posting keep_alive:0 only asks Ollama to unload; measured here, the HTTP call returns in 63ms while the driver takes a further 77ms to release 14.9GB. Returning inside that window is how ComfyUI ends up allocating into VRAM that is still occupied. instant_free_ollama_vram() polls NVML until the allocation is actually gone and reports request/confirm split. 2. ComfyUI VRAM is no longer purged 1.5s after every prompt, which forced a full checkpoint reload on each workflow iteration. It is held for 30s of genuinely empty queue, with an immediate purge when Ollama actually asks for the memory. 3. Cache-hit classification uses achieved bandwidth (size / load duration) rather than a fixed `load_duration < 2500ms`. That constant called a 12.9GB model read at 2.9GB/s a cold load, and a 0.5GB model read from NVMe a cache hit. 4. Page-cache residency is measured, not assumed. mincore(2) reported 128GB resident on a box with 46GB of page cache: the kernel only permits page-cache introspection on files you own, and the Ollama blobs are owned by uid ollama, for which mincore answers "all resident" instead of failing. Uses cachestat(2) where permitted and a randomised read-rate probe elsewhere, labelling which was used. Fixed-offset probing was self-fulfilling, so windows are random and cold ones are returned with FADV_DONTNEED. 5. Warming is budgeted and ranked by recency/frequency instead of reading every file top-to-bottom, which on 64GB of RAM just evicts whatever was warmed first. 6. Telemetry and events persist to SQLite (~0.38 MB/hour) instead of living in a 50-entry in-memory deque, so /api/analytics/profiles can finally answer whether an overclock profile actually delivers more tok/s. 7. Thermal governor walks the overclock back on sustained heat or hardware throttling, with hysteresis, fed from the existing sampler. 8. Autotune sweeps a clock offset, benchmarks decode at each step, watches for Xid errors and degenerate output, and restores the profile in a finally block. 9. Stock clocks/power/fans are restored on shutdown and via systemd ExecStopPost. Nothing previously undid a locked clock or a manually pinned fan. Also: one shared 1Hz telemetry sampler fanned out to SSE subscribers rather than every client re-running the whole snapshot; wall-clock timestamps in place of the event loop's monotonic clock; cached nvidia-smi shell-outs; quieter httpx logging. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
@@ -2,8 +2,8 @@
|
||||
"ollama": {
|
||||
"label": "Ollama \u2014 LLM decode (memory-bandwidth bound)",
|
||||
"power_limit_w": 370,
|
||||
"core_offset_mhz": 150,
|
||||
"mem_offset_mhz": 825,
|
||||
"core_offset_mhz": 35,
|
||||
"mem_offset_mhz": 200,
|
||||
"lock_core_min": 0,
|
||||
"lock_core_max": 0,
|
||||
"lock_mem_mhz": 0,
|
||||
@@ -14,12 +14,12 @@
|
||||
"label": "ComfyUI \u2014 diffusion (core-compute bound)",
|
||||
"power_limit_w": 370,
|
||||
"core_offset_mhz": 100,
|
||||
"mem_offset_mhz": 500,
|
||||
"mem_offset_mhz": 150,
|
||||
"lock_core_min": 2900,
|
||||
"lock_core_max": 3105,
|
||||
"lock_mem_mhz": 0,
|
||||
"fan_mode": "manual",
|
||||
"fan_speed_pct": 75
|
||||
"fan_speed_pct": 100
|
||||
},
|
||||
"balanced": {
|
||||
"label": "Balanced \u2014 stock boost, power unlocked",
|
||||
|
||||
Reference in New Issue
Block a user