Add barrier-confirmed yielding, measured residency, persistence and closed-loop tuning
Nine changes, in rough order of how much they affect real behaviour: 1. VRAM yield is now a barrier. Posting keep_alive:0 only asks Ollama to unload; measured here, the HTTP call returns in 63ms while the driver takes a further 77ms to release 14.9GB. Returning inside that window is how ComfyUI ends up allocating into VRAM that is still occupied. instant_free_ollama_vram() polls NVML until the allocation is actually gone and reports request/confirm split. 2. ComfyUI VRAM is no longer purged 1.5s after every prompt, which forced a full checkpoint reload on each workflow iteration. It is held for 30s of genuinely empty queue, with an immediate purge when Ollama actually asks for the memory. 3. Cache-hit classification uses achieved bandwidth (size / load duration) rather than a fixed `load_duration < 2500ms`. That constant called a 12.9GB model read at 2.9GB/s a cold load, and a 0.5GB model read from NVMe a cache hit. 4. Page-cache residency is measured, not assumed. mincore(2) reported 128GB resident on a box with 46GB of page cache: the kernel only permits page-cache introspection on files you own, and the Ollama blobs are owned by uid ollama, for which mincore answers "all resident" instead of failing. Uses cachestat(2) where permitted and a randomised read-rate probe elsewhere, labelling which was used. Fixed-offset probing was self-fulfilling, so windows are random and cold ones are returned with FADV_DONTNEED. 5. Warming is budgeted and ranked by recency/frequency instead of reading every file top-to-bottom, which on 64GB of RAM just evicts whatever was warmed first. 6. Telemetry and events persist to SQLite (~0.38 MB/hour) instead of living in a 50-entry in-memory deque, so /api/analytics/profiles can finally answer whether an overclock profile actually delivers more tok/s. 7. Thermal governor walks the overclock back on sustained heat or hardware throttling, with hysteresis, fed from the existing sampler. 8. Autotune sweeps a clock offset, benchmarks decode at each step, watches for Xid errors and degenerate output, and restores the profile in a finally block. 9. Stock clocks/power/fans are restored on shutdown and via systemd ExecStopPost. Nothing previously undid a locked clock or a manually pinned fan. Also: one shared 1Hz telemetry sampler fanned out to SSE subscribers rather than every client re-running the whole snapshot; wall-clock timestamps in place of the event loop's monotonic clock; cached nvidia-smi shell-outs; quieter httpx logging. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
drjones
@@ -665,6 +665,144 @@
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- ============ NEXT-LEVEL PANELS: governor / residency / analytics / autotune ============ -->
|
||||
<div class="grid grid-cols-1 xl:grid-cols-2 gap-5 mt-5">
|
||||
|
||||
<!-- Thermal Governor -->
|
||||
<div class="bg-slate-900/80 border border-slate-800 rounded-2xl p-5">
|
||||
<div class="flex items-center justify-between pb-3 border-b border-slate-800">
|
||||
<div class="flex items-center space-x-2">
|
||||
<div class="p-2 rounded-lg bg-rose-950/80 border border-rose-800 text-rose-400">
|
||||
<i class="fa-solid fa-temperature-arrow-down text-sm"></i>
|
||||
</div>
|
||||
<div>
|
||||
<h3 class="font-bold text-slate-100 text-sm">Thermal Governor</h3>
|
||||
<p class="text-xs text-slate-400">Walks the overclock back when the card complains</p>
|
||||
</div>
|
||||
</div>
|
||||
<div class="flex items-center space-x-2">
|
||||
<button onclick="toggleGovernor()" id="gov-toggle" class="px-2.5 py-1 text-xs font-semibold rounded-lg bg-slate-800 border border-slate-700 text-slate-300 hover:bg-slate-700 transition">Enabled</button>
|
||||
<button onclick="resetGovernor()" class="px-2.5 py-1 text-xs font-semibold rounded-lg bg-slate-800 border border-slate-700 text-slate-300 hover:bg-slate-700 transition">Reset</button>
|
||||
</div>
|
||||
</div>
|
||||
<div class="mt-4 space-y-3">
|
||||
<div class="flex items-baseline justify-between">
|
||||
<span id="gov-label" class="text-2xl font-bold text-emerald-400">full</span>
|
||||
<span id="gov-level" class="text-xs font-mono text-slate-500">level 0 / 3</span>
|
||||
</div>
|
||||
<div class="w-full bg-slate-950 rounded-full h-2 overflow-hidden border border-slate-800">
|
||||
<div id="gov-bar" class="h-full bg-gradient-to-r from-emerald-500 to-rose-500 transition-all duration-500" style="width:0%"></div>
|
||||
</div>
|
||||
<div class="grid grid-cols-3 gap-2 text-center pt-1">
|
||||
<div class="bg-slate-950/60 rounded-lg p-2 border border-slate-800">
|
||||
<div class="text-[10px] text-slate-500 uppercase">Escalate</div>
|
||||
<div id="gov-esc" class="text-sm font-mono text-rose-400">83°C</div>
|
||||
</div>
|
||||
<div class="bg-slate-950/60 rounded-lg p-2 border border-slate-800">
|
||||
<div class="text-[10px] text-slate-500 uppercase">Recover</div>
|
||||
<div id="gov-rec" class="text-sm font-mono text-cyan-400">72°C</div>
|
||||
</div>
|
||||
<div class="bg-slate-950/60 rounded-lg p-2 border border-slate-800">
|
||||
<div class="text-[10px] text-slate-500 uppercase">Offset Scale</div>
|
||||
<div id="gov-scale" class="text-sm font-mono text-slate-300">100%</div>
|
||||
</div>
|
||||
</div>
|
||||
<p class="text-xs text-slate-400"><span class="text-slate-500">Last action:</span> <span id="gov-reason" class="font-mono">cold start</span></p>
|
||||
<div id="gov-history" class="space-y-1 max-h-24 overflow-y-auto text-[11px] font-mono"></div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Measured page-cache residency -->
|
||||
<div class="bg-slate-900/80 border border-slate-800 rounded-2xl p-5">
|
||||
<div class="flex items-center justify-between pb-3 border-b border-slate-800">
|
||||
<div class="flex items-center space-x-2">
|
||||
<div class="p-2 rounded-lg bg-amber-950/80 border border-amber-800 text-amber-400">
|
||||
<i class="fa-solid fa-memory text-sm"></i>
|
||||
</div>
|
||||
<div>
|
||||
<h3 class="font-bold text-slate-100 text-sm">Measured Page-Cache Residency</h3>
|
||||
<p class="text-xs text-slate-400">What is genuinely in RAM, not what we hope is</p>
|
||||
</div>
|
||||
</div>
|
||||
<div class="flex items-center space-x-2">
|
||||
<button onclick="fetchWarmPlan()" class="px-2.5 py-1 text-xs font-semibold rounded-lg bg-slate-800 border border-slate-700 text-slate-300 hover:bg-slate-700 transition">Plan</button>
|
||||
<button onclick="fetchCacheReport(true)" class="px-2.5 py-1 text-xs font-semibold rounded-lg bg-amber-950/70 border border-amber-800 text-amber-300 hover:bg-amber-900 transition">Rescan</button>
|
||||
</div>
|
||||
</div>
|
||||
<div class="mt-4">
|
||||
<div class="flex items-baseline justify-between mb-2">
|
||||
<span id="cache-total" class="text-2xl font-bold text-amber-400">— GB</span>
|
||||
<span id="cache-meta" class="text-xs font-mono text-slate-500">—</span>
|
||||
</div>
|
||||
<div id="cache-capability" class="text-[11px] text-slate-500 mb-3"></div>
|
||||
<div id="cache-files" class="space-y-1.5 max-h-56 overflow-y-auto pr-1"></div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Profile analytics -->
|
||||
<div class="bg-slate-900/80 border border-slate-800 rounded-2xl p-5">
|
||||
<div class="flex items-center justify-between pb-3 border-b border-slate-800">
|
||||
<div class="flex items-center space-x-2">
|
||||
<div class="p-2 rounded-lg bg-cyan-950/80 border border-cyan-800 text-cyan-400">
|
||||
<i class="fa-solid fa-chart-column text-sm"></i>
|
||||
</div>
|
||||
<div>
|
||||
<h3 class="font-bold text-slate-100 text-sm">Is the Overclock Actually Working?</h3>
|
||||
<p class="text-xs text-slate-400">Decode throughput per profile, from persisted history</p>
|
||||
</div>
|
||||
</div>
|
||||
<select id="analytics-days" onchange="fetchAnalytics()" class="bg-slate-950 border border-slate-700 text-xs text-slate-300 rounded-lg px-2 py-1">
|
||||
<option value="1">24h</option><option value="7" selected>7d</option><option value="30">30d</option>
|
||||
</select>
|
||||
</div>
|
||||
<div id="analytics-body" class="mt-4 text-xs text-slate-400">Collecting data…</div>
|
||||
</div>
|
||||
|
||||
<!-- Autotune -->
|
||||
<div class="bg-slate-900/80 border border-slate-800 rounded-2xl p-5">
|
||||
<div class="flex items-center justify-between pb-3 border-b border-slate-800">
|
||||
<div class="flex items-center space-x-2">
|
||||
<div class="p-2 rounded-lg bg-fuchsia-950/80 border border-fuchsia-800 text-fuchsia-400">
|
||||
<i class="fa-solid fa-wave-square text-sm"></i>
|
||||
</div>
|
||||
<div>
|
||||
<h3 class="font-bold text-slate-100 text-sm">Overclock Autotune</h3>
|
||||
<p class="text-xs text-slate-400">Sweep a clock offset, measure tok/s, stop at instability</p>
|
||||
</div>
|
||||
</div>
|
||||
<span id="autotune-state" class="text-xs font-mono text-slate-500">idle</span>
|
||||
</div>
|
||||
<div class="mt-4 grid grid-cols-2 gap-3">
|
||||
<label class="text-xs text-slate-400">Knob
|
||||
<select id="at-knob" class="w-full mt-1 bg-slate-950 border border-slate-700 text-slate-300 rounded-lg px-2 py-1.5">
|
||||
<option value="mem_offset_mhz">Memory offset</option>
|
||||
<option value="core_offset_mhz">Core offset</option>
|
||||
</select>
|
||||
</label>
|
||||
<label class="text-xs text-slate-400">Profile
|
||||
<select id="at-profile" class="w-full mt-1 bg-slate-950 border border-slate-700 text-slate-300 rounded-lg px-2 py-1.5">
|
||||
<option value="ollama">ollama</option><option value="comfy">comfy</option><option value="balanced">balanced</option>
|
||||
</select>
|
||||
</label>
|
||||
<label class="text-xs text-slate-400">Start<input id="at-start" type="number" value="0" class="w-full mt-1 bg-slate-950 border border-slate-700 text-slate-300 rounded-lg px-2 py-1.5 font-mono"></label>
|
||||
<label class="text-xs text-slate-400">Stop<input id="at-stop" type="number" value="600" class="w-full mt-1 bg-slate-950 border border-slate-700 text-slate-300 rounded-lg px-2 py-1.5 font-mono"></label>
|
||||
<label class="text-xs text-slate-400">Step<input id="at-step" type="number" value="150" class="w-full mt-1 bg-slate-950 border border-slate-700 text-slate-300 rounded-lg px-2 py-1.5 font-mono"></label>
|
||||
<label class="text-xs text-slate-400 flex items-end space-x-2 pb-1">
|
||||
<input id="at-apply" type="checkbox" class="accent-fuchsia-500"><span>Save winner to profile</span>
|
||||
</label>
|
||||
</div>
|
||||
<div class="flex items-center space-x-2 mt-3">
|
||||
<button onclick="runSweep()" id="at-run" class="px-4 py-2 bg-gradient-to-r from-fuchsia-600 to-purple-600 hover:from-fuchsia-500 hover:to-purple-500 text-white text-xs font-bold rounded-lg shadow-md transition">
|
||||
<i class="fa-solid fa-play mr-1"></i>Run Sweep
|
||||
</button>
|
||||
<button onclick="cancelSweep()" class="px-3 py-2 bg-slate-800 border border-slate-700 text-slate-300 text-xs font-semibold rounded-lg hover:bg-slate-700 transition">Cancel</button>
|
||||
<span class="text-[11px] text-slate-500">Restores the profile when done, even on error.</span>
|
||||
</div>
|
||||
<div id="autotune-body" class="mt-3 text-xs"></div>
|
||||
</div>
|
||||
|
||||
</div>
|
||||
|
||||
</main>
|
||||
|
||||
<script src="/static/app.js"></script>
|
||||
|
||||
Reference in New Issue
Block a user