diff --git a/README.md b/README.md index fb9f53f..d187b98 100644 --- a/README.md +++ b/README.md @@ -9,6 +9,14 @@ --- +## Real-Time Telemetry & Control Dashboard + +![HyperSwap Dashboard](assets/dashboard.png) + +*The HyperSwap live dashboard running on `:9090`, demonstrating real-time VRAM allocation tracking (Ollama 14.14 GB, ComfyUI 0.38 GB, Desktop 0.6 GB), 33.89 GB of models resident in 64GB host RAM cache, and sub-25ms model VRAM purges and soft-yields.* + +--- + ## 1. Architectural Overview & Physics of High-Speed Switching ```mermaid @@ -20,8 +28,8 @@ flowchart TD subgraph GPU["NVIDIA GeForce RTX 4080 SUPER (16 GB VRAM)"] direction LR - ActiveLLM["Active LLM
(0–14 GB VRAM)"] - ActiveDiffusion["Active Diffusion Pipeline
(0–14 GB VRAM)"] + ActiveLLM["Active LLM
(0–14.5 GB VRAM)"] + ActiveDiffusion["Active Diffusion Pipeline
(0–14.5 GB VRAM)"] end subgraph Orchestrator["HyperSwap Control Plane (:9090)"] @@ -48,6 +56,11 @@ When running both Ollama and ComfyUI on a 16 GB GPU: * **PCIe Bus Hot-Swap Speed**: Reloading from host RAM over the PCIe 4.0 x16 bus achieves **~31.5 GB/s** transfer bandwidth, bringing model swap times down to **hundreds of milliseconds**. * **15ms Soft-Yield**: When ComfyUI triggers an image generation, Ollama executes an instant soft-yield (`keep_alive: 0`), dropping VRAM allocation from 14.5 GB to 0 MB in **~15 milliseconds** without discarding model pages from system RAM. +### Hardware Optimization Tip: Offloading Display to iGPU +If your CPU has an integrated GPU (such as Intel UHD Graphics 750): +* Plugging your display monitor into the motherboard's HDMI/DisplayPort offloads the desktop display server (`gnome-shell`, `firefox`, `Xwayland`) to the iGPU (shared system RAM). +* This **reclaims ~0.7 to 1.5 GB of dedicated GDDR6X VRAM** on the RTX 4080 SUPER, giving AI models 100% dedicated access to the full **16.0 GB VRAM**. + --- ## 2. REST API Reference @@ -59,52 +72,6 @@ The HyperSwap server runs on port `9090` by default. Interactive OpenAPI/Swagger #### `GET /api/stats` Returns a unified JSON snapshot of all system sensors, GPU processes, host RAM, Ollama status, ComfyUI queue, and transition logs. -**Response (200 OK):** -```json -{ - "timestamp": 59583.16, - "gpu": { - "available": true, - "device_name": "NVIDIA GeForce RTX 4080 SUPER", - "vram_total_gb": 15.99, - "vram_used_gb": 1.46, - "vram_free_gb": 14.53, - "vram_used_pct": 9.1, - "gpu_util_pct": 11, - "temperature_c": 48, - "power_w": 31.4, - "fan_pct": 0, - "breakdown": { - "ollama_gb": 0.0, - "comfyui_gb": 0.24, - "system_gb": 0.67, - "free_gb": 14.53, - "processes": [...] - } - }, - "ram": { - "total_gb": 60.34, - "used_gb": 6.72, - "cached_gb": 36.21, - "free_gb": 17.41, - "cache_ratio_pct": 60.0 - }, - "ollama": { - "online": true, - "active_model_name": null, - "active_model_vram_gb": 0.0, - "installed_models": [...] - }, - "comfyui": { - "online": true, - "executing": false, - "queue_remaining": 0, - "vram_free_mb": 14882.4 - }, - "history": [...] -} -``` - #### `GET /api/gpu` Returns hardware sensors (utilization %, temperature, power draw in Watts, fan %, GPU graphics/memory clocks, and active PIDs). @@ -129,20 +96,6 @@ Hot-swaps the active Ollama LLM in VRAM and tracks transition timing. } ``` -**Response (200 OK):** -```json -{ - "success": true, - "prev_model": "None", - "target_model": "qwen3.8fast:latest", - "total_duration_ms": 1420.5, - "load_duration_ms": 839.5, - "tokens_per_sec": 42.0, - "is_ram_hit": true, - "response": "Ready." -} -``` - #### `POST /api/free-vram` Instructs Ollama to soft-yield VRAM down to 0 MB in ~15 milliseconds while keeping model weights in 64GB RAM cache. @@ -155,14 +108,6 @@ Pre-faults and reads all installed Ollama models and ComfyUI Safetensors into th #### `POST /api/warm-model` Pre-warms a specific model or file into RAM. -**Request Body:** -```json -{ - "model_name": "gemma4:26b", - "filepath": null -} -``` - #### `POST /api/benchmark` Runs an automated back-and-forth model swap benchmark and computes average transition latency. @@ -170,7 +115,7 @@ Runs an automated back-and-forth model swap benchmark and computes average trans ## 3. Model Context Protocol (MCP) Reference -HyperSwap includes a native **MCP 2.0 server** ([mcp_server.py](file:///home/drjones/unified-model-manager/mcp_server.py)) that exposes all orchestration and telemetry functions as agentic tools. +HyperSwap includes a native **MCP 2.0 server** (`mcp_server.py`) that exposes all orchestration and telemetry functions as agentic tools. ### MCP Tools List @@ -223,19 +168,7 @@ HyperSwap includes a native **MCP 2.0 server** ([mcp_server.py](file:///home/drj --- -## 4. Web Dashboard & Real-Time Telemetry - -The Web Dashboard is hosted at `http://localhost:9090`. - -* **Hero Memory Gauges**: Visual multi-segment bars representing VRAM allocation across Ollama, ComfyUI, and Desktop, alongside the 64GB host RAM page cache. -* **Ollama Control Card**: Real-time active model indicator, hot-swap selector, context size, and 1-click VRAM yield button. -* **ComfyUI Pipeline Card**: Live execution state (Idle vs Generating), active prompt queue counter, and VRAM purge controls. -* **GPU Hardware Card**: Live gauges for GPU Core Utilization, Temperature (°C), Power Draw (W), Fan Speed (%), and active compute process table. -* **Switch Timeline**: Real-time event feed detailing swap durations in milliseconds and RAM cache hit flags. - ---- - -## 5. Linux Kernel & Host Tuning +## 4. Linux Kernel & Host Tuning To ensure that 45–50 GB of model weights remain permanently in RAM without kernel eviction: @@ -253,7 +186,7 @@ echo "vm.swappiness = 10" | sudo tee -a /etc/sysctl.d/99-hyperswap.conf --- -## 6. Systemd Service Management +## 5. Systemd Service Management The manager runs as a persistent systemd user service: @@ -270,6 +203,6 @@ journalctl --user -u hyperswap-manager.service -f --- -## 7. License +## 6. License MIT License. Developed for Google Antigravity & High-Throughput Linux AI Deployments. diff --git a/assets/dashboard.png b/assets/dashboard.png new file mode 100644 index 0000000..630e723 Binary files /dev/null and b/assets/dashboard.png differ