Complete photo deletion on disk, snappy lightbox, keyboard shortcuts, prefetching, and perfected documentation

README.md

@@ -1,116 +1,93 @@
 # PHOTON — local photo intelligence console
 
-A local web app that walks through a photo folder, has an Ollama vision model
+A local web application that scans your photo collection, uses local Ollama vision models to describe + categorize each photo, and embeds standard metadata inside photo files so everything becomes searchable across Spotlight, Finder, Photos, Lightroom, etc.
-describe + categorize each photo, and embeds the result as standard metadata
-inside the photo file so everything becomes searchable (Spotlight, Photos,
-Lightroom, etc.). Nothing is ever deleted, moved, or renamed.
 
-## Run it — zero config
+Original photos remain completely untouched unless you explicitly choose to edit or delete them.
 
-Drop this whole folder *inside* the photo collection you want to organize,
+---
-then just run it:
+
+## Quick Start (Zero-Config)
+
+Drop this directory inside the photo collection you want to organize, then run:
 
 ```bash
 cd "your-photo-library/photon"
 python3 server.py
-# then open http://localhost:8765
+# Open http://localhost:8765 in your browser
 ```
 
-It auto-detects the folder to scan as **the parent of wherever this app
+### Automatic Folder Detection & Overrides
-lives** — so if you dropped it into `~/Pictures/Vacation2026/photon`, it
+- **Auto-Detection**: Scans the parent directory of wherever `server.py` lives.
-defaults to scanning `~/Pictures/Vacation2026`. The folder it last scanned is
+- **Persistence**: Remembers your last scanned folder (`photon_folder.json`).
-also remembered automatically (`photon_folder.json`), so on every future
+- **CLI & Environment Overrides**:
-launch it just picks up where you left off — no retyping paths.
+  ```bash
+  python3 server.py /path/to/photos                       # CLI argument override
+  PHOTON_FOLDER=/path/to/photos python3 server.py         # Environment variable override
+  PHOTON_PORT=8766 python3 server.py                      # Custom port
+  ```
 
-Want to point it somewhere else? Any of these work, in priority order:
+### Prerequisites
-```bash
+- **Ollama** running locally with a vision model (e.g. `ollama pull qwen3.5:9b` or `ollama pull qwen3.5:4b`).
-python3 server.py /path/to/photos        # one-off CLI override
+- **`exiftool`** (installed via Homebrew: `brew install exiftool`).
-PHOTON_FOLDER=/path/to/photos python3 server.py   # env var override
+- **macOS** (`sips` built-in for fast downscaling & pixel integrity verification).
-```
+- **`ffmpeg` / `ffprobe`** (optional: for video frame extraction & video thumbnails).
-Or just type a new path into the folder field in the Console tab and hit
-**Scan Directory** — that becomes the new remembered default too. Running
-two libraries at once? `PHOTON_PORT=8766 python3 server.py` avoids a port clash.
 
-Requires: Ollama running with a vision model, `exiftool` (installed via
+---
-brew), macOS (`sips` for fast downscaling), and `ffmpeg`/`ffprobe` (for video
-thumbnails and tagging — optional, everything else works without it).
 
-## How it works
+## Key Features & Capabilities
 
-1. **Scan** — recursively finds images (`jpg/jpeg/png/heic/tiff/webp/bmp`).
+### 1. Multithreaded 3-Stage Pipeline
-   Videos are counted but skipped. Hidden files and `._*` AppleDouble sidecars
+1. **Downscale Stage**: Uses macOS `sips` to create fast temp copies (original files are read-only).
-   are never touched.
+2. **Inference Stage**: Calls local Ollama vision models to determine `{description, category}`.
-2. **Analyze** — each photo is downscaled with `sips` to a temp copy (original
+3. **Write Stage**: Uses `exiftool` to embed metadata with atomic renames and date preservation (`-P`).
-   is only ever *read*), sent to the chosen Ollama vision model with a JSON
-   schema that forces `{description, category}` output.
-3. **Write** — `exiftool` embeds:
-   - `EXIF:ImageDescription`, `IPTC:Caption-Abstract`, `XMP-dc:Description` — the description
-   - `XMP-dc:Subject` + `IPTC:Keywords` — the category, plus a `photon-tagged` marker
-   - Writes use exiftool's temp-file + atomic-rename mode; file dates preserved with `-P`.
-4. **Journal** — every processed photo is appended to `photon_journal.jsonl`
-   (path, description, category, model, timing). Restarting the app resumes
-   where it left off ("skip already tagged").
 
-## The 10 categories
+### 2. Search & Interactive Library Browser
+- **Instant Search**: Full-text keyword search across descriptions, categories, filenames, and paths.
+- **Filter Chips**: 1-click taxonomy pills (`People`, `Animals`, `Screenshots`, `Vehicles`, `Objects`, etc.).
+- **Sub-Views**:
+  - **Tagged**: Explore and filter all processed photos.
+  - **Untagged**: Browse photos awaiting tagging.
+  - **Failed**: View and retry failed operations.
+  - **Videos**: Frame extraction, AI tagging, and native video player.
+  - **Duplicates**: Perceptual hash index (pHash) visual duplicate grouping.
 
-People · Animals · Food & Drink · Nature & Outdoors · City & Buildings ·
+### 3. Full-Screen Lightbox & Organic Browsing
-Vehicles · Screenshots & Documents · Events & Parties · Objects & Stuff ·
+- **Snappy Viewer**: Full-resolution image/video lightbox with metadata inspector.
-Art & Miscellaneous
+- **0ms Image Prefetching**: Pre-caches adjacent images in memory for instant switching.
+- **Keyboard Shortcuts**:
+  - `←` / `→` : Navigate previous / next photo.
+  - `Esc` : Close Lightbox.
+  - `Delete` / `Backspace` : Delete current photo on disk.
 
-## Smart router (recommended)
+### 4. Disk Photo Deletion & Management
+- **Single & Bulk Deletion**: Click "Delete Photo" or select multiple photos to permanently delete them on disk (or send to macOS Trash).
+- **Automated Cleanup**: Deleting a photo purges its entry from `photon_journal.jsonl`, removes `_organized/` symlinks, and clears thumbnail & view caches.
 
-With the **smart router** toggle on, a fast scout model (glm-ocr, 1.1B) first
+### 5. Bulk Operations & Export
-classifies each image as *screenshot* or *photo*, then hands it to the right
+- **Select Mode**: Range selection via `Shift`-Click or "Select All Matching".
-describer with a specialized prompt. Screenshots also get an **OCR text embed**:
+- **Exporting**: Export catalog metadata to CSV or JSON.
-glm-ocr transcribes the visible words and they're appended to the description
+- **ZIP Downloads**: Stream original-quality files into a single ZIP archive.
-(`… | text: …`), so you can find a screenshot by searching the exact words in it.
 
-Tested defaults: scout `glm-ocr` (6/6 routing accuracy) → describer
+### 6. Trust & Safety Safeguards
-`qwen3.5:4b` for both branches (8/8 accuracy, reads product labels correctly).
+- **Verify Pixel Integrity**: Option to double-hash image pixels via raw BMP conversions before and after writes. Guarantees 100% zero image corruption.
+- **Organized Symlinks**: Generates relative portable Finder aliases in `[folder]/_organized/[category]/[name]`.
+- **Undo All Tags**: One-click exiftool pass to cleanly remove all `photon-tagged` metadata and categories.
+- **Persistent Failure Retries**: Saves errored paths to `photon_failures.jsonl` with 1-click retry.
 
-## Settings that affect speed
+### 7. Quality Control & Custom Taxonomies
+- **Blind Accuracy Grader**: Interactive 100-photo audit mode to score description quality.
+- **Custom Categories**: Edit, add, or customize category schemas (`photon_categories.json`).
 
-| Setting | Effect |
+---
-|---|---|
-| Vision model | `glm-ocr` (1.1B) ≈ 7 s/photo; `qwen3.5:9b` slower but smarter |
-| Image feed resolution | 512 px is fastest; originals are untouched either way |
-| Description length | brief/standard/detailed — caps the model's output tokens |
-| Keep-alive | "forever" keeps the model in RAM between photos (fastest) |
-| Dry run | full pipeline but no metadata written |
-| Keep `_original` backups | exiftool keeps a backup copy of every file (doubles disk usage) |
 
-## Searching afterwards
+## Standard Metadata Specifications
 
-Spotlight: just type a word from a description in Finder search.
+`exiftool` embeds the following tags into image files:
-Or from terminal: `mdfind -onlyin "/path/to/your/photos" "scooter"`
+- `EXIF:ImageDescription`, `IPTC:Caption-Abstract`, `XMP-dc:Description` — Description string
-Or grep the journal: `grep -i scooter photon_journal.jsonl`
+- `XMP-dc:Subject`, `IPTC:Keywords` — Category string + `photon-tagged` keyword
 
-Or use the app itself — the **Search & Edit** tab is a full photo library browser:
+---
 
-- **Tagged** — instant multi-word search across description/category/filename/path,
+## License & Safety Notice
-  filter by category (click a chip), date range, or file type (photos/videos),
-  sort by newest/name/category. Click any result for a full-resolution
-  viewer + editor with Save / AI Redo / Reveal-in-Finder.
-- **Untagged** / **Failed** — see what's left to do or what errored, tag or
-  retry one at a time without leaving the grid.
-- **Videos** — thumbnails via `ffmpeg` frame-grab, native playback, AI tagging
-  off an extracted frame.
-- **Duplicates** — one-time background scan builds a perceptual-hash index and
-  groups exact visual matches (re-saves, burst duplicates). No auto-delete —
-  just Reveal-in-Finder per copy so you decide.
 
-**Selecting & downloading photos in bulk:** click **Select** to enter select
+Original photo files are never deleted or modified unless you explicitly trigger **Delete Photo**, **Metadata Writing**, or **Undo All Tags**.
-mode, then:
-- click a photo to toggle it, **shift-click** to select a range
-- **Select all loaded** / **Select all N matching** (grabs everything matching
-  your current search, not just what's rendered) / **Clear selection**
-- **Download selected (ZIP)** or **Download all matches (ZIP)** — streams a
-  real zip of the original files, byte-for-byte, no re-compression (capped at
-  3000 files per zip)
-- **Apply category to selected**, or **Export** the set as CSV/JSON
-
-## Appearance
-
-Settings & Safety → **Appearance**: pick an accent color (native color
-picker), grid density (compact/comfortable/large), and results-per-page.
-Saved per-browser, doesn't touch anything on disk.

index.html

@@ -218,6 +218,15 @@ input[type=color]{width:46px;height:32px;border:1px solid var(--line);border-rad
 .result-desc2{font-size:11px;color:var(--txt);line-height:1.4;display:-webkit-box;-webkit-line-clamp:3;-webkit-box-orient:vertical;overflow:hidden}
 mark{background:rgba(34,211,238,.28);color:#fff;border-radius:2px;padding:0 1px}
 
+/* ---------- toast notifications ---------- */
+.toast-container{position:fixed;bottom:24px;right:24px;z-index:9999;display:flex;flex-direction:column;gap:8px;pointer-events:none}
+.toast-msg{background:var(--panel2);border:1px solid var(--line);color:var(--txt);padding:10px 18px;border-radius:6px;font-size:12px;font-family:var(--font-sans);font-weight:500;box-shadow:0 8px 24px rgba(0,0,0,0.6);opacity:0;transform:translateY(12px);transition:all .25s ease;pointer-events:auto;display:flex;align-items:center;gap:8px}
+.toast-msg.show{opacity:1;transform:translateY(0)}
+.toast-msg.ok{border-color:var(--grn);color:var(--grn);box-shadow:0 0 16px rgba(74,222,128,.2)}
+.toast-msg.error{border-color:var(--red);color:var(--red);box-shadow:0 0 16px rgba(248,113,113,.2)}
+.toast-msg.info{border-color:var(--cyan);color:var(--cyan);box-shadow:0 0 16px rgba(34,211,238,.2)}
+
+
 .load-more-row{display:flex;justify-content:center;padding:6px 14px 20px}
 
 /* ---------- fullscreen viewer / editor lightbox ---------- */
@@ -448,6 +457,7 @@ mark{background:rgba(34,211,238,.28);color:#fff;border-radius:2px;padding:0 1px}
           <button id="btnExportSelected">Export selected (JSON)</button>
           <button id="btnExportAllCsv">Export all matches (CSV)</button>
           <button id="btnDownloadAllZip">⬇ Download all matches (ZIP)</button>
+          <button id="btnBulkDelete" class="btn-danger">🗑 Delete selected</button>
         </div>
         <div class="search-results-grid" id="searchResultsGrid">
           <div class="search-empty">Loading tagged photos…</div>
@@ -714,6 +724,7 @@ mark{background:rgba(34,211,238,.28);color:#fff;border-radius:2px;padding:0 1px}
   <div class="lightbox-topbar">
     <div class="lightbox-pos" id="lbPos">1 / 1</div>
     <div class="lightbox-topbar-actions">
+      <button id="lbDeleteTop" class="btn-danger" title="Delete photo from disk (Delete / Backspace key)">🗑 Delete</button>
       <button id="lbReveal">Reveal in Finder</button>
       <button id="lbClose" class="btn-danger">✕ Close</button>
     </div>
@@ -745,6 +756,7 @@ mark{background:rgba(34,211,238,.28);color:#fff;border-radius:2px;padding:0 1px}
       <div class="editor-actions">
         <button id="lbSave" class="btn-primary">✓ Save</button>
         <button id="lbRedo">⌁ AI Redo</button>
+        <button id="lbDelete" class="btn-danger" style="grid-column:1/-1">🗑 Delete photo from disk</button>
       </div>
     </div>
   </div>
@@ -860,6 +872,25 @@ function addLog(e){
 }
 function escapeHtml(s){return s.replace(/[&<>]/g,c=>({"&":"&amp;","<":"&lt;",">":"&gt;"}[c]))}
 
+function showToast(msg, type="info", duration=3000){
+  let container = $("toastContainer");
+  if(!container){
+    container = document.createElement("div");
+    container.id = "toastContainer";
+    container.className = "toast-container";
+    document.body.appendChild(container);
+  }
+  const toast = document.createElement("div");
+  toast.className = `toast-msg ${type}`;
+  toast.innerHTML = `<span>${escapeHtml(msg)}</span>`;
+  container.appendChild(toast);
+  setTimeout(()=>toast.classList.add("show"), 10);
+  setTimeout(()=>{
+    toast.classList.remove("show");
+    setTimeout(()=>toast.remove(), 300);
+  }, duration);
+}
+
 function setBadge(st){
   state=st;
   const b=$("badge"); b.textContent=st.toUpperCase(); b.className="badge "+st;
@@ -1264,6 +1295,16 @@ function renderLightbox(){
     img.onerror = ()=>{ $("lbLoading").textContent = "failed to load full image"; };
     img.src = "/api/photo?path=" + encodeURIComponent(rec.path) + "&_=" + Date.now();
   }
+
+  // Instant 0ms prefetching for adjacent photos
+  if(lbMode === "grid"){
+    if(lbIndex + 1 < searchResults.length && !VIDEO_EXT_RE.test(searchResults[lbIndex + 1].path)){
+      new Image().src = "/api/photo?path=" + encodeURIComponent(searchResults[lbIndex + 1].path);
+    }
+    if(lbIndex - 1 >= 0 && !VIDEO_EXT_RE.test(searchResults[lbIndex - 1].path)){
+      new Image().src = "/api/photo?path=" + encodeURIComponent(searchResults[lbIndex - 1].path);
+    }
+  }
 }
 
 async function lbGoto(delta){
@@ -1280,15 +1321,51 @@ async function lbGoto(delta){
   renderLightbox();
 }
 
+async function deleteCurrentPhoto(){
+  const rec = currentLbRec();
+  if(!rec) return;
+  if(!confirm(`Permanently delete this photo from disk?\n\n${rec.path}`)) return;
+  
+  const j = await api("/api/delete_photo", { path: rec.path });
+  if(j.ok){
+    showToast("Deleted " + os_basename(rec.path) + " from disk", "ok");
+    addLog({level:"ok", msg:`Deleted ${os_basename(rec.path)} from disk.`, ts:new Date().toTimeString().slice(0,8)});
+    if(lbMode === "grid"){
+      searchResults.splice(lbIndex, 1);
+      searchTotal = Math.max(0, searchTotal - 1);
+      if(searchResults.length === 0){
+        closeLightbox();
+        doSearch(true);
+      } else {
+        if(lbIndex >= searchResults.length) lbIndex = searchResults.length - 1;
+        renderLightbox();
+        renderSearchResults();
+      }
+    } else {
+      closeLightbox();
+    }
+  } else {
+    showToast(j.error || "Failed to delete photo", "error");
+  }
+}
+
 $("lbPrev").onclick = ()=>lbGoto(-1);
 $("lbNext").onclick = ()=>lbGoto(1);
 $("lbClose").onclick = closeLightbox;
+if($("lbDelete")) $("lbDelete").onclick = deleteCurrentPhoto;
+if($("lbDeleteTop")) $("lbDeleteTop").onclick = deleteCurrentPhoto;
 
 document.addEventListener("keydown", e=>{
   if(!$("lightbox").classList.contains("active")) return;
+  const tag = (e.target.tagName || "").toLowerCase();
+  if(tag === "input" || tag === "textarea" || tag === "select") return;
   if(e.key === "Escape") closeLightbox();
   else if(e.key === "ArrowLeft") lbGoto(-1);
   else if(e.key === "ArrowRight") lbGoto(1);
+  else if(e.key === "Delete" || e.key === "Backspace"){
+    e.preventDefault();
+    deleteCurrentPhoto();
+  }
 });
 
 $("lbSave").onclick = async ()=>{
@@ -1300,6 +1377,7 @@ $("lbSave").onclick = async ()=>{
   if(j.ok){
     rec.desc = j.desc; rec.category = j.category;
     if(lbMode === "grid") renderSearchResults();
+    showToast("Saved metadata for " + os_basename(rec.path), "ok");
     addLog({level:"ok", msg:`Updated ${os_basename(rec.path)} metadata manually.`, ts:new Date().toTimeString().slice(0,8)});
   }
 };
@@ -1315,6 +1393,7 @@ $("lbRedo").onclick = async ()=>{
     rec.desc = j.desc; rec.category = j.category; rec.route = j.route;
     renderLightbox();
     if(lbMode === "grid") renderSearchResults();
+    showToast("AI re-evaluated " + os_basename(rec.path), "ok");
     addLog({level:"ok", msg:`AI re-evaluated ${os_basename(rec.path)} successfully.`, ts:new Date().toTimeString().slice(0,8)});
   }
 };
@@ -1329,6 +1408,7 @@ $("lbCopyPath").onclick = async ()=>{
   if(!rec) return;
   try{
     await navigator.clipboard.writeText(rec.path);
+    showToast("File path copied to clipboard", "ok");
     const btn = $("lbCopyPath");
     const orig = btn.textContent;
     btn.textContent = "✓ Copied";
@@ -1518,6 +1598,24 @@ $("btnExportAllCsv").onclick = ()=>{
   const params = currentSearchParams({format:"csv"});
   window.open("/api/export?" + params.toString(), "_blank");
 };
+if($("btnBulkDelete")){
+  $("btnBulkDelete").onclick = async ()=>{
+    if(selectedPaths.size === 0) return;
+    if(!confirm(`Permanently delete ${selectedPaths.size} selected photo(s) from disk?\n\nThis cannot be undone.`)) return;
+    $("btnBulkDelete").disabled = true;
+    const j = await api("/api/delete_photos", { paths: Array.from(selectedPaths) });
+    $("btnBulkDelete").disabled = false;
+    if(j.ok){
+      showToast(`Deleted ${j.deleted} photo(s) from disk`, "ok");
+      addLog({level:"ok", msg:`Batch deleted ${j.deleted} photos from disk.`, ts:new Date().toTimeString().slice(0,8)});
+      selectedPaths.clear();
+      updateBulkBar();
+      doSearch(true);
+    } else {
+      showToast(j.error || "Batch delete failed", "error");
+    }
+  };
+}
 
 $("btnDownloadSelected").onclick = ()=>{
   if(selectedPaths.size === 0) return;

server.py

@@ -242,6 +242,101 @@ def journal_write(rec):
     with open(JOURNAL, "a", encoding="utf-8") as f:
         f.write(json.dumps(rec, ensure_ascii=False) + "\n")
 
+def trash_or_remove_file(path):
+    if not os.path.exists(path):
+        return True
+    try:
+        # Try macOS Trash via AppleScript
+        cmd = ["osascript", "-e", f'tell application "Finder" to delete POSIX file "{os.path.abspath(path)}"']
+        r = subprocess.run(cmd, capture_output=True, timeout=10)
+        if r.returncode == 0:
+            return True
+    except Exception:
+        pass
+    try:
+        os.remove(path)
+        return True
+    except Exception as e:
+        log("error", f"Failed to delete file {path}: {e}")
+        return False
+
+def remove_photo_records(path):
+    abs_path = os.path.abspath(path)
+    # 1. Remove symlink in _organized
+    with S.lock:
+        folder = S.folder
+    org_root = os.path.join(folder, "_organized")
+    if os.path.exists(org_root):
+        for root, dirs, files in os.walk(org_root):
+            dirs[:] = [d for d in dirs if not d.startswith(".")]
+            for name in files:
+                p = os.path.join(root, name)
+                if os.path.islink(p):
+                    try:
+                        if os.path.realpath(p) == abs_path:
+                            os.remove(p)
+                    except Exception:
+                        pass
+
+    # 2. Clean thumbnail and view caches
+    try:
+        h = hashlib.md5(abs_path.encode()).hexdigest()
+        t1 = os.path.join(tempfile.gettempdir(), "photon_thumbs", f"{h}.jpg")
+        t2 = os.path.join(tempfile.gettempdir(), "photon_view", f"{h}.jpg")
+        for t in (t1, t2):
+            if os.path.exists(t):
+                try: os.remove(t)
+                except Exception: pass
+    except Exception:
+        pass
+
+    # 3. Rewrite journal without this path
+    if os.path.exists(JOURNAL):
+        lines_to_keep = []
+        with open(JOURNAL, "r", encoding="utf-8") as f:
+            for line in f:
+                line_str = line.strip()
+                if not line_str:
+                    continue
+                try:
+                    rec = json.loads(line_str)
+                    if os.path.abspath(rec.get("path", "")) != abs_path:
+                        lines_to_keep.append(line_str)
+                except Exception:
+                    lines_to_keep.append(line_str)
+        with open(JOURNAL, "w", encoding="utf-8") as f:
+            for l in lines_to_keep:
+                f.write(l + "\n")
+
+    # 4. Remove from State
+    with S.lock:
+        if abs_path in S.done_paths:
+            S.done_paths.remove(abs_path)
+        if abs_path in S.files:
+            S.files.remove(abs_path)
+        if abs_path in S.video_files:
+            S.video_files.remove(abs_path)
+        if abs_path in S.failed_paths:
+            S.failed_paths.remove(abs_path)
+            write_failures()
+    
+    global SEARCH_CACHE
+    SEARCH_CACHE = {"mtime": None, "records": []}
+    load_journal()
+    push_stats()
+
+def delete_photo_file_and_record(path):
+    if not path:
+        return False, "path is required"
+    abs_path = os.path.abspath(path)
+    deleted_disk = False
+    if os.path.exists(abs_path):
+        deleted_disk = trash_or_remove_file(abs_path)
+    else:
+        deleted_disk = True
+    remove_photo_records(abs_path)
+    return deleted_disk, None
+
 SEARCH_CACHE = {"mtime": None, "records": []}
 
 def load_search_index():
@@ -1864,6 +1959,32 @@ class Handler(BaseHTTPRequestHandler):
             except Exception as e:
                 self._json({"error": str(e)}, 500)
 
+        elif self.path == "/api/delete_photo":
+            path = body.get("path")
+            if not path:
+                self._json({"error": "path is required"}, 400); return
+            success, err = delete_photo_file_and_record(path)
+            if success:
+                log("ok", f"Deleted photo from disk: {os.path.basename(path)}")
+                self._json({"ok": True, "path": path})
+            else:
+                self._json({"error": f"Failed deleting file: {err}"}, 500)
+
+        elif self.path == "/api/delete_photos":
+            paths = body.get("paths") or []
+            if not paths:
+                self._json({"error": "paths array is required"}, 400); return
+            deleted_count = 0
+            failed = []
+            for p in paths:
+                success, err = delete_photo_file_and_record(p)
+                if success:
+                    deleted_count += 1
+                else:
+                    failed.append({"path": p, "error": err})
+            log("ok", f"Batch deleted {deleted_count} photos from disk" + (f" ({len(failed)} failed)" if failed else ""))
+            self._json({"ok": True, "deleted": deleted_count, "failed": failed})
+
         elif self.path == "/api/dedupe/build":
             with S.lock:
                 already_running = S.dedupe_status == "running"