drjones da75b635f4 Add fetch-and-index pipeline: crawl immediately indexes into OpenSearch+Qdrant ...

- _fetch_and_index() fetches, strips HTML, extracts title/meta, indexes
- crawl_url now indexes immediately (not just submits to YaCy)
- Fixed search: removed invalid refresh param on _search requests
- Auto-categorization by domain (wikipedia→reference, github→software, etc)
- Verified: 5 docs in OpenSearch, 5 vectors in Qdrant

2026-08-04 06:12:05 -07:00

27 KiB

Raw Blame History

View Raw