diff --git a/Documentation/Bugs/fswatcher-create-event-swallowed-linux.md b/Documentation/Bugs/fswatcher-create-event-swallowed-linux.md new file mode 100644 index 0000000..9e48921 --- /dev/null +++ b/Documentation/Bugs/fswatcher-create-event-swallowed-linux.md @@ -0,0 +1,54 @@ +# Bug: watcher swallows Create events on Linux (debounce replaces instead of merges) + +**Date found:** 2026-07-16 +**Found during:** local-embeddings epic, Phase 5 Linux verification (first-ever test run on Linux) +**Status:** Fix submitted — PR #5 (`fix/watcher-create-swallowed-linux`, based on `main`, independent of the epic PR stack) +**Severity:** Low today (no user-visible breakage), latent correctness risk + +## Symptom + +`go test ./internal/watcher/` fails deterministically on Linux (3/3 runs, Docker +`golang:1.26-bookworm`, arm64): + +``` +--- FAIL: TestOnCreate (2.01s) + fswatcher_test.go:121: expected OnCreate to be called +``` + +`TestOnModify` and `TestOnDelete` pass. The full suite is green on macOS. + +## Root cause + +Writing a new file on Linux (inotify) emits **two events** for the same path within +milliseconds: `CREATE`, then `WRITE`. In `internal/watcher/fswatcher.go`, +`handleEvent` → `debounce(path, fn)` stores **one pending closure per path and +replaces it** on each new event (`fswatcher.go:114-128`): the `WRITE` event cancels +the `CREATE` timer and substitutes a closure that only sees `WRITE`. When the +debounce fires, the handler classifies the event as a modification — +`handler.OnCreate` is never called for newly written files. + +macOS (fsnotify kqueue/FSEvents backend) delivers/coalesces these events differently, +so the swallow never manifests there. + +## Impact + +- **Today: effectively none for users.** `engine.OnCreate` and `engine.OnModify` + are identical (`engine.go:717-747` — both call `IndexFile`), so new files on + Linux are still indexed, merely misclassified as modifications. +- **Latent risk:** any future divergence between create and modify handling + (e.g. create-only bookkeeping, activity-log semantics, per-event UX) silently + breaks on Linux only. +- Blocks a fully green `go test ./...` on Linux (Phase 5 verify gate) until fixed. + +## Fix (applied in PR #5) + +In the debouncer, **accumulate the fsnotify op bits per path** instead of replacing +the closure — e.g. keep `pendingOps map[string]fsnotify.Op`, OR-ing each event's op; +when the timer fires, classify with precedence Remove/Rename > Create > Write/Chmod +from the merged bits, then clear the entry. Semantics on macOS are unchanged +(single-op case degenerates to today's behavior); Linux create+write merges to +Create. `TestOnCreate` then passes on both platforms. + +Fix delivered as its own small PR per working rules (the watcher is outside the +local-embeddings epic's scope): **PR #5**. Verified there: watcher suite 3/3 pass +on Linux arm64 (was 3/3 fail) and 3/3 pass on macOS. diff --git a/Documentation/Epics/local-embeddings.md b/Documentation/Epics/local-embeddings.md index 36c9a28..93386c9 100644 --- a/Documentation/Epics/local-embeddings.md +++ b/Documentation/Epics/local-embeddings.md @@ -365,8 +365,10 @@ and the indexing-progress UX all already exist and are the extension points. ### Explicitly unchanged `internal/watcher/`, `internal/extractor/`, `internal/mcp/dispatch.go`, `internal/mcp/stdio.go` -(interface types unchanged — `ReadOnlyEngineService` signature is stable), `tray.go`, +(interface types unchanged — `ReadOnlyEngineService` signature is stable), `internal/store/sqlite_readonly.go`, all search/KNN logic in `store.Search`. +(~~`tray.go`~~ — removed from this list in Phase 5: it required a `//go:build darwin` guard ++ non-darwin stub to compile anywhere but macOS; see Phase 5 Linux findings.) ### Known dead-code risks to check at the end @@ -441,6 +443,25 @@ and the indexing-progress UX all already exist and are the extension points. - `dylibCandidates` gains the versioned Linux name (`libonnxruntime.so.1.26.0`) that the official Linux tarball actually ships. +**Linux findings (recorded 2026-07-16, branch `feat/xplat-linux`):** + +- **`tray.go` was never darwin-guarded** — its Cocoa CGo preamble compiled on every platform, + making *any* non-macOS build impossible. Fixed here (`//go:build darwin` + no-op + `tray_stub.go`); this removes tray.go from the "Explicitly unchanged" list (deviation + recorded per the rules — CLAUDE.md already declared it macOS-only in intent). +- **Pre-existing watcher bug surfaced by first-ever Linux test run:** inotify emits + CREATE+WRITE for a new file; the per-path debouncer replaces rather than merges events, so + OnCreate is swallowed (OnModify fires instead — no user impact today since both index the + file). Documented in `Documentation/Bugs/fswatcher-create-event-swallowed-linux.md`; + fix deliberately kept out of this epic's PRs (watcher is out of scope). +- **linux-arm64 verified end-to-end** in Docker (golang:1.26-bookworm): `make assets` + checksums, test suite (watcher known-fail excepted), real int8 inference (cosine ordering + 0.140 < 0.171 < 0.286, matching Phase 0), full Wails build (webkit2gtk-4.1 via the + `webkit2_41` tag), and an offline (`--network none`) MCP search that retrieved correct + semantic matches from a macOS-indexed DB — cross-platform vector compatibility confirmed. +- **linux-amd64**: artifacts pinned and checksum-verified; runtime smoke still pending + (needs an x64 Linux env — Docker on the arm64 dev Mac would run it under slow emulation). + 1. Linux x64/arm64: assets manifest entries, CI build, smoke test. 2. Windows x64: CI job builds `libtokenizers.a` with Rust toolchain (no published binary); ORT DLL from official release; smoke test. diff --git a/assets/manifest.json b/assets/manifest.json index 805c099..1d31955 100644 --- a/assets/manifest.json +++ b/assets/manifest.json @@ -1,6 +1,6 @@ { "schema": 1, - "comment": "Pinned local-embedding artifacts (URLs + SHA-256). Weights/libs are NEVER committed; `make assets` downloads and checksum-verifies these into internal/embeddings/local/embedded/ (go:embed runtime assets) and internal/embeddings/local/lib/ (link-time static lib). For tarball artifacts, `sha256` is the checksum of the downloaded archive and `member_sha256` is the checksum of the extracted member. `dest` is relative to internal/embeddings/local/. Only darwin-arm64 is populated in Phase 3a; other platforms land in Phase 5.", + "comment": "Pinned local-embedding artifacts (URLs + SHA-256). Weights/libs are NEVER committed; `make assets` downloads and checksum-verifies these into internal/embeddings/local/embedded/ (go:embed runtime assets) and internal/embeddings/local/lib/ (link-time static lib). For tarball artifacts, `sha256` is the checksum of the downloaded archive and `member_sha256` is the checksum of the extracted member. `dest` is relative to internal/embeddings/local/. Model + tokenizer entries are identical bytes on every platform (per-platform keyed for a uniform `make assets`). Remaining Phase 5 platforms: windows-amd64, darwin-amd64.", "platforms": { "darwin-arm64": { "model": { @@ -27,6 +27,58 @@ "member_sha256": "c91ae814afb8fe4f000099972208f60c3aa2d13899a7cd5a31fee2e9e2efbac7", "dest": "lib/libtokenizers.a" } + }, + "linux-arm64": { + "model": { + "url": "https://huggingface.co/Xenova/multilingual-e5-small/resolve/main/onnx/model_quantized.onnx", + "sha256": "f80102d3f2a1229f387d3c81909990d8945513e347b0eab049f7de3c6f98c193", + "dest": "embedded/model_quantized.onnx" + }, + "tokenizer": { + "url": "https://huggingface.co/Xenova/multilingual-e5-small/resolve/main/tokenizer.json", + "sha256": "0b44a9d7b51c3c62626640cda0e2c2f70fdacdc25bbbd68038369d14ebdf4c39", + "dest": "embedded/tokenizer.json" + }, + "onnxruntime": { + "url": "https://github.com/microsoft/onnxruntime/releases/download/v1.26.0/onnxruntime-linux-aarch64-1.26.0.tgz", + "sha256": "34ff1c2d0f12e2cf3d33a0c5f82e39792e1d581fbd6968fd7c30d173654be01a", + "member": "onnxruntime-linux-aarch64-1.26.0/lib/libonnxruntime.so.1.26.0", + "member_sha256": "115ecb838e703d390262b8b4d07d5248e6693c67658d4c98c48f94905ab27af4", + "dest": "embedded/libonnxruntime.so.1.26.0" + }, + "tokenizers_static": { + "url": "https://github.com/daulet/tokenizers/releases/download/v1.27.0/libtokenizers.linux-arm64.tar.gz", + "sha256": "e96545ad05930c26f51f63d932ee6d3bbd32bbed149e102c5290d587a2293067", + "member": "libtokenizers.a", + "member_sha256": "aa4d3a7d28023e439fa7a645fb606195acafb5c33c8cd01928ed1aacfc4c074e", + "dest": "lib/libtokenizers.a" + } + }, + "linux-amd64": { + "model": { + "url": "https://huggingface.co/Xenova/multilingual-e5-small/resolve/main/onnx/model_quantized.onnx", + "sha256": "f80102d3f2a1229f387d3c81909990d8945513e347b0eab049f7de3c6f98c193", + "dest": "embedded/model_quantized.onnx" + }, + "tokenizer": { + "url": "https://huggingface.co/Xenova/multilingual-e5-small/resolve/main/tokenizer.json", + "sha256": "0b44a9d7b51c3c62626640cda0e2c2f70fdacdc25bbbd68038369d14ebdf4c39", + "dest": "embedded/tokenizer.json" + }, + "onnxruntime": { + "url": "https://github.com/microsoft/onnxruntime/releases/download/v1.26.0/onnxruntime-linux-x64-1.26.0.tgz", + "sha256": "1254da24fb389cf39dc0ff3451ab48301740ffbfcbaf646849df92f80ee92c57", + "member": "onnxruntime-linux-x64-1.26.0/lib/libonnxruntime.so.1.26.0", + "member_sha256": "5bd5bedf736fc501692435d0ec4f6e8b2bdf48cd30af8e6d00d61b3ddc9a7ab8", + "dest": "embedded/libonnxruntime.so.1.26.0" + }, + "tokenizers_static": { + "url": "https://github.com/daulet/tokenizers/releases/download/v1.27.0/libtokenizers.linux-amd64.tar.gz", + "sha256": "72556cdca798dd4ea7cdaba308e5f0d68a8cb93b67c96edf485b7a0edd7b07f4", + "member": "libtokenizers.a", + "member_sha256": "e6862b31745bb7d07980fcee70e49cd3b4318097609180f5d2d3fb394f305d50", + "dest": "lib/libtokenizers.a" + } } } } diff --git a/internal/embeddings/local/assets_embed_linux.go b/internal/embeddings/local/assets_embed_linux.go new file mode 100644 index 0000000..bbffb69 --- /dev/null +++ b/internal/embeddings/local/assets_embed_linux.go @@ -0,0 +1,18 @@ +//go:build localembed && linux + +package local + +import "embed" + +// Per-platform ONNX Runtime shared library (Linux). One binary can embed only +// one platform's ORT lib (epic Open Concern #8). Both Linux arches ship the +// same versioned file name — the arm64/x64 distinction is which artifact +// `make assets` downloads into embedded/ (see assets/manifest.json), so a +// single build-constrained file covers linux/arm64 and linux/amd64. +// +//go:embed embedded/libonnxruntime.so.1.26.0 +var embeddedORTLib embed.FS + +// ortLibFile is the base name of this platform's embedded ONNX Runtime shared +// library, both inside embedded/ and after extraction to the runtime dir. +const ortLibFile = "libonnxruntime.so.1.26.0" diff --git a/tray.go b/tray.go index 467a595..e0ab2d0 100644 --- a/tray.go +++ b/tray.go @@ -1,3 +1,5 @@ +//go:build darwin + package main /* diff --git a/tray_stub.go b/tray_stub.go new file mode 100644 index 0000000..10d8c1b --- /dev/null +++ b/tray_stub.go @@ -0,0 +1,9 @@ +//go:build !darwin + +package main + +// setupTray is a no-op on platforms without the macOS status-bar integration +// (tray.go is darwin-only Objective-C via CGo). Returns a no-op cleanup func. +func (a *App) setupTray() func() { + return func() {} +}