Replace docs-mcp-server and Ollama with qmd for memory search - #27
Merged
Conversation
- Ships a single 610 MB embedding model and no daemon; reranking and query expansion are disabled by pointing their model slots at the embedder, so a search with default arguments cannot trigger a mid-query model download. - Uses a named qmd index under .claude/.kb-index/ instead of a project-local .qmd/, which is trust-gated and silently substitutes a weaker embedding model for non-interactive callers. - Records KB searches sent as typed searches[] rather than only a bare query field, so the delegation gate still sees them. Claude-Session: https://claude.ai/code/session_01M2kgbNZQeStEs1E8PjBWLs
- qmd-install now carries a single doctor check that asserts version 2.8.3. A second, broader check made isAlreadyInstalled short-circuit, so the pinned install was skipped on any machine that already had a different qmd — which then ignored QMD_CONFIG_DIR and read the wrong index config. - Both memory-loop doctor checks harden PATH the way the MCP launcher already did, and stay quiet when qmd is missing rather than reporting a second failure. - global_context is now three lines: qmd echoes it into every search result, so the long version was over half of a ten-result response. Claude-Session: https://claude.ai/code/session_01M2kgbNZQeStEs1E8PjBWLs
- Searches now ask for limit:5. Ranks past 5 never contributed a hit on the retrieval fixture, and the default of 10 nearly doubled the response size. - The lex and vec lines are described by what they take — keywords versus prose — since each is the only one that finds a whole class of memory. - States that result scores are 1/rank rather than confidence, so a poor match still scores 1.00 and the snippets have to be judged on their own. Claude-Session: https://claude.ai/code/session_01M2kgbNZQeStEs1E8PjBWLs
…repo - Drops the retrieval-bench row from the command table; the harness is kept outside the repo, so the command was unrunnable from a clean checkout. - Says where the quoted retrieval numbers came from, since removing the row left them with no visible source and nothing re-checks them. - Records limit:5 as part of the search call shape the four copies must share. Claude-Session: https://claude.ai/code/session_01M2kgbNZQeStEs1E8PjBWLs
There was a problem hiding this comment.
Pull request overview
Replaces the Ollama/docs-mcp-server retrieval stack with project-local qmd indexing and MCP search.
Changes:
- Adds pinned qmd installation, model setup, indexing, and health checks.
- Updates hooks, skills, and templates for qmd’s typed search API.
- Documents migration and expands gate tests.
Reviewed changes
Copilot reviewed 11 out of 12 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
.gitignore |
Ignores local index data. |
CLAUDE.md |
Documents qmd maintenance invariants. |
README.md |
Updates setup and migration guidance. |
SYNC-BLOCKS.md |
Updates synchronized indexing terminology. |
hooks/continuous-learning-activator.sh |
References qmd search. |
hooks/kb-gate.sh |
Records qmd queries and updates guidance. |
hooks/sync-memories.sh |
Implements qmd configuration and indexing. |
skills/continuous-learning/SKILL.md |
Migrates memory lookup workflow. |
skills/memory-audit/SKILL.md |
Migrates audit tools to qmd. |
techpack.yaml |
Defines qmd installation, MCP, and checks. |
templates/continuous-learning.md |
Teaches typed qmd searches. |
tests/kb-gate-test.sh |
Tests query payload recording. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
- The pack's gitignore component already contributes .claude/.kb-index/, so the repo-level copy only duplicated what mcs sync installs. Claude-Session: https://claude.ai/code/session_01M2kgbNZQeStEs1E8PjBWLs
- The gate no longer treats `intent` as evidence of a search. It never retrieves on its own, so a request that returned nothing could satisfy the barrier. - The indexer writes the memories path as a quoted YAML scalar, and detects deletions inside subdirectories — a nested memory could stay searchable after being removed, and a project path containing "#" indexed the wrong directory. - The version check is anchored, so 2.8.30 and 12.8.3 no longer pass as 2.8.3. Comments that restated invariants already owned by CLAUDE.md are gone; the ones explaining a specific line stayed. Claude-Session: https://claude.ai/code/session_01M2kgbNZQeStEs1E8PjBWLs
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 11 out of 12 changed files in this pull request and generated 2 comments.
Suppressed comments (2)
Previously missed (1) — in code that hasn't changed since the last review.
README.md:21
- Node 22+ is listed both as a prerequisite and as something
mcsinstalls automatically, leaving the installation contract contradictory. If users must supply the required version before syncing, remove Node from the automatic-install list (or instead remove it from the prerequisites ifmcsguarantees the version).
**Prerequisites:** macOS, [Claude Code](https://docs.anthropic.com/en/docs/claude-code), and Node 22 or newer. `mcs` installs the rest (Node, `gh`, `jq`, [qmd](https://github.com/tobi/qmd)) automatically, and downloads a ~610 MB embedding model once on first sync — shared by every project, with no daemon left running afterwards.
hooks/sync-memories.sh:116
- The staleness scan does not traverse a symlink supplied as
find's starting path. Since this hook explicitly supports.claude/memoriesbeing a shared-memories symlink, changes beneath that target make both probes empty after the first index, so this branch exits and leaves the search index stale. Follow the command-line symlink for both scans (for example,find -L) and also detect when the symlink itself is repointed.
newest=$(find "$MEMORIES_DIR" -name "*.md" -newer "$TIMESTAMP_FILE" -print -quit 2>/dev/null)
- Serialises reindexing and confirms nothing is left pending before touching the freshness timestamp. Overlapping async runs hit qmd's embed lock, which reports contention as success, so the index could be marked fresh with zero vectors and then skipped indefinitely. - The install check probes a throwaway index. A bare `qmd status` adopts a project-local .qmd/ by walking up from the working directory, so it was editing the user's own qmd config and creating their index. - The MCP server starts in projects with no memories yet, and the collection is registered on the first hook run, so a first memory is searchable without waiting for the next session. Claude-Session: https://claude.ai/code/session_01M2kgbNZQeStEs1E8PjBWLs
- A failed or incomplete run keeps its output in .claude/.kb-index/memory-loop.log and deletes it on success, so the file's presence is the signal and it holds the reason. Gating the timestamp alone stopped failures concealing themselves, but left a retry every prompt with no way to see why. - A new doctor check reports that log, since a broken index is otherwise visible only as searches quietly returning nothing. - Records the hook/check coupling through that file, and corrects the project-root invariant, which named three copies of a ladder that now appears six times. Claude-Session: https://claude.ai/code/session_01M2kgbNZQeStEs1E8PjBWLs
- `find` does not descend a symlinked root, and .claude/memories is a symlink whenever the shared-memories pack is installed — so the staleness gate saw zero files and froze the index for days with no failure log. The stamp was also written after the work, so files created during a run were skipped too. - Dropping the gate costs 0.42s of async work per prompt against the 0.056s it saved, and makes folder-vs-symlink a non-question: qmd resolves the path. - `.claude/.memories-last-indexed` is now unused; it is inert and can be deleted.
- Stubs qmd on PATH, so CI needs no model. The stub creates $INDEX_PATH on `update`, which is what lets the suite tell a skipping hook from a working one: a staleness gate guarded on that file falls through without it. - Asserts the second run, not the first — a first run reindexes under any version of the hook because the config file does not exist yet. Verified the suite fails 6 assertions against the pre-fix hook. - Bumps actions/checkout v6 to v7 across all three workflows.
- A result snippet is capped at 300 characters and collapses to the file's first three lines when the lex terms are not in the matched text, so all four instruction copies now say to get the document before relying on it. - Trims global_context 412 to 313 bytes by dropping the call-shape example the query tool's own schema already carries; it is echoed on every result, so the response at limit 5 drops from 4931 to ~4200 bytes despite saying more. - Keeps "never the bare query field" there — the tool schema recommends it.
- `lex` now asks for terms expected verbatim in the target memory: a guessed keyword misses and also picks the snippet, which is why 28% of results came back showing only a title. Documents prefix matching, which was unstated. - `intent` now says to name what to avoid, not just what to find; it steers ranking and selects which part of a document the snippet shows. - Keeps limit:5 with one carve-out — raising it appends a tail and never reorders the top 5, so re-query instead unless surveying a topic.
- The rerank fallback is not a strict no-op: a document RRF ranked 9th was dropped when `rerank: true` was passed. Top ranks held, so the structural disable stands, but "nothing changes" was wrong. - `qmd bench` measures the shipped path when its fixture query uses the structured form; only a bare query string is auto-expanded. - Replaces the manifest-parse command with `mcs pack validate`; the documented `/usr/bin/python3` has no PyYAML on a current macOS.
- Editing a memory re-chunks it and strands the old vectors; one audit that removed 4 memories left 275 orphaned chunks, 47% of the index. Reclaimed 966KB on first run. - Unconditional, at 0.116s — the same reasoning that removed the staleness check. The LLM cache it also clears is empty here, since nothing populates it while rerank and query expansion are disabled. - Best-effort: a failed cleanup writes no failure log and still exits 0.
- Drops four clauses that explain rather than instruct: the "costs twice" aside, the prefix-matching example, the limit survey carve-out, and the lex/intent paragraph in the skill that restated the CLAUDE.md template. - No behavioural change; all four copies still carry verbatim-lex, intent, limit:5 and the retrieve step.
- global_context carries identity only. qmd serves it both as the MCP server's instructions and as the context field of every result, so guidance there is billed per result while reaching no consumer the isRequired CLAUDE.md template misses. 342 to 126 bytes; a six-result response drops 5279B to 4001B, less than limit 5 cost before. - limit 5 to 6, matching what the previous backend defaulted to and what Claude already chose unprompted. The call shape is now stated in three places, not four — CLAUDE.md records why global_context is not one of them. - Cuts two explanatory clauses from the template and records the snippet mechanism in CLAUDE.md instead, where a future cleanup will read it.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Memory search ran through docs-mcp-server backed by an Ollama daemon — a stack with a documented run of failures in this repo: cold starts overrunning Claude's 30-second MCP budget, a vector store that refused to start non-interactively whenever the embedding model changed, and a runtime crash that surfaced only as unparseable errors from the embeddings endpoint. qmd replaces it with one binary that embeds in-process, so there is no service to install, start, or keep healthy.
Retrieval improves rather than merely simplifying. On a 20-query fixture over this repo's own memories the shipped configuration scores MRR 0.800 / R@5 0.875, against 0.764 / 0.850 for the previous stack, and answers in a fraction of the time. Upgrading needs no action: the old MCP server is deregistered on the next sync, and anything left behind is optional cleanup.
Changes
The rerank and query-expansion model slots deliberately point at the embedding model. That is not a copy-paste slip. The search tool defaults reranking on and offers no way to disable it server-side, and a missing rerank model is downloaded mid-query with no progress output — indistinguishable from a hang. An embedding model has no ranking head, so the engine warns and falls back to its fusion scores: a structural disable that costs no extra bytes and cannot be overridden by a caller.
Test plan
mcs sync --global, thenmcs doctor→ expect the memory checks green, including the embedding-model assertion.memory-loopconnected inclaude mcp list, a search returning memories, and any.qmdindex of your own left untouched.https://claude.ai/code/session_01M2kgbNZQeStEs1E8PjBWLs