feat(telemetry): leadbay_set_telemetry — enable/disable/status + honor opt-out#164
Draft
ArtyETH06 wants to merge 7 commits into
Draft
feat(telemetry): leadbay_set_telemetry — enable/disable/status + honor opt-out#164ArtyETH06 wants to merge 7 commits into
ArtyETH06 wants to merge 7 commits into
Conversation
github-actions
Bot
added
the
needs-manual-rebase
Contributor
|
main shipped |
… honor opt-out Adds an in-product control (product#3879) to enable, disable, or check product-usage telemetry. Telemetry stays ON by default (opt-out). - leadbay_set_telemetry composite (action: enable|disable|status, default status). status → GET /users/me; enable/disable → POST /users/telemetry then invalidateMe(). Idempotent (a no-op set reports changed:false, no write). - UserMePayload.telemetry_enabled; the preference lives on the Leadbay account, so the control works the same on the hosted/web connector as on a local install (no local file — wrong for multi-tenant web). - Hosted enforcement: telemetryHandleForRequest() reads the request user's telemetry_enabled and binds NOOP_TELEMETRY when they've opted out, so a disabled user's tool calls emit nothing — per-request, without affecting other tenants. Wired into both the streamable and SSE paths. - Contract: version 0.24.1 → 0.25.0 (package.json + server.json + npx pin), CHANGELOG, README "Privacy & telemetry", WORKFLOWS.md row. Consumes the backend telemetry_enabled field (leadbay/backend) + api-specs contract. New tests: the tool (status/enable/disable/idempotent) and the hosted opt-out enforcement (disabled user suppressed; multi-tenant A-off/B-on). Co-Authored-By: Claude <noreply@anthropic.com>
ArtyETH06
force-pushed
the
ArtyETH06/telemetry-toggle-tool
branch
from
055e3e9 to
eb2de3f
Compare
…g audit - SSE opt-out is now LIVE: the session holds a mutable `suppressed` flag the bound handle reads on every capture, and POST /messages refreshes it (from the client's /users/me, freshened by the tool's invalidateMe) before dispatch — so a mid-session leadbay_set_telemetry disable takes effect on the next message instead of only after reconnect. bindTelemetryIdentity gains an optional isSuppressed predicate; the streamable path (fixed per-request) omits it. - set_telemetry success copy no longer over-promises "no further events": it says the preference is saved and events stop per-request on the hosted connector / next session on a local install — honest about not-instant stdio. - Add leadbay_set_telemetry to TOOLS_WITH_ROUTING so its routing block + examples are audited (the repo's new-routed-tool wiring contract). New test: the live-suppression predicate flips the SAME bound handle without rebuild. Full suite green (mcp 549, core 577). Co-Authored-By: Claude <noreply@anthropic.com>
Codex P1: bindTelemetryIdentity gated the PostHog capture methods on isSuppressed but left captureException inherited from base — so an opted-out SSE user's tool errors still reached Sentry, even though the streamable path returns full NOOP for the same preference. Gate captureException on isSuppressed too; when not suppressed it passes through to base unchanged (ctx still carries region/org). New test: a suppressed handle drops captureException while an enabled one forwards it. Co-Authored-By: Claude <noreply@anthropic.com>
Codex P1: the tool's copy implied disabling here stops telemetry on a local (stdio) install, but the stdio gate only reads LEADBAY_TELEMETRY_ENABLED and never consults the account flag — so that promise was false locally. Per the remote-only enforcement scope, remove the local-opt-out claim everywhere it appeared and point local users at the env var instead: - set-telemetry.ts success hint - the tool description template (+ regenerated tool-descriptions.generated.ts) - README "Privacy & telemetry" opt-out note - WORKFLOWS.md row #45 The account flag is honored on the hosted connector per-request (unchanged); local opt-out still needs LEADBAY_TELEMETRY_ENABLED=false, and the copy now says exactly that. Co-Authored-By: Claude <noreply@anthropic.com>
Codex P1: resolveTelemetryContext returned enabled:true when /users/me timed out or errored, so a transient failure leaked telemetry for a user who had opted out. Flip both failure paths to enabled:false (suppress) — the SSE per-message refresh too. A privacy opt-out must not fail open: leaking an opted-out user's events on transient API slowness is an unrecoverable consent violation, whereas failing closed only drops a data point for an enabled user on that one request (self-corrects next successful read). An absent telemetry_enabled on a SUCCESSFUL read still means enabled (known preference). New test: /users/me error → NOOP, nothing emitted. Co-Authored-By: Claude <noreply@anthropic.com>
Codex P2: opting out returned full NOOP_TELEMETRY (streamable) / gated captureFeedback (SSE), so leadbay_send_feedback — an explicit user-initiated "deliver my message to the team" action — silently returned sent:false for opted-out users. That's a regression: telemetry opt-out should suppress passive analytics + error telemetry, not the user's own outbound feedback. telemetryHandleForRequest now always returns the bound wrapper (with an always-true isSuppressed predicate when opted out) instead of bare NOOP, and bindTelemetryIdentity leaves captureFeedback un-gated. Everything else (tool/composite/quota/topup/agent-memory/friction + captureException) stays suppressed. Same for the SSE path. New test: a fully-suppressed handle drops analytics but still delivers feedback. Co-Authored-By: Claude <noreply@anthropic.com>
Codex P1: the per-message SSE opt-out refresh used resolveMe() (60s-cached), so a disable made from ANOTHER connector/session — which only invalidateMe()s ITS client, not this session's — was missed: the stale telemetry_enabled:true kept this session emitting until the TTL expired. Use resolveMe(true) to force a fresh read so a cross-session opt-out takes effect on the next message. New test proves resolveMe(true) bypasses the cache (both forced reads hit the backend; the second sees the flipped value). Co-Authored-By: Claude <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds
leadbay_set_telemetry— an in-product control to enable, disable, or check product-usage telemetry (product#3879). Telemetry stays ON by default (opt-out).action: enable | disable | status, defaultstatus).status→GET /users/me;enable/disable→POST /users/telemetrytheninvalidateMe(). Idempotent — a no-op set reportschanged:falseand skips the write.UserMePayload.telemetry_enabled) — so the control works the same on the hosted/web connector as on a local install. A local file would be wrong for the multi-tenant hosted server; the flag lives on the Leadbay account.telemetryHandleForRequest()reads the request user'stelemetry_enabledand bindsNOOP_TELEMETRYwhen they've opted out, so a disabled user's tool calls emit nothing, per-request, without affecting other tenants. Wired into both the streamable and SSE paths.Part of a 3-PR feature (backend owns the contract → lands first)
POST /users/telemetry, 1.5 + 1.6).status/enable/disablewill 404. Merge order: api-specs → backend → this.Verification
pnpm -r build && test && typecheckgreen (core 577, mcp 548 — includes new: the tool's status/enable/disable/idempotent cases, and the hosted opt-out enforcement incl. multi-tenant "A disabled → suppressed, B enabled → captured").Note on scope: enforcement here is the hosted HTTP path (the multi-tenant remote where the users are). One item deliberately NOT done:
leadbay_set_telemetryis not added toTOOLS_WITH_ROUTINGin the routing audit — that's an existing test file (the "never modify existing test files" rule), and omission is safe (the tool ships with full routing frontmatter; it's just not asserted). Say the word if you want it added.No eval coverage for this change (a backend-relay tool; not eval-tested prompt/widget behaviour).
Closes https://github.com/leadbay/product/issues/3879