[superlog] Log recoverable AI tool errors as WARN instead of ERROR

[superlog-app]

Summary

Scheduled insights generation jobs complete successfully but are logged at ERROR severity whenever the AI agent encounters a recoverable tool failure (e.g., a SQL query error or an annotation NOT_FOUND). This generates false-positive ERROR alerts in Superlog, masking real failures.

The insights service wires the AI tool logger to the job's wide-event logger (setAiRequestLoggerProvider(getActiveInsightsLog)). When any tool logged an error via requestLogger.error(new Error(message), context), evlog recorded the highest severity as ERROR. The job's success-path logger.emit({ job_status: "succeeded" }) then emitted the wide event at ERROR level — despite the job completing successfully and the AI agent having already recovered from the tool failure.

The fix changes createToolLogger.error() to call requestLogger.warn(message, context) instead of requestLogger.error(new Error(message), context) when a request-scoped logger is active. Tool-level errors are recoverable by design (the AI agent retries or works around them), so WARN is the appropriate severity. Actual job failures remain at ERROR because jobs.ts calls logger.error(err) directly on the unhandled-exception path before emitting with job_status: "failed".

An alternative approach would be to use requestLogger.set() to add error fields to the wide event without any severity change at all. That would suppress even WARN-level recording of tool errors; the current approach of using WARN keeps the tool failure visible in the job's wide event while preventing false ERROR alerts.

Incident on Superlog

---

Was this PR helpful? Leave feedback — goes straight to the Superlog team.

---

Summary by cubic

Log recoverable tool failures as WARN instead of ERROR to stop false alerts and keep successful jobs at the right severity. True job failures still log as ERROR.

• Bug Fixes
  • In packages/ai/src/ai/tools/utils/logger.ts, tool logger now calls request-scoped warn when available, preventing wide-event severity from escalating to ERROR.

^{Written for commit 9f68681. Summary will update on new commits.}

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
databuddy-status	Ready	Preview, Comment	Jun 26, 2026 3:38pm

2 Skipped Deployments

Project	Deployment	Actions	Updated (UTC)
dashboard	Skipped		Jun 26, 2026 3:38pm
documentation	Skipped		Jun 26, 2026 3:38pm

[unkey-deploy]

The latest updates on your projects. Learn more about Unkey Deploy

Name	Status	Preview	Inspect	Updated (UTC)
api (preview)	Ready	Visit Preview	Inspect	Jun 26, 2026 3:39pm

[greptile-apps]

Greptile Summary

This PR fixes false-positive ERROR alerts caused by recoverable AI tool failures (SQL errors, NOT_FOUND responses, etc.) escalating a job's wide-event severity to ERROR even when the job completed successfully. The fix is a one-line change in createToolLogger.error() that routes through requestLogger.warn(message, ...) instead of requestLogger.error(new Error(message), ...) when a request-scoped logger is active.

• The global log.error(...) fallback (no active request logger) is left untouched, preserving ERROR visibility outside of job scope.
• The WARN downgrade applies uniformly to all call-sites of logger.error() across every tool file (annotations.ts, funnels.ts, goals.ts, flags.ts, links.ts, insight-digest.ts, rpc.ts, query.ts), so any tool error that is genuinely unrecoverable will also be silently downgraded to WARN rather than ERROR in the request-scoped path.

Confidence Score: 4/5

The change is safe to merge — it eliminates a confirmed source of false-positive ERROR alerts without touching any job-failure or real-error paths.

The fix correctly addresses the described issue and the global fallback and job-level error paths are untouched. The .error() method on createToolLogger now unconditionally produces WARN in the request-scoped path, so if a tool ever surfaces a truly unrecoverable infrastructure error it would also be silently downgraded to WARN inside a job scope.

No files require special attention beyond packages/ai/src/ai/tools/utils/logger.ts, where the single-line change lives.

Important Files Changed

Filename

Overview

packages/ai/src/ai/tools/utils/logger.ts

Changes requestLogger.error(new Error(message), ctx) to requestLogger.warn(message, ctx) in the request-scoped branch of createToolLogger.error(), preventing recoverable tool failures from escalating job wide-event severity to ERROR. The global log.error fallback is unchanged. All callers of .error() across 8+ tool files will now produce WARN (not ERROR) in the request-scoped path, including any hypothetically unrecoverable tool errors.

Sequence Diagram

%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
    participant Job as insights job (jobs.ts)
    participant Logger as createToolLogger.error()
    participant RL as requestLogger (evlog)
    participant GL as log (global evlog)

    Job->>RL: setAiRequestLoggerProvider(getActiveInsightsLog)
    Job->>Logger: toolLogger.error("SQL query failed", ctx)

    alt Request-scoped (BEFORE this PR)
        Logger->>RL: requestLogger.error(new Error(message), ctx)
        Note over RL: wide-event severity to ERROR
        Job->>RL: logger.emit - job_status succeeded
        Note over RL: emits at ERROR - false-positive alert
    end

    alt Request-scoped (AFTER this PR)
        Logger->>RL: requestLogger.warn(message, ctx)
        Note over RL: wide-event severity to WARN
        Job->>RL: logger.emit - job_status succeeded
        Note over RL: emits at WARN - no false alert
    end

    alt No active request logger (unchanged)
        Logger->>GL: log.error service api aiTool message ctx
        Note over GL: global ERROR log preserved
    end

    alt Real job failure (unchanged, jobs.ts)
        Job->>RL: logger.error(err)
        Job->>RL: logger.emit - job_status failed
        Note over RL: emits at ERROR - real alert fires
    end

Loading

%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
    participant Job as insights job (jobs.ts)
    participant Logger as createToolLogger.error()
    participant RL as requestLogger (evlog)
    participant GL as log (global evlog)

    Job->>RL: setAiRequestLoggerProvider(getActiveInsightsLog)
    Job->>Logger: toolLogger.error("SQL query failed", ctx)

    alt Request-scoped (BEFORE this PR)
        Logger->>RL: requestLogger.error(new Error(message), ctx)
        Note over RL: wide-event severity to ERROR
        Job->>RL: logger.emit - job_status succeeded
        Note over RL: emits at ERROR - false-positive alert
    end

    alt Request-scoped (AFTER this PR)
        Logger->>RL: requestLogger.warn(message, ctx)
        Note over RL: wide-event severity to WARN
        Job->>RL: logger.emit - job_status succeeded
        Note over RL: emits at WARN - no false alert
    end

    alt No active request logger (unchanged)
        Logger->>GL: log.error service api aiTool message ctx
        Note over GL: global ERROR log preserved
    end

    alt Real job failure (unchanged, jobs.ts)
        Job->>RL: logger.error(err)
        Job->>RL: logger.emit - job_status failed
        Note over RL: emits at ERROR - real alert fires
    end

Loading

Comments Outside Diff (1)

1. packages/ai/src/ai/tools/utils/logger.ts, line 21-31 (link)

   error() unconditionally downgrades to WARN for all request-scoped callers

   Every call-site of logger.error() across the tool files (RPC errors, DB query failures, annotation/funnel/goal/flag mutation errors) will now produce WARN — not ERROR — whenever a request logger is active. That's correct for recoverable query failures, but any infrastructure-level tool error (e.g., an unhandled RPC transport failure in rpc.ts) will also be silently downgraded. If a future tool genuinely needs to surface an ERROR within a job scope, there's currently no way to do that through createToolLogger. Consider whether a two-tier API (e.g., a separate fatalError or an isRecoverable flag) would be useful, or at minimum add a comment on the public method signature noting that in request scope all errors are recorded at WARN.

   Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

_{Reviews (1): Last reviewed commit: "[superlog] Log recoverable AI tool error..." | Re-trigger Greptile}