Skip to content

Add performance instrumentation and metrics #11

Description

@Lucasmind

Priority: LOW

Problem

There are no performance metrics, making it impossible to identify bottlenecks or optimize latency. Key measurements like time-to-first-token, TTS synthesis time, and end-to-end latency are not tracked.

Suggested Metrics

Metric Where Why
llm.time_to_first_token LLMClient.js Measures LLM responsiveness
llm.total_tokens LLMClient.js Tracks context usage
llm.tokens_per_second LLMClient.js GPU inference performance
tts.synthesis_time TTSManager.js TTS bottleneck identification
tts.chars_per_second TTSManager.js TTS throughput
tts.queue_depth TTSManager.js Queue saturation
stt.transcription_time STTManager.js STT latency
pipeline.end_to_end Orchestrator.js Total user-perceived latency
pipeline.time_to_speech Orchestrator.js Time from input to first audio

Suggested Implementation

// utils/metrics.js
class Metrics {
  constructor() {
    this.timers = new Map();
  }
  
  start(name) {
    this.timers.set(name, performance.now());
  }
  
  end(name, metadata = {}) {
    const start = this.timers.get(name);
    if (!start) return;
    const duration = performance.now() - start;
    this.timers.delete(name);
    log('info', `metric.${name}`, { duration_ms: Math.round(duration), ...metadata });
    return duration;
  }
}

export const metrics = new Metrics();

Files Affected

  • New file: frontend/src/utils/metrics.js
  • frontend/src/pipeline/Orchestrator.js
  • frontend/src/llm/LLMClient.js
  • frontend/src/tts/TTSManager.js
  • frontend/src/stt/STTManager.js

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions