Priority: LOW
Problem
There are no performance metrics, making it impossible to identify bottlenecks or optimize latency. Key measurements like time-to-first-token, TTS synthesis time, and end-to-end latency are not tracked.
Suggested Metrics
| Metric |
Where |
Why |
llm.time_to_first_token |
LLMClient.js |
Measures LLM responsiveness |
llm.total_tokens |
LLMClient.js |
Tracks context usage |
llm.tokens_per_second |
LLMClient.js |
GPU inference performance |
tts.synthesis_time |
TTSManager.js |
TTS bottleneck identification |
tts.chars_per_second |
TTSManager.js |
TTS throughput |
tts.queue_depth |
TTSManager.js |
Queue saturation |
stt.transcription_time |
STTManager.js |
STT latency |
pipeline.end_to_end |
Orchestrator.js |
Total user-perceived latency |
pipeline.time_to_speech |
Orchestrator.js |
Time from input to first audio |
Suggested Implementation
// utils/metrics.js
class Metrics {
constructor() {
this.timers = new Map();
}
start(name) {
this.timers.set(name, performance.now());
}
end(name, metadata = {}) {
const start = this.timers.get(name);
if (!start) return;
const duration = performance.now() - start;
this.timers.delete(name);
log('info', `metric.${name}`, { duration_ms: Math.round(duration), ...metadata });
return duration;
}
}
export const metrics = new Metrics();
Files Affected
- New file:
frontend/src/utils/metrics.js
frontend/src/pipeline/Orchestrator.js
frontend/src/llm/LLMClient.js
frontend/src/tts/TTSManager.js
frontend/src/stt/STTManager.js
Priority: LOW
Problem
There are no performance metrics, making it impossible to identify bottlenecks or optimize latency. Key measurements like time-to-first-token, TTS synthesis time, and end-to-end latency are not tracked.
Suggested Metrics
llm.time_to_first_tokenllm.total_tokensllm.tokens_per_secondtts.synthesis_timetts.chars_per_secondtts.queue_depthstt.transcription_timepipeline.end_to_endpipeline.time_to_speechSuggested Implementation
Files Affected
frontend/src/utils/metrics.jsfrontend/src/pipeline/Orchestrator.jsfrontend/src/llm/LLMClient.jsfrontend/src/tts/TTSManager.jsfrontend/src/stt/STTManager.js